Annual losses witnessed by online retailers due to inadequate A/B testing could reach USD 13 billion - study

A/B testing stands for testing A against B in terms of making a change to a website, and attempting to measure whether that theoretical change (B) makes a positive impact on sales , as compared to the original scheme of things (A) on the currently live platform. Qubit points out that despite the transparency of the concept, there are a number of ways that A/B testing can be executed erroneously, leading to a potential positive impact being missed, or a false positive impact being indicated. Qubit has done some research into the figures, and claims that properly carried out testing has demonstrated a 12% uplift in sales, which would total a USD 13 billion increase concerning US businesses.

Qubit lays out three major reasons for A/B testing failures, among which regression to the mean and insufficient statistical power, the latter being caused by failing to correctly judge the sample size for the test. There’s also the practice of running multiple simultaneous tests, which is an approach more likely to lead to false positives.