A/B testing stands for testing A against B in terms of making a change to a website, and attempting to measure whether that theoretical change (B) makes a positive impact on sales , as compared to the original scheme of things (A) on the currently live platform. Qubit points out that despite the transparency of the concept, there are a number of ways that A/B testing can be executed erroneously, leading to a potential positive impact being missed, or a false positive impact being indicated. Qubit has done some research into the figures, and claims that properly carried out testing has demonstrated a 12% uplift in sales, which would total a USD 13 billion increase concerning US businesses.
Qubit lays out three major reasons for A/B testing failures, among which regression to the mean and insufficient statistical power, the latter being caused by failing to correctly judge the sample size for the test. There’s also the practice of running multiple simultaneous tests, which is an approach more likely to lead to false positives.
Every day we send out a free e-mail with the most important headlines of the last 24 hours.
Subscribe now
We welcome comments that add value to the discussion. We attempt to block comments that use offensive language or appear to be spam, and our editors frequently review the comments to ensure they are appropriate. If you see a comment that you believe is inappropriate to the discussion, you can bring it to our attention by using the report abuse links. As the comments are written and submitted by visitors of the The Paypers website, they in no way represent the opinion of The Paypers.