What is A/B Testing?

We use different words for A/B Testing from split testing to bucket testing and also with another form multi-variate testing. At the core, it is a method of comparing two versions of a web page or a mobile app screen. Both of the versions are displayed to different users or the same users at different times to track which one performs better. AB Testing this way is essentially running experiments at random. As these experiments are run, statistical analysis is applied to identify which version has better performance. Which version has a beter chance of reaching the conversion goals. This technique can be reliably applied on websites with significant enough traffic to produce meaningful results.

Featured web applications

Click on one of our projects below to view a detailed case study

• 

  eBay multi-sided platform

• 

  Online Event Marketplace

• 

  A bespoke Web Application

• 

  Web application in a router

What components can be A/B tested?

For web platform owners, A/B testing brings cost savings and development benefits. Here we list of some areas that can benefit from A/B testing:

- Email subject lines
- CTAs or Call to Action on websites
- Hero images on the home page
- Images, audio and video on pages
- Product layout page
- Social proof & Sensitivity analysis
- Email marketing
- Landing pages

A/B Testing Frameworks & Technology used

There are different approaches we use. In some cases, we use own web frameworks to run the A/B testing. In other cases we rely on external companies to run the A/B testing for us. The choice is made based on the web project. If the website or web application requires testing of various components at scale and these experiments are to be run at different intervals. They might suit external tools like Crazy Egg, Optimizely, etc. For the project with eBay called SecretGuru, we carried out large scale testing using an internal framework that was purpose built.

A/B testing permits people, groups, and organisations to roll out cautious improvements to their users while gathering information on the outcomes. This permits a web project owner to develop theories, and to learn better why certain components of their encounters sway customer behaviour. In another manner, their hyphothesis can be refuted, the conclusion and best guesses for a given objective can be refuted through an A/B test.

How is SEO impacted by A/B Testing

Popular search engines themselves use A/B testing and are seen to encourage ways to engage with users to increase conversions. These multivariate tests and A/B testing on web pages and web applications do not pose inherent risk to lower your website's rank. However we have notice that search engines are wary to cloaking. Cloaking means showing a search engine a specific variety of a web page while showing the user another variation. To prevent cloaking, do not segment visitors based on detection of a web page bot. Also we recommend that the web page experiments are only run as long as it is an experiment and not continuously. Finally for the sake of a website user, we prefer to give them some continuity for their web experience. We recommend the use of cookies (browser side to avoid GDPR issues) to keep a consistent page and avoiding displaying different web page content each time they load the page. A lot of these challenges do not apply to A/B testing on mobile apps compared to web pages and web applications.

About Webdigi & Services

We are a web development company in London. We specialise in developing bespoke web applications & unique web platforms. Over the years we have helped clients build great web applications. From time to time we publish open source tools, projects and articles like this to raise awareness and contribute back to the open source community that we rely on so much!