Successful A/B testing can increase your understanding of customers needs and lead to increased conversions. A recent study shows better UX design as a result of user testing can increase a company’s conversion rate by 400 percent. If you've always wanted to A/B test your ecommerce store but not sure where to start, we'll give you some tips to get started.
What is A/B Testing & Why is it Important?
A/B testing is a technique in which two versions of the same page are shown to users at random. The goal is to determine which version performs better. Many ecommerce businesses use A/B testing to help optimize their website experience and increase conversion rates.
Common ecommerce A/B test examples:
There's an infinite number of variations that can be tested to improve each user's experience. Choosing the right elements is key to a successful experiment. Here's a few common A/B test examples you can try:
- Checkout page - How can you adjust your checkout flow? Try one page vs multi-page checkout.
- Product page - How are you displaying your products? Test different product image sizing.
- CTAs - Does shopper behavior change based on CTA phrasing, color, or placement?
- Promotions - Track how shoppers interact with promotions vs no promotions.
Now that you have an idea of what you can test, here's some best practices to run an effective A/B test on your ecommerce website:
8 Ecommerce A/B Testing Best Practices:
Have a hypothesis
Think of this test like a science experiment. You'll need to create a hypothesis for what you think will happen when you implement the change. For example, you can A/B test a new payment option for customers to see if it will increase conversion.
Calculate your A/B test sample size
Before starting your A/B test, make sure you have enough traffic to generate a statistically meaningful test in a reasonable amount of time. Your A/B testing tool may offer a sample size calculator. If not, there are many free sample size calculators that you can use. Calculate the estimated amount of traffic needed in the sample size calculator and divide that by your average daily traffic to estimate how long the test will take.
Run the test when you have normal traffic (or as close as possible)
It's best to not run an A/B test if you're expecting an irregular influx of traffic to your site. Customers may behave differently around Black Friday or major holidays, shopping with increased urgency. During the holiday season, ecommerce websites may receive an increase in visitors that don't normally shop on their site. It's more effective to test regular visitors because what works during the holidays may not work during the rest of the year.
Set up the test
Once you have enough traffic to complete the test in a reasonable amount of time, set up and start the test. If you have an A/B testing tool, it’s generally pretty easy to set up A/B tests that are straightforward like changing the text for CTAs or testing different images. More advanced tests (e.g., testing a new sight navigation) may require a developer. If you're A/B testing in Google Analytics, Google Optimize is a free tool that works well for most A/B tests.
Check for duplicate transactions
Google Analytics is one of the most popular web analytics tool in the ecommerce industry. However, unless you take the necessary steps to prevent Google Analytics from recording duplicate transactions, your numbers may be inflated. Every time a customer lands on the page that fires the transaction, Google Analytics will record a duplicate transaction for that transaction ID, as well as duplicate revenue.
Duplicate transactions typically happen when users restore the order confirmation page from a closed browser session on their desktop or mobile device. This occurrence can greatly skew data in general, especially for A/B tests. It's best to eliminate duplicate transactions before running an experiment.
More information on how to identify and prevent duplicate transactions in Google Analytics can be found here.
Remove non-human website traffic
Excluding bots and spiders from your ecommerce website is a great way to see how much actual human visitors you're getting. Bot traffic is any non-human traffic coming to your website. Similarly, Spiders are search engine bots that crawl webpages. Once removed you may see a significant decrease in traffic, however, this will make your data much more accurate.
Google analytics makes filtering out known bots simple. In your Admin go to View Settings and select the check box to begin filtering out bot traffic.
Filter out internal traffic
It's important to exclude internal traffic so that you're only measuring the reaction of shoppers on your website. Internal traffic can include call centers, sales reps, or development teams. Due to the pandemic's increase in remote work, you may want to also consider filtering out staff's IP addresses.
To filter out internal traffic in Google Analytics, go to Admin>All Filters then add a new filter. Leave Filter Type as predefined. It should read: Exclude + traffic from the IP addresses + that are equal to. Then, add the internal IP addresses you'd like to exclude. See Google's in depth tutorial.
Analyze Your Results
Your A/B testing tool should indicate if your test has reached a statistically significant conclusion, based on all visitors during the experiment. It can be very useful to drill down to see the results by sub-groups that are important to your business. Try viewing the test results by New and Returning Visitors, by device category, and for your key acquisition channels. You can also use you results to run a more advanced A/B test or try a similar test on other pages or CTAs.
Conclusion
If executed properly, A/B testing can be a powerful tool to drive ecommerce growth. A/B testing should be done regularly so that you can always give your customers an optimal experience, ultimately increasing conversions.
