5 Steps to A/B Testing Your eCommerce Store

A/B Testing

I recently published an ebook for LemonStand on A/B Testing for eCommerce stores. You can read the full version here, but in this post I wanted to cover the basics of A/B testing and how it can increase your conversion rates and lead to more sales for your online store.

For many small business owners, especially eCommerce businesses, terms like conversion rate optimisation and A/B testing sound pretty complicated. The fact is, technology has made it easier technology has made it easier than ever to optimise your conversion rates. It’s no longer a growth strategy that’s only accessible by large companies with huge marketing budgets.

Simply put conversion rate optimisation is all about improving your conversion rates, the number of people out of every 100 visitors who purchase something from your store. For example, if you get 1000 visitors a month and 20 people buy, your conversion rates are 2%.

A/B Testing 101

A/B testing is a scientific way to optimise conversion rates. Instead of guessing what changes on your site will lead to higher conversions, you methodically test changes.

To illustrate, let’s say you have a banner on your product page saying ‘fast shipping’. You think if you changed it to ‘next day shipping’ more people will be inclined to buy the product. Instead of changing it permanently and hoping it works, you can test the two against each other.

All you have to do is create a copy of your original product page but with only one change – the wording on the banner. Your original page is the control (A) and the duplicate page with the change is the treatment (B).

Now here’s where the magic happens. Both pages, A and B (see why it’s called A/B testing?), are actually live, but only half your visitors see A and the other half see B. Because you’re running the pages at the same time any external factors that could affect conversion rates apply to both pages. That means, if there’s a difference in conversion rates between the two pages, it has to be because of the change you made.

A/B Testing

Ok, it’s not really magic, but it also doesn’t require hiring developers or spending a lot of money. There are softwares out there that do all this automatically, so you don’t even need any technical know-how.

A/B Testing Softwares

Google Content Experiments – This is a free tool by Google and can be integrated with Google Analytics to create and test conversion goals. However, you’ll have to manually code the variations and changes yourself.

Optimizely – An easy to use tool with a drag-and-drop interface that lets you create changes on your site without any coding. The software then automatically redirects traffic to each variation and collects data based on goals you set up in the dashboard. Pricing starts at $17/month.

Visual Web Optimizer – Another drag-and-drop software, VWO also has integrated heat maps and click maps. Pricing starts at $49/month.

Check out each one to see which best fits your needs. Optimizely and VWO also have free trials so you can create tests with both and see how they work. Once you have a software, it’s time to create the test.

The Process

Step 1: Research

Before you figure out what to test, you need to do a bit of research. Start by looking at your analytics data for your store. Find out what your conversion rates are for each product, what pages people visit the most, and what pages show the largest exit rates (people leaving your site from the page).

Conducting interviews and collecting feedback from customers will give you insights into what they’re thinking when they are on your site. Find out why they visit your site, and why they leave to explain the analytics data you collected.

Step 2: Create a Hypothesis

Now that you have all this information, you can figure out what to test. Let’s say your analytics tells you that many people visit your homepage and then leave the site immediately afterwards. Clearly, something is wrong with the homepage.

When you conduct interviews, you find out that people aren’t sure if the products you sell are authentic or fake. So obviously your homepage isn’t doing a good job of reassuring your customers.

You can now hypothesise that adding a badge on your homepage saying ‘Certified Dealer’, will ease visitors’ concerns. You can’t say for certain unless you test it out, but at least you have an idea of what to test instead of wasting time testing colours.

Step 3: Run the Test

Now you can actually test your hypothesis. With Optimizely and VWO, you can add a badge, an image or a few words with just a few clicks. When you’re ready, start the test and let the software do it’s thing.

As visitors come to your site, the software will split traffic randomly so that they see one or the other. You can see how each page is doing in terms of conversions. Are people still leaving the homepage with the badge or are they clicking through to the product pages and buying?

Step 4: Analyzing the Results

Don’t be too eager to call a variation a winner and conclude the test. Initally, there will be a lot of volatility in your conversion rates for both variations. As time passes and more people come to your site, you will find that the volatility decreases till there’s stability.

At this point, usually after a week or a few thousand visitors, you can decide which variation had a better conversion rate. Your software will help you decide this. You can then permanently make the change on your site so that all visitors see it and your store’s conversion rates increase.

Step 5: Repeat

Conversion rate optimisation is not a one-off thing you do on your site. It’s a continuous process that will lead to constant improvements in your conversion rates. Each test counts, no matter how small the improvement, and ultimately they will all add up. You’ll never convert 100% of your visitors, but there’s no harm trying to get close, right?

A Case Study

SmartWool, an online retailer of socks, wanted to increase their conversion rates on their category pages. These pages saw high traffic, but not many people were clicking on the products. They thought that changing the design might help.

Smartwool A/B Testing


As you can see, the original design showcased product images in different sizes. It’s a good design, but they thought that a more standard design, like the one on the right, might perform better.

They used Optimizely to create the variation and ran the two pages simultaneously. They tested it against 25,000 visitors and it became clear that their hypothesis was correct. The variation design actually performed better, and not only did it improve conversion rates, it also improved the revenue per visitor by 17.1%.

Take Action

It’s time for you to start taking your conversion rates seriously. With testing something as simple as image sizes, SmartWool dramatically increased their revenues.

If you think 17.1% is too small to be worth your time, think again. Let’s say they have 10,000 visitors a month with an average revenue per visitor of $10. That means their monthly revenue was $100,000.

After the test, their new average revenue per visitor was $11.7. Conversion rate optimisation doesn’t increase your traffic, it increases the number of people who buy from you. So SmartWool still has 10,000 visitors a month. That means, their new monthly revenue is $117,000.

That’s an increase of $17,000 every month from one test! A test that you can do without any major investment. Are you ready to start testing your store? Check out the full ebook by clicking the link below. It shows you exactly what you need to do for each step and gives you some great ideas about what you can start testing on your store.

Get the full ebook here!

Did you enjoy this article?
Get FREE Updates!
Actionable content and product reviews to boost your store's traffic and earnings. Straight to your inbox!

Leave A Response

* Denotes Required Field