Running an A/B test on your website (also called split testing) require so knowledge and preparation before you start. It's also important that you know the "rules of A/B testing" before you draw conclusions of your results.
The three stages of A/B testing
The stages of A/B testing are usually split up into three different categories.
The preparation stage is essentially where you will be planning and structuring your experiment. You will figure out what you want to test and how you're going to do it.
You will then be able to run you A/B experiment, which can take some time.
In the end, you need to evaluate the results of your experiment and identify any lessons that can be applied to future marketing efforts.
Preparation
Run experiement
Conclusion
The preparation stage
A crucial part of running an an A/B test on your website is doing the work and preparing before you actually do the experiment. Here are 5 things that you should have an answer for before you go to the next step and run the experiment.
Objective
What type of knowledge do you want to gain from this?
Do you want to increase conversion rate on you website? Do you want to decrease bounce rate? It is essential to be precise about the purpose of the test.
Variables
It is necessary to determine which factors you are assessing during your experiment. Are you evaluating different copy or a different color on your CTA button? Make sure you are clear on what the experiment is is designed to test.
Metrics
Identify which metrics you would want to measure to find out if the variables your testing will outperform others. Some examples could be conversion rate, amount of conversions etc.
You would need two type of metrics here. First, you need an exposure metric, in this case it would be page visits or sessions. The second metric you would need is a conversion metric or engagement metric, for example conversion rate.
Hypothesis
You always want to have an hypothesis going into the test. Is landing page A going to perform better with a new hero images than landing page B? It's OK going into a split test with an open mind and without a strong opinion on which will perform better.
Parameters
One parameter you need to set is the required sample size to draw any reasonable conclusion from the experiment. It could be like 1000 visitors for each landing page.
Another parameter is significant level. How much error in your data are you willing to accept? Typically you would set a confidence level at 5% which means there is a 5% chance that the result is due to randomness.
Running the A/B test on you website
Now you've prepared and everything is in place for a successful A/B website test. Running the experiment could take days, weeks or months depending on your parameters and metrics you are measuring.
Drawing conclusions from your test result
In the final stage you will declare a winner. Note that in some instances there might not be a clear winner.
You will take what you learn from this experiment and use it to improve your website. Perhaps the hypothesis you thought would outperform ended up underperforming. That can be disappointing, but it's all about keeping an open mind and learn from the experiment.
Tips on how to run a successful A/B test
Test only one variable at the time
When doing an experiment is important to only change one variable at the time. Let's say you change both a button color and a headline and the new variation wins, you have no idea which one of those made the difference. Was it the button color or was it the new headline?
Make sure the page have had enough visitors before ending your test
A/B testing websites needs a decent sample size of visitors to actually declare a solid winner. If you only get like 100 visitors a month to the page you're trying to split test, then you need to run the experiment for a few months before you have enough data to draw a conclusion.
Make sure both landing pages has equal amount of visits
Going back to exposure metrics here. You really need both pages to have equal or at least close to equal amount of visitors. You don't want to be comparing page A with 1000 visitors against Page B with 200 visitors.
Start with simple tests when you're doing it for the first time
A/B testing can get complicated so start doing simple tests in the beginning. It's easy to run split tests in the wrong way and drawing false conclusions based on either not enough data or simply doing it the wrong way.
Here are some experiments you can run on your website
- Changing colors
- Changing font size
- Different font styles
- Position of call to action
- Call to action copy
- Headline
- Price
- Short form vs long form
- Images