Personal Notes of A/B testing
I have helped my friend conducting A/B testing for her graduation. An online learning website.
At its simplest, A/B testing is a method for comparing 2 versions of something against each other to discover which is the most successful. The something can be an image, a button, a headline, or beyond.Multivariate testing is an expansion of A/B testing where more than 2 versions are compared and (often) more variation is included. The aim of A/B testing is to enable you to make incremental improvements to your website or app.
https://www.optimizely.com/ab-testing/
AB testing uses data & statistics to validate new design changes and improve your conversion rates.
Then, half of your traffic is shown the original version of the page (known as the control) and half are shown the modified version of the page (the variation).By testing ad copy, marketers can learn which version attracts more clicks. By testing the subsequent landing page, they can learn which layout converts visitors to customers best.
As visitors are served either the control or variation, their engagement with each experience is measured and collected in an analytics dashboard and analyzed through a statistical engine. You can then determine whether changing the experience had a positive, negative, or no effect on visitor behaviour.
AB testing can be used consistently to continually improve a given experience, improving a single goal like conversion rate over time.
A/B TESTING PROCESSTo conduct a basic A/B test, you first need an existing website or app. (
A/B testing facilitates incremental improvements to an existing product
and is not suitable for testing redesigns or new products and services).
Collect Data: Your analytics will often provide insight into where you can begin optimizing. It helps to begin with high traffic areas of your site or app, as that will allow you to gather data faster. Look for pages with low conversion rates or high drop-off rates that can be improved.
Identify Goals: Your conversion goals are the metrics that you are using to determine whether or not the variation is more successful than the original version.
Generate Hypothesis: Once you've identified a goal you can begin generating A/B testing ideas and hypotheses for why you think they will be better than the current version. Once you have a list of ideas, prioritize them in terms of expected impact and difficulty of implementation.
Create Variations: Using your A/B testing software (like Optimizely), make the desired changes to an element of your website or mobile app experience. Many leading A/B testing tools have a visual editor that will make these changes easy. Make sure to QA your experiment to make sure it works as expected.
Run Experiment: Kick off your experiment and wait for visitors to participate! Their interaction with each experience is measured, counted, and compared to determine how each performs.
Analyze Results: Your A/B testing software will present the data from the experiment and show you the difference between how the two versions of your page performed, and whether there is a statistically significant difference.
(Some tools (such as Google Analytics) don’t allow you to set who will see the original version versus who will see the alternative, or even how long the test is. This can be a useful feature for a beginner as it simplifies the overall process.)
Visual Website Optimiser https://vwo.com/ab-testing/
http://blog.invisionapp.com/ab-testing-beginners-guide/
“A/B testing can provide value, but it shouldn’t come at the expense of other areas of design.”One of the key aspects of A/B testing is that you change
only one variable at a time
. At first glance, this seems like a simple task, but it can be quite easy to overstep the mark and add more variables.Whatever test you decide to do, you also must understand and outline the key measurement (or metric) you’ll track.In the case of the button example, it’s most likely you’d measure the number of people who click on the button. For something like a headline change, you may wish to track the bounce rate or time on site.
When considering A/B testing, it’s important to understand what it cannot do:
Tell you why. A/B testing is a fantastic tool to understand what works and what doesn’t. What it cannot tell you however is why. For that you’ll need to conduct qualitative user research. This is a crucial element to understand—data does not equal understanding.
Let you test drastic redesigns of your website or app. While in theory, you could pit an entire page design against an alternative and get data on its performance—you wouldn’t be able to understand what about that design was causing any change in performance. Was it the design, the copy, the links? Unless you coupled such an exercise with user research, the results would be meaningless.
Tell you if you’re solving the right/wrong problem. Due to incremental nature of A/B testing, it can be a powerful tool to continuously improve your website or app. A/B testing can’t tell you if you’re solving the correct problem. You may be focusing your tests on the homepage and seeing improvement, but another area of the site might be the real problem. This is a concept known as the local maximum.
discoholicmusic
prguitarman
jacobvanloon
ellievsbear
