Why is A/B testing so important?
Of course, there are people in the industry and among the developers who have such a perfect sense of the expectations of fans and players that they always "hit the spot". However, over the years, it has been recognized that the best results always come from testing every point in the production where different paths to solve a given problem appear - if only to confirm the opinion of an "industry genius".
The most popular and easiest in "execution" form of testing aspects of our production are A/B tests - mainly due to their implementation in most of the tools that accompany the mobile developer's way - both in Google Play and App Store, as well as development environments such as Unity.
Their principle is simple - we take anything in our game in which we hesitate over two versions of the same solution - the variable can be color, text, sound, absolutely anything. Then, we divide our test group into two parts - one is given solution A, the other solution B. If one of the groups is more promising - it either wins in case of a crushing advantage or gives an opportunity for further tests, with more comparative versions.
However, the availability of tools is one thing, another issue is the mistakes that are often made. So it is worth mentioning a few aspects that will protect us from a bad A/B test.
Take it easy!
Remember that testing is a slow process. It involves permanent iteration - small changes to bring the best possible results. If you don't know how to go about it, it's a good idea to start with a simple "check-list" that you can loop through to improve your results:
- Identify the goal - identify what you want to improve by changes (lower number of rejections, more conversions, more retention, etc.).
- Build a hypothesis - think about what potential impact your change could have on the final outcome
- Create a test - set the appropriate variants in your tool and...
- Wait - run the test for the intended time and wait until the end - don't be influenced by the residual data obtained from the test
- Analyze the result - if you achieved your goal - great, if not, it's time to go back to square one.
Patience is the basis for success.
Ea tenetur sed molestiae ut eos tempore. Dolorum praesentium quo voluptatem. Dolorem aliquam distinctio sit quia iure magnam cum dolorem. Id cupiditate fugit sit This is a task for the persistent people - if you don't have this attribute, delegate this task to someone else. Ending the test too early, for example because of unsatisfactory results, will not give the results back in full and you will never see the full picture. That's why you should always bring your tests to the end.
Maintain uniformity throughout the test period.
You can only draw meaningful conclusions about a test if you change one thing in one test. If you change several variables during one test, you will never get the correct intention.
For example, if it is an application icon - during one test change either only the background or only the object in the icon. If you change both at the same time, you will not find out which background fits better, nor which object looks better.
Don't change target groups between tests of the same thing.
Testing will be pointless if you suddenly decide to change the demographics of the test group by changing any of the variables. If you would like to change target groups - maybe treat them as a variable in the test - i.e. test the same thing in the game on different target groups and see where our product fits more.
To learn more about the necessary steps in mobile game development, join Playstrict early access today!
Fill out the contact form at playstrict.com and look forward to starting an amazing adventure!
