If we don't want to join the pantheon of games known from the compilation of funny glitches on YouTube, we must not underestimate the process of thorough testing. Although the automated testing department is great for testing "real" software, such as business tools, office tools, etc., in the case of games (and mobile games as well) the main role must still be played by so-called manual testers.
They must meticulously check every detail of the production and its compliance with the assumptions made during the design. How such tests will be run depends on the given production and its complexity - that's why it's hard to put it into some general frames of "rules".
On the other hand, there are a few mobile market-specific tests that should be mandatory if you want your app to match the industry standards. They can be divided into three more general categories:
- Performance tests
- Server/API communication tests
- Network performance tests
Below you will find an expansion of these categories with a number of basic factors, the checking, and optimization of which will make your production pleasant to use and not cause problems for players, and will save you hours spent on support of such production.
Performance tests
A basic and obvious category of testing - a user may abandon a production that behaves unstably through normal and ordinary annoyance. What might be the "hot spots" to look out for?
- Game start time
Does our game take a bit too long to start?
Does the user have the impression that his phone "stutters" after clicking on the icon of our application?
Generally, it is assumed (based on surveys and studies of user behavior in the application) that the maximum time of "black screen" which the user is able to accept is only 1-2 seconds! After this time something should obligatorily appear (at least our studio logo). If we are not able to skip loading some elements at the start (because, for example, we are limited by the engine we are using), it would be good if the loading screen contained some animation that would "enliven" the waiting time and convince the user that something is happening. If it's the first launch, during which we download additional content from the Internet, it's also a great place for, for example, a comic story introducing the player to the world presented in the game.
Of course, as much as possible, you should give the user access to the game as soon as possible - or at least to its menu, where the initial loading can continue, unnoticed by the user (e.g. ad networks, first board textures, etc.) which will later shorten the loading time of even the gameplay screen.
- Battery life
Graphically demanding productions are definitely a challenge for the power-efficient and passively cooled chips of mobile devices. This means that a poorly optimized production will "suck the juice" out of the battery at an express pace, and will additionally heat up the phone, resulting in a self-perpetuating spiral that will either drive the gamer to seek a charger (which we wish for everyone), but may also drive them to turn off the demanding production.
How to overcome this? First of all, optimize the resources used in the game - maybe not all the textures need to have a huge resolution, we do not need to load them all at once, we can disable unused objects in the game - there is a lot of it and it also depends on the engine used and the efforts of their creators in this matter. Nevertheless, we have to do "whatever we can" not to expose ourselves to unpleasant comments from users who want to cool down their phones in a bowl of water.
- Memory consumption
This point is also somewhat a result of the previous point - the use of memory optimization will also positively affect battery life. In general - the more additional functionality our production gains, the more memory it will use. If we write these functionalities ourselves, we have control over it, but nowadays we often use various ready-made APIs and libraries that someone has already written in some way and we have less control over it.
When should the red light go on? Definitely when the application consumes more memory than the operating system, and especially when it is turned off. This is a common cause of so-called social games, which have active mechanisms for push notifications and continuous integration with social networking sites.
Of course, the obvious case is when the game uses so much memory that it is turned off by the operating system. Unfortunately - this means a lot of work on optimization.
- Different hardware and software versions
A problem primarily concerns Android and the associated fragmentation. It is necessary, unfortunately, to stock a number of devices from different manufacturers - Samsung, Xiaomi, Huawei, LG, Lenovo, and whatever comes to mind - and to have a number of devices with more "historical" versions of this system. Of course, it is worth using current statistics on the most popular devices - although, of course, this 0.01% still using Android 4 KitKat will appreciate our efforts to optimize the production for this system, so it is worth thinking about whether they are worth the cost ;)
This problem affects the Apple ecosystem to a lesser extent - although, of course, it's worth getting a few devices, but you can be tempted to say that here - "if it works smoothly and without any problems on an older iPhone (on the same software version!) it will also work safely on a newer one. If we have the means to do so - of course, it would not be a bad idea to check for sure.
- The application running in the background
An application that goes into the background must retain all the information with which we left it so that returning to it allows us to resume the game immediately. Ensuring this is not so easy, due to the various operating system tools that "scour" memory for resources. It is necessary to properly protect our application data from such behavior.
Server/API communication tests
When a game communicates with an external server through an API, communication in the shortest possible time remains a priority. What factors should you pay special attention to in order to make it work that way?
- Data transfer to/from server
The data received from the server must be processed in an efficient manner. It is very important here to avoid the time-consuming conversion of data from one format to another. As far as possible - store data in the format as it comes from the server. The conversion process is often visible to the user as "clipping" of the application and negatively affects the reception.
- As few API requests as possible
Since each query has to be processed by the server, and it may have to deal with a huge number of them in case of bigger productions, we should limit the number of queries sent by our application. If it is possible, we should use mechanisms of grouping many potential queries into one. Of course, it is not always possible, but it is worth trying. Especially if our API is located on the cloud of a larger provider, who accounts for the number of queries ;)
- Server availability
If the server goes down, there are two paths. For smaller productions, we can use local backup mechanisms - so have a small local database inside the application that will serve as temporary storage for our local data and then upload it to the server when the time is right - or deny users from using the network functions if we don't have bots implemented that can pretend to be online players ;)
The second solution is, of course, to have a backup server (synchronized with the main server), or even a network of servers able to seamlessly take over traffic from each other depending on demand and the state of the network. However, this is a solution for large studios that can afford it. Alternatively, using cloud services gives us such possibilities in the package, but has its other disadvantages. However, this is an issue for a dedicated post.
Network performance tests
Communication on the application-server level is one thing. It should also be remembered that not everyone will be in our studio, which probably has a pretty good internet connection with a good transfer. People will be playing in trains, buses, on the green meadow and underground too. This makes it worth having some things secured already at the beginning - of course handling the total lack of internet is a priority and it is worth it that the application does not hang because of it. Nevertheless, there are several other factors related to impaired network access to consider:
- Jitter
When there is a delay in receiving information from the network, this term is referred to as jitter. Since network data comes in packets, they may be in the wrong order, for example. It is the task of our application to deal with this, although of course the network mechanisms of commonly used engines (Unity, Unreal) already have integrated mechanisms to protect against this. Nevertheless, if the case is specific (e.g. real-time network game), we have to pay a lot of attention to the implementation of proper protection solutions. And good communication for the user - he must know that his connection is unstable, not to blame us ;)
- Packet loss
Sometimes, however, the packets do not arrive at all and there is no way to "recreate" them without asking again. Therefore, it is worth informing the user about this (laconically, of course) and encouraging him to try again.
- Connection speed
The last factor is to test different connection conditions. So, Wi-Fi connection should be considered, as well as packet data. 3G, LTE, 5G networks - under different conditions. It is worth preparing on the test bench a configurable router "weakening" the signal, as well as physical barriers such as aluminum covers.
It is also important to check how the game behaves when switching between connection types - from WiFi network to packet data and vice versa. Our game can then not crash, but have programmed exception prepared for this occasion and send the required data again using a new connection - and the user does not really need to know about it unless something went wrong.
Of course, each production is governed by its own laws, and it is worth noting that this is only a rough list, and the specific features resulting from the technologies used by you may impose further necessary points. Nevertheless, you can put the above points "permanently" into your test checklist, as a solid starting point - and if in doubt, you can also use the help of our Playstrict platform, where the appropriate tools (and experts in the chat) will point out the weak points of your production and suggest improvements.
