The benefits

Managing user states correctly is crucial to mobile test automation. It helps remove any manual step from the test automation process, but it also unlocks other benefits of automation, such as enabling concurrent runs.

The basics

A key aspect of automated testing is ensuring that each test is deterministic and isolated.

• Isolated means that it should not matter if a test runs before, after, or at the same time as another test.
• Deterministic means that the same input produces the same output (for example, clicking on button A always shows the same screen as a result).

How do you ensure this behavior? This is where state management comes into play. Continue reading to dive deeper into what it means.

State management

Usually, the state of an app comes from who is currently logged into it. Getting this right is critical for your testing strategy. There are two ways to be logged into your app: “Log into an existing account” or “Create a pristine account for your test.” In the following walkthrough, we’ll sometimes use the example of a social app where the user experience differs based on whether you have zero friends, or you have ten friends.

With this example in mind, let’s examine the pros and cons of relying on Login or Sign Up to generate the desired state.

Login

Pros:

• Your test can load some relatively complex states (for instance, an account that already has 10 friends).

Cons:

• The states from all previous runs accumulate with each run of your test. For example, if a user has 10 friends, and the test adds a new friend, the account will open with 11 friends on the next run instead of the intended 10.
• Thus, for every action, your test would need to undo that action for next run to pass (in this case, the test must delete the newly added friend to maintain the proper state). This is not ideal because you cannot rely on a test to always succeed. (The test could fail after the new friend has been added, but before that friend is deleted.)
• Even if you undo action for the next run, you are still blocked from running the test in parallel 😱.
• The account could be deleted arbitrarily (for instance, during a database reset on the staging environment).

❗️

Warning

We strongly recommend against using the login approach, ever. You will always get into trouble down the line.

Sign Up

There is one super simple way to make sure that you get the same fresh state every single time: create a new account via sign up.

Pros:

• You get a clean state every time.

Cons:

• It can require many actions to get the account into the correct initial state you want to test from (for instance, you want to test what happens when a user has 10 friends).

The right way to manage state

The simple approach to Sign Up

In some simple cases, starting from Sign Up will work just fine (for instance, if all the user interactions in your app are accessible from the home screen immediately after sign up completes or do not require specific conditions to be accessible). Some of our customers do not even need to think past that.

The advanced approach to Sign Up

In other cases, things can get more complex. Looking at our social app example, if you start from a fresh state, you would need to run the "add a friend" test 10 times to add 10 friends to a pristine new account. It makes the test run longer, and requires you to wait before being able to test what comes once a user has 10 friends.
Good news is, there are methods to make your test suite more efficient. We recommend these two seeding method options:

1. On-creation seeding

• Based on the pattern of the username.
• You can load a specific state for a particular account.

📘

Example

On the sign up screen for a social app, you can set up your backend so that any username with the pattern “username+lonely” will log in as a user with zero friends. Alternatively, a username with the pattern “username+friends” will log in as a user with 10 friends loaded.

2. On-demand seeding

• At any step in your test, you can load some assets/state into the account.
• Options include using deep links or a debug menu. The latter option achieves the same thing but has the advantage of allowing you to access all commands actionable by deep links from a single location.

📘

Example

Using our social app example, you might want to test interactions only available to users with 10 friends. In this case, simply start with a fresh sign up, and on the desired step, use the deep link/debug menu to add friends to that user and test these interactions.

Side-by-side comparison of seeding methods