Challenges to consider

Challenges to consider when building your test

When building your first test, it is important to keep the technical limitations of automation in mind. Not everything is meant to be automated, or can be automated effectively.

📘

While building tests, keep in mind:

Waldo is designed to test your app from the experience of a brand new user coming into the application for the first time. For more information on recommended best practices, visit our methodology page.

Waldo helps you balance the benefits of automation without compromising the integrity of your tests, but there are some limitations to be aware of while building your suite.

We will lay out a few common examples below:

2FA or SMS verification

This step has a level of variability that is not conducive to automation. Waldo cannot confirm or perform random SMS/2FA verification steps in our simulator.

👍

Workaround for 2FA/SMS

There are two options for handling this in Waldo: disable these secondary verification steps OR auto-verify recognizable patterns for numbers in your tests. For more information, you can reach out to your support contact.

Date selection

When randomly selecting a date, any requirements applied to that date selection cannot be automated for example: if you need to book a hotel, the date selected would have to occur in the future for the app to progress appropriately. Waldo has a date picker that can help with this action.

Random marketing modals or pop-ups

Since these are randomly generated and not repeatable, they cannot occur during a test. This means they cannot be fully automated.

A/B testing variations

Much like the random pop-ups from above, you need to have repeatability to have success in automation. The nature of A/B tests is that they are not identical, allowing for different variations of acceptable behaviors, and thus: not repeatable.

👍

Workaround for A/B Testing

This can be solved in Waldo in a few different ways. One solution is to maintain separate test suites: creating an explicit test for each variation. For example: “Signup – Variation A” and “Signup – Variation B”.

For more solutions on A/B Testing and Waldo, click here.

User State Management
Tests are often dependent on the state of the "user" that is navigating through the flow. Understanding how to maintain required user states is critical to building a long-lasting test suite.

👍

For more on User State Management

Check out our article on the importance of User State Management or navigate to our Waldo Academy site to see several examples and best practice tips.