Automate UI tests

Testing user interactions helps ensure users do not encounter unexpected results or have a poor experience when interacting with your app. You should get into the habit of creating user interface (UI) tests if you need to verify that the UI of your app is functioning correctly.

One approach to UI testing is to simply have a human tester perform a set of user operations on the target app and verify that it is behaving correctly. However, this manual approach can be time-consuming and error-prone. A more efficient approach is to write your UI tests such that user actions are performed in an automated way. The automated approach allows you to run your tests quickly and reliably in a repeatable manner.

UI tests launch an app (or part of it), then simulate user interactions, and finally check that the app reacted appropriately. They are integration tests that can range from verifying the behavior of a small component to a large navigation test that traverses a whole user flow. They are useful to check for regressions and to verify compatibility with different API levels and physical devices.

Instrumented UI tests in Android Studio

To run instrumented UI tests using Android Studio, you implement your test code in a separate Android test folder - src/androidTest/java. The Android Plug-in for Gradle builds a test app based on your test code, then loads the test app on the same device as the target app. In your test code, you can use UI testing frameworks to simulate user interactions on the target app, in order to perform testing tasks that cover specific usage scenarios.

Jetpack frameworks

Jetpack includes various frameworks that provide APIs for writing UI tests:

  • The Espresso testing framework provides APIs for writing UI tests to simulate user interactions with Views within a single target app. Espresso tests can run on devices running Android 4.0.1 (API level 14) and higher. A key benefit of using Espresso is that it provides automatic synchronization of test actions with the UI of the app you are testing. Espresso detects when the main thread is idle, so it is able to run your test commands at the appropriate time, improving the reliability of your tests.
  • Jetpack Compose provides a set of testing APIs to launch and interact with Compose screens and components. Interactions with Compose elements are synchronized with tests and have complete control over time, animations and recompositions.
  • UI Automator (Android 4.3, API level 18 or higher) is a UI testing framework suitable for cross-app functional UI testing across system and installed apps. The UI Automator APIs allows you to perform operations such as opening the Settings menu or the app launcher on a test device.
  • Robolectric lets you create local tests that run on your workstation or continuous integration environment in a regular JVM, instead of on an emulator or device. It can use Espresso or Compose testing APIs to interact with UI components.

Flakiness and synchronization

The asynchronous nature of mobile applications and frameworks oftentimes makes it challenging to write reliable and repeatable tests. When a user event is injected, the testing framework must wait for the app to finish reacting to it, which could range from changing some text on screen to a complete recreation of an activity. When a test doesn't have a deterministic behavior, it's flaky.

Modern frameworks like Compose or Espresso are designed with testing in mind so there's a certain guarantee that the UI will be idle before the next test action or assertion. This is synchronization.

Test synchronization

Issues can still arise when you run asynchronous or background operations unknown to the test, such as loading data from a database or showing infinite animations.

flow diagram showing a loop that checks if the app is idle before making a test pass
Figure 1: Test synchronization.

To increase the reliability of your test suite, you can install a way to track background operations, such as Espresso Idling Resources. Also, you can replace modules for testing versions that you can query for idleness or that improve synchronization, such as TestCoroutineDispatcher for coroutines or RxIdler for RxJava.

Diagram showing a test failure when the synchronization is based on waiting for a fixed time
Figure 2: Using sleep in tests leads to slow or flaky tests.

Architecture and test setup

The architecture of your app should let tests replace parts of it for testing doubles and you should use libraries that provide utilities to help with testing. For example, you can replace a data repository module with an in-memory version of it that provides fake, deterministic data to the test.

Production and testing architectural diagrams. Production diagram shows local and remote data sources providing data to repository, which in turn provides it asynchronously to the UI. Testing diagram shows a Fake repository which provides its data synchronously to the UI.
Figure 3: Testing a UI by replacing its dependencies with fakes.

The recommended approach to enable this functionality is using dependency injection. You can create your own system manually but we recommend using a DI framework like Hilt for this.

Why test automatically?

An Android app can target thousands of different devices across many API levels and form factors, and the high level of customization that the OS brings to the user means your app could be rendered incorrectly or even crash on some devices.

UI testing lets you do compatibility testing, verifying the behavior of an app in different contexts. You might want to run your UI tests on devices that vary in the following ways:

  • API level: 21, 25, and 30.
  • Locale: English, Arabic, and Chinese.
  • Orientation: Portrait, landscape.

Moreover, apps should check the behavior beyond phones. You should test on tablets, foldables, and other devices.

Additional resources

For more information about creating UI tests, consult the following resources.

Documentation

Codelabs