Scale your tests with Gradle Managed Devices

Gradle Managed Devices improve consistency, performance, and reliability for your automated instrumented tests. This feature, available for API levels 27 and higher, allows you to configure virtual test devices in your project's Gradle files. The build system uses the configurations to fully manage—that is, create, deploy, and tear down—those devices when executing your automated tests.

This feature grants Gradle visibility into not only the tests you’re running, but also the lifecycle of the devices, thus improving the quality of your testing experience in the following ways:

• Handles device-related issues in order to ensure your tests are executed
• Utilizes emulator snapshots to improve device startup time and memory usage and restore devices to a clean state between tests
• Caches test results and reruns only tests that are likely to provide different results
• Provides a consistent environment for running your tests between local and remote test runs

Additionally, Gradle Managed Devices introduce a new type of emulator device, called the Automated Test Device (ATD), that is optimized to improve performance when running your instrumented tests. Combined with support for test sharding, you can experiment with splitting your test suite across multiple ATD instances in order to reduce overall test execution time.

Create a Gradle managed device

You can specify a virtual device that you want Gradle to use for testing your app in your module-level build.gradle file. The following code sample creates a Pixel 2 running API level 30 as a Gradle managed device.

android {
  testOptions {
    managedDevices {
      devices {
        pixel2api30 (com.android.build.api.dsl.ManagedVirtualDevice) {
          // Use device profiles you typically see in Android Studio.
          device = "Pixel 2"
          // Use only API levels 27 and higher.
          apiLevel = 30
          // To include Google services, use "google".
          systemImageSource = "aosp"
        }
      }
    }
  }
}

android {
  testOptions {
    managedDevices {
      devices {
        maybeCreate<com.android.build.api.dsl.ManagedVirtualDevice>("pixel2api30").apply {
          // Use device profiles you typically see in Android Studio.
          device = "Pixel 2"
          // Use only API levels 27 and higher.
          apiLevel = 30
          // To include Google services, use "google".
          systemImageSource = "aosp"
        }
      }
    }
  }
}

To run your tests using the Gradle managed devices you configured, use the following command. device-name is the name of the device you configured in your Gradle build script (such as pixel2api30), and BuildVariant is the build variant of your app you want to test.

gradlew device-nameBuildVariantAndroidTest

Define groups of devices

To help you scale your tests across multiple device configurations, such as different API levels and form factors, you can define multiple Gradle managed devices and add them to a named group. Gradle can then execute your tests across all the devices in the group in parallel.

The example below shows two managed devices added to a device group called phoneAndTablet.

testOptions {
  managedDevices {
    devices {
      pixel2api29 (com.android.build.api.dsl.ManagedVirtualDevice) { ... }
      nexus9api30 (com.android.build.api.dsl.ManagedVirtualDevice) { ... }
    }
    groups {
      phoneAndTablet {
        targetDevices.add(devices.pixel2api29)
        targetDevices.add(devices.nexus9api30)
      }
    }
  }
}

testOptions {
  managedDevices {
    devices {
      maybeCreate<com.android.build.api.dsl.ManagedVirtualDevice>("pixel2api29").apply { ... }
      maybeCreate<com.android.build.api.dsl.ManagedVirtualDevice>("nexus9api30").apply { ... }
    }
    groups {
      maybeCreate("phoneAndTablet").apply {
        targetDevices.add(devices["pixel2api29"])
        targetDevices.add(devices["nexus9api30"])
      }
    }
  }
}

To run your tests using the group of Gradle managed devices, use the following command:

gradlew group-nameGroupBuildVariantAndroidTest

Run tests using Automated Test Devices

Gradle Managed Devices supports a new type of emulator device, called the Automated Test Device (ATD), which is optimized to reduce CPU and memory resources when running your instrumented tests. ATDs improve runtime performance in a few ways:

• Remove pre-installed apps that are typically not useful for testing your app
• Disable certain background services that are typically not useful for testing your app
• Disable hardware rendering

Before getting started, make sure you update the Android Emulator to the latest available version. Then, specify an "-atd" image when defining a Gradle Managed Device in your build.gradle, as shown below:

android {
  testOptions {
    managedDevices {
      devices {
        pixel2api30 (com.android.build.api.dsl.ManagedVirtualDevice) {
          // Use device profiles you typically see in Android Studio.
          device = "Pixel 2"
          // ATDs currently support only API level 30.
          apiLevel = 30
          // You can also specify "google-atd" if you require Google Play Services.
          systemImageSource = "aosp-atd"
        }
      }
    }
  }
}

android {
  testOptions {
    managedDevices {
      devices {
        maybeCreate<com.android.build.api.dsl.ManagedVirtualDevice>("pixel2api30").apply {
          // Use device profiles you typically see in Android Studio.
          device = "Pixel 2"
          // ATDs currently support only API level 30.
          apiLevel = 30
          // You can also specify "google-atd" if you require Google Play Services.
          systemImageSource = "aosp-atd"
        }
      }
    }
  }
}

You can also create device groups as you can with other Gradle Managed Devices. To further leverage the performance improvements, you can also use ATDs with test sharding to reduce the total test execution time of your test suite.

What's removed from ATD images?

In addition to operating in a headless mode, ATDs also optimize performance by removing or disabling apps and services that are typically not required for testing your app's code. The table below provides an overview of the components we've removed or disabled in ATD images and descriptions of why they might not be useful.

Enable test sharding

Gradle Managed Devices support test sharding, which allows you to split your test suite across a number of identical virtual device instances, called shards, that run in parallel. Using test sharding can help reduce overall test execution time at the cost of additional computational resources.

To set the number of shards you want to use in a given test run, set the following in your gradle.properties file:

android.experimental.androidTest.numManagedDeviceShards=<number_of_shards>

When running your tests using this option, Gradle Managed Devices provision the number of shards you specify for each device profile in the test run. So, for example, if you deployed your tests to a device group of three devices and set numManagedDeviceShards to two, Gradle Managed Devices will provision a total of six virtual devices for your test run.

When your tests are complete, Gradle outputs test results in a .proto file for each shard used in the test run.