Improve app performance with Baseline Profiles

1. Before you begin

In this codelab, you'll learn how to generate Baseline Profiles to optimize the performance of your application and how to verify the performance benefits of using Baseline Profiles.

Prerequisites

This codelab builds on the Inspect app performance with Macrobenchmark codelab that shows you how to measure app performance with Macrobenchmark library.

What you'll need

• Android Studio Dolphin (2021.3.1) or newer
• Knowledge of Kotlin
• Basic understanding of Jetpack Macrobenchmark
• A physical Android device with Android 6 (API level 23) or higher
• An Android emulator without Google Play with Android 9 (API level 28) or higher

What you'll do

• Generate Baseline Profiles to optimize the performance
• Verify the performance gains with the Macrobenchmark library

What you'll learn

• Generating baseline profiles
• Understanding performance gains of Baseline Profiles

2. Getting set up

To get started, clone the Github repository from the command line by using the following command:

$ git clone --branch baselineprofiles-main  https://github.com/googlecodelabs/android-performance.git

Alternatively, you can download two zip files:

• Starting point
• Solution

Open Project into Android Studio

1. On the Welcome to Android Studio window select Open an Existing Project
2. Select the folder [Download Location]/android-performance/benchmarking (tip: make sure you select the benchmarking directory containing build.gradle)
3. When Android Studio has imported the project, ensure that you can run the app module to build the sample application we'll benchmark.

3. What are Baseline Profiles

Baseline Profiles improve code execution speed by around 30% from the first launch by avoiding interpretation and just-in-time (JIT) compilation steps for included code paths. By shipping a Baseline Profile in an app or library, Android Runtime (ART) can optimize included code paths through Ahead of Time (AOT) compilation, providing performance enhancements for every new user and on every app update. This profile guided optimization (PGO) lets apps optimize startup, reduce interaction jank, and improve overall runtime performance for end users from the first launch.

Benefits of Baseline Profiles

With a Baseline Profile, all user interactions (such as app startup, navigating between screens, or scrolling through content) are smoother from the first time they run. Increasing the speed and responsiveness of an app leads to more daily active users and a higher average return visit rate.

Baseline Profiles help guide optimization beyond app startup by providing common user interactions that improve app runtime from the first launch. Guided AOT compilation doesn't rely on user devices and can be done once per release on a development machine instead of a mobile device. By shipping releases with a Baseline Profile, app optimizations become available much faster than by relying on Cloud Profiles alone.

When not using a Baseline Profile, all app code is JIT compiled in memory after being interpreted, or into an odex file in the background when the device is idle. Users have a less optimal experience for the first time they run an app after installing or updating it, until the new paths have been optimized. This performance boost has been measured at around 30% for many apps.

4. Set up the benchmarking module

As an app developer, you can automatically generate Baseline Profiles using the Jetpack Macrobenchmark library. To generate Baseline Profiles you can use the same module created for benchmarking your application with some additional changes.

Disable obfuscation for Baseline Profiles

If your app has obfuscation enabled, you need to disable it for the benchmarks.

You can do this by adding an additional proguard file into your :app module and disabling the obfuscation there and adding the proguard file to the benchmark buildType.

Create a new file named benchmark-rules.pro in the :app module. The file should be placed in the /app/ folder next to the module specific build.gradle file.

In this file, disable obfuscation by adding -dontobfuscate as in the following snippet:

# Disables obfuscation for benchmark builds.
-dontobfuscate

Next, modify the benchmark buildType in the :app module specific build.gradle and add the file you created. Since we're using initWith release buildType, this line will add the benchmark-rules.pro proguard file to the release proguard files.

buildTypes {
   release {
      // ...
   }

   benchmark {
      initWith buildTypes.release
      // ...
      proguardFiles('benchmark-rules.pro')
   }
}

Now, let's write a Baseline Profiles generator class.

5. Write a Baseline Profile generator

Usually, you'd generate Baseline Profiles for the typical user journeys of your app.

In our example, you could identify these three journeys:

1. Start the application (this will be critical for most applications)
2. Scroll the snack list
3. Go to snack detail

For generating the Baseline Profiles, we'll add a new test class BaselineProfileGenerator in the :macrobenchmark module. This class will use a BaselineProfileRule test rule and will contain one test method for generating the profile. The entrypoint for generating the profile is the collectBaselineProfile function. It requires only two parameters:

• packageName, which is the package of your app
• profileBlock (the last lambda parameter)

@RunWith(AndroidJUnit4::class)
class BaselineProfileGenerator {

   @get:Rule
   val rule = BaselineProfileRule()

   @Test
   fun generate() {
       rule.collectBaselineProfile("com.example.macrobenchmark_codelab") {
           // TODO Add interactions for the typical user journeys
       }
   }
}

In the profileBlock lambda you specify the interactions that cover the typical user journeys of your app. The library will run the profileBlock several times and it will collect the called classes and functions to be optimized and generate the Baseline Profile on device.

You can check the outline of our Baseline Profile generator covering the typical journeys in the following snippet:

@RunWith(AndroidJUnit4::class)
class BaselineProfileGenerator {

   @get:Rule
   val rule = BaselineProfileRule()

   @Test
   fun generate() {
       rule.collectBaselineProfile("com.example.macrobenchmark_codelab") {
           startApplicationJourney() // TODO Implement
           scrollSnackListJourney() // TODO Implement
           goToSnackDetailJourney() // TODO Implement
       }
   }
}

Now, let's write interactions for each mentioned journey. You can write it as the extension function of the MacrobenchmarkScope so you have access to the parameters and functions it provides. Writing it this way allows you to reuse the interactions with the benchmarks to verify the performance gains.

Start application journey

For the app startup journey (startApplicationJourney), you need to cover the following interactions:

1. Press home to be sure the state of the app is restarted
2. Start the default Activity and wait for the first frame to be rendered
3. Wait until the content is loaded and rendered and the user can interact with it

fun MacrobenchmarkScope.startApplicationJourney() {
   pressHome()
   startActivityAndWait()
   val contentList = device.findObject(By.res("snack_list"))
   // Wait until a snack collection item within the list is rendered
   contentList.wait(Until.hasObject(By.res("snack_collection")), 5_000)
}

Scrolling list journey

For the scrolling snack list journey (scrollSnackListJourney) you can follow these interactions:

1. Find the snack list UI element
2. Set the gesture margins not to trigger the system navigation
3. Scroll the list and wait until the UI settles

fun MacrobenchmarkScope.scrollSnackListJourney() {
   val snackList = device.findObject(By.res("snack_list"))
   // Set gesture margin to avoid triggering gesture navigation
   snackList.setGestureMargin(device.displayWidth / 5)
   snackList.fling(Direction.DOWN)
   device.waitForIdle()
}

Go to detail journey

The last journey (goToSnackDetailJourney) implements these interactions:

1. Find the snack list and find all snack items that you can work with
2. Select an item from the list
3. Click on the item and wait until the detail screen is loaded. You can leverage the fact that snack list won't be on screen anymore

fun MacrobenchmarkScope.goToSnackDetailJourney() {
   val snackList = device.findObject(By.res("snack_list"))
   val snacks = snackList.findObjects(By.res("snack_item"))
   // Select random snack from the list
   snacks[Random.nextInt(snacks.size)].click()
   // Wait until the screen is gone = the detail is shown
   device.wait(Until.gone(By.res("snack_list")), 5_000)
}

Now you have defined all the interactions needed for our Baseline Profile generator to be ready to run, but first you need to define the device it should run on.

6. Prepare a Gradle Managed Device

To generate Baseline Profiles you first need to have an userdebug emulator ready. To automate the process of creating the Baseline Profile, you can use Gradle Managed Devices. You can learn more about Gradle Managed Devices in our documentation.

First, define Gradle Managed Device in the :macrobenchmark module build.gradle file as in the following snippet:

testOptions {
    managedDevices {
        devices {
            pixel2Api31(com.android.build.api.dsl.ManagedVirtualDevice) {
                device = "Pixel 2"
                apiLevel = 31
                systemImageSource = "aosp"
            }
        }
    }
}

For generating Baseline Profiles you need to use rooted Android 9 (API 28) or higher.

In our case, we'll use Android 11 (API level 31) and the aosp system image is capable of rooted access.

The Gradle Managed Device allows you to run tests on an Android emulator without the need to manually launch it and tear it down. After adding the definition to build.gradle, the new pixel2Api31[BuildVariant]AndroidTest task will be available to run. We'll use that task in the next step to generate the Baseline Profile.

7. Generate the Baseline Profile

Once you have the Gradle Managed Device ready, you can start the generator test.

Run the generator from run configuration

The Gradle Managed Devices require running the test as a Gradle task. To get started quickly, we've created a run configuration that specifies the task with all necessary parameters to run.

To run it, locate the generateBaselineProfile run configuration and click the Run button .

The test will create the emulator image defined earlier, run the interactions several times and afterwards tear down the emulator and provide the output to the Android Studio.

(Optional) Run the generator from the command line

To run the generator from the command line, you can leverage the task created by the Gradle Managed Device – :macrobenchmark:pixel2Api31BenchmarkAndroidTest.

This command runs all tests in the project, which would fail, because the module also contains Benchmarks for later verification of the performance gains.

For that, you can filter the class you want to run with parameter -P android.testInstrumentationRunnerArguments.class and specify the com.example.macrobenchmark.BaselineProfileGenerator you wrote earlier.

The whole command looks as follows:

./gradlew :macrobenchmark:pixel2Api31BenchmarkAndroidTest -P android.testInstrumentationRunnerArguments.class=com.example.macrobenchmark.BaselineProfileGenerator

8. Apply the generated Baseline Profile

Once the generator finishes with success, you need to do several things to make the Baseline Profile work with your app.

You need to place the generated Baseline Profiles file into your src/main folder (next to AndroidManifest.xml). To retrieve the file, you can copy it from the managed_device_android_test_additional_output/ folder, which is located in /macrobenchmark/build/outputs/ as shown in the following screenshot.

Alternatively, you can click on the results link in the Android Studio output and save the content, or ​​use the adb pull command printed in the output.

Next, you need to rename the file to baseline-prof.txt.

Then, add the profileinstaller dependency to your :app module.

dependencies {
  implementation("androidx.profileinstaller:profileinstaller:1.2.0")
}

Adding this dependency allows you to:

• Locally benchmark the Baseline Profiles.
• Use Baseline Profiles on Android 7 (API level 24) and Android 8 (Api level 26), which don't support Cloud profiles.
• Use Baseline Profiles on devices that don't have Google Play Services.

FInally, sync the project with Gradle Files by clicking on the icon.

In the next step, we'll see how to verify how much better the app performance is with Baseline Profiles.

9. Verify startup performance improvement

Now, we've generated the Baseline Profile and added it to our app. Let's verify it has the desired effect on our app's performance.

Let's get back to our ExampleStartupBenchmark class that contains a benchmark to measure app startup. You need to slightly change the startup() test to be reused with different compilation modes. This will allow you to compare the difference when using Baseline Profiles.

CompilationMode

The CompilationMode parameter defines how the application is pre-compiled into machine code. It has the following options:

• DEFAULT – It partially pre-compiles the app using Baseline Profiles if available (this is used if no compilationMode parameter is applied)
• None() – It resets the app compilation state and doesn't pre-compile the app. Just in time compilation (JIT) is still enabled during execution of the app.
• Partial() – It pre-compiles the app with Baseline Profiles and/or warm up runs.
• Full() – It pre-compiles the whole application code. This is the only option on Android 6 (API 23) and lower.

If you want to start optimizing your application performance, you can choose DEFAULT compilation mode, because the performance will be similar to when the app is installed from Google Play. If you want to compare the performance benefits provided by Baseline Profiles, you can do it by comparing the results of compilation mode None and Partial.

Modify startup test with different CompilationMode

First, let's remove the @Test annotation from our startup method (because JUnit tests can't have parameters) and add the compilationMode parameter and use it in our measureRepeated function:

// Remove @Test annotation and add the compilationMode parameter.
fun startup(compilationMode: CompilationMode) = benchmarkRule.measureRepeated(
   packageName = "com.google.samples.apps.sunflower",
   metrics = listOf(StartupTimingMetric()),
   iterations = 5,
   compilationMode = compilationMode, // Set the compilation mode
   startupMode = StartupMode.COLD
) {
   pressHome()
   startActivityAndWait()
}

Now when you have that, let's add two test functions with different CompilationMode. The first will use CompilationMode.None, which means that before each benchmark, the state of the app will be reset and the app will have no pre-compiled code.

@Test
fun startupCompilationNone() = startup(CompilationMode.None())

The second test will leverage CompilationMode.Partial, which loads the Baseline Profiles and pre-compiles the specified classes and functions from the profile before running the benchmark.

@Test
fun startupCompilationPartial() = startup(CompilationMode.Partial())

Optionally, you can add a third method that would pre-compile the whole application using CompilationMode.Full. This is the only option on Android 6 (API 23) or lower, because the system runs only fully pre-compiled apps.

@Test
fun startupCompilationFull() = startup(CompilationMode.Full())

Next, run the benchmarks as you did before (on a physical device) and wait for the Macrobenchmark to measure the startup timings with different compilation modes.

After the benchmarks complete, you can see the timings in the Android Studio output as in the following screenshot:

From the screenshot you can see that the app startup time is different for each CompilationMode. The median values are shown in the following table:

Intuitively, the None compilation performs the worst, because the device has to do the most JIT compiling during startup of the app. What may be counterintuitive is that Full compilation doesn't perform the best. Since everything is compiled in this case, the app's odex files are large, and therefore the system usually has to do significantly more IO during app startup. The best performance is in the Partial case that uses the Baseline Profile. That's because partial compilation strikes a balance between compiling the code the user is most likely to use, but leaves the non-critical code not pre-compiled so it doesn't have to be loaded immediately.

10. Verify scrolling performance improvement

Similarly to what you did in the previous step, you can measure and verify the scrolling benchmarks. Let's modify the ScrollBenchmarks class similarly as before – adding parameter to the scroll test and adding more tests with different compilation mode parameter.

Open the ScrollBenchmarks.kt file, modify the scroll() function to add the compilationMode parameter:

fun scroll(compilationMode: CompilationMode) {
        benchmarkRule.measureRepeated(
            compilationMode = compilationMode, // Set the compilation mode
            // ... 

And now define multiple tests that uses different parameter:

@Test
fun scrollCompilationNone() = scroll(CompilationMode.None())

@Test
fun scrollCompilationPartial() = scroll(CompilationMode.Partial())

And run the benchmarks as before to get results as in the following screenshot:

From the results you can see that the CompilationMode.Partial has on average shorter frame timing by 0.4ms, which may not be noticeable for users, but for the other percentiles the results are more obvious. For P99 the difference is 36.9ms, which is more than 3 skipped frames (Pixel 7 runs 90 FPS, so it's ~11ms).

11. Congratulations

Congratulations, you've successfully completed this codelab and improved the performance of your app by using Baseline Profiles!

What's next?

Check our performance samples Github repository, which contains Macrobenchmark and other performance samples. Also, check Now In Android sample app – a real world application that uses benchmarking and Baseline Profiles to improve performance.

Reference docs

• https://developer.android.com/studio/profile/macrobenchmark-overview
• https://developer.android.com/studio/profile/baselineprofiles