Register now for Android Dev Summit 2019!

Use pre-launch reports, Firebase features, and Android vitals to improve your app

  • Launch
  • Test
  • Develop

Your app business relies on delivering high-quality apps. Part of achieving this quality is testing your apps on a range of devices. The pre-launch reports use automated tests on real devices that can identify layout issues, provide crash diagnostics, locate security vulnerabilities, and more. You can then run custom tests using Firebase Test Lab, get details about errors in your app from Firebase Crashlytics, and use Android vitals to search more deeply for quality issues.

Why it works

Identifying where your app has problems will ensure you're presenting the highest quality possible to users at launch. By correcting any issues quickly, you may be able to reverse negative reviews (particularly if you respond to let users know you've fixed the issue). You'll also provide a better experience for new users, which should lead to better ratings and reviews.

Features

  • Pre-launch reports: Summarizes the issues found in apps you upload to the default closed testing track. These apps are tested automatically for stability, performance, compatibility, display and accessibility issues, and security vulnerabilities, on a range of devices in Firebase Test Lab.

    When issues are found, the report gives you details to help you address them, including stack traces and (if relevant) screenshots or video replays of the crawled screens. If a crash detected in the report is also found in Android vitals, a link is created so you can see information on the impact of the crash, helping you to prioritize crash fixes. Also, find out about accessibility issues you should fix.

  • Custom tests using Firebase Test Lab: Catch defects throughout the development process by running scripted tests on your app.
  • Firebase Crashlytics: Get detailed reports about the errors in your app. Errors are grouped into clusters with similar stack traces and triaged by the severity of impact on your users. In addition to receiving automatic reports, you can log custom events to help capture the steps leading up to a crash.
  • Android vitals: Get app performance metrics for stability, power, jank, startup times, and permission denials. Use these metrics to identify and fix bad app behavior so that your app maintains a good user experience.

Best practices

  • Use the default closed testing track and open testing track to run a pre-launch report. Any additional closed tracks that you create will not receive a pre-launch report.
  • Track progress by comparing builds. As part of the pre-launch report, the Crashes tab shows a chart with your app's compatibility compared to previous builds. Each bar on the report represents a recently tested build of your app in the closed or open testing tracks, along with the number of successful and failed launches found during testing.
  • Check how your app runs on different versions of Android, and in particular on the latest version. With the pre-launch report, near the bottom half of your app's Crashes tab, you can view reports showing the test device and the device's Android version with a red icon showing whether your app had any testing issues.
  • Find layout issues. Screenshots in each test summary let you see how your app appears on test devices with different Android versions, languages, and screen resolutions.
  • Consider uploading another version of your app. A test may show 'unavailable devices' in the results, even though your app was compatible with those devices. Upload another version of your app to the closed or open testing tracks to make sure it works with all devices.
  • Identify security vulnerabilities. Known security vulnerabilities are flagged so you can act to ensure your app is safe for users. The report even scans for issues in third-party libraries. The names and descriptions of any security vulnerabilities found in your APK are included in the pre-launch report for each test.
  • Identify the use of unsupported (non-SDK) APIs. Beginning in Android 9 (API level 28), restrictions have been applied to the use of interfaces that aren't in the public Android SDK. The Stability tab of the pre-launch report shows the unsupported platform interfaces that Android vitals detects for your app, categorized by severity to help you prioritize. At least one stack trace is shown for every interface, to help you find where the API is invoked.
  • Remove location requirements. Your app may have a dependency on geolocation and this dependency might not be met by the location at which the test devices are based. If this is the case, publish an app to the closed or open testing tracks that removes the location requirements, especially if your app has content restrictions based on country.
  • Embed data you want included in the tests. If you want to run tests on an app with pre-loaded data, publish a version of your app to the closed or open testing tracks with your media files embedded in it.
  • Disable Google Play licensing services. If your app hasn't reached the production track, publish a version to the closed testing track with Google Play licensing services disabled to ensure your app properly identifies the users on the test devices. Otherwise, you'll get limited results from the tests.
  • Test from behind a login. If your app includes a sign-in screen, consider providing credentials representing a test account to Android vitals. That way, your tests can run in full, allowing you to better understand how your app performs across the most popular devices.
  • Create demo loops to test predefined actions in your game. Demo loops make the pre-launch report relevant for testing game play, as the standard crawler doesn't support OpenGL. For games created using OpenGL, publish a version of your game with demo loops to the closed or open testing tracks.
  • Take more control over tests and ensure key features are tested. Set up Robo test scripts to perform specific actions. Record them in Android Studio with Espresso Test Recorder then load them into your app's pre-launch report settings. The test crawler then runs your scripts and includes results in the report.
  • Identify deep links. Add your app's deep links to the pre-launch report setting and the test crawler will exercise them in the tests.
  • Review your test summary for accessibility errors, warnings, and minor issues. During testing, any accessibility issues will be flagged. Check the report's Accessibility tab to see screen clusters that identify where issues were found in your app. Also, learn how to use Accessibility Scanner to identify accessibility suggestions.