Test Release Note Archive

AndroidX Test Espresso 3.1.1, Runner 1.1.1, Rules 1.1.1, Monitor 1.1.1 AndroidTestOrchestrator 1.1.1, Core 1.1.0, Truth 1.1.0, JUnit 1.1.0 (2018-12-13)

This is the stable release of AndroidX Test 1.1.0

  • Core
    • Make ActivityScenario support activities which start another activity

AndroidX Test Espresso 3.1.1-beta01, Runner 1.1.1-beta01, Rules 1.1.1-beta01, Monitor 1.1.1-beta01 AndroidTestOrchestrator 1.1.1-beta01, Core 1.1.0-beta01 Truth 1.1.0-beta01, JUnit 1.1.0-beta01 (2018-12-06)

  • Core
    • New core-ktx kotlin extension artifact! Includes a kotlin-friendly ActivityScenario.launchActivity API
    • New ActivityScenario API for launching activities with custom intents
    • New ActivityScenario API for receiving an Activity result
    • Make ActivityScenario closeable
  • Espresso
    • Modify withResourceNameMatcher and HumanReadables to be API 28 compatible.
    • Update ReplaceTextAction's description to include the stringToBeSet
    • Support Espresso in Robolectric paused looper mode.
  • JUnit
    • New ActivityScenarioRule API, for auto-launching and closing an Activity on test setup and teardown
    • New junit-ktx kotlin extension artifact! Includes a kotlin-friendly ActivityScenarioRule API
  • Runner
    • Make -e package and -e testFile consistent in behavior when receiving packages
  • Truth
    • Add bool, parcelable, and parcelableAsType BundleSubject APIs

AndroidX Test Espresso 3.1.0, Runner 1.1.0, Rules 1.1.0, Monitor 1.1.0 AndroidTestOrchestrator 1.1.0, Core 1.0.0 Truth 1.0.0, JUnit 1.0.0 (2018-10-24)

  • All
    • Set minSdkVersion to 14 and targetSdkVersion to 28
  • Espresso
    • Fix withContentDescription to work with non-string types
    • Add support for using Espresso on Robolectric
    • Issue 72798625: Espresso ViewMatchers.withText doesn't work when textAllCaps is enabled
    • Add support for injecting a sequence of motion events
  • Intents
    • Add beta API for retrieving list of intents. Intended for use with new truth assertions
  • Runner
    • Add support for instant apps
    • Deprecate androidx.test.runner.AndroidJUnit4 and replace with androidx.test.ext.junit.runners.AndroidJUnit4
  • Monitor
    • Deprecate androidx.test.InstrumentationRegistry and replace with androidx.test.platform.app.InstrumentationRegistry and androidx.test.core.app.ApplicationProvider
  • AndroidTestOrchestrator
    • Only enable orchestrator coverage handling if both 'coverage' and 'coverageFilePath' arguments are passed.
    • Only wait for debugger when the -debug is set but not for listing ATO test cases. A new orchestratorDebug flag was added for debugging orchestrator itself
  • Core
    • New artifact! Includes new APIs that support both local and on-device tests for:
      • Retrieving context: ApplicationProvider
      • Controlling activity lifecycles: ActivityScenario(beta)
      • Builders for MotionEvent, PackageInfo
      • Parceables utility class
  • Truth
    • New artifact! Includes custom truth subjects for Notification, Intent, Bundle, Parcelable, and MotionEvent
  • JUnit
    • New artifact! Includes JUnit runner class androidx.test.ext.junit.runners.AndroidJUnit4 that supports both local and on-device tests.

Espresso 3.0.2-beta1, Runner 1.0.2-beta1, Rules 1.0.2-beta1, Monitor 1.0.2-beta1, AndroidTestOrchestrator 1.0.2-beta1 (2018-04-16)

  • Espresso
    • Breaking API Change:
      • It was brought up to our attention in Issue 64062890 that we were using Guava Optional in our public API. This was terrible oversight on our part . As a result, there is a breaking public API change in this release to address this issue. We introduced an ugly wrapper around Guava Optional class, named EspressoOptional which lives under "android.support.test.espresso.util" namespace. Developers that were using the leaked Guava Optional API need to change their imports and references to use EspressoOptional during the update to this new version. Sorry for the inconvenience this may cause.
    • onView() and onData() APIs are now marked @CheckReturnValue to prevent errors
    • Fixed espresso-core POM file to not pull in "rules" dependency, instead have espresso-intents POM pull it. This should be a NoOp change for developers since espresso-intents cannot be used without espresso-core.
    • Issue 65486414: Espresso missing guava dependency
    • Issue 65576174: Espresso IdlingResourceRegistry.sync causes second test fail
    • Issue 65568629: Espresso.onIdle not using IdlingRegistry
    • Issue 69333598: espresso 3.0.1 incompatible with play-services-auth:11.6.0 in android library module.
    • Issue 64062890: Internal Optional type exposed by AdapterViewProtocol interface
    • Issue 64091847: Espresso 3.0.0 should NOT depend on test runner
    • Issue 73722050: espresso-contrib 3.0.2-alpha1 packages android.arch.{lifecycle/core} classes
  • Espresso-remote
    • This is a brand new artifact. We decoupled all of Espresso's multi process functionality outside of espresso-core artifact. This is cleaner and should significantly reduce espresso-core's overall size and method count.
  • Runner
    • Truncate stack trace if too large for a binder transaction. Since AJUR needs to report failures back to AM via a binder IPC, we need to make sure that we don't exceed the Binder transaction limit - which is 1MB per process.
    • Issue 65828576: TestRequestBuilder crash when running test in class with @Ignore
    • Issue 37057596: We don't handle failures in @BeforeClass
  • Rules
    • Ensure to release a reference on the activity under test after lifecycle changes. During the duration of the test one is now able to manipulate the Activity directly using the reference obtained from #getActivity() If the Activity is finished and relaunched, the reference returned by #getActivity() now always points to the current instance of the Activity.
    • Issue 64389280: GrantPermissionRule doesn't provide WRITE_EXTERNAL_STORAGE
    • Issue 37065965: ActivityTestRule leaks activity after orientation change
    • Issue 75254050: ActivityTestRule doesn't update Activity instance during configuration changes
    • Issue 64464625: Cannot do UI work in method finish() of Activity
  • AndroidTestOrchestrator
    • Pass -e coverage true -e coverageFilePath /sdcard/foo/ flags to generate coverage files in the given location (The app must have permission to write to the given location). The coverage file naming convention now looks like this com.foo.Class#method1.ec. Note, this is only supported when running in isolated mode. Also, it cannot be used together with AndroidJUnitRunner's coverageFile flag. Since the generated coverage files overwrite each other.
    • Pass -e clearPackageData flag if you wish the orchestrator to run pm clear context.getPackageName() and pm clear targetContext.getPackageName() commands in between test invocations. Note, the context in the clear command is the App under test context.
    • Fixed - When running an empty test, aka. no @Test inside the target, the test result is different than legacy mode.
    • Issue 72758547: Test Orchestrator causes Jacoco Coverage Data to be incomplete, only has last test run data
    • Issue 67916042: Android Test Orchestrator : Execution stopped on Process crash due to OutOfMemory
    • Issue 77752735: Orchestrator crashes for TransactionTooLargeException
    • Issue 77549481: Test Orchestrator Should Run "pm clear" After Each Test

Espresso 3.0.2-alpha1, Runner 1.0.2-alpha1, Rules 1.0.2-alpha1, AndroidTestOrchestrator 1.0.2-alpha1 (2017-12-05)

  • Espresso

    • Intents now has a callable response, allowing tests to execute after capturing a fired intent but before returning an Instrumentation.ActivityResult object.

  • Runner

    • Split out monitor maven artifact com.android.support.test:monitor:<version>, for users who need MonitoringInstrumentation without test running and JUnit features.

      If you use com.android.support.test:runner:<version>, everything works as expected, because Gradle automatically pulls in the monitor module as a dependency of the runner module.

    • Added flag newRunListenerOrderMode. When true, user-defined listeners run before default listeners. (We expect this behavior to eventually become the default.)

    • Issue 65828576: TestRequestBuilder crashes when running tests in a class annotated with @Ignore (standalone without test running or JUnit features).

  • AndroidTestOrchestrator

    • Now handles empty tests in the same way as non-orchestrated AndroidJUnitRunner.
    • Orchestrator can now handle very large standard output from AndroidJUnitRunner.

Espresso 3.0.1, Runner 1.0.1, Rules 1.0.1, AndroidTestOrchestrator 1.0.1 (2017-08-28)

  • Espresso

    • Added a IdlingPolicy option to suppress onTimeout() when a step debugger is attached to the VM.
    • Issues 64024656, 64247586, and 64525881: Don't report failures for unsuccessful class loading unless a user is loading specific classes by including the -e class runner argument.
    • Issue 64877246: Add missing classes to sources JAR file.

  • Runner

    • Simplified ShardingFilter logic.
    • Issue 65025743: @RequiresDevice filter now supports FTL emulators.

  • AndroidTestOrchestrator

    • Don't duplicate report failures. When a test process crashes after failing, now only one failure is reported for the test.
    • Fixed Javadoc for Orchestrator.
    • Tests now indicated as missed if the remote process crashes.
    • Now handles ignored test cases.
    • Now excludes ignored test cases from footer to match legacy results.
    • Fixed runtime permission issues. Test reports are now written to SD card on Android 7.0 (API level 24) and higher.

Espresso 3.0.0, Runner 1.0.0, Rules 1.0.0, AndroidTestOrchestrator 1.0.0 (2017-07-25, Announcement)

Breaking changes

  • All artifacts
    • Dropping support for API levels lower than 15 – however, the min SDK still points to API level 9 to give users time to upgrade
  • Espresso

Known issues

  • AndroidTestOrchestrator
    • -e numShards and -e shardIndex runner arguments aren't currently supported
    • Parameterized tests aren't currently supported

New features

  • Rules
  • AndroidTestOrchestrator
    • Android Test Orchestrator provides a new way of collecting and running tests, with an emphasis on correctness and isolation. Orchestrator is an independent instrumentation process, spawning one instrumentation runner process for each test and collecting the results.
      • Application crashes take down the runner instrumentation but not the orchestrator, allowing your test suite to continue
      • Requires installation of the orchestrator APK – 'com.android.support.test:orchestrator:1.0.0'
      • Version 1.0 has a command-line interface only; integration with Android Studio and Firebase Test Labs is planned

Bug fixes

  • Espresso
    • Improved root view synchronization to significantly reduce flakiness
    • Fix corruption of IdlingResourceRegistry
    • Better synchronization against IdlingResource objects
    • Issue 37132680: Espresso doesn't wait for dialog creation to complete before performing the next action
    • Issue 37103280: Espresso should ship ProGuard consumer rules so consumers aren't required to add them
    • Issue 37094726: Espresso Intents contains unnecessary application label
    • Issue 37093953: Espresso: No available check for null/empty error text
    • Issue 37071776: espresso-core embeds Guava's .pom files
    • Issue 37062612: NPE in release()
    • Issue 37063389: Having Guava and espresso-web as androidTest dependencies doesn't compile
    • Issue 37070533: Add support for NavigationView in Android Design Support Library

Other notable changes

  • Binaries are now published via Google Maven
  • Reduced size of espresso-core and espresso-web JAR files – embedded dependencies now have ProGuard applied
  • All .aar files now include ProGuard rules
  • Add proguard_library.cfg files to released artifacts
  • Tapper interface has a new version of sendTap() to implement

External contributions

Espresso 2.2.2, Runner/Rules 0.5 (2016-02-22, silent release)

New features

  • espresso
    • Issue 194253: Add support for NavigationView in android support design library
    • Added checks for enabled animations and transitions
    • New ViewMatcher API: withResourceName()

Bug fixes

  • espresso
    • Issue 195331: espresso-core embeds guava’s pom files
    • Moved counting idling resource out of espresso-contrib
  • rules
    • Issue 187249: NPE in Intents.release()
  • runner
    • Issue 196066: The -e log true argument in AndroidJUnitRunner does not bypass actual test
    • Wait for debugger in onCreate() of the runner
    • Moved all supported test annotation out of platform and into ATSL
    • Removed the stack trace dump about no JSBridge
    • Fixed AndroidAnnotatedBuilder

Other notable changes

  • ActivityTestRule, UiThreadTestRule, IntentsTestRule and ServiceTestRule are out of beta
  • Add code style settings file for uniform code formatting

Espresso 2.2.1, Runner/Rules 0.4 (2015-09-15)

New features

  • rules
    • Added new IntentsTestRule constructor to be fully compatible with ActivityTestRule
  • runner
    • Added special case multidex installation for API levels 15 and lower
    • Added exclude filters to class and package:
      • Running all tests except those in a particular class: adb shell am instrument -w -e notClass com.android.foo.FooTest
      • Running all but a single test: adb shell am instrument -w -e notClass com.android.foo.FooTest#testFoo
      • Running all tests except a particular package: adb shell am instrument -w -e notPackage com.android.foo.bar

External contributions

  • espresso
    • 157911: Add view matcher for input type on an EditText object
    • 157912: Add view matcher for matching error text on an EditText object
    • 150674: Add DrawerActions support for drawers with arbitrary gravity
    • 150744: DrawerActions no longer leak parentListener
    • 153303: Gravity specified on the "is the drawer open or closed" checks
    • 157910: Add DrawerLayout open and close action factories

Bug fixes

  • espresso
    • ViewActions.closeSoftKeyboard() now ensures that soft keyboard is completely gone
    • Fixed synchronization issue with Espresso's Espresso.pressBack() method on API level 21 and higher
    • Fixed synchronization for keyboard closure animations on API level 23
  • rules
    • Fixed ServiceTestRule on API level 23, startService() must always be called with an explicit Intent
  • runner
    • Fixed broken gradle JaCoCo support
    • Fixed broken test sharding support
    • Fixed inconsistent state in test runner after JUnit3 style test timeouts

Other notable changes

  • Javadoc fixes and error message improvements
  • Ignore suite() methods and don't ignore init errors when using method filters

Espresso 2.2 / ATSL 0.3 (2015-06-09)

New features

  • espresso-web 2.2
    • New WebView support
  • espresso-core 2.2
    • Migrated to use dagger v2
    • Migrated to use hamcrest v1.3
  • espresso-contrib 2.2
    • Accessibility checks
    • DrawerActions gravity support
  • rules 0.3
    • DisableOnAndroidDebug rule
  • runner 0.3
    • Upgrade from JUnit v4.10 to JUnit v4.12
    • Migrated to use Hamcrest v1.3

Bug fixes

  • Fixed DrawerActions leaking ParentListener
  • Assumption failure is now treated as an ignore test rather than a failing test
  • Fixed MonitoringInstrumentation leaking activity instances through ExecutorService
  • Fixed for orphan activities being stuck in stopped stage
  • Update Until.scrollFinished() to return true if no scroll events were generated. Guard against potential NPE in UiObject2#setText().

Espresso 2.1, Test Runner/Rules 0.2 and UIAutomator 2.1.0 (2015-04-21)

Breaking changes

  • Test runner artifact split into two and the name changed from com.android.support.test:testing-support-lib:0.1 to com.android.support.test:runner:0.2 and com.android.support.test:rules:0.2.

New features

  • espresso-intents: A Mockito-like API that enables hermetic inter-activity testing by allowing test authors to verify and stub outgoing intents
    • IntentsTestRule: extends ActivityTestRule, initialized and releases Espresso-Intents in functional UI tests
  • espresso-core
    • ViewActions: Added ability to run global assertions prior to running actions. This is useful for other frameworks that build on top of Espresso to validate state of the view hierarchy while existing Espresso test suite is executed
    • ViewMatchers.withContentDescription() resId overload
  • rules
    • ActivityTestRule: This rule provides functional testing of a single activity
    • UiThreadRule and UiThreadTest annotations: This rule allows the test method annotated with UiThreadTest to execute on the application's main thread (or UI thread)
    • ServiceTestRule: This rule provides functional testing of a service
  • runner
    • ApplicationLifecycleCallback: Callback for monitoring application lifecycle events
    • All runner arguments can now be also specified in the Android manifest file using a <meta-data> tag
  • UIAutomator
    • UiDevice.dumpWindowHierarchy() can now accept a File or an OutputStream

Bug fixes

  • espresso
    • Cursor matcher now returns false if the column wasn't found so Hamcrest can proceed to the next cursor
    • NullPointerException with PreferenceMatchers withTitle no longer occurs
    • Unregistering idling resource no longer causes Espresso to think we have busy idling resources
    • Updated Support Annotations version used by Espresso Contrib
  • runner
    • AndroidJUnit4 now skips tests with failing assumptions
  • UIAutomator
    • Run watchers to prevent StaleObjectException

Other notable changes

  • Add better error message when we can't typeText with a non-Latin string

UIAutomator 2.0 (2015-03-12)

UI Automator is now based on Android Instrumentation, and you can build and run tests using the ./gradlew connectedCheck command.

Espresso Version 2.0, Test Runner 0.1 (Released on: 2014-12-19)

Breaking changes

  • Espresso has moved to a new namespace, from android.support.test.espresso to android.support.test.espresso
  • Espresso artifacts have been renamed
    • espresso-1.1.jar is now espresso-core-release-2.0.jar
    • IdlingResource interface has been moved into a separate library: espresso-idling-resource-release-2.0.jar
    • CountingIdlingResource now resides in espresso-contrib-release-2.0.jar (as it always should have)
  • Optional (a guava dependency) has been removed from the public API in order to support repackaging the guava dependency and avoid DEX collision (a major source of development pain). Affected methods include the following:
    • ViewAssertion.check()
    • HumanReadables.getViewHierarchyErrorMessage()

New features

  • Actions
    • ViewActions
      • replaceText()
      • openLink()
      • Swipe up and down
    • espresso-contrib
      • RecyclerViewActions: Handles interactions with RecyclerViews
      • PickerActions: Handles interactions with Date and Time pickers
  • Matchers
    • RootMatchers
      • isPlatformPopup()
    • ViewMatchers
      • isJavascriptEnabled()
      • withSpinnerText()
      • withHint()
      • isSelected()
      • hasLinks()
    • LayoutMatchers: Matchers for i18n-related layout testing
    • CursorMatchers: A collection of matchers for Cursor objects
  • Assertions
    • PositionAssertions, including isLeftOf() and isAbove(): Colleciton of ViewAssertions for checking relative position of elements on the screen
    • LayoutAssertions: Assertions for i18n-related layout testing
  • Test app: Many new sample activities/tests
  • Other
    • Espresso.unregisterIdlingResources() and Espresso.getIdlingResources(): Provides additional flexibility for working with IdlingResources
    • ViewInteraction.withFailureHandler(): Allows overriding the failure handler from onView()
    • onData() support for AdapterViews backed by CursorAdapters

Bug fixes

  • ViewMatchers.isDisplayed() matches views that take up the entire screen, but are no longer less than 90% displayed
  • Performing swipe action call to DrawerActions.openDrawer() no longer results in IdlingResourceTimeoutException

Other notable changes

  • Switched from building with Maven to Gradle
  • Moved Espresso dependencies (Guava, Dagger, Hamcrest) out of the way to avoid DEX collisions
  • Changed to return success or failure when registering and unregistering idling resources
  • Lollipop support: Place message.recycle() behind an interface to account for version-related changes
  • Switched target SDK level to 21 – mostly affects the test app

Version 1.1 (Released on: 2014-01-08)

Espresso

  • New swipeLeft and swipeRight ViewActions
  • Multi-window support: An advanced feature that enables picking the target window on which Espresso should run the operation
  • Improvements to TypeTextAction: Allows typing text into a pre-focused view, which makes it easier to append text
  • Numerous bug fixes

Espresso Contrib Library

  • This new library contains features that supplement Espresso, but aren't part of the core library
  • New DrawerActions for operating on DrawerLayout: Has a dependency on Android Support Library, hence we are keeping it outside of the core Espresso library

Sample Tests

  • These tests have been relocated to live in the same package as the test app
  • Maven POMs have been fixed to remove duplicate guava deps, so mvn install should work now