Android fundamentals 03.2: Unit tests

1. Welcome

Introduction

Testing your code can help you catch bugs early in development, when bugs are the least expensive to address. As your app gets larger and more complex, testing improves your code's robustness. With tests in your code, you can exercise small portions of your app in isolation, and you can test in ways that are automatable and repeatable.

Android Studio and the Android Testing Support Library support several different kinds of tests and testing frameworks. In this practical you explore Android Studio's built-in testing functionality, and you learn how to write and run local unit tests.

Local unit tests are tests that are compiled and run entirely on your local machine with the Java Virtual Machine (JVM). You use local unit tests to test the parts of your app that don't need access to the Android framework or an Android-powered device or emulator, for example the internal logic. You also use local unit tests to test parts of your app for which you can create fake ("mock" or stub) objects that pretend to behave like the framework equivalents.

Unit tests are written with JUnit, a common unit testing framework for Java.

What you should already know

You should be able to:

Create an Android Studio project.
Build and run your app in Android Studio, on both an emulator and on a device.
Navigate the Project > Android pane in Android Studio.
Find the major components of an Android Studio project, including AndroidManifest.xml, resources, Java files, and Gradle files.

What you'll learn

How to organize and run tests in Android Studio.
Understand what a unit test is.
Write unit tests for your code.

What you'll do

Run the initial tests in the SimpleCalc app.
Add more tests to the SimpleCalc app.
Run the unit tests to see the results.

2. App overview

This practical uses the SimpleCalc app from the previous practical codelab ( Android fundamentals 3.1: The debugger). You can modify that app in place, or make a copy of your project folder before proceeding.

3. Task 1: Explore and run CalculatorTest

You write and run your tests (both unit tests and instrumented tests ) inside Android Studio, alongside the code for your app. Every new Android project includes basic sample classes for testing that you can extend or replace for your own uses.

In this task you return to the SimpleCalc app, which includes a basic unit testing class.

1.1 Explore source sets and CalculatorTest

Source sets are collections of code in your project that are for different build targets or other "flavors" of your app. When Android Studio creates your project, it creates three source sets:

The main source set, for your app's code and resources.
The (test) source set, for your app's local unit tests. The source set shows (test) after the package name.
The (androidTest) source set, for Android instrumented tests. The source set shows (androidTest) after the package name.

In this task you'll explore how source sets are displayed in Android Studio, examine the Gradle configuration for testing, and run the unit tests for the SimpleCalc app.

Open the SimpleCalc project in Android Studio, if you have not already done so.
Open the Project > Android pane, and expand the app and java folders.

The java folder in the Android view lists all the source sets in the app by package name. In this case (as shown below), the app code is in the com.android.example.SimpleCalc source set. The test code is in the source set with test appearing in parentheses after the package name: com.android.example.SimpleCalc (test).

Source sets

Expand the com.android.example.SimpleCalc (test) folder.

This folder is where you put your app's local unit tests. Android Studio creates a sample test class for you in this folder for new projects, but for SimpleCalc the test class is called CalculatorTest.

Open CalculatorTest.

Examine the code and note the following:

The only imports are from the org.junit, org.hamcrest, and android.test packages. There are no dependencies on the Android framework classes.
The @RunWith(JUnit4.class) annotation indicates the runner that will be used to run the tests in this class. A test runner is a library or set of tools that enables testing to occur and the results to be printed to a log. For tests with more complicated setup or infrastructure requirements (such as Espresso) you'll use different test runners. For this example we're using the basic JUnit4 test runner.
The @SmallTest annotation indicates that all the tests in this class are unit tests that have no dependencies, and run in milliseconds. The @SmallTest, @MediumTest, and @LargeTest annotations are conventions that make it easier to bundle groups of tests into suites of similar functionality.
The setUp() method is used to set up the environment before testing, and includes the @Before annotation. In this case the setup creates a new instance of the Calculator class and assigns it to the mCalculator member variable.
The addTwoNumbers() method is an actual test, and is annotated with @Test. Only methods in a test class that have an @Test annotation are considered tests to the test runner. Note that by convention test methods do not include the word "test."
The first line of addTwoNumbers() calls the add() method from the Calculator class. You can only test methods that are public or package-protected. In this case the Calculator is a public class with public methods, so all is well.
The second line is the assertion for the test. Assertions are expressions that must evaluate and result in true for the test to pass. In this case the assertion is that the result you got from the add method (1 + 1) matches the given number 2. You'll learn more about how to create assertions later in this practical.

1.2 Run tests in Android Studio

In this task you'll run the unit tests in the test folder and view the output for both successful and failed tests.

In the Project > Android pane, right-click (or Control-click) CalculatorTest and select Run ‘CalculatorTest'.

The project builds, if necessary, and the CalculatorTest pane appears at the bottom of the screen. At the top of the pane, the drop-down list for available execution configurations also changes to CalculatorTest.

All the tests in the CalculatorTest class run, and if those tests are successful, the progress bar at the top of the view turns green. (In this case, there is currently only one test.) A status message in the footer also reports "Tests Passed."

All tests successful

Open CalculatorTest if it is not already open, and change the assertion in addTwoNumbers() to:

assertThat(resultAdd, is(equalTo(3d)));

In the run configurations dropdown menu at the top of the screen, select CalculatorTest (if it is not already selected) and click Run .

The test runs again as before, but this time the assertion fails (3 is not equal to 1 + 1). The progress bar in the run view turns red, and the testing log indicates where the test (assertion) failed and why.

Change the assertion in addTwoNumbers() back to the correct test and run your tests again to ensure they pass.
In the run configurations dropdown, select app to run your app normally.

4. Task 2: Add more unit tests to CalculatorTest

With unit testing, you take a small bit of code in your app such as a method or a class, and isolate it from the rest of your app, so that the tests you write makes sure that one small bit of the code works in the way you'd expect. Typically, a unit test calls a method with a variety of different inputs, and verifies that the method does what you expect and returns what you expect it to return.

In this task you learn more about how to construct unit tests. You'll write additional unit tests for the Calculator utility methods in the SimpleCalc app, and run those tests to make sure that they produce the output you expect.

Note: Unit testing, test-driven development, and the JUnit 4 API are all large and complex topics and outside the scope of this course.

2.1 Add more tests for the add() method

Although it is impossible to test every possible value that the add() method may ever see, it's a good idea to test for input that might be unusual. For example, consider what happens if the add() method gets arguments:

With negative operands
With floating-point numbers
With exceptionally large numbers
With operands of different types (a float and a double, for example)
With an operand that is zero
With an operand that is infinity

In this task we'll add more unit tests for the add() method to test different kinds of inputs.

Add a new method to CalculatorTest called addTwoNumbersNegative(). Use this skeleton:

@Test
public void addTwoNumbersNegative() {
}

This test method has a similar structure to addTwoNumbers(): it is a public method, with no parameters, that returns void. It is annotated with @Test, which indicates it is a single unit test.

Why not just add more assertions to addTwoNumbers()? Grouping more than one assertion into a single method can make your tests harder to debug if only one assertion fails, and obscures the tests that do succeed. The general rule for unit tests is to provide a test method for every individual assertion.

Run all tests in CalculatorTest, as before.

In the test window both addTwoNumbers and addTwoNumbersNegative are listed as available (and passing) tests in the left panel. The addTwoNumbersNegative test still passes even though it doesn't contain any code—a test that does nothing is still considered a successful test.

Add a line to addTwoNumbersNegative() to invoke the add() method in the Calculator class with a negative operand.

double resultAdd = mCalculator.add(-1d, 2d);

The d notation after each operand indicates that these are numbers of type double. Because the add() method is defined with double parameters, a float or int will also work. Indicating the type explicitly enables you to test other types separately, if you need to.

Add an assertion with assertThat().

assertThat(resultAdd, is(equalTo(1d)));

The assertThat() method is a JUnit4 assertion that claims the expression in the first argument is equal to the one in the second argument. Older versions of JUnit used more specific assertion methods (assertEquals(), assertNull(), or assertTrue()), but assertThat() is a more flexible, more debuggable and often easier to read format.

The assertThat() method is used with matchers. Matchers are the chained method calls in the second operand of this assertion, is(equalto(). The Hamcrest framework defines the available matchers you can use to build an assertion. ("Hamcrest" is an anagram for "matchers.") Hamcrest provides many basic matchers for most basic assertions. You can also define your own custom matchers for more complex assertions.

In this case the assertion is that the result of the add() operation (-1 + 2) equals 1.

Add a new unit test to CalculatorTest for floating-point numbers:

@Test
public void addTwoNumbersFloats() {
  double resultAdd = mCalculator.add(1.111f, 1.111d);
  assertThat(resultAdd, is(equalTo(2.222d)));
}

Again, a very similar test to the previous test method, but with one argument to add() that is explicitly type float rather than double. The add() method is defined with parameters of type double, so you can call it with a float type, and that number is promoted to a double.

Click Run to run all the tests again.

This time the test failed, and the progress bar is red. This is the important part of the error message:

java.lang.AssertionError: 
Expected: is <2.222>
     but: was <2.2219999418258665>

Arithmetic with floating-point numbers is inexact, and the promotion resulted in a side effect of additional precision. The assertion in the test is technically false: the expected value is not equal to the actual value.

The question this raises is: When you have a precision problem with promoting float arguments, is that a problem with your code, or a problem with your test? In this particular case both input arguments to the add() method from the SimpleCalc app will always be type double, so this is an arbitrary and unrealistic test. However, if your app was written such that the input to the add() method could be either double or float, and you only care about some precision, you need to provide some wiggle room to the test so that "close enough" counts as a success.

Change the assertThat() method to use the closeTo() matcher:

assertThat(resultAdd, is(closeTo(2.222, 0.01)));

You need to make a choice for the matcher. Click on closeTo twice (until the entire expression is underlined), and press Alt+Enter (Option+Return on a Mac). Choose isCloseTo.closeTo (org.hamcrest.number).

Click Run to run all the tests again.

This time the test passes.

With the closeTo() matcher, rather than testing for exact equality you can test for equality within a specific delta. In this case the closeTo() matcher method takes two arguments: the expected value and the amount of delta. In the example above, that delta is just two decimal points of precision.

2.2 Add unit tests for the other calculation methods

Use what you learned in the previous task to fill out the unit tests for the Calculator class.

Add a unit test called subTwoNumbers() that tests the sub() method.
Add a unit test called subWorksWithNegativeResults() that tests the sub() method where the given calculation results in a negative number.
Add a unit test called mulTwoNumbers() that tests the mul() method.
Add a unit test called mulTwoNumbersZero() that tests the mul() method with at least one argument as zero.
Add a unit test called divTwoNumbers() that tests the div() method with two non-zero arguments.
Add a unit test called divTwoNumbersZero() that tests the div() method with a double dividend and zero as the divider.

All of these tests should pass, except divTwoNumbersZero() which causes an illegal argument exception for dividing by zero. If you run the app, enter zero as Operand 2, and click Div to divide, the result is an error.

Task 2 solution code

Android Studio project: SimpleCalcTest

The following code snippet shows the tests for this task:

@Test
public void addTwoNumbers() {
  double resultAdd = mCalculator.add(1d, 1d);
  assertThat(resultAdd, is(equalTo(2d)));
}

@Test
public void addTwoNumbersNegative() {
  double resultAdd = mCalculator.add(-1d, 2d);
  assertThat(resultAdd, is(equalTo(1d)));
}

@Test
public void addTwoNumbersFloats() {
  double resultAdd = mCalculator.add(1.111f, 1.111d);
  assertThat(resultAdd, is(closeTo(2.222, 0.01)));
}

@Test
public void subTwoNumbers() {
  double resultSub = mCalculator.sub(1d, 1d);
  assertThat(resultSub, is(equalTo(0d)));
}

@Test
public void subWorksWithNegativeResult() {
  double resultSub = mCalculator.sub(1d, 17d);
  assertThat(resultSub, is(equalTo(-16d)));
}

@Test
public void mulTwoNumbers() {
  double resultMul = mCalculator.mul(32d, 2d);
  assertThat(resultMul, is(equalTo(64d)));
}

@Test
public void divTwoNumbers() {
  double resultDiv = mCalculator.div(32d,2d);
  assertThat(resultDiv, is(equalTo(16d)));
}

@Test
public void divTwoNumbersZero() {
  double resultDiv = mCalculator.div(32d,0);
  assertThat(resultDiv, is(equalTo(Double.POSITIVE_INFINITY)));
}

5. Coding challenges

Challenge 1: Dividing by zero is always worth testing for, because it is a special case in arithmetic. How might you change the app to more gracefully handle divide by zero? To accomplish this challenge, start with a test that shows what the right behavior should be.

Remove the divTwoNumbersZero() method from CalculatorTest, and add a new unit test called divByZeroThrows() that tests the div() method with a second argument of zero, with the expected result as IllegalArgumentException.class. This test will pass, and as a result it will demonstrate that any division by zero will result in this exception.

After you learn how to write code for an Exception handler, your app can handle this exception gracefully by, for example, displaying a Toast message to the user to change Operand 2 from zero to another number.

Challenge 2: Sometimes it's difficult to isolate a unit of code from all of its external dependencies. Rather than organize your code in complicated ways just so you can test it more easily, you can use a mock framework to create fake ("mock") objects that pretend to be dependencies. Research the Mockito framework, and learn how to set it up in Android Studio. Write a test class for the calcButton() method in SimpleCalc, and use Mockito to simulate the Android context in which your tests will run.

6. Summary

Android Studio has built-in features for running local unit tests:

Local unit tests use the JVM of your local machine. They don't use the Android framework.
Unit tests are written with JUnit, a common unit testing framework for Java.
JUnit tests are located in the (test) folder in the Android Studio Project > Android pane.
Local unit tests only need these packages: org.junit, org.hamcrest, and android.test.
The @RunWith(JUnit4.class) annotation tells the test runner to run tests in this class.
@SmallTest, @MediumTest, and @LargeTest annotations are conventions that make it easier to bundle similar groups of tests
The @SmallTest annotation indicates all the tests in a class are unit tests that have no dependencies and run in milliseconds.
Instrumented tests are tests that run on an Android-powered device or emulator. Instrumented tests have access to the Android framework.
A test runner is a library or set of tools that enables testing to occur and the results to be printed to the log.

7. Related concept

The related concept documentation is in 3.2: App testing.

8. Learn more

Android Studio documentation:

Android developer documentation:

Other:

9. Homework

This section lists possible homework assignments for students who are working through this codelab as part of a course led by an instructor. It's up to the instructor to do the following:

Assign homework if required.
Communicate to students how to submit homework assignments.
Grade the homework assignments.

Instructors can use these suggestions as little or as much as they want, and should feel free to assign any other homework they feel is appropriate.

If you're working through this codelab on your own, feel free to use these homework assignments to test your knowledge.

Build and run an app

Open the SimpleCalc app from the practical on using the debugger. You're going to add a POW button to the layout. The button calculates the first operand raised to the power of the second operand. For example, given operands of 5 and 4, the app calculates 5 raised to the power of 4, or 625.

Before you write the implementation of your power button, consider the kind of tests you might want to perform using this calculation. What unusual values may occur in this calculation?

Update the Calculator class in the app to include a pow() method. Hint: Consult the documentation for the java.lang.Math class.
Update the MainActivity class to connect the POW Button to the calculation.

Now write each of the following tests for your pow() method. Run your test suite each time you write a test, and fix the original calculation in your app if necessary:

A test with positive integer operands.
A test with a negative integer as the first operand.
A test with a negative integer as the second operand.
A test with 0 as the first operand and a positive integer as the second operand.
A test with 0 as the second operand.
A test with 0 as the first operand and -1 as the second operand. (Hint: consult the documentation for Double.POSITIVE_INFINITY.)
A test with -0 as the first operand and any negative number as the second operand.

Answer these questions

Question 1

Which statement best describes a local unit test? Choose one:

Tests that run on an Android-powered device or emulator and have access to the Android framework.
Tests that enable you to write automated UI test methods.
Tests that are compiled and run entirely on your local machine with the Java Virtual Machine (JVM).

Question 2

Source sets are collections of related code. In which source set are you likely to find unit tests? Choose one:

app/res
com.example.android.SimpleCalcTest
com.example.android.SimpleCalcTest (test)
com.example.android.SimpleCalcTest (androidTest)

Question 3

Which annotation is used to mark a method as an actual test? Choose one:

@RunWith(JUnit4.class)
@SmallTest
@Before
@Test

Submit your app for grading

Guidance for graders

Check that the app has the following features:

It displays a POW Button that provides an exponential ("power of") calculation.
The implementation of MainActivity includes a click handler for the POW Button.
The implementation of Calculator includes a pow() method that performs the calculation.
The CalculatorTest() method includes separate test methods for the pow() method in the Calculator class that perform tests for negative and 0 operands, and for the case of 0 and -1 as the operands.

10. Next codelab

To find the next practical codelab in the Android Developer Fundamentals (V2) course, see Codelabs for Android Developer Fundamentals (V2).

For an overview of the course, including links to the concept chapters, apps, and slides, see Android Developer Fundamentals (Version 2).