Android Paging

1. Introduction

What you'll learn

  • What the main components of Paging 3.0 are.
  • How to add Paging 3.0 to your project.
  • How to add a header or footer to your list using the Paging 3.0 API.
  • How to add list separators using the Paging 3.0 API.
  • How to page from network and database

What you'll build

In this codelab, you start with a sample app that already displays a list of GitHub repositories. Whenever the user scrolls to the end of the displayed list, a new network request is triggered and its result is displayed on the screen.

You will add code through a series of steps, to achieve the following:

  • Migrate to the Paging library components.
  • Add a loading status header and footer to your list.
  • Show loading progress between every new repository search.
  • Add separators in your list.
  • Add database support for paging from network and database

Here's what your app will look like in the end:

23643514cb9cf43e.png

What you'll need

For an introduction to Architecture Components, check out the Room with a View codelab. For an introduction to Flow, check out the Advanced Coroutines with Kotlin Flow and LiveData codelab.

2. Setup Your Environment

In this step, you will download the code for the entire codelab and then run a simple example app.

To get you started as quickly as possible, we have prepared a starter project for you to build on.

If you have git installed, you can simply run the command below. (You can check by typing git --version in the terminal / command line and verify it executes correctly.)

 git clone https://github.com/googlecodelabs/android-paging

The initial state is on the master branch. For certain steps you can see the solution as follows:

  • Branch step5-9_paging_3.0 - you'll find the solution for steps 5 - 9, where we're adding Paging 3.0 to our project.
  • Branch step10_loading_state_footer - you'll find the solution for step 10, where we're adding a footer that displays a loading state.
  • Branch step11_loading_state - you'll find the solution for step 11, where we're adding a display for the loading state between queries.
  • Branch step12_separators - you'll find the solution for step 12, where we're adding separators to our app.
  • Branch step13-19_network_and_database - you'll find the solution for steps 13 - 19, where we're adding offline support to our app

If you do not have git, you can click the following button to download all the code for this codelab:

Download source code

  1. Unzip the code, and then open the project Android Studio version 3.6.1 or newer.
  2. Run the app run configuration on a device or emulator.

b3c0dfdb92dfed77.png

The app runs and displays a list of GitHub repositories similar to this one:

50d1d2aa6e79e473.png

3. Project overview

The app allows you to search GitHub for repositories whose name or description contains a specific word. The list of repositories is displayed in descending order based on the number of stars, then alphabetically by name.

The app follows the architecture recommended in the " Guide to app architecture". Here's what you will find in each package:

  • api - Github API calls, using Retrofit.
  • data - the repository class, responsible for triggering API requests and caching the responses in memory.
  • model - the Repo data model, which is also a table in the Room database; and RepoSearchResult, a class that is used by the UI to observe both search results data and network errors.
  • ui - classes related to displaying an Activity with a RecyclerView.

The GithubRepository class retrieves the list of repository names from the network every time the user scrolls towards the end of the list, or when the user searches for a new repository. The list of results for a query is kept in memory in the GithubRepository in a ConflatedBroadcastChannel and exposed as a Flow.

SearchRepositoriesViewModel requests the data from GithubRepository and exposes it to the SearchRepositoriesActivity. Because we want to ensure that we're not requesting the data multiple times on configuration change (e.g. rotation), we're converting the Flow to LiveData in the ViewModel using the liveData() builder method. That way, the LiveData caches the latest list of results in memory, and when the SearchRepositoriesActivity gets recreated, the content of the LiveData will be displayed on the screen. The ViewModel exposes:

  1. A LiveData<UiState>
  2. A function (UiAction) -> Unit

The UiState is a representation of everything needed to render the app's UI, with different fields corresponding to different UI components. It is an immutable object, which means it can't be changed; however, new versions of it can be produced and observed by the UI. In our case, new versions of it are produced as a result of the user's actions: either searching for a new query, or scrolling the list to fetch more.

The user actions are aptly represented by the UiAction type. Enclosing the API for interactions to the ViewModel in a single type has the following benefits:

  • Small API surface: Actions can be added, removed, or changed, but the method signature of the ViewModel never changes. This makes refactoring local and less likely to leak abstractions or interface implementations.
  • Easier concurrency management: As you'll see later in the codelab, it's important to be able to guarantee the execution order of certain requests. By typing the API strongly with UiAction, we can write code with strict requirements for what can happen, and when it can happen.

From a usability perspective, we have the following issues:

  • The user has no information on the list loading state: they see an empty screen when they search for a new repository or just an abrupt end of the list while more results for the same query are being loaded.
  • The user can't retry a failed query.
  • The list always scrolls to the top after orientation changes or after process death.

From an implementation perspective, we have the following issues:

  • The list grows unbounded in memory, wasting memory as the user scrolls.
  • We have to convert our results from Flow to LiveData to cache them, increasing the complexity of our code.
  • If our app needed to show multiple lists, we'd see that there is a lot of boilerplate to write for each list.

Let's find out how the Paging library can help us with these issues and what components it includes.

4. Paging library components

The Paging library makes it easier for you to load data incrementally and gracefully within your app's UI. The Paging API provides support for many of the functionalities that you would otherwise need to implement manually when you need to load data in pages:

  • Keeps track of the keys to be used for retrieving the next and previous page.
  • Automatically requests the correct page when the user has scrolled to the end of the list.
  • Ensures that multiple requests aren't triggered at the same time.
  • Allows you to cache data: if you're using Kotlin, this is done in a CoroutineScope; if you're using Java, this can be done with LiveData.
  • Tracks loading state and allows you to display it in a RecyclerView list item or elsewhere in your UI, and easily retry failed loads.
  • Allows you to execute common operations like map or filter on the list that will be displayed, independently of whether you're using Flow, LiveData, or RxJava Flowable or Observable.
  • Provides an easy way of implementing list separators.

The Guide to app architecture proposes an architecture with the following main components:

  • A local database that serves as a single source of truth for data presented to the user and manipulated by the user.
  • A web API service.
  • A repository that works with the database and the web API service, providing a unified data interface.
  • A ViewModel that provides data specific to the UI.
  • The UI, which shows a visual representation of the data in the ViewModel.

The Paging library works with all of these components and coordinates the interactions between them, so that it can load "pages" of content from a data source and display that content in the UI.

This codelab introduces you to the Paging library and its main components:

  • PagingData - a container for paginated data. Each refresh of data will have a separate corresponding PagingData.
  • PagingSource - a PagingSource is the base class for loading snapshots of data into a stream of PagingData.
  • Pager.flow - builds a Flow<PagingData>, based on a PagingConfig and a function that defines how to construct the implemented PagingSource.
  • PagingDataAdapter - a RecyclerView.Adapter that presents PagingData in a RecyclerView. The PagingDataAdapter can be connected to a Kotlin Flow, a LiveData, an RxJava Flowable, or an RxJava Observable. The PagingDataAdapter listens to internal PagingData loading events as pages are loaded and uses DiffUtil on a background thread to compute fine-grained updates as updated content is received in the form of new PagingData objects.
  • RemoteMediator - helps implement pagination from network and database.

In this codelab, you will implement examples of each of the components described above.

5. Define the source of data

The PagingSource implementation defines the source of data and how to retrieve data from that source. The PagingData object queries data from the PagingSource in response to loading hints that are generated as the user scrolls in a RecyclerView.

Currently, the GithubRepository has a lot of the responsibilities of a data source that the Paging library will handle once we're done adding it:

  • Loads the data from GithubService, ensuring that multiple requests aren't triggered at the same time.
  • Keeps an in-memory cache of the retrieved data.
  • Keeps track of the page to be requested.

To build the PagingSource you need to define the following:

  • The type of the paging key - in our case, the Github API uses 1-based index numbers for pages, so the type is Int.
  • The type of data loaded - in our case, we're loading Repo items.
  • Where is the data retrieved from - we're getting the data from GithubService. Our data source will be specific to a certain query, so we need to make sure we're also passing the query information to GithubService.

So, in the data package, let's create a PagingSource implementation called GithubPagingSource:

class GithubPagingSource(
        private val service: GithubService,
        private val query: String
) : PagingSource<Int, Repo>() { 
    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Repo> {
        TODO("Not yet implemented")
    }
   override fun getRefreshKey(state: PagingState<Int, Repo>): Int? { 
        TODO("Not yet implemented")
    }

}

We'll see that PagingSource requires us to implement two functions: load() and getRefreshKey().

The load() function will be called by the Paging library to asynchronously fetch more data to be displayed as the user scrolls around. The LoadParams object keeps information related to the load operation, including the following:

  • Key of the page to be loaded. If this is the first time that load is called, LoadParams.key will be null. In this case, you will have to define the initial page key. For our project, you'll have to move GITHUB_STARTING_PAGE_INDEX constant from GithubRepository to your PagingSource implementation since this is the initial page key.
  • Load size - the requested number of items to load.

The load function returns a LoadResult. This will replace the usage of RepoSearchResult in our app, as LoadResult can take one of the following types:

  • LoadResult.Page, if the result was successful.
  • LoadResult.Error, in case of error.

When constructing the LoadResult.Page, pass null for nextKey or prevKey if the list can't be loaded in the corresponding direction. For example, in our case, we could consider that if the network response was successful but the list was empty, we don't have any data left to be loaded; so the nextKey can be null.

Based on all of this information, we should be able to implement the load() function!

Next we need to implement getRefreshKey(). The refresh key is used for subsequent refresh calls to PagingSource.load() (the first call is initial load which uses initialKey provided by Pager). A refresh happens whenever the Paging library wants to load new data to replace the current list, e.g., on swipe to refresh or on invalidation due to database updates, config changes, process death, etc. Typically, subsequent refresh calls will want to restart loading data centered around PagingState.anchorPosition which represents the most recently accessed index.

The GithubPagingSource implementation looks like this:

// GitHub page API is 1 based: https://developer.github.com/v3/#pagination
private const val GITHUB_STARTING_PAGE_INDEX = 1

class GithubPagingSource(
        private val service: GithubService,
        private val query: String
) : PagingSource<Int, Repo>() {

    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Repo> {
        val position = params.key ?: GITHUB_STARTING_PAGE_INDEX
        val apiQuery = query + IN_QUALIFIER
        return try {
            val response = service.searchRepos(apiQuery, position, params.loadSize)
            val repos = response.items
            val nextKey = if (repos.isEmpty()) {
                null
            } else {
                // initial load size = 3 * NETWORK_PAGE_SIZE
                // ensure we're not requesting duplicating items, at the 2nd request
                position + (params.loadSize / NETWORK_PAGE_SIZE)
            }
            LoadResult.Page(
                    data = repos,
                    prevKey = if (position == GITHUB_STARTING_PAGE_INDEX) null else position - 1,
                    nextKey = nextKey
            )
        } catch (exception: IOException) {
            return LoadResult.Error(exception)
        } catch (exception: HttpException) {
            return LoadResult.Error(exception)
        }
    }
    // The refresh key is used for subsequent refresh calls to PagingSource.load after the initial load
    override fun getRefreshKey(state: PagingState<Int, Repo>): Int? {
        // We need to get the previous key (or next key if previous is null) of the page
        // that was closest to the most recently accessed index.
        // Anchor position is the most recently accessed index
        return state.anchorPosition?.let { anchorPosition ->
            state.closestPageToPosition(anchorPosition)?.prevKey?.plus(1)
                ?: state.closestPageToPosition(anchorPosition)?.nextKey?.minus(1)
        }
    }

}

6. Build and configure PagingData

In our current implementation, we use a Flow<RepoSearchResult> in the GitHubRepository to get the data from the network and pass it to the ViewModel. The ViewModel then transforms it into a LiveData and exposes it to the UI. Whenever we get to the end of the displayed list and more data is loaded from the network, the Flow<RepoSearchResult> will contain the entire list of previously retrieved data for that query in addition to the latest data.

RepoSearchResult encapsulates both the success and error cases. The success case holds the repository data. The error case contains the Exception reason. With Paging 3.0 we don't need the RepoSearchResult anymore, as the library models both the success and error cases with LoadResult. Feel free to delete RepoSearchResult as in the next few steps we'll replace it.

To construct the PagingData, we first need to decide what API we want to use to pass the PagingData to other layers of our app:

  • Kotlin Flow - use Pager.flow.
  • LiveData - use Pager.liveData.
  • RxJava Flowable - use Pager.flowable.
  • RxJava Observable - use Pager.observable.

As we're already using Flow in our app, we'll continue with this approach; but instead of using Flow<RepoSearchResult>, we'll use Flow<PagingData<Repo>>.

No matter which PagingData builder you use, you'll have to pass the following parameters:

  • PagingConfig. This class sets options regarding how to load content from a PagingSource such as how far ahead to load, the size request for the initial load, and others. The only mandatory parameter you have to define is the page size—how many items should be loaded in each page. By default, Paging will keep in memory all the pages you load. To ensure that you're not wasting memory as the user scrolls, set the maxSize parameter in PagingConfig. By default Paging will return null items as a placeholder for content that is not yet loaded if Paging can count the unloaded items and if the enablePlaceholders config flag is true. Like this, you will be able to display a placeholder view in your adapter. To simplify the work in this codelab, let's disable the placeholders by passing enablePlaceholders = false.
  • A function that defines how to create the PagingSource. In our case, we'll create a new GithubPagingSource for each new query.

Let's modify our GithubRepository!

Update GithubRepository.getSearchResultStream

  • Remove the suspend modifier.
  • Return Flow<PagingData<Repo>>.
  • Construct Pager.
fun getSearchResultStream(query: String): Flow<PagingData<Repo>> {
    return Pager(
          config = PagingConfig(
            pageSize = NETWORK_PAGE_SIZE,
            enablePlaceholders = false
         ),
          pagingSourceFactory = { GithubPagingSource(service, query) }
    ).flow
}

Cleanup GithubRepository

Paging 3.0 does a lot of things for us:

  • Handles in-memory cache.
  • Requests data when the user is close to the end of the list.

This means that everything else in our GithubRepository can be removed, except getSearchResultStream and the companion object where we defined the NETWORK_PAGE_SIZE. Your GithubRepository should now look like this:

class GithubRepository(private val service: GithubService) {

    fun getSearchResultStream(query: String): Flow<PagingData<Repo>> {
        return Pager(
                config = PagingConfig(
                pageSize = NETWORK_PAGE_SIZE,
                enablePlaceholders = false
             ),
                pagingSourceFactory = { GithubPagingSource(service, query) }
        ).flow
    }

    companion object {
        const val NETWORK_PAGE_SIZE = 50
    }
}

You should now have compile errors in the SearchRepositoriesViewModel. Let's see what changes need to be made there!

7. Request and cache PagingData in the ViewModel

Before addressing the compile errors, let's review the types in the ViewModel:

sealed class UiAction {
    data class Search(val query: String) : UiAction()
    data class Scroll(
        val visibleItemCount: Int,
        val lastVisibleItemPosition: Int,
        val totalItemCount: Int
    ) : UiAction()
}

data class UiState(
    val query: String,
    val searchResult: RepoSearchResult
)

In our UiState we expose a searchResult; the role of searchResult is to be an in-memory cache for result searches that survives configuration changes. With Paging 3.0 we don't need to convert our Flow to LiveData anymore. Instead, SearchRepositoriesViewModel will now expose a StateFlow<UiState>. Furthermore, we drop the searchResult val entirely, opting instead to expose a separate Flow<PagingData<Repo>> that serves the same purpose that searchResult did.

PagingData is a self-contained type that contains a mutable stream of updates to the data to be displayed in the RecyclerView. Each emission of PagingData is completely independent, and multiple PagingData may be emitted for a single query. As such, Flows of PagingData should be exposed independent of other Flows.

Furthermore, as a user experience perk, for every new query entered, we want to scroll to the top of the list to show the first search result. However, as paging data may be emitted multiple times we only want to scroll to the top of the list if the user has not begun scrolling.

To do this, let's update UiState and add fields for the lastQueryScrolled and hasNotScrolledForCurrentSearch. These flags will prevent us from scrolling to the top of the list when we shouldn't:

data class UiState(
    val query: String = DEFAULT_QUERY,
    val lastQueryScrolled: String = DEFAULT_QUERY,
    val hasNotScrolledForCurrentSearch: Boolean = false
)

Let's revisit our architecture. Because all requests to the ViewModel go through a single entry point - the accept field defined as a (UiAction) -> Unit - we need to do the following:

  • Convert that entry point into streams containing types we're interested in.
  • Transform those streams.
  • Combine the streams back into a StateFlow<UiState>.

In more functional terms, we are going to reduce emissions of UiAction into UiState. It's sort of like an assembly line: the UiAction types are the raw materials that come in, they cause effects (sometimes called mutations), and the UiState is the finished output ready to be bound to the UI. This is sometimes called making the UI a function of the UiState.

Let's rewrite the ViewModel to handle each UiAction type in two different streams, and then transform them into a StateFlow<UiState> using a few Kotlin Flow operators.

First we update the definitions for state in the ViewModel to use a StateFlow instead of a LiveData while also adding a field for exposing a Flow of PagingData:

   /**
     * Stream of immutable states representative of the UI.
     */
    val state: StateFlow<UiState>

    val pagingDataFlow: Flow<PagingData<Repo>>

Next, we update the definition for the UiAction.Scroll subclass:

sealed class UiAction {
    ...
    data class Scroll(val currentQuery: String) : UiAction()
}

Notice that we removed all of the fields in the UiAction.Scroll data class and replaced them with the single currentQuery string. This lets us associate a scroll action with a particular query. We also delete the shouldFetchMore extension because it is no longer used. This is also something that needs to be restored after process death, so we make sure we update the onCleared() method in the SearchRepositoriesViewModel:

class SearchRepositoriesViewModel{
  ...
   override fun onCleared() {
        savedStateHandle[LAST_SEARCH_QUERY] = state.value.query
        savedStateHandle[LAST_QUERY_SCROLLED] = state.value.lastQueryScrolled
        super.onCleared()
    }
}

// This is outside the ViewModel class, but in the same file
private const val LAST_QUERY_SCROLLED: String = "last_query_scrolled"

This is also a good time to introduce the method that will actually create the pagingData Flow from the GithubRepository:

class SearchRepositoriesViewModel(
    ...
) : ViewModel() {

    override fun onCleared() {
        ...
    }

    private fun searchRepo(queryString: String): Flow<PagingData<Repo>> =
        repository.getSearchResultStream(queryString)
}

Flow<PagingData> has a handy cachedIn() method that allows us to cache the content of a Flow<PagingData> in a CoroutineScope. Since we're in a ViewModel, we will use the androidx.lifecycle.viewModelScope.

Now, we can start converting the accept field in the ViewModel into a UiAction stream. Replace the init block of SearchRepositoriesViewModel with the following:

class SearchRepositoriesViewModel(
    ...
) : ViewModel() {
    ...
    init {
        val initialQuery: String = savedStateHandle.get(LAST_SEARCH_QUERY) ?: DEFAULT_QUERY
        val lastQueryScrolled: String = savedStateHandle.get(LAST_QUERY_SCROLLED) ?: DEFAULT_QUERY
        val actionStateFlow = MutableSharedFlow<UiAction>()
        val searches = actionStateFlow
            .filterIsInstance<UiAction.Search>()
            .distinctUntilChanged()
            .onStart { emit(UiAction.Search(query = initialQuery)) }
        val queriesScrolled = actionStateFlow
            .filterIsInstance<UiAction.Scroll>()
            .distinctUntilChanged()
            // This is shared to keep the flow "hot" while caching the last query scrolled,
            // otherwise each flatMapLatest invocation would lose the last query scrolled,
            .shareIn(
                scope = viewModelScope,
                started = SharingStarted.WhileSubscribed(stopTimeoutMillis = 5000),
                replay = 1
            )
            .onStart { emit(UiAction.Scroll(currentQuery = lastQueryScrolled)) }
   }
}

Let's go over the code snippet above. We start with two items, the initialQuery String, pulled from saved state or a default, along with lastQueryScrolled, a String representing the last searched term where the user has interacted with the list. Next we begin to split the Flow into specific UiAction types:

  1. UiAction.Search for each time the user enters a particular query.
  2. UiAction.Scroll for each time the user scrolls the list with a particular query in focus.

The UiAction.Scroll Flow has some extra transformations applied to it. Let's go over them:

  1. shareIn: This is needed because when this Flow is ultimately consumed, it is consumed using a flatmapLatest operator. Each time the upstream emits, flatmapLatest will cancel the last Flow it was operating on, and start working based on the new flow it was given. In our case, this would make us lose the value of the last query the user has scrolled through. So, we use the Flow operator with a replay value of 1 to cache the last value so that it isn't lost when a new query comes in.
  2. onStart: Also used for caching. If the app was killed, but the user had already scrolled through a query, we don't want to scroll the list to the top causing them to lose their place again.

There should still be compile errors because we have not defined the state, pagingDataFlow, and accept fields yet. Let's fix that. With the transformations applied to each UiAction, we can now use them to create flows for both our PagingData and the UiState.

init {
        ...
        pagingDataFlow = searches
            .flatMapLatest { searchRepo(queryString = it.query) }
            .cachedIn(viewModelScope)

        state = combine(
            searches,
            queriesScrolled,
            ::Pair
        ).map { (search, scroll) ->
            UiState(
                query = search.query,
                lastQueryScrolled = scroll.currentQuery,
                // If the search query matches the scroll query, the user has scrolled
                hasNotScrolledForCurrentSearch = search.query != scroll.currentQuery
            )
        }
            .stateIn(
                scope = viewModelScope,
                started = SharingStarted.WhileSubscribed(stopTimeoutMillis = 5000),
                initialValue = UiState()
            )

        accept = { action ->
            viewModelScope.launch { actionStateFlow.emit(action) }
        }
    }
}

We use the flatmapLatest operator on the searches flow because each new search query requires a new Pager to be created. Next, we apply the cachedIn operator to the PagingData flow to keep it active within the viewModelScope and assign the result to the pagingDataFlow field. On the UiState side of things, we use the combine operator to populate the required UiState fields and assign the resulting Flow to the exposed state field. We also define accept as a lambda that launches a suspending function that feeds our state machine.

That's it! We now have a functional ViewModel from both a literal and reactive programming point of view!

8. Make the Adapter work with PagingData

To bind a PagingData to a RecyclerView, use a PagingDataAdapter. The PagingDataAdapter gets notified whenever the PagingData content is loaded and then it signals the RecyclerView to update.

Update the ui.ReposAdapter to work with a PagingData stream:

  • Right now, ReposAdapter implements ListAdapter. Make it implement PagingDataAdapter instead. The rest of the class body remains unchanged:
class ReposAdapter : PagingDataAdapter<Repo, RepoViewHolder>(REPO_COMPARATOR) {
// body is unchanged
}

We've been making a lot of changes so far, but now we're just one step away from being able to run the app—we just need to connect the UI!

9. Trigger network updates

Replace LiveData with Flow

Let's update SearchRepositoriesActivity to work with Paging 3.0. To be able to work with Flow<PagingData>, we need to launch a new coroutine. We will do that in the lifecycleScope, which is responsible for canceling the request when the activity is recreated.

Fortunately, we needn't change much. Rather than observe() a LiveData, we'll instead launch() a coroutine and collect() a Flow. The UiState will be combined with the PagingAdapter LoadState Flow to give us the guarantee that we will not scroll the list back up to the top with new emissions of PagingData if the user has already scrolled.

First off, as we are now returning state as a StateFlow instead of a LiveData, all references in the Activity to LiveData should be replaced with a StateFlow, making sure to add an argument for the pagingData Flow as well. The first place is in the bindState method:

   private fun ActivitySearchRepositoriesBinding.bindState(
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        uiActions: (UiAction) -> Unit
    ) {
        ...
    }

This change has a cascading effect, as we now have to update bindSearch() and bindList(). bindSearch() has the smallest change, so let's start there:

   private fun ActivitySearchRepositoriesBinding.bindSearch(
        uiState: StateFlow<UiState>,
        onQueryChanged: (UiAction.Search) -> Unit
    ) {
        searchRepo.setOnEditorActionListener {...}
        searchRepo.setOnKeyListener {...}

        lifecycleScope.launch {
            uiState
                .map { it.query }
                .distinctUntilChanged()
                .collect(searchRepo::setText)
        }
    }

The major change here is the need to launch a coroutine, and collect the query change from the UiState Flow.

Address the scrolling issue and bind data

Now for the scrolling part. First, like the last two changes, we replace the LiveData with a StateFlow and add an argument for the pagingData Flow. With that done, we can move on to the scroll listener. Notice that previously, we used an OnScrollListener attached to the RecyclerView to know when to trigger more data. The Paging library handles list scrolling for us, but we still need the OnScrollListener as a signal for if the user has scrolled the list for the current query. In the bindList() method, let's replace setupScrollListener() with an inline RecyclerView.OnScrollListener. We also delete the setupScrollListener() method entirely.

   private fun ActivitySearchRepositoriesBinding.bindList(
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        list.addOnScrollListener(object : RecyclerView.OnScrollListener() {
            override fun onScrolled(recyclerView: RecyclerView, dx: Int, dy: Int) {
                if (dy != 0) onScrollChanged(UiAction.Scroll(currentQuery = uiState.value.query))
            }
        })
        // the rest of the code is unchanged
    }

Next, we set up the pipeline to create a shouldScrollToTop boolean flag. With that done, we have two flows we can collect from: The PagingData Flow and the shouldScrollToTop Flow.

    private fun ActivitySearchRepositoriesBinding.bindList(
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        list.addOnScrollListener(...)
        val notLoading = repoAdapter.loadStateFlow
            // Only emit when REFRESH LoadState for the paging source changes.
            .distinctUntilChangedBy { it.source.refresh }
            // Only react to cases where REFRESH completes i.e., NotLoading.
            .map { it.source.refresh is LoadState.NotLoading }

        val hasNotScrolledForCurrentSearch = uiState
            .map { it.hasNotScrolledForCurrentSearch }
            .distinctUntilChanged()

        val shouldScrollToTop = combine(
            notLoading,
            hasNotScrolledForCurrentSearch,
            Boolean::and
        )
            .distinctUntilChanged()

        lifecycleScope.launch {
            pagingData.collectLatest(repoAdapter::submitData)
        }

        lifecycleScope.launch {
            shouldScrollToTop.collect { shouldScroll ->
                if (shouldScroll) list.scrollToPosition(0)
            }
        }
    }

In the above, we use collectLatest on the pagingData Flow so we can cancel collection on previous pagingData emissions on new emissions of pagingData. For the shouldScrollToTop flag, the emissions of PagingDataAdapter.loadStateFlow are synchronous with what is displayed in the UI, so it's safe to immediately call list.scrollToPosition(0) as soon as the boolean flag emitted is true.

The type in a LoadStateFlow is a CombinedLoadStates object.

CombinedLoadStates allows us to get the load state for the three different types of load operations:

  • CombinedLoadStates.refresh - represents the load state for loading the PagingData for the first time.
  • CombinedLoadStates.prepend - represents the load state for loading data at the start of the list.
  • CombinedLoadStates.append - represents the load state for loading data at the end of the list.

In our case, we want to reset the scroll position only when the refresh has completed i.e LoadState is refresh, NotLoading.

We can now remove binding.list.scrollToPosition(0) from updateRepoListFromInput().

With all that done, your activity should look like:

class SearchRepositoriesActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        val binding = ActivitySearchRepositoriesBinding.inflate(layoutInflater)
        val view = binding.root
        setContentView(view)

        // get the view model
        val viewModel = ViewModelProvider(this, Injection.provideViewModelFactory(owner = this))
            .get(SearchRepositoriesViewModel::class.java)

        // add dividers between RecyclerView's row items
        val decoration = DividerItemDecoration(this, DividerItemDecoration.VERTICAL)
        binding.list.addItemDecoration(decoration)

        // bind the state
        binding.bindState(
            uiState = viewModel.state,
            pagingData = viewModel.pagingDataFlow,
            uiActions = viewModel.accept
        )
    }

    /**
     * Binds the [UiState] provided  by the [SearchRepositoriesViewModel] to the UI,
     * and allows the UI to feed back user actions to it.
     */
    private fun ActivitySearchRepositoriesBinding.bindState(
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        uiActions: (UiAction) -> Unit
    ) {
        val repoAdapter = ReposAdapter()
        list.adapter = repoAdapter

        bindSearch(
            uiState = uiState,
            onQueryChanged = uiActions
        )
        bindList(
            repoAdapter = repoAdapter,
            uiState = uiState,
            pagingData = pagingData,
            onScrollChanged = uiActions
        )
    }

    private fun ActivitySearchRepositoriesBinding.bindSearch(
        uiState: StateFlow<UiState>,
        onQueryChanged: (UiAction.Search) -> Unit
    ) {
        searchRepo.setOnEditorActionListener { _, actionId, _ ->
            if (actionId == EditorInfo.IME_ACTION_GO) {
                updateRepoListFromInput(onQueryChanged)
                true
            } else {
                false
            }
        }
        searchRepo.setOnKeyListener { _, keyCode, event ->
            if (event.action == KeyEvent.ACTION_DOWN && keyCode == KeyEvent.KEYCODE_ENTER) {
                updateRepoListFromInput(onQueryChanged)
                true
            } else {
                false
            }
        }

        lifecycleScope.launch {
            uiState
                .map { it.query }
                .distinctUntilChanged()
                .collect(searchRepo::setText)
        }
    }

    private fun ActivitySearchRepositoriesBinding.updateRepoListFromInput(onQueryChanged: (UiAction.Search) -> Unit) {
        searchRepo.text.trim().let {
            if (it.isNotEmpty()) {
                list.scrollToPosition(0)
                onQueryChanged(UiAction.Search(query = it.toString()))
            }
        }
    }

    private fun ActivitySearchRepositoriesBinding.bindList(
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        list.addOnScrollListener(object : RecyclerView.OnScrollListener() {
            override fun onScrolled(recyclerView: RecyclerView, dx: Int, dy: Int) {
                if (dy != 0) onScrollChanged(UiAction.Scroll(currentQuery = uiState.value.query))
            }
        })
        val notLoading = repoAdapter.loadStateFlow
            // Only emit when REFRESH LoadState for the paging source changes.
            .distinctUntilChangedBy { it.source.refresh }
            // Only react to cases where REFRESH completes i.e., NotLoading.
            .map { it.source.refresh is LoadState.NotLoading }

        val hasNotScrolledForCurrentSearch = uiState
            .map { it.hasNotScrolledForCurrentSearch }
            .distinctUntilChanged()

        val shouldScrollToTop = combine(
            notLoading,
            hasNotScrolledForCurrentSearch,
            Boolean::and
        )
            .distinctUntilChanged()

        lifecycleScope.launch {
            pagingData.collectLatest(repoAdapter::submitData)
        }

        lifecycleScope.launch {
            shouldScrollToTop.collect { shouldScroll ->
                if (shouldScroll) list.scrollToPosition(0)
            }
        }
    }
}

Our app should compile and run, but without the loading state footer and the Toast that displays on error. In the next step, we will see how to display the loading state footer.

You can find the full code for the steps done so far in branch step5-9_paging_3.0.

10. Displaying the loading state in a footer

In our app, we want to be able to display a footer based on the load status: when the list is loading, we want to show a progress spinner. In case of an error, we want to show the error and a retry button.

3f6f2cd47b55de92.png 661da51b58c32b8c.png

The header/footer that we need to build follows the idea of a list that needs to be appended either at the beginning (as a header) or at the end (as a footer) of the actual list of items we're displaying. The header/footer is a list with only one element: a view that displays a progress bar or an error with a retry button, based on the Paging LoadState.

As displaying a header/footer based on the loading state and implementing a retry mechanism are common tasks, the Paging 3.0 API helps us with both of these.

For header/footer implementation we'll use a LoadStateAdapter. This implementation of RecyclerView.Adapter is automatically notified of changes in load state. It makes sure that only Loading and Error states lead to items being displayed and notifies the RecyclerView when an item is removed, inserted, or changed, depending on the LoadState.

For the retry mechanism we use adapter.retry(). Under the hood, this method ends up calling your PagingSource implementation for the right page. The response will be automatically propagated via Flow<PagingData>.

Let's see what our header/footer implementation looks like!

Like with any list, we have 3 files to create:

  • The layout file containing the UI elements for displaying progress, the error and the retry button
  • The **ViewHolder** **file** making the UI items visible based on the Paging LoadState
  • The adapter file defining how to create and bind the ViewHolder. Instead of extending a RecyclerView.Adapter, we will extend LoadStateAdapter from Paging 3.0.

Create the view layout

Create the repos_load_state_footer_view_item layout for our repo load state. It should have a ProgressBar, a TextView (to display the error), and a retry Button. The necessary strings and dimensions are already declared in the project.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    xmlns:tools="http://schemas.android.com/tools"
    android:orientation="vertical"
    android:padding="8dp">
    <TextView
        android:id="@+id/error_msg"
        android:textColor="?android:textColorPrimary"
        android:textSize="@dimen/error_text_size"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="center"
        android:textAlignment="center"
        tools:text="Timeout"/>
    <ProgressBar
        android:id="@+id/progress_bar"
        style="?android:attr/progressBarStyle"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_gravity="center"/>
    <Button
        android:id="@+id/retry_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="center"
        android:text="@string/retry"/>
</LinearLayout>

Create the ViewHolder

Create a new ViewHolder called ReposLoadStateViewHolder in the ui folder**.** It should receive a retry function as a parameter, to be called when the retry button is pressed. Create a bind() function that receives the LoadState as a parameter and sets the visibility of each view depending on the LoadState. An implementation of ReposLoadStateViewHolder using ViewBinding looks like this:

class ReposLoadStateViewHolder(
        private val binding: ReposLoadStateFooterViewItemBinding,
        retry: () -> Unit
) : RecyclerView.ViewHolder(binding.root) {

    init {
        binding.retryButton.setOnClickListener { retry.invoke() }
    }

    fun bind(loadState: LoadState) {
        if (loadState is LoadState.Error) {
            binding.errorMsg.text = loadState.error.localizedMessage
        }
        binding.progressBar.isVisible = loadState is LoadState.Loading
        binding.retryButton.isVisible = loadState is LoadState.Error
        binding.errorMsg.isVisible = loadState is LoadState.Error
    }

    companion object {
        fun create(parent: ViewGroup, retry: () -> Unit): ReposLoadStateViewHolder {
            val view = LayoutInflater.from(parent.context)
                    .inflate(R.layout.repos_load_state_footer_view_item, parent, false)
            val binding = ReposLoadStateFooterViewItemBinding.bind(view)
            return ReposLoadStateViewHolder(binding, retry)
        }
    }
}

Create the LoadStateAdapter

Create a ReposLoadStateAdapter that extends LoadStateAdapter in the ui folder as well. The adapter should receive the retry function as a parameter, since the retry function will be passed to the ViewHolder when constructed.

As with any Adapter, we need to implement the onBind() and onCreate() methods. LoadStateAdapter makes it easier as it passes the LoadState in both of these functions. In onBindViewHolder(), bind your ViewHolder. In onCreateViewHolder(), define how to create the ReposLoadStateViewHolder based on the parent ViewGroup and the retry function:

class ReposLoadStateAdapter(private val retry: () -> Unit) : LoadStateAdapter<ReposLoadStateViewHolder>() {
    override fun onBindViewHolder(holder: ReposLoadStateViewHolder, loadState: LoadState) {
        holder.bind(loadState)
    }

    override fun onCreateViewHolder(parent: ViewGroup, loadState: LoadState): ReposLoadStateViewHolder {
        return ReposLoadStateViewHolder.create(parent, retry)
    }
}

Now that we have all of the elements of our footer, let's bind them to our list. To do this, the PagingDataAdapter has 3 useful methods:

  • withLoadStateHeader - if we only want to display a header—this should be used when your list only supports adding items at the beginning of the list.
  • withLoadStateFooter - if we only want to display a footer—this should be used when your list only supports adding items at the end of the list.
  • withLoadStateHeaderAndFooter—if we want to display a header and a footer - if the list can be paged in both directions.

Update the ActivitySearchRepositoriesBinding.bindState() method and call withLoadStateHeaderAndFooter() on the adapter. As a retry function, we can call adapter.retry().

   private fun ActivitySearchRepositoriesBinding.bindState(
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        uiActions: (UiAction) -> Unit
    ) {
        val repoAdapter = ReposAdapter()
        list.adapter = repoAdapter.withLoadStateHeaderAndFooter(
            header = ReposLoadStateAdapter { repoAdapter.retry() },
            footer = ReposLoadStateAdapter { repoAdapter.retry() }
        )
        ...
    }

Since we have an infinite scrolling list, one easy way to get to see the footer is by putting your phone or emulator in airplane mode and scrolling until the end of the list.

Let's run the app!

You can find the full code for the steps done so far in branch step10_loading_state_footer.

11. Displaying the loading state in Activity

You might have noticed that we currently have two problems:

  • While migrating to Paging 3.0 we lost the ability to display a message when the list of results in empty.
  • Whenever you search for a new query, the current query result stays on screen until we get a network response. That's bad user experience! Instead we should display a progress bar or a retry button.

ab9ff1b8b033179e.png bd744ff3ddc280c3.png

The solution to both of these problems is to react to load state changes in our SearchRepositoriesActivity.

Display empty list message

First, let's bring back the empty list message. This message should be displayed only once the list is loaded and the number of items in the list is 0. To know when the list was loaded we will use the PagingDataAdapter.loadStateFlow property. This Flow emits every time there's a change in the load state via a CombinedLoadStates object.

CombinedLoadStates gives us the load state for the PageSource we defined or for the RemoteMediator needed for network and database case (more about this later).

In SearchRepositoriesActivity.bindList() we collect from loadStateFlow directly. The list is empty when the refresh state of CombinedLoadStates is NotLoading and adapter.itemCount == 0. Then we toggle the visibility of emptyList and list respectively:

private fun ActivitySearchRepositoriesBinding.bindList(
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        ...
        lifecycleScope.launch {
            repoAdapter.loadStateFlow.collect { loadState ->
                val isListEmpty = loadState.refresh is LoadState.NotLoading && repoAdapter.itemCount == 0
                // show empty list
                emptyList.isVisible = isListEmpty
                // Only show the list if refresh succeeds.
                list.isVisible = !isListEmpty
                }
            }
        }
    }

Display loading state

Let's update our activity_search_repositories.xml to include a retry button and a progress bar UI elements:

<androidx.constraintlayout.widget.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".ui.SearchRepositoriesActivity">
    <com.google.android.material.textfield.TextInputLayout
        android:id="@+id/input_layout"
        android:layout_width="0dp"
        android:layout_height="wrap_content"
        android:layout_marginEnd="8dp"
        android:layout_marginStart="8dp"
        android:layout_marginTop="8dp"
        android:layout_marginLeft="8dp"
        android:layout_marginRight="8dp"
        style="@style/Widget.MaterialComponents.TextInputLayout.OutlinedBox"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent">
        <EditText
            android:id="@+id/search_repo"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:hint="@string/search_hint"
            android:imeOptions="actionSearch"
            android:inputType="textNoSuggestions"
            android:selectAllOnFocus="true"
            tools:text="Android"/>
    </com.google.android.material.textfield.TextInputLayout>

    <androidx.recyclerview.widget.RecyclerView
        android:id="@+id/list"
        android:layout_width="0dp"
        android:layout_height="0dp"
        android:paddingVertical="@dimen/row_item_margin_vertical"
        android:scrollbars="vertical"
        app:layoutManager="LinearLayoutManager"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/input_layout"
        tools:ignore="UnusedAttribute"/>

    <ProgressBar
        android:id="@+id/progress_bar"
        style="?android:attr/progressBarStyle"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_gravity="center"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"/>

    <Button
        android:id="@+id/retry_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="@string/retry"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"/>

    <TextView android:id="@+id/emptyList"
        android:layout_width="0dp"
        android:layout_height="match_parent"
        android:gravity="center"
        android:text="@string/no_results"
        android:textSize="@dimen/repo_name_size"
        android:visibility="gone"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"/>

</androidx.constraintlayout.widget.ConstraintLayout>

Our retry button should trigger a reload of the PagingData. To do this, we call adapter.retry() in the onClickListener implementation, like we did for the header/footer:

// SearchRepositoriesActivity.kt

   private fun ActivitySearchRepositoriesBinding.bindList(
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        retryButton.setOnClickListener { repoAdapter.retry() }
        ...
}

Next, let's react to load state changes in SearchRepositoriesActivity.bindList. Since we only want our progress bar to be displayed when we have a new query, we need to rely on the load type from our paging source, specifically CombinedLoadStates.source.refresh and on the LoadState: Loading or Error. Also, one piece of functionality we commented out in a previous step was displaying a Toast when we got an error, so let's make sure we bring that in as well. To display the error message we will have to check whether CombinedLoadStates.prepend or CombinedLoadStates.append is an instance of LoadState.Error and retrieve the error message from the error.

Let's update our ActivitySearchRepositoriesBinding.bindList in SearchRepositoriesActivity method to have this functionality:

private fun ActivitySearchRepositoriesBinding.bindList(
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        pagingData: Flow<PagingData<Repo>>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        ...
        lifecycleScope.launch {
            repoAdapter.loadStateFlow.collect { loadState ->
                val isListEmpty = loadState.refresh is LoadState.NotLoading && repoAdapter.itemCount == 0
                // show empty list
                emptyList.isVisible = isListEmpty
                // Only show the list if refresh succeeds.
                list.isVisible = !isListEmpty
                // Show loading spinner during initial load or refresh.
                progressBar.isVisible = loadState.source.refresh is LoadState.Loading
                // Show the retry state if initial load or refresh fails.
                retryButton.isVisible = loadState.source.refresh is LoadState.Error

                // Toast on any error, regardless of whether it came from RemoteMediator or PagingSource
                val errorState = loadState.source.append as? LoadState.Error
                    ?: loadState.source.prepend as? LoadState.Error
                    ?: loadState.append as? LoadState.Error
                    ?: loadState.prepend as? LoadState.Error
                errorState?.let {
                    Toast.makeText(
                        this@SearchRepositoriesActivity,
                        "\uD83D\uDE28 Wooops ${it.error}",
                        Toast.LENGTH_LONG
                    ).show()
                }
            }
        }
    }

Now let's run the app and check out how it works!

That's it! With the current setup, the Paging library components are the ones triggering the API requests at the right time, handling the in-memory cache, and displaying the data. Run the app and try searching for repositories.

You can find the full code for the steps done so far in branch step11_loading_state.

12. Adding list separators

One way to improve your list's readability is to add separators. For example, in our app, since the repositories are ordered by the number of stars descending, we could have separators every 10k stars. To help implement this, the Paging 3.0 API allows inserting separators into PagingData.

573969750b4c719c.png

Adding separators in PagingData will lead to the modification of the list we display on our screen. We no longer display just Repo objects but also separator objects. Therefore, we have to change the UI model we're exposing from the ViewModel from Repo to another type that can encapsulate both types: RepoItem and SeparatorItem. Next, we'll have to update our UI to support separators:

  • Add a layout and ViewHolder for separators.
  • Update RepoAdapter to support creating and binding both separators and repositories.

Let's take this step by step and see what the implementation looks like.

Change the UI model

Currently SearchRepositoriesViewModel.searchRepo() returns Flow<PagingData<Repo>>. To support both repositories and separators, we'll create a UiModel sealed class in the same file with SearchRepositoriesViewModel. We can have 2 types of UiModel objects: RepoItem and SeparatorItem.

sealed class UiModel {
    data class RepoItem(val repo: Repo) : UiModel()
    data class SeparatorItem(val description: String) : UiModel()
}

Because we want to separate repositories based on 10k stars, let's create an extension property on RepoItem that rounds up the number of stars for us:

private val UiModel.RepoItem.roundedStarCount: Int
    get() = this.repo.stars / 10_000

Insert separators

SearchRepositoriesViewModel.searchRepo() should now return Flow<PagingData<UiModel>>.

class SearchRepositoriesViewModel(
    private val repository: GithubRepository,
    private val savedStateHandle: SavedStateHandle
) : ViewModel() {

    ...

    fun searchRepo(queryString: String): Flow<PagingData<UiModel>> {
        ... 
    }
}

Let's see how the implementation changes! Currently, repository.getSearchResultStream(queryString) returns a Flow<PagingData<Repo>>, so the first operation we need to add is to transform each Repo into a UiModel.RepoItem. To do this, we can use the Flow.map operator and then map each PagingData to build a new UiModel.Repo from the current Repo item, resulting in a Flow<PagingData<UiModel.RepoItem>>:

...
val newResult: Flow<PagingData<UiModel>> = repository.getSearchResultStream(queryString)
                .map { pagingData -> pagingData.map { UiModel.RepoItem(it) } }
... 

Now we can insert the separators! For each emission of the Flow, we'll call PagingData.insertSeparators(). This method returns a PagingData containing each original element, with an optional separator that you will generate, given the elements before and after. In boundary conditions (at the beginning or end of the list) the respective before or after elements will be null. If a separator doesn't need to be created, return null.

Because we're changing the type of PagingData elements from UiModel.Repo to UiModel, make sure you explicitly set the type arguments of the insertSeparators() method.

Here's what the searchRepo() method should look like:

   private fun searchRepo(queryString: String): Flow<PagingData<UiModel>> =
        repository.getSearchResultStream(queryString)
            .map { pagingData -> pagingData.map { UiModel.RepoItem(it) } }
            .map {
                it.insertSeparators { before, after ->
                    if (after == null) {
                        // we're at the end of the list
                        return@insertSeparators null
                    }

                    if (before == null) {
                        // we're at the beginning of the list
                        return@insertSeparators UiModel.SeparatorItem("${after.roundedStarCount}0.000+ stars")
                    }
                    // check between 2 items
                    if (before.roundedStarCount > after.roundedStarCount) {
                        if (after.roundedStarCount >= 1) {
                            UiModel.SeparatorItem("${after.roundedStarCount}0.000+ stars")
                        } else {
                            UiModel.SeparatorItem("< 10.000+ stars")
                        }
                    } else {
                        // no separator
                        null
                    }
                }
            }

Support multiple view types

SeparatorItem objects need to be displayed in our RecyclerView. We're only displaying a string here, so let's create a separator_view_item layout with a TextView in the res/layout folder:

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout 
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:background="@color/separatorBackground">

    <TextView
        android:id="@+id/separator_description"
        android:layout_width="0dp"
        android:layout_height="wrap_content"
        android:padding="@dimen/row_item_margin_horizontal"
        android:textColor="@color/separatorText"
        android:textSize="@dimen/repo_name_size"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        tools:text="10000+ stars" />
</androidx.constraintlayout.widget.ConstraintLayout>

Let's create a SeparatorViewHolder in the ui folder, where we just bind a string to the TextView:

class SeparatorViewHolder(view: View) : RecyclerView.ViewHolder(view) {
    private val description: TextView = view.findViewById(R.id.separator_description)

    fun bind(separatorText: String) {
        description.text = separatorText
    }

    companion object {
        fun create(parent: ViewGroup): SeparatorViewHolder {
            val view = LayoutInflater.from(parent.context)
                    .inflate(R.layout.separator_view_item, parent, false)
            return SeparatorViewHolder(view)
        }
    }
}

Update ReposAdapter to support a UiModel instead of a Repo:

  • Update the PagingDataAdapter parameter from Repo to UiModel.
  • Implement a UiModel comparator and replace the REPO_COMPARATOR with it.
  • Create the SeparatorViewHolder and bind it with the description of UiModel.SeparatorItem.

As we now need to display 2 different ViewHolders, replace RepoViewHolder with ViewHolder:

  • Update the PagingDataAdapter parameter
  • Update the onCreateViewHolder return type
  • Update the onBindViewHolder holder parameter

Here's what your final ReposAdapter will look like:

class ReposAdapter : PagingDataAdapter<UiModel, ViewHolder>(UIMODEL_COMPARATOR) {

    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): ViewHolder {
        return if (viewType == R.layout.repo_view_item) {
            RepoViewHolder.create(parent)
        } else {
            SeparatorViewHolder.create(parent)
        }
    }

    override fun getItemViewType(position: Int): Int {
        return when (getItem(position)) {
            is UiModel.RepoItem -> R.layout.repo_view_item
            is UiModel.SeparatorItem -> R.layout.separator_view_item
            null -> throw UnsupportedOperationException("Unknown view")
        }
    }

    override fun onBindViewHolder(holder: ViewHolder, position: Int) {
        val uiModel = getItem(position)
        uiModel.let {
            when (uiModel) {
                is UiModel.RepoItem -> (holder as RepoViewHolder).bind(uiModel.repo)
                is UiModel.SeparatorItem -> (holder as SeparatorViewHolder).bind(uiModel.description)
            }
        }
    }

    companion object {
        private val UIMODEL_COMPARATOR = object : DiffUtil.ItemCallback<UiModel>() {
            override fun areItemsTheSame(oldItem: UiModel, newItem: UiModel): Boolean {
                return (oldItem is UiModel.RepoItem && newItem is UiModel.RepoItem &&
                        oldItem.repo.fullName == newItem.repo.fullName) ||
                        (oldItem is UiModel.SeparatorItem && newItem is UiModel.SeparatorItem &&
                                oldItem.description == newItem.description)
            }

            override fun areContentsTheSame(oldItem: UiModel, newItem: UiModel): Boolean =
                    oldItem == newItem
        }
    }
}

That's it! When running the app you should be able to see the separators!

You can find the full code for the steps done so far in branch step12_separators.

13. Paging from network and database

Let's add offline support to our app by saving the data in a local database. That way, the database will be the source of truth for our app and we will always load data from there. Whenever we don't have any more data, we request more from the network and then save it in the database. Because the database is the source of truth, the UI will be automatically updated when more data is saved.

Here's what we need to do to add offline support:

  1. Create a Room database, a table to save the Repo objects in, and a DAO that we'll use to work with the Repo objects.
  2. Define how to load data from the network when we reach the end of the data in the database by implementing a RemoteMediator.
  3. Build a Pager based on the Repos table as a data source and the RemoteMediator for loading and saving data.

Let's take each of these steps!

14. Define the Room database, table, and DAO

Our Repo objects need to be saved in the database, so let's start by making the Repo class an entity, with tableName = "repos", where the Repo.id is the primary key. To do this, annotate the Repo class with @Entity(tableName = "repos") and add the @PrimaryKey annotation to id. This is what your Repo class should look like now:

@Entity(tableName = "repos")
data class Repo(
    @PrimaryKey @field:SerializedName("id") val id: Long,
    @field:SerializedName("name") val name: String,
    @field:SerializedName("full_name") val fullName: String,
    @field:SerializedName("description") val description: String?,
    @field:SerializedName("html_url") val url: String,
    @field:SerializedName("stargazers_count") val stars: Int,
    @field:SerializedName("forks_count") val forks: Int,
    @field:SerializedName("language") val language: String?
)

Create a new db package. This is where we will implement the class that accesses data in the database and the class that defines the database.

Implement the data access object (DAO) to access the repos table by creating a RepoDao interface, annotated with @Dao. We need the following actions on Repo:

  • Insert a list of Repo objects. If the Repo objects are already in the table, then replace them.
 @Insert(onConflict = OnConflictStrategy.REPLACE)
 suspend fun insertAll(repos: List<Repo>)
  • Query for repos that contain the query string in the name or in the description and sort those results in descending order by the number of stars and then alphabetically by name. Instead of returning a List<Repo>, return PagingSource<Int, Repo>. That way, the repos table becomes the source of data for Paging.
@Query("SELECT * FROM repos WHERE " +
  "name LIKE :queryString OR description LIKE :queryString " +
  "ORDER BY stars DESC, name ASC")
fun reposByName(queryString: String): PagingSource<Int, Repo>
  • Clear all data in the Repos table.
@Query("DELETE FROM repos")
suspend fun clearRepos()

This is what your RepoDao should look like:

@Dao
interface RepoDao {

    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertAll(repos: List<Repo>)

    @Query("SELECT * FROM repos WHERE " +
   "name LIKE :queryString OR description LIKE :queryString " +
   "ORDER BY stars DESC, name ASC")
    fun reposByName(queryString: String): PagingSource<Int, Repo>

    @Query("DELETE FROM repos")
    suspend fun clearRepos()
}

Implement the Repo database:

  • Create an abstract class RepoDatabase that extends RoomDatabase.
  • Annotate the class with @Database, set the list of entities to contain the Repo class, and set the database version to 1. For the purpose of this codelab we don't need to export the schema.
  • Define an abstract function that returns the ReposDao.
  • Create a getInstance() function in a companion object that builds the RepoDatabase object if it doesn't exist already.

Here's what your RepoDatabase looks like:

@Database(
    entities = [Repo::class],
    version = 1,
    exportSchema = false
)
abstract class RepoDatabase : RoomDatabase() {

    abstract fun reposDao(): RepoDao

    companion object {

        @Volatile
        private var INSTANCE: RepoDatabase? = null

        fun getInstance(context: Context): RepoDatabase =
                INSTANCE ?: synchronized(this) {
                    INSTANCE
                            ?: buildDatabase(context).also { INSTANCE = it }
                }

        private fun buildDatabase(context: Context) =
                Room.databaseBuilder(context.applicationContext,
                        RepoDatabase::class.java, "Github.db")
                        .build()
    }
}

Now that we've set up our database, let's see how we request data from the network and save it in the database.

15. Request and save data - overview

The Paging library uses the database as a source of truth for the data that needs to be displayed in the UI. Whenever we don't have any more data in the database, we need to request more from the network. To help with this, Paging 3.0 defines the RemoteMediator abstract class, with one method that needs to be implemented: load(). This method will be called whenever we need to load more data from the network. This class returns a MediatorResult object, that can either be:

  • Error - if we got an error while requesting data from the network.
  • Success - If we successfully got data from the network. Here, we also need to pass in a signal that tells whether more data can be loaded or not. For example, if the network response was successful but we got an empty list of repositories, it means that there is no more data to be loaded.

In the data package, let's create a new class called GithubRemoteMediator that extends RemoteMediator. This class will be recreated for every new query, so it will receive the following as parameters:

  • The query String.
  • The GithubService - so we can make network requests.
  • The RepoDatabase - so we can save data we got from the network request.
@OptIn(ExperimentalPagingApi::class)
class GithubRemoteMediator(
    private val query: String,
    private val service: GithubService,
    private val repoDatabase: RepoDatabase
) : RemoteMediator<Int, Repo>() {

    override suspend fun load(loadType: LoadType, state: PagingState<Int, Repo>): MediatorResult {

   }
}

To be able to build the network request, the load method has 2 parameters that should give us all the information we need:

  • PagingState - this gives us information about the pages that were loaded before, the most recently accessed index in the list, and the PagingConfig we defined when initializing the paging stream.
  • LoadType - this tells us whether we need to load data at the end (LoadType.APPEND) or at the beginning of the data (LoadType.PREPEND) that we previously loaded, or if this the first time we're loading data (LoadType.REFRESH).

For example, if the load type is LoadType.APPEND then we retrieve the last item that was loaded from the PagingState. Based on that we should be able to find out how to load the next batch of Repo objects, by computing the next page to be loaded.

In the next section you'll find out how to compute keys for the next and previous pages to be loaded.

16. Compute and save remote page keys

For the purposes of the Github API, the page key that we use to request pages of repos is just a page index that is incremented when getting the next page. This means that given a Repo object, the next batch of Repo objects can be requested based on page index + 1. The previous batch of Repo objects can be requested based on page index - 1. All Repo objects received on a certain page response will have the same next and previous keys.

When we get the last item loaded from the PagingState, there's no way to know the index of the page it belonged to. To solve this problem, we can add another table that stores the next and previous page keys for each Repo; we can call it remote_keys. While this can be done in the Repo table, creating a new table for the next and previous remote keys associated with a Repo allows us to have a better separation of concerns.

In the db package, let's create a new data class called RemoteKeys, annotate it with @Entity, and add 3 properties: the repo id (which is also the primary key), and the previous and next keys (which can be null when we can't append or prepend data).

@Entity(tableName = "remote_keys")
data class RemoteKeys(
    @PrimaryKey
    val repoId: Long,
    val prevKey: Int?,
    val nextKey: Int?
)

Let's create a RemoteKeysDao interface. We will need the following capabilities:

  • Insert a list of **RemoteKeys**, as whenever we get Repos from the network we will generate the remote keys for them.
  • Get a **RemoteKey** based on a Repo id.
  • Clear the **RemoteKeys**, which we will use whenever we have a new query.
@Dao
interface RemoteKeysDao {

    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertAll(remoteKey: List<RemoteKeys>)

    @Query("SELECT * FROM remote_keys WHERE repoId = :repoId")
    suspend fun remoteKeysRepoId(repoId: Long): RemoteKeys?

    @Query("DELETE FROM remote_keys")
    suspend fun clearRemoteKeys()
}

Let's add the RemoteKeys table to our database and provide access to the RemoteKeysDao. To do this, update the RepoDatabase as follows:

  • Add RemoteKeys to the list of entities.
  • Expose the RemoteKeysDao as an abstract function.
@Database(
        entities = [Repo::class, RemoteKeys::class],
        version = 1,
        exportSchema = false
)
abstract class RepoDatabase : RoomDatabase() {

    abstract fun reposDao(): RepoDao
    abstract fun remoteKeysDao(): RemoteKeysDao

    ... 
    // rest of the class doesn't change
}

17. Request and save data - implementation

Now that we saved the remote keys, let's get back to GithubRemoteMediator and see how to use them. This class will replace our GithubPagingSource. Let's copy the GITHUB_STARTING_PAGE_INDEX declaration from GithubPagingSource in our GithubRemoteMediator and delete the GithubPagingSource class.

Let's see how we can implement the GithubRemoteMediator.load() method:

  1. Find out what page we need to load from the network, based on the LoadType.
  2. Trigger the network request.
  3. Once the network request completes, if the received list of repositories is not empty, then do the following:
  4. We compute the RemoteKeys for every Repo.
  5. If this a new query (loadType = REFRESH) then we clear the database.
  6. Save the RemoteKeys and Repos in the database.
  7. Return MediatorResult.Success(endOfPaginationReached = false).
  8. If the list of repos was empty then we return MediatorResult.Success(endOfPaginationReached = true). If we get an error requesting data we return MediatorResult.Error.

Here's how the code looks like overall. We'll replace the TODOs later on.

override suspend fun load(loadType: LoadType, state: PagingState<Int, Repo>): MediatorResult {
    val page = when (loadType) {
        LoadType.REFRESH -> {
         // TODO
        }
        LoadType.PREPEND -> {
        // TODO
        }
        LoadType.APPEND -> {
        // TODO
        }
    }
    val apiQuery = query + IN_QUALIFIER

    try {
        val apiResponse = service.searchRepos(apiQuery, page, state.config.pageSize)

        val repos = apiResponse.items
        val endOfPaginationReached = repos.isEmpty()
        repoDatabase.withTransaction {
            // clear all tables in the database
            if (loadType == LoadType.REFRESH) {
                repoDatabase.remoteKeysDao().clearRemoteKeys()
                repoDatabase.reposDao().clearRepos()
            }
            val prevKey = if (page == GITHUB_STARTING_PAGE_INDEX) null else page - 1
            val nextKey = if (endOfPaginationReached) null else page + 1
            val keys = repos.map {
                RemoteKeys(repoId = it.id, prevKey = prevKey, nextKey = nextKey)
            }
            repoDatabase.remoteKeysDao().insertAll(keys)
            repoDatabase.reposDao().insertAll(repos)
        }
        return MediatorResult.Success(endOfPaginationReached = endOfPaginationReached)
    } catch (exception: IOException) {
        return MediatorResult.Error(exception)
    } catch (exception: HttpException) {
        return MediatorResult.Error(exception)
    }
}

Let's see how we find the page to load based on the LoadType.

18. Getting the page based on the LoadType

Now that we know what happens in the GithubRemoteMediator.load() method once we have the page key, let's see how we compute it. This will depend on the LoadType.

LoadType.APPEND

When we need to load data at the end of the currently loaded data set, the load parameter is LoadType.APPEND. So now, based on the last item in the database we need to compute the network page key.

  1. We need to get the remote key of the last Repo item loaded from the database—let's separate this in a function:
    private suspend fun getRemoteKeyForLastItem(state: PagingState<Int, Repo>): RemoteKeys? {
        // Get the last page that was retrieved, that contained items.
        // From that last page, get the last item
        return state.pages.lastOrNull() { it.data.isNotEmpty() }?.data?.lastOrNull()
                ?.let { repo ->
                    // Get the remote keys of the last item retrieved
repoDatabase.remoteKeysDao().remoteKeysRepoId(repo.id)
                }
    }
  1. If remoteKeys is null, that means the refresh result is not in the database yet. We can return Success with endOfPaginationReached = false because Paging will call this method again if RemoteKeys becomes non-null. If remoteKeys is not null but its nextKey is null, that means we've reached the end of pagination for append.
val page = when (loadType) {
    LoadType.APPEND -> {
        val remoteKeys = getRemoteKeyForLastItem(state)
        // If remoteKeys is null, that means the refresh result is not in the database yet.
        // We can return Success with endOfPaginationReached = false because Paging
        // will call this method again if RemoteKeys becomes non-null.
        // If remoteKeys is NOT NULL but its nextKey is null, that means we've reached
        // the end of pagination for append.
        val nextKey = remoteKeys?.nextKey
        if (nextKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        nextKey
    }
      ...
  }

LoadType.PREPEND

When we need to load data at the beginning of the currently loaded data set, the load parameter is LoadType.PREPEND. Based on the first item in the database we need to compute the network page key.

  1. We need to get the remote key of the first Repo item loaded from the database—let's separate this in a function:
private suspend fun getRemoteKeyForFirstItem(state: PagingState<Int, Repo>): RemoteKeys? {
    // Get the first page that was retrieved, that contained items.
    // From that first page, get the first item
    return state.pages.firstOrNull { it.data.isNotEmpty() }?.data?.firstOrNull()
            ?.let { repo ->
                // Get the remote keys of the first items retrieved
                repoDatabase.remoteKeysDao().remoteKeysRepoId(repo.id)
            }
}
  1. If remoteKeys is null, that means the refresh result is not in the database yet. We can return Success with endOfPaginationReached = false because Paging will call this method again if RemoteKeys becomes non-null. If remoteKeys is not null but its prevKey is null, that means we've reached the end of pagination for prepend.
val page = when (loadType) {
    LoadType.PREPEND -> {
        val remoteKeys = getRemoteKeyForFirstItem(state)
        // If remoteKeys is null, that means the refresh result is not in the database yet.
        val prevKey = remoteKeys?.prevKey
        if (prevKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        prevKey
    }

      ...
  }

LoadType.REFRESH

LoadType.REFRESH gets called when it's the first time we're loading data, or when PagingDataAdapter.refresh() is called; so now the point of reference for loading our data is the state.anchorPosition. If this is the first load, then the anchorPosition is null. When PagingDataAdapter.refresh() is called, the anchorPosition is the first visible position in the displayed list, so we will need to load the page that contains that specific item.

  1. Based on the anchorPosition from the state, we can get the closest Repo item to that position by calling state.closestItemToPosition().
  2. Based on the Repo item, we can get the RemoteKeys from the database.
private suspend fun getRemoteKeyClosestToCurrentPosition(
        state: PagingState<Int, Repo>
): RemoteKeys? {
    // The paging library is trying to load data after the anchor position
    // Get the item closest to the anchor position
    return state.anchorPosition?.let { position ->
        state.closestItemToPosition(position)?.id?.let { repoId -> 
   repoDatabase.remoteKeysDao().remoteKeysRepoId(repoId)
        }
    }
}
  1. If remoteKey is not null, then we can get the nextKey from it. In the Github API the page keys are incremented sequentially. So to get the page that contains the current item, we just subtract 1 from remoteKey.nextKey.
  2. If RemoteKey is null (because the anchorPosition was null), then the page we need to load is the initial one: GITHUB_STARTING_PAGE_INDEX

Now, the full page computation looks like this:

val page = when (loadType) {
    LoadType.REFRESH -> {
        val remoteKeys = getRemoteKeyClosestToCurrentPosition(state)
        remoteKeys?.nextKey?.minus(1) ?: GITHUB_STARTING_PAGE_INDEX
    }
    LoadType.PREPEND -> {
        val remoteKeys = getRemoteKeyForFirstItem(state)
        val prevKey = remoteKeys?.prevKey
        if (prevKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        prevKey
    }
    LoadType.APPEND -> {
        val remoteKeys = getRemoteKeyForLastItem(state)
        val nextKey = remoteKeys?.nextKey
        if (nextKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        nextKey
    }
}

19. Update the paging Flow creation

Now that we have the GithubRemoteMediator and the PagingSource in our ReposDao implemented, we need to update GithubRepository.getSearchResultStream to use them.

In order to do this, GithubRepository needs access to the database. Let's pass the database as a parameter in the constructor. Also, since this class will use GithubRemoteMediator:

class GithubRepository(
        private val service: GithubService,
        private val database: RepoDatabase
) { ... }

Update the Injection file:

  • provideGithubRepository method should get a context as a parameter and in the GithubRepository constructor invoke RepoDatabase.getInstance.
  • provideViewModelFactory method should get a context as a parameter and pass it to provideGithubRepository.
object Injection {
    private fun provideGithubRepository(context: Context): GithubRepository {
        return GithubRepository(GithubService.create(), RepoDatabase.getInstance(context))
    }

    fun provideViewModelFactory(context: Context, owner: SavedStateRegistryOwner): ViewModelProvider.Factory {
        return ViewModelFactory(owner, provideGithubRepository(context))
    }
}

Update the SearchRepositoriesActivity.onCreate() method and pass the context to Injection.provideViewModelFactory():

       // get the view model
        val viewModel = ViewModelProvider(
            this, Injection.provideViewModelFactory(
                context = this,
                owner = this
            )
        )
            .get(SearchRepositoriesViewModel::class.java)

Let's get back to GithubRepository. First, to be able to search for repos by name, we'll have to add % to the beginning and end of the query string. Then, when calling the reposDao.reposByName, we get a PagingSource. Because the PagingSource is invalidated every time we make a change in the database, we need to tell Paging how to get a new instance of the PagingSource. To do this, we just create a function that calls the database query:

// appending '%' so we can allow other characters to be before and after the query string
val dbQuery = "%${query.replace(' ', '%')}%"
val pagingSourceFactory =  { database.reposDao().reposByName(dbQuery)}

Now we can change the Pager builder, to use a GithubRemoteMediator and the pagingSourceFactory. The Pager is an experimental API so we'll have to annotate it with @OptIn:

@OptIn(ExperimentalPagingApi::class)
return Pager(
        config = PagingConfig(
            pageSize = NETWORK_PAGE_SIZE,
            enablePlaceholders = false
         ),
        remoteMediator = GithubRemoteMediator(
                query,
                service,
                database
        ),
        pagingSourceFactory = pagingSourceFactory
).flow

That's it! Let's run the app!

Reacting to load states when using a RemoteMediator

Until now, when reading from CombinedLoadStates, we've always read from CombinedLoadStates.source. When using a RemoteMediator however, accurate loading information can only be obtained by checking both CombinedLoadStates.source and CombinedLoadStates.mediator. In particular, we currently trigger a scroll to the top of the list on new queries when the source LoadState is NotLoading. We also have to make sure that our newly-added RemoteMediator has a LoadState of NotLoading as well.

To do this, we define an enum that summarizes the presentation states of our list as fetched by the Pager:

enum class RemotePresentationState {
    INITIAL, REMOTE_LOADING, SOURCE_LOADING, PRESENTED
}

With the above definition, we can then compare consecutive emissions of CombinedLoadStates, and use them to determine the exact state of the items in the list.

@OptIn(ExperimentalCoroutinesApi::class)
fun Flow<CombinedLoadStates>.asRemotePresentationState(): Flow<RemotePresentationState> =
    scan(RemotePresentationState.INITIAL) { state, loadState ->
        when (state) {
            RemotePresentationState.PRESENTED -> when (loadState.mediator?.refresh) {
                is LoadState.Loading -> RemotePresentationState.REMOTE_LOADING
                else -> state
            }
            RemotePresentationState.INITIAL -> when (loadState.mediator?.refresh) {
                is LoadState.Loading -> RemotePresentationState.REMOTE_LOADING
                else -> state
            }
            RemotePresentationState.REMOTE_LOADING -> when (loadState.source.refresh) {
                is LoadState.Loading -> RemotePresentationState.SOURCE_LOADING
                else -> state
            }
            RemotePresentationState.SOURCE_LOADING -> when (loadState.source.refresh) {
                is LoadState.NotLoading -> RemotePresentationState.PRESENTED
                else -> state
            }
        }
    }
        .distinctUntilChanged()

The above lets us update the definition of the notLoading Flow we use to check whether we can scroll to the top of the list:

       val notLoading = repoAdapter.loadStateFlow
            .asRemotePresentationState()
            .map { it == RemotePresentationState.PRESENTED }

Similarly, when it comes to displaying a loading spinner during the initial page load (in the bindList extension in SearchRepositoriesActivity) , the app still relies on LoadState.source. What we want right now is to show a loading spinner only for loads from RemoteMediator. Other UI elements whose visibility depends on the LoadStates also share this concern. We therefore update the binding of the LoadStates to the UI elements as follows:

private fun ActivitySearchRepositoriesBinding.bindList(
        header: ReposLoadStateAdapter,
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        ...

        lifecycleScope.launch {
            repoAdapter.loadStateFlow.collect { loadState ->
                ...
                val isListEmpty = loadState.refresh is LoadState.NotLoading && repoAdapter.itemCount == 0
                // show empty list
                emptyList.isVisible = isListEmpty
                // Only show the list if refresh succeeds, either from the the local db or the remote.
                list.isVisible =  loadState.source.refresh is LoadState.NotLoading || loadState.mediator?.refresh is LoadState.NotLoading
                // Show loading spinner during initial load or refresh.
                progressBar.isVisible = loadState.mediator?.refresh is LoadState.Loading
                // Show the retry state if initial load or refresh fails.
                retryButton.isVisible = loadState.mediator?.refresh is LoadState.Error && repoAdapter.itemCount == 0
                }
            }
        }
    }

Furthermore, because we have the database as the single source of truth, it is possible to launch the app in a state where we have data in the database, but a refresh with the RemoteMediator fails. This is an interesting edge case, but one we can easily handle. To do this, we can keep a reference to the header LoadStateAdapter and override its LoadState to be that of the RemoteMediator's if and only if its refresh state has an error. Otherwise, we go with the default.

private fun ActivitySearchRepositoriesBinding.bindList(
        header: ReposLoadStateAdapter,
        repoAdapter: ReposAdapter,
        uiState: StateFlow<UiState>,
        onScrollChanged: (UiAction.Scroll) -> Unit
    ) {
        ...

        lifecycleScope.launch {
            repoAdapter.loadStateFlow.collect { loadState ->
                // Show a retry header if there was an error refreshing, and items were previously
                // cached OR default to the default prepend state
                header.loadState = loadState.mediator
                    ?.refresh
                    ?.takeIf { it is LoadState.Error && repoAdapter.itemCount > 0 }
                    ?: loadState.prepend
                ...
            }
        }
    }

You can find the full code for the steps done so far in branch step13-19_network_and_database.

20. Wrap up

Now that we have added all the components, let's recap what we've learned!

  • The PagingSource asynchronously loads the data from a source you define.
  • The Pager.flow creates a Flow<PagingData> based on a configuration and a function that defines how to instantiate the PagingSource.
  • The Flow emits a new PagingData whenever new data is loaded by the PagingSource.
  • The UI observes the changed PagingData and uses a PagingDataAdapter to update the RecyclerView that presents the data.
  • To retry a failed load from the UI, use the PagingDataAdapter.retry method. Under the hood, the Paging library will trigger the PagingSource.load() method.
  • To add separators to your list, create a high-level type with separators as one of the supported types. Then use the PagingData.insertSeparators() method to implement your separator generation logic.
  • To display the load state as header or footer, use PagingDataAdapter.withLoadStateHeaderAndFooter() method and implement a LoadStateAdapter. If you want to execute other actions based on the load state, use the PagingDataAdapter.addLoadStateListener() callback.
  • To work with the network and database, implement a RemoteMediator.
  • Adding a RemoteMediator causes updates to the mediator field in the LoadStatesFlow.