Android Paging

What you'll learn

• What the main components of Paging 3.0 are.
• How to add Paging 3.0 to your project.
• How to add a header or footer to your list using the Paging 3.0 API.
• How to add list separators using the Paging 3.0 API.
• How to page from network and database

What you'll build

In this codelab, you start with a sample app that already displays a list of GitHub repositories. Whenever the user scrolls to the end of the displayed list, a new network request is triggered and its result is displayed on the screen.

You will add code through a series of steps, to achieve the following:

• Migrate to the Paging library components.
• Add a loading status header and footer to your list.
• Show loading progress between every new repository search.
• Add separators in your list.
• Add database support for paging from network and database

Here's what your app will look like in the end:

What you'll need

• Android Studio 3.6 or higher.
• Familiarity with the following Architecture Components: LiveData, ViewModel, View Binding and with the architecture suggested in the " Guide to App Architecture".
• Familiarity with coroutines and Kotlin Flow.

For an introduction to Architecture Components, check out the Room with a View codelab. For an introduction to Flow, check out the Advanced Coroutines with Kotlin Flow and LiveData codelab.

In this step, you will download the code for the entire codelab and then run a simple example app.

To get you started as quickly as possible, we have prepared a starter project for you to build on.

If you have git installed, you can simply run the command below. (You can check by typing git --version in the terminal / command line and verify it executes correctly.)

 git clone https://github.com/googlecodelabs/android-paging

The initial state is on the master branch. For certain steps you can see the solution as follows:

• Branch step5-9_paging_3.0 - you'll find the solution for steps 5 - 9, where we're adding Paging 3.0 to our project.
• Branch step10_loading_state_footer - you'll find the solution for step 10, where we're adding a footer that displays a loading state.
• Branch step11_loading_state - you'll find the solution for step 11, where we're adding a display for the loading state between queries.
• Branch step12_separators - you'll find the solution for step 12, where we're adding separators to our app.
• Branch step13-19_network_and_database - you'll find the solution for steps 13 - 19, where we're adding offline support to our app

If you do not have git, you can click the following button to download all the code for this codelab:

Download source code

1. Unzip the code, and then open the project Android Studio version 3.6.1 or newer.
2. Run the app run configuration on a device or emulator.

The app runs and displays a list of GitHub repositories similar to this one:

The app allows you to search GitHub for repositories whose name or description contains a specific word. The list of repositories is displayed in descending order based on the number of stars, then alphabetically by name.

The app follows the architecture recommended in the " Guide to app architecture". Here's what you will find in each package:

• api - Github API calls, using Retrofit.
• data - the repository class, responsible for triggering API requests and caching the responses in memory.
• model - the Repo data model, which is also a table in the Room database; and RepoSearchResult, a class that is used by the UI to observe both search results data and network errors.
• ui - classes related to displaying an Activity with a RecyclerView.

The GithubRepository class retrieves the list of repository names from the network every time the user scrolls towards the end of the list, or when the user searches for a new repository. The list of results for a query is kept in memory in the GithubRepository in a ConflatedBroadcastChannel and exposed as a Flow.

SearchRepositoriesViewModel requests the data from GithubRepository and exposes it to the SearchRepositoriesActivity. Because we want to ensure that we're not requesting the data multiple times on configuration change (e.g. rotation), we're converting the Flow to LiveData in the ViewModel using the liveData() builder method. That way, the LiveData caches the latest list of results in memory, and when the SearchRepositoriesActivity gets recreated, the content of the LiveData will be displayed on the screen.

From a usability perspective, we have the following issues:

• The user has no information on the list loading state: they see an empty screen when they search for a new repository or just an abrupt end of the list while more results for the same query are being loaded.
• The user can't retry a failed query.

From an implementation perspective, we have the following issues:

• The list grows unbounded in memory, wasting memory as the user scrolls.
• We have to convert our results from Flow to LiveData to cache them, increasing the complexity of our code.
• If our app needed to show multiple lists, we'd see that there is a lot of boilerplate to write for each list.

Let's find out how the Paging library can help us with these issues and what components it includes.

The Paging library makes it easier for you to load data incrementally and gracefully within your app's UI. The Paging API provides support for many of the functionalities that you would otherwise need to implement manually when you need to load data in pages:

• Keeps track of the keys to be used for retrieving the next and previous page.
• Automatically requests the correct page when the user has scrolled to the end of the list.
• Ensures that multiple requests aren't triggered at the same time.
• Allows you to cache data: if you're using Kotlin, this is done in a CoroutineScope; if you're using Java, this can be done with LiveData.
• Tracks loading state and allows you to display it in a RecyclerView list item or elsewhere in your UI, and easily retry failed loads.
• Allows you to execute common operations like map or filter on the list that will be displayed, independently of whether you're using Flow, LiveData, or RxJava Flowable or Observable.
• Provides an easy way of implementing list separators.

The Guide to app architecture proposes an architecture with the following main components:

• A local database that serves as a single source of truth for data presented to the user and manipulated by the user.
• A web API service.
• A repository that works with the database and the web API service, providing a unified data interface.
• A ViewModel that provides data specific to the UI.
• The UI, which shows a visual representation of the data in the ViewModel.

The Paging library works with all of these components and coordinates the interactions between them, so that it can load "pages" of content from a data source and display that content in the UI.

This codelab introduces you to the Paging library and its main components:

• PagingData - a container for paginated data. Each refresh of data will have a separate corresponding PagingData.
• PagingSource - a PagingSource is the base class for loading snapshots of data into a stream of PagingData.
• Pager.flow - builds a Flow<PagingData>, based on a PagingConfig and a function that defines how to construct the implemented PagingSource.
• PagingDataAdapter - a RecyclerView.Adapter that presents PagingData in a RecyclerView. The PagingDataAdapter can be connected to a Kotlin Flow, a LiveData, an RxJava Flowable, or an RxJava Observable. The PagingDataAdapter listens to internal PagingData loading events as pages are loaded and uses DiffUtil on a background thread to compute fine-grained updates as updated content is received in the form of new PagingData objects.
• RemoteMediator - helps implement pagination from network and database.

In this codelab, you will implement examples of each of the components described above.

The PagingSource implementation defines the source of data and how to retrieve data from that source. The PagingData object queries data from the PagingSource in response to loading hints that are generated as the user scrolls in a RecyclerView.

Currently, the GithubRepository has a lot of the responsibilities of a data source that the Paging library will handle once we're done adding it:

• Loads the data from GithubService, ensuring that multiple requests aren't triggered at the same time.
• Keeps an in-memory cache of the retrieved data.
• Keeps track of the page to be requested.

To build the PagingSource you need to define the following:

• The type of the paging key - in our case, the Github API uses 1-based index numbers for pages, so the type is Int.
• The type of data loaded - in our case, we're loading Repo items.
• Where is the data retrieved from - we're getting the data from GithubService. Our data source will be specific to a certain query, so we need to make sure we're also passing the query information to GithubService.

So, in the data package, let's create a PagingSource implementation called GithubPagingSource:

class GithubPagingSource(
        private val service: GithubService,
        private val query: String
) : PagingSource<Int, Repo>() { 
    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Repo> {
        TODO("Not yet implemented")
    }
   override fun getRefreshKey(state: PagingState<Int, Repo>): Int? { 
        TODO("Not yet implemented")
    }

}

We'll see that PagingSource requires us to implement two functions: load and getRefreshKey.

The load() function will be called by the Paging library to asynchronously fetch more data to be displayed as the user scrolls around. The LoadParams object keeps information related to the load operation, including the following:

• Key of the page to be loaded. If this is the first time that load is called, LoadParams.key will be null. In this case, you will have to define the initial page key. For our project, you'll have to move GITHUB_STARTING_PAGE_INDEX constant from GithubRepository to your PagingSource implementation since this is the initial page key.
• Load size - the requested number of items to load.

The load function returns a LoadResult. This will replace the usage of RepoSearchResult in our app, as LoadResult can take one of the following types:

• LoadResult.Page, if the result was successful.
• LoadResult.Error, in case of error.

When constructing the LoadResult.Page, pass null for nextKey or prevKey if the list can't be loaded in the corresponding direction. For example, in our case, we could consider that if the network response was successful but the list was empty, we don't have any data left to be loaded; so the nextKey can be null.

Based on all of this information, we should be able to implement the load() function!

Next we need to implement getRefreshKey(). The refresh key is used for subsequent refresh calls to PagingSource.load() (the first call is initial load which uses initialKey provided by Pager). A refresh happens whenever the Paging library wants to load new data to replace the current list, e.g., on swipe to refresh or on invalidation due to database updates, config changes, process death, etc. Typically, subsequent refresh calls will want to restart loading data centered around PagingState.anchorPosition which represents the most recently accessed index.

The GithubPagingSource implementation looks like this:

// GitHub page API is 1 based: https://developer.github.com/v3/#pagination
private const val GITHUB_STARTING_PAGE_INDEX = 1

class GithubPagingSource(
        private val service: GithubService,
        private val query: String
) : PagingSource<Int, Repo>() {

    override suspend fun load(params: LoadParams<Int>): LoadResult<Int, Repo> {
        val position = params.key ?: GITHUB_STARTING_PAGE_INDEX
        val apiQuery = query + IN_QUALIFIER
        return try {
            val response = service.searchRepos(apiQuery, position, params.loadSize)
            val repos = response.items
            val nextKey = if (repos.isEmpty()) {
                null
            } else {
                // initial load size = 3 * NETWORK_PAGE_SIZE
                // ensure we're not requesting duplicating items, at the 2nd request
                position + (params.loadSize / NETWORK_PAGE_SIZE)
            }
            LoadResult.Page(
                    data = repos,
                    prevKey = if (position == GITHUB_STARTING_PAGE_INDEX) null else position - 1,
                    nextKey = nextKey
            )
        } catch (exception: IOException) {
            return LoadResult.Error(exception)
        } catch (exception: HttpException) {
            return LoadResult.Error(exception)
        }
    }
    // The refresh key is used for subsequent refresh calls to PagingSource.load after the initial load
    override fun getRefreshKey(state: PagingState<Int, Repo>): Int? {
        // We need to get the previous key (or next key if previous is null) of the page
        // that was closest to the most recently accessed index.
        // Anchor position is the most recently accessed index
        return state.anchorPosition?.let { anchorPosition ->
            state.closestPageToPosition(anchorPosition)?.prevKey?.plus(1)
                ?: state.closestPageToPosition(anchorPosition)?.nextKey?.minus(1)
        }
    }

}

In our current implementation, we use a Flow<RepoSearchResult> in the GitHubRepository to get the data from the network and pass it to the ViewModel. The ViewModel then transforms it into a LiveData and exposes it to the UI. Whenever we get to the end of the displayed list and more data is loaded from the network, the Flow<RepoSearchResult> will contain the entire list of previously retrieved data for that query in addition to the latest data.

RepoSearchResult encapsulates both the success and error cases. The success case holds the repository data. The error case contains the Exception reason. With Paging 3.0 we don't need the RepoSearchResult anymore, as the library models both the success and error cases with LoadResult. Feel free to delete RepoSearchResult as in the next few steps we'll replace it.

To construct the PagingData, we first need to decide what API we want to use to pass the PagingData to other layers of our app:

• Kotlin Flow - use Pager.flow.
• LiveData - use Pager.liveData.
• RxJava Flowable - use Pager.flowable.
• RxJava Observable - use Pager.observable.

As we're already using Flow in our app, we'll continue with this approach; but instead of using Flow<RepoSearchResult>, we'll use Flow<PagingData<Repo>>.

No matter which PagingData builder you use, you'll have to pass the following parameters:

• PagingConfig. This class sets options regarding how to load content from a PagingSource such as how far ahead to load, the size request for the initial load, and others. The only mandatory parameter you have to define is the page size—how many items should be loaded in each page. By default, Paging will keep in memory all the pages you load. To ensure that you're not wasting memory as the user scrolls, set the maxSize parameter in PagingConfig. By default Paging will return null items as a placeholder for content that is not yet loaded if Paging can count the unloaded items and if the enablePlaceholders config flag is true. Like this, you will be able to display a placeholder view in your adapter. To simplify the work in this codelab, let's disable the placeholders by passing enablePlaceholders = false.
• A function that defines how to create the PagingSource. In our case, we'll create a new GithubPagingSource for each new query.

Let's modify our GithubRepository!

Update GithubRepository.getSearchResultStream

• Remove the suspend modifier.
• Return Flow<PagingData<Repo>>.
• Construct Pager.

fun getSearchResultStream(query: String): Flow<PagingData<Repo>> {
    return Pager(
          config = PagingConfig(
            pageSize = NETWORK_PAGE_SIZE,
            enablePlaceholders = false
         ),
          pagingSourceFactory = { GithubPagingSource(service, query) }
    ).flow
}

Cleanup GithubRepository

Paging 3.0 does a lot of things for us:

• Handles in-memory cache.
• Requests data when the user is close to the end of the list.

This means that everything else in our GithubRepository can be removed, except getSearchResultStream and the companion object where we defined the NETWORK_PAGE_SIZE. Your GithubRepository should now look like this:

class GithubRepository(private val service: GithubService) {

    fun getSearchResultStream(query: String): Flow<PagingData<Repo>> {
        return Pager(
                config = PagingConfig(
                pageSize = NETWORK_PAGE_SIZE,
                enablePlaceholders = false
             ),
                pagingSourceFactory = { GithubPagingSource(service, query) }
        ).flow
    }

    companion object {
        private const val NETWORK_PAGE_SIZE = 50
    }
}

You should now have compile errors in the SearchRepositoriesViewModel. Let's see what changes need to be made there!

From our SearchRepositoriesViewModel we expose a repoResult: LiveData<RepoSearchResult>. The role of repoResult is to be an in-memory cache for result searches that survives configuration changes. With Paging 3.0 we don't need to convert our Flow to LiveData anymore. Instead, SearchRepositoriesViewModel will have a private Flow<PagingData<Repo>> member that serves the same purpose that repoResult did..

Instead of using a LiveData object for each new query, we can just use a String. This will help us ensure that whenever we get a new search query that is the same as the current query, we will just return the existing Flow. We only need to call repository.getSearchResultStream() if the new search query is different.

Flow<PagingData> has a handy cachedIn() method that allows us to cache the content of a Flow<PagingData> in a CoroutineScope. Since we're in a ViewModel, we will use the androidx.lifecycle.viewModelScope.

We'll rewrite most of SearchRepositoriesViewModel to leverage the built-in functionality from Paging 3.0. Your SearchRepositoriesViewModel will look like this:

class SearchRepositoriesViewModel(private val repository: GithubRepository) : ViewModel() {

    private var currentQueryValue: String? = null

    private var currentSearchResult: Flow<PagingData<Repo>>? = null

    fun searchRepo(queryString: String): Flow<PagingData<Repo>> {
        val lastResult = currentSearchResult
        if (queryString == currentQueryValue && lastResult != null) {
            return lastResult
        }
        currentQueryValue = queryString
        val newResult: Flow<PagingData<Repo>> = repository.getSearchResultStream(queryString)
                .cachedIn(viewModelScope)
        currentSearchResult = newResult
        return newResult
    }
}

Now, let's see what changes we made to SearchRepositoriesViewModel:

• Added the new query String and search result Flow members.
• Updated the searchRepo() method with the functionality described previously.
• Removed queryLiveData and repoResult as their purpose is covered by Paging 3.0 and Flow.
• Removed the listScrolled() since the Paging library will handle this for us.
• Removed the companion object because VISIBLE_THRESHOLD is no longer needed.

To bind a PagingData to a RecyclerView, use a PagingDataAdapter. The PagingDataAdapter gets notified whenever the PagingData content is loaded and then it signals the RecyclerView to update.

Update the ui.ReposAdapter to work with a PagingData stream:

• Right now, ReposAdapter implements ListAdapter. Make it implement PagingDataAdapter instead. The rest of the class body remains unchanged:

class ReposAdapter : PagingDataAdapter<Repo, RepoViewHolder>(REPO_COMPARATOR) {
// body is unchanged
}

We've been making a lot of changes so far, but now we're just one step away from being able to run the app—we just need to connect the UI!

Let's update SearchRepositoriesActivity to work with Paging 3.0. To be able to work with Flow<PagingData>, we need to launch a new coroutine. We will do that in the lifecycleScope, which is responsible for canceling the request when the activity is recreated.

We also want to ensure that whenever the user searches for a new query, the previous query is cancelled. To do this, our SearchRepositoriesActivity can hold a reference to a new Job that will be cancelled every time we search for a new query.

Let's create a new search function that gets a query as a parameter. The function should do the following:

• Cancel the previous search job.
• Launch a new job in lifecycleScope.
• Call viewModel.searchRepo.
• Collect the PagingData result.
• Pass the PagingData to the ReposAdapter by calling adapter.submitData(pagingData).

private var searchJob: Job? = null

private fun search(query: String) {
   // Make sure we cancel the previous job before creating a new one
   searchJob?.cancel()
   searchJob = lifecycleScope.launch {
       viewModel.searchRepo(query).collectLatest {
           adapter.submitData(it)
       }
   }
}

The search function should be called in SearchRepositoriesActivity in the onCreate() method, In updateRepoListFromInput(), replace the viewModel and adapter calls with search():

private fun updateRepoListFromInput() {
    binding.searchRepo.text.trim().let {
        if (it.isNotEmpty()) {
            binding.list.scrollToPosition(0)
            search(it.toString())
        }
    }
}

Because we wanted to make sure that the scroll position is reset for each new search, we had: binding.list.scrollToPosition(0). But, instead of resetting the position on new search, we should reset the position when the list adapter is updated with the result of a new search. To achieve this, we can use the PagingDataAdapter.loadStateFlow API. This Flow emits us every time there's a change in the load state via a CombinedLoadStates object.

CombinedLoadStates allows us to get the load state for the 3 different types of load operations:

• CombinedLoadStates.refresh - represents the load state for loading the PagingData for the first time.
• CombinedLoadStates.prepend - represents the load state for loading data at the start of the list.
• CombinedLoadStates.append - represents the load state for loading data at the end of the list.

In our case, we want to reset the scroll position only when the refresh has completed i.e LoadState is refresh, NotLoading.

Let's collect from this flow when we initialize the search, in the initSearch method and at every new emission of the flow, let's scroll to position 0.

private fun initSearch(query: String) {
    ...
    // First part of the method is unchanged

        // Scroll to top when the list is refreshed from network.
        lifecycleScope.launch {
            adapter.loadStateFlow
                   // Only emit when REFRESH LoadState changes.
                   .distinctUntilChangedBy { it.refresh }
                   // Only react to cases where REFRESH completes i.e., NotLoading.
                   .filter { it.refresh is LoadState.NotLoading }
                   .collect { binding.list.scrollToPosition(0) }
        }
}

We can now remove binding.list.scrollToPosition(0) from updateRepoListFromInput().

Currently, we use an OnScrollListener attached to the RecyclerView to know when to trigger more data. We can let the Paging library handle list scrolling for us. Remove the setupScrollListener() method and all references to it.

Let's also remove repoResult usage. This is what your activity should look like:

class SearchRepositoriesActivity : AppCompatActivity() {

    private lateinit var binding: ActivitySearchRepositoriesBinding
    private lateinit var viewModel: SearchRepositoriesViewModel
    private val adapter = ReposAdapter()

    private var searchJob: Job? = null

    private fun search(query: String) {
        // Make sure we cancel the previous job before creating a new one
        searchJob?.cancel()
        searchJob = lifecycleScope.launch {
            viewModel.searchRepo(query).collect {
                adapter.submitData(it)
            }
        }
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivitySearchRepositoriesBinding.inflate(layoutInflater)
        val view = binding.root
        setContentView(view)

        // get the view model
        viewModel = ViewModelProvider(this, Injection.provideViewModelFactory())
                .get(SearchRepositoriesViewModel::class.java)

        // add dividers between RecyclerView's row items
        val decoration = DividerItemDecoration(this, DividerItemDecoration.VERTICAL)
        binding.list.addItemDecoration(decoration)

        initAdapter()
        val query = savedInstanceState?.getString(LAST_SEARCH_QUERY) ?: DEFAULT_QUERY
        search(query)
        initSearch(query)
    }

    override fun onSaveInstanceState(outState: Bundle) {
        super.onSaveInstanceState(outState)
        outState.putString(LAST_SEARCH_QUERY, binding.searchRepo.text.trim().toString())
    }

    private fun initAdapter() {
        binding.list.adapter = adapter
    }

    private fun initSearch(query: String) {
        binding.searchRepo.setText(query)

        binding.searchRepo.setOnEditorActionListener { _, actionId, _ ->
            if (actionId == EditorInfo.IME_ACTION_GO) {
                updateRepoListFromInput()
                true
            } else {
                false
            }
        }
        binding.searchRepo.setOnKeyListener { _, keyCode, event ->
            if (event.action == KeyEvent.ACTION_DOWN && keyCode == KeyEvent.KEYCODE_ENTER) {
                updateRepoListFromInput()
                true
            } else {
                false
            }
        }

        // Scroll to top when the list is refreshed from network.
        lifecycleScope.launch {
            adapter.loadStateFlow
                    // Only emit when REFRESH LoadState for RemoteMediator changes.
                    .distinctUntilChangedBy { it.refresh }
                    // Only react to cases where Remote REFRESH completes i.e., NotLoading.
                    .filter { it.refresh is LoadState.NotLoading }
                    .collect { binding.list.scrollToPosition(0) }
        }
    }

    private fun updateRepoListFromInput() {
        binding.searchRepo.text.trim().let {
            if (it.isNotEmpty()) {
                search(it.toString())
            }
        }
    }

    private fun showEmptyList(show: Boolean) {
        if (show) {
            binding.emptyList.visibility = View.VISIBLE
            binding.list.visibility = View.GONE
        } else {
            binding.emptyList.visibility = View.GONE
            binding.list.visibility = View.VISIBLE
        }
    }

    companion object {
        private const val LAST_SEARCH_QUERY: String = "last_search_query"
        private const val DEFAULT_QUERY = "Android"
    }
}

Our app should compile and run, but without the loading state footer and the Toast that displays on error. In the next step, we will see how to display the loading state footer.

You can find the full code for the steps done so far in branch step5-9_paging_3.0.

In our app, we want to be able to display a footer based on the load status: when the list is loading, we want to show a progress spinner. In case of an error, we want to show the error and a retry button.

The header/footer that we need to build follows the idea of a list that needs to be appended either at the beginning (as a header) or at the end (as a footer) of the actual list of items we're displaying. The header/footer is a list with only one element: a view that displays a progress bar or an error with a retry button, based on the Paging LoadState.

As displaying a header/footer based on the loading state and implementing a retry mechanism are common tasks, the Paging 3.0 API helps us with both of these.

For header/footer implementation we'll use a LoadStateAdapter. This implementation of RecyclerView.Adapter is automatically notified of changes in load state. It makes sure that only Loading and Error states lead to items being displayed and notifies the RecyclerView when an item is removed, inserted, or changed, depending on the LoadState.

For the retry mechanism we use adapter.retry(). Under the hood, this method ends up calling your PagingSource implementation for the right page. The response will be automatically propagated via Flow<PagingData>.

Let's see what our header/footer implementation looks like!

Like with any list, we have 3 files to create:

• The layout file containing the UI elements for displaying progress, the error and the retry button
• The ViewHolder file making the UI items visible based on the Paging LoadState
• The adapter file defining how to create and bind the ViewHolder. Instead of extending a RecyclerView.Adapter, we will extend LoadStateAdapter from Paging 3.0.

Create the view layout

Create the repos_load_state_footer_view_item layout for our repo load state. It should have a ProgressBar, a TextView (to display the error), and a retry Button. The necessary strings and dimensions are already declared in the project.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    xmlns:tools="http://schemas.android.com/tools"
    android:orientation="vertical"
    android:padding="8dp">
    <TextView
        android:id="@+id/error_msg"
        android:textColor="?android:textColorPrimary"
        android:textSize="@dimen/error_text_size"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="center"
        android:textAlignment="center"
        tools:text="Timeout"/>
    <ProgressBar
        android:id="@+id/progress_bar"
        style="?android:attr/progressBarStyle"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_gravity="center"/>
    <Button
        android:id="@+id/retry_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="center"
        android:text="@string/retry"/>
</LinearLayout>

Create the ViewHolder

Create a new ViewHolder called ReposLoadStateViewHolder in the ui folder**.** It should receive a retry function as a parameter, to be called when the retry button is pressed. Create a bind() function that receives the LoadState as a parameter and sets the visibility of each view depending on the LoadState. An implementation of ReposLoadStateViewHolder using ViewBinding looks like this:

class ReposLoadStateViewHolder(
        private val binding: ReposLoadStateFooterViewItemBinding,
        retry: () -> Unit
) : RecyclerView.ViewHolder(binding.root) {

    init {
        binding.retryButton.setOnClickListener { retry.invoke() }
    }

    fun bind(loadState: LoadState) {
        if (loadState is LoadState.Error) {
            binding.errorMsg.text = loadState.error.localizedMessage
        }
        binding.progressBar.isVisible = loadState is LoadState.Loading
        binding.retryButton.isVisible = loadState is LoadState.Error
        binding.errorMsg.isVisible = loadState is LoadState.Error
    }

    companion object {
        fun create(parent: ViewGroup, retry: () -> Unit): ReposLoadStateViewHolder {
            val view = LayoutInflater.from(parent.context)
                    .inflate(R.layout.repos_load_state_footer_view_item, parent, false)
            val binding = ReposLoadStateFooterViewItemBinding.bind(view)
            return ReposLoadStateViewHolder(binding, retry)
        }
    }
}

Create the LoadStateAdapter

Create a ReposLoadStateAdapter that extends LoadStateAdapter in the ui folder as well. The adapter should receive the retry function as a parameter, since the retry function will be passed to the ViewHolder when constructed.

As with any Adapter, we need to implement the onBind() and onCreate() methods. LoadStateAdapter makes it easier as it passes the LoadState in both of these functions. In onBindViewHolder(), bind your ViewHolder. In onCreateViewHolder(), define how to create the ReposLoadStateViewHolder based on the parent ViewGroup and the retry function:

class ReposLoadStateAdapter(private val retry: () -> Unit) : LoadStateAdapter<ReposLoadStateViewHolder>() {
    override fun onBindViewHolder(holder: ReposLoadStateViewHolder, loadState: LoadState) {
        holder.bind(loadState)
    }

    override fun onCreateViewHolder(parent: ViewGroup, loadState: LoadState): ReposLoadStateViewHolder {
        return ReposLoadStateViewHolder.create(parent, retry)
    }
}

Bind the footer adapter with the list

Now that we have all of the elements of our footer, let's bind them to our list. To do this, the PagingDataAdapter has 3 useful methods:

• withLoadStateHeader - if we only want to display a header—this should be used when your list only supports adding items at the beginning of the list.
• withLoadStateFooter - if we only want to display a footer—this should be used when your list only supports adding items at the end of the list.
• withLoadStateHeaderAndFooter—if we want to display a header and a footer - if the list can be paged in both directions.

Update the SearchRepositoriesActivity.initAdapter() method and call withLoadStateHeaderAndFooter() on the adapter. As a retry function, we can call adapter.retry().

private fun initAdapter() {
    binding.list.adapter = adapter.withLoadStateHeaderAndFooter(
            header = ReposLoadStateAdapter { adapter.retry() },
            footer = ReposLoadStateAdapter { adapter.retry() }
    )
}

Since we have an infinite scrolling list, one easy way to get to see the footer is by putting your phone or emulator in airplane mode and scrolling until the end of the list.

Let's run the app!

You can find the full code for the steps done so far in branch step10_loading_state_footer.

You might have noticed that we currently have two problems:

• While migrating to Paging 3.0 we lost the ability to display a message when the list of results in empty.
• Whenever you search for a new query, the current query result stays on screen until we get a network response. That's bad user experience! Instead we should display a progress bar or a retry button.

The solution to both of these problems is to need to react to load state changes in our SearchRepositoriesActivity.

Display empty list message

First, let's bring back the empty list message. This message should be displayed only once the list was loaded and the number of items in the list is 0. To know when the list was loaded we will use the PagingDataAdapter.addLoadStateListener() method. This callback notifies us every time there's a change in the load state via a CombinedLoadStates object.

CombinedLoadStates gives us the load state for the PageSource we defined or for the RemoteMediator needed for network and database case (more about this later).

In SearchRepositoriesActivity.initAdapter() we call addLoadStateListener. The list is empty when the refresh state of CombinedLoadStates is NotLoading and adapter.itemCount == 0. Then we call showEmptyList:

private fun initAdapter() {
    binding.list.adapter = adapter.withLoadStateHeaderAndFooter(
            header = ReposLoadStateAdapter { adapter.retry() },
            footer = ReposLoadStateAdapter { adapter.retry() }
    )
    adapter.addLoadStateListener { loadState ->
        // show empty list
        val isListEmpty = loadState.refresh is LoadState.NotLoading && adapter.itemCount == 0
        showEmptyList(isListEmpty)
  }
}

Display loading state

Let's update our activity_search_repositories.xml to include a retry button and a progress bar UI elements:

<androidx.constraintlayout.widget.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".ui.SearchRepositoriesActivity">
    <com.google.android.material.textfield.TextInputLayout
        android:id="@+id/input_layout"
        android:layout_width="0dp"
        android:layout_height="wrap_content"
        android:layout_marginEnd="8dp"
        android:layout_marginStart="8dp"
        android:layout_marginTop="8dp"
        android:layout_marginLeft="8dp"
        android:layout_marginRight="8dp"
        style="@style/Widget.MaterialComponents.TextInputLayout.OutlinedBox"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent">
        <EditText
            android:id="@+id/search_repo"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:hint="@string/search_hint"
            android:imeOptions="actionSearch"
            android:inputType="textNoSuggestions"
            android:selectAllOnFocus="true"
            tools:text="Android"/>
    </com.google.android.material.textfield.TextInputLayout>

    <androidx.recyclerview.widget.RecyclerView
        android:id="@+id/list"
        android:layout_width="0dp"
        android:layout_height="0dp"
        android:paddingVertical="@dimen/row_item_margin_vertical"
        android:scrollbars="vertical"
        app:layoutManager="LinearLayoutManager"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@+id/input_layout"
        tools:ignore="UnusedAttribute"/>

    <ProgressBar
        android:id="@+id/progress_bar"
        style="?android:attr/progressBarStyle"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_gravity="center"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"/>

    <Button
        android:id="@+id/retry_button"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="@string/retry"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"/>

    <TextView android:id="@+id/emptyList"
        android:layout_width="0dp"
        android:layout_height="match_parent"
        android:gravity="center"
        android:text="@string/no_results"
        android:textSize="@dimen/repo_name_size"
        android:visibility="gone"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"/>

</androidx.constraintlayout.widget.ConstraintLayout>

Our retry button should trigger a reload of the PagingData. To do this, we call adapter.retry() in the onClickListener implementation, like we did for the header/footer:

// SearchRepositoriesActivity.kt

override fun onCreate(savedInstanceState: Bundle?) {
    ...
    binding.retryButton.setOnClickListener { adapter.retry() }
}

Next, let's react on load state changes, in SearchRepositoriesActivity.initAdapter. Since we only want our progress bar to be displayed when we have a new query, we need to rely on load from our paging source, specifically CombinedLoadStates.source.refresh and on the LoadState: Loading or Error rely. Also, one piece of functionality we commented out in a previous step was displaying a Toast when we got an error, so let's make sure we bring that in as well. To display the error message we will have to check whether CombinedLoadStates.prepend or CombinedLoadStates.append is an instance of LoadState.Error and retrieve the error message from the error.

Let's update our SearchRepositoriesActivity.initAdapter method to have this functionality:

private fun initAdapter() {
    binding.list.adapter = adapter.withLoadStateHeaderAndFooter(
            header = ReposLoadStateAdapter { adapter.retry() },
            footer = ReposLoadStateAdapter { adapter.retry() }
    )
    adapter.addLoadStateListener { loadState ->
        // show empty list
        val isListEmpty = loadState.refresh is LoadState.NotLoading && adapter.itemCount == 0
        showEmptyList(isListEmpty)

        // Only show the list if refresh succeeds.
        binding.list.isVisible = loadState.source.refresh is LoadState.NotLoading
        // Show loading spinner during initial load or refresh.
        binding.progressBar.isVisible = loadState.source.refresh is LoadState.Loading
        // Show the retry state if initial load or refresh fails.
        binding.retryButton.isVisible = loadState.source.refresh is LoadState.Error

        // Toast on any error, regardless of whether it came from RemoteMediator or PagingSource
        val errorState = loadState.source.append as? LoadState.Error
                ?: loadState.source.prepend as? LoadState.Error
                ?: loadState.append as? LoadState.Error
                ?: loadState.prepend as? LoadState.Error
        errorState?.let {
            Toast.makeText(
                    this,
                    "\uD83D\uDE28 Wooops ${it.error}",
                    Toast.LENGTH_LONG
            ).show()
        }
    }
}

Now let's run the app and check out how it works!

That's it! With the current setup, the Paging library components are the ones triggering the API requests at the right time, handling the in-memory cache, and displaying the data. Run the app and try searching for repositories.

You can find the full code for the steps done so far in branch step11_loading_state.

One way to improve your list's readability is to add separators. For example, in our app, since the repositories are ordered by number of stars descending, we could have separators every 10k stars. To help implement this, the Paging 3.0 API allows inserting separators into PagingData.

Adding separators in PagingData will lead to the modification of the list we display on our screen. We no longer display just Repo objects but also separator objects. Therefore, we have to change the UI model we're exposing from the ViewModel from Repo to another type that can encapsulate both types: RepoItem and SeparatorItem. Next, we'll have to update our UI to support separators:

• Add a layout and ViewHolder for separators.
• Update RepoAdapter to support creating and binding both separators and repositories.

Let's take this step by step and see what the implementation looks like.

Change the UI model

Currently SearchRepositoriesViewModel.searchRepo() returns Flow<PagingData<Repo>>. To support both repositories and separators, we'll create a UiModel sealed class in the same file with SearchRepositoriesViewModel. We can have 2 types of UiModel objects: RepoItem and SeparatorItem.

sealed class UiModel {
    data class RepoItem(val repo: Repo) : UiModel()
    data class SeparatorItem(val description: String) : UiModel()
}

Because we want to separate repositories based on 10k stars, let's create an extension property on RepoItem that rounds up the number of stars for us:

private val UiModel.RepoItem.roundedStarCount: Int
    get() = this.repo.stars / 10_000

Insert separators

SearchRepositoriesViewModel.searchRepo() should now return Flow<PagingData<UiModel>>. Make currentSearchResult the same type.

class SearchRepositoriesViewModel(private val repository: GithubRepository) : ViewModel() {

    private var currentQueryValue: String? = null

    private var currentSearchResult: Flow<PagingData<UiModel>>? = null

    fun searchRepo(queryString: String): Flow<PagingData<UiModel>> {
        ... 
    }
}

Let's see how the implementation changes! Currently, repository.getSearchResultStream(queryString) returns a Flow<PagingData<Repo>>, so the first operation we need to add is to transform each Repo into a UiModel.RepoItem. To do this, we can use the Flow.map operator and then map each PagingData to build a new UiModel.Repo from the current Repo item, resulting in a Flow<PagingData<UiModel.RepoItem>>:

...
val newResult: Flow<PagingData<UiModel>> = repository.getSearchResultStream(queryString)
                .map { pagingData -> pagingData.map { UiModel.RepoItem(it) } }
... 

Now we can insert the separators! For each emission of the Flow, we'll call PagingData.insertSeparators(). This method returns a PagingData containing each original element, with an optional separator that you will generate, given the elements before and after. In boundary conditions (at the beginning or end of the list) the respective before or after elements will be null. If a separator doesn't need to be created, return null.

Because we're changing the type of PagingData elements from UiModel.Repo to UiModel, make sure you explicitly set the type arguments of the insertSeparators() method.

Here's what the searchRepo() method should look like:

fun searchRepo(queryString: String): Flow<PagingData<UiModel>> {
    val lastResult = currentSearchResult
    if (queryString == currentQueryValue && lastResult != null) {
        return lastResult
    }
    currentQueryValue = queryString
    val newResult: Flow<PagingData<UiModel>> = repository.getSearchResultStream(queryString)
            .map { pagingData -> pagingData.map { UiModel.RepoItem(it) } }
            .map {
                it.insertSeparators<UiModel.RepoItem, UiModel> { before, after ->
                    if (after == null) {
                        // we're at the end of the list
                        return@insertSeparators null
                    }

                    if (before == null) {
                        // we're at the beginning of the list
                        return@insertSeparators UiModel.SeparatorItem("${after.roundedStarCount}0.000+ stars")
                    }
                    // check between 2 items
                    if (before.roundedStarCount > after.roundedStarCount) {
                        if (after.roundedStarCount >= 1) {
                            UiModel.SeparatorItem("${after.roundedStarCount}0.000+ stars")
                        } else {
                            UiModel.SeparatorItem("< 10.000+ stars")
                        }
                    } else {
                        // no separator
                        null
                    }
                }
            }
            .cachedIn(viewModelScope)
    currentSearchResult = newResult
    return newResult
}

Support multiple view types

SeparatorItem objects need to be displayed in our RecyclerView. We're only displaying a string here, so let's create a separator_view_item layout with a TextView in the res/layout folder:

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout 
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:background="@color/separatorBackground">

    <TextView
        android:id="@+id/separator_description"
        android:layout_width="0dp"
        android:layout_height="wrap_content"
        android:padding="@dimen/row_item_margin_horizontal"
        android:textColor="@color/separatorText"
        android:textSize="@dimen/repo_name_size"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent"
        tools:text="10000+ stars" />
</androidx.constraintlayout.widget.ConstraintLayout>

Let's create a SeparatorViewHolder in the ui folder, where we just bind a string to the TextView:

class SeparatorViewHolder(view: View) : RecyclerView.ViewHolder(view) {
    private val description: TextView = view.findViewById(R.id.separator_description)

    fun bind(separatorText: String) {
        description.text = separatorText
    }

    companion object {
        fun create(parent: ViewGroup): SeparatorViewHolder {
            val view = LayoutInflater.from(parent.context)
                    .inflate(R.layout.separator_view_item, parent, false)
            return SeparatorViewHolder(view)
        }
    }
}

Update ReposAdapter to support a UiModel instead of a Repo:

• Update the PagingDataAdapter parameter from Repo to UiModel.
• Implement a UiModel comparator and replace the REPO_COMPARATOR with it.
• Create the SeparatorViewHolder and bind it with the description of UiModel.SeparatorItem.

As we now need to display 2 different ViewHolders, replace RepoViewHolder with ViewHolder:

• Update the PagingDataAdapter parameter
• Update the onCreateViewHolder return type
• Update the onBindViewHolder holder parameter

Here's what your final ReposAdapter will look like:

class ReposAdapter : PagingDataAdapter<UiModel, ViewHolder>(UIMODEL_COMPARATOR) {

    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): ViewHolder {
        return if (viewType == R.layout.repo_view_item) {
            RepoViewHolder.create(parent)
        } else {
            SeparatorViewHolder.create(parent)
        }
    }

    override fun getItemViewType(position: Int): Int {
        return when (getItem(position)) {
            is UiModel.RepoItem -> R.layout.repo_view_item
            is UiModel.SeparatorItem -> R.layout.separator_view_item
            null -> throw UnsupportedOperationException("Unknown view")
        }
    }

    override fun onBindViewHolder(holder: ViewHolder, position: Int) {
        val uiModel = getItem(position)
        uiModel.let {
            when (uiModel) {
                is UiModel.RepoItem -> (holder as RepoViewHolder).bind(uiModel.repo)
                is UiModel.SeparatorItem -> (holder as SeparatorViewHolder).bind(uiModel.description)
            }
        }
    }

    companion object {
        private val UIMODEL_COMPARATOR = object : DiffUtil.ItemCallback<UiModel>() {
            override fun areItemsTheSame(oldItem: UiModel, newItem: UiModel): Boolean {
                return (oldItem is UiModel.RepoItem && newItem is UiModel.RepoItem &&
                        oldItem.repo.fullName == newItem.repo.fullName) ||
                        (oldItem is UiModel.SeparatorItem && newItem is UiModel.SeparatorItem &&
                                oldItem.description == newItem.description)
            }

            override fun areContentsTheSame(oldItem: UiModel, newItem: UiModel): Boolean =
                    oldItem == newItem
        }
    }
}

That's it! When running the app you should be able to see the separators!

You can find the full code for the steps done so far in branch step12_separators.

Let's add offline support to our app by saving the data in a local database. That way, the database will be the source of truth for our app and we will always load data from there. Whenever we don't have any more data, we request more from the network and then save it in the database. Because the database is the source of truth, the UI will be automatically updated when more data is saved.

Here's what we need to do to add offline support:

1. Create a Room database, a table to save the Repo objects in, and a DAO that we'll use to work with the Repo objects.
2. Define how to load data from the network when we reach the end of the data in the database by implementing a RemoteMediator.
3. Build a Pager based on the Repos table as a data source and the RemoteMediator for loading and saving data.

Let's take each of these steps!

Our Repo objects need to be saved in the database, so let's start by making the Repo class an entity, with tableName = "repos", where the Repo.id is the primary key. To do this, annotate the Repo class with @Entity(tableName = "repos") and add the @PrimaryKey annotation to id. This is what your Repo class should look like now:

@Entity(tableName = "repos")
data class Repo(
    @PrimaryKey @field:SerializedName("id") val id: Long,
    @field:SerializedName("name") val name: String,
    @field:SerializedName("full_name") val fullName: String,
    @field:SerializedName("description") val description: String?,
    @field:SerializedName("html_url") val url: String,
    @field:SerializedName("stargazers_count") val stars: Int,
    @field:SerializedName("forks_count") val forks: Int,
    @field:SerializedName("language") val language: String?
)

Create a new db package. This is where we will implement the class that accesses data in the database and the class that defines the database.

Implement the data access object (DAO) to access the repos table by creating a RepoDao interface, annotated with @Dao. We need the following actions on Repo:

• Insert a list of Repo objects. If the Repo objects are already in the table, then replace them.

 @Insert(onConflict = OnConflictStrategy.REPLACE)
 suspend fun insertAll(repos: List<Repo>)

• Query for repos that contain the query string in the name or in the description and sort those results in descending order by the number of stars and then alphabetically by name. Instead of returning a List<Repo>, return PagingSource<Int, Repo>. That way, the repos table becomes the source of data for Paging.

@Query("SELECT * FROM repos WHERE " +
  "name LIKE :queryString OR description LIKE :queryString " +
  "ORDER BY stars DESC, name ASC")
fun reposByName(queryString: String): PagingSource<Int, Repo>

• Clear all data in the Repos table.

@Query("DELETE FROM repos")
suspend fun clearRepos()

This is what your RepoDao should look like:

@Dao
interface RepoDao {

    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertAll(repos: List<Repo>)

    @Query("SELECT * FROM repos WHERE " +
   "name LIKE :queryString OR description LIKE :queryString " +
   "ORDER BY stars DESC, name ASC")
    fun reposByName(queryString: String): PagingSource<Int, Repo>

    @Query("DELETE FROM repos")
    suspend fun clearRepos()
}

Implement the Repo database:

• Create an abstract class RepoDatabase that extends RoomDatabase.
• Annotate the class with @Database, set the list of entities to contain the Repo class, and set the database version to 1. For the purpose of this codelab we don't need to export the schema.
• Define an abstract function that returns the ReposDao.
• Create a getInstance() function in a companion object that builds the RepoDatabase object if it doesn't exist already.

Here's what your RepoDatabase looks like:

@Database(
    entities = [Repo::class],
    version = 1,
    exportSchema = false
)
abstract class RepoDatabase : RoomDatabase() {

    abstract fun reposDao(): RepoDao

    companion object {

        @Volatile
        private var INSTANCE: RepoDatabase? = null

        fun getInstance(context: Context): RepoDatabase =
                INSTANCE ?: synchronized(this) {
                    INSTANCE
                            ?: buildDatabase(context).also { INSTANCE = it }
                }

        private fun buildDatabase(context: Context) =
                Room.databaseBuilder(context.applicationContext,
                        RepoDatabase::class.java, "Github.db")
                        .build()
    }
}

Now that we've set up our database, let's see how we request data from the network and save it in the database.

The Paging library uses the database as a source of truth for the data that needs to be displayed in the UI. Whenever we don't have any more data in the database, we need to request more from the network. To help with this, Paging 3.0 defines the RemoteMediator abstract class, with one method that needs to be implemented: load(). This method will be called whenever we need to load more data from the network. This class returns a MediatorResult object, that can either be:

• Error - if we got an error while requesting data from the network.
• Success - If we successfully got data from the network. Here, we also need to pass in a signal that tells whether more data can be loaded or not. For example, if the network response was successful but we got an empty list of repositories, it means that there is no more data to be loaded.

In the data package, let's create a new class called GithubRemoteMediator that extends RemoteMediator. This class will be recreated for every new query, so it will receive the following as parameters:

• The query String.
• The GithubService - so we can make network requests.
• The RepoDatabase - so we can save data we got from the network request.

@OptIn(ExperimentalPagingApi::class)
class GithubRemoteMediator(
    private val query: String,
    private val service: GithubService,
    private val repoDatabase: RepoDatabase
) : RemoteMediator<Int, Repo>() {

    override suspend fun load(loadType: LoadType, state: PagingState<Int, Repo>): MediatorResult {

   }
}

To be able to build the network request, the load method has 2 parameters that should give us all the information we need:

• PagingState - this gives us information about the pages that were loaded before, the most recently accessed index in the list, and the PagingConfig we defined when initializing the paging stream.
• LoadType - this tells us whether we need to load data at the end (LoadType.APPEND) or at the beginning of the data (LoadType.PREPEND) that we previously loaded, or if this the first time we're loading data (LoadType.REFRESH).

For example, if the load type is LoadType.APPEND then we retrieve the last item that was loaded from the PagingState. Based on that we should be able to find out how to load the next batch of Repo objects, by computing the next page to be loaded.

In the next section you'll find out how to compute keys for the next and previous pages to be loaded.

For the purposes of the Github API, the page key that we use to request pages of repos is just a page index that is incremented when getting the next page. This means that given a Repo object, the next batch of Repo objects can be requested based on page index + 1. The previous batch of Repo objects can be requested based on page index - 1. All Repo objects received on a certain page response will have the same next and previous keys.

When we get the last item loaded from the PagingState, there's no way to know the index of the page it belonged to. To solve this problem, we can add another table that stores the next and previous page keys for each Repo; we can call it remote_keys. While this can be done in the Repo table, creating a new table for the next and previous remote keys associated with a Repo allows us to have a better separation of concerns.

In the db package, let's create a new data class called RemoteKeys, annotate it with @Entity, and add 3 properties: the repo id (which is also the primary key), and the previous and next keys (which can be null when we can't append or prepend data).

@Entity(tableName = "remote_keys")
data class RemoteKeys(
    @PrimaryKey
    val repoId: Long,
    val prevKey: Int?,
    val nextKey: Int?
)

Let's create a RemoteKeysDao interface. We will need the following capabilities:

• Insert a list of RemoteKeys, as whenever we get Repos from the network we will generate the remote keys for them.
• Get a RemoteKey based on a Repo id.
• Clear the RemoteKeys, which we will use whenever we have a new query.

@Dao
interface RemoteKeysDao {

    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertAll(remoteKey: List<RemoteKeys>)

    @Query("SELECT * FROM remote_keys WHERE repoId = :repoId")
    suspend fun remoteKeysRepoId(repoId: Long): RemoteKeys?

    @Query("DELETE FROM remote_keys")
    suspend fun clearRemoteKeys()
}

Let's add the RemoteKeys table to our database and provide access to the RemoteKeysDao. To do this, update the RepoDatabase as follows:

• Add RemoteKeys to the list of entities.
• Expose the RemoteKeysDao as an abstract function.

@Database(
        entities = [Repo::class, RemoteKeys::class],
        version = 1,
        exportSchema = false
)
abstract class RepoDatabase : RoomDatabase() {

    abstract fun reposDao(): RepoDao
    abstract fun remoteKeysDao(): RemoteKeysDao

    ... 
    // rest of the class doesn't change
}

Now that we saved the remote keys, let's get back to GithubRemoteMediator and see how to use them. This class will replace our GithubPagingSource. Let's copy the GITHUB_STARTING_PAGE_INDEX declaration from GithubPagingSource in our GithubRemoteMediator and delete the GithubPagingSource class.

Let's see how we can implement the GithubRemoteMediator.load() method:

1. Find out what page we need to load from the network, based on the LoadType.
2. Trigger the network request.
3. Once the network request completes, if the received list of repositories is not empty, then do the following:
4. We compute the RemoteKeys for every Repo.
5. If this a new query (loadType = REFRESH) then we clear the database.
6. Save the RemoteKeys and Repos in the database.
7. Return MediatorResult.Success(endOfPaginationReached = false).
8. If the list of repos was empty then we return MediatorResult.Success(endOfPaginationReached = true). If we get an error requesting data we return MediatorResult.Error.

Here's how the code looks like overall. We'll replace the TODOs later on.

override suspend fun load(loadType: LoadType, state: PagingState<Int, Repo>): MediatorResult {
    val page = when (loadType) {
        LoadType.REFRESH -> {
         // TODO
        }
        LoadType.PREPEND -> {
        // TODO
        }
        LoadType.APPEND -> {
        // TODO
        }
    }
    val apiQuery = query + IN_QUALIFIER

    try {
        val apiResponse = service.searchRepos(apiQuery, page, state.config.pageSize)

        val repos = apiResponse.items
        val endOfPaginationReached = repos.isEmpty()
        repoDatabase.withTransaction {
            // clear all tables in the database
            if (loadType == LoadType.REFRESH) {
                repoDatabase.remoteKeysDao().clearRemoteKeys()
                repoDatabase.reposDao().clearRepos()
            }
            val prevKey = if (page == GITHUB_STARTING_PAGE_INDEX) null else page - 1
            val nextKey = if (endOfPaginationReached) null else page + 1
            val keys = repos.map {
                RemoteKeys(repoId = it.id, prevKey = prevKey, nextKey = nextKey)
            }
            repoDatabase.remoteKeysDao().insertAll(keys)
            repoDatabase.reposDao().insertAll(repos)
        }
        return MediatorResult.Success(endOfPaginationReached = endOfPaginationReached)
    } catch (exception: IOException) {
        return MediatorResult.Error(exception)
    } catch (exception: HttpException) {
        return MediatorResult.Error(exception)
    }
}

Let's see how we find the page to load based on the LoadType.

Now that we know what happens in the GithubRemoteMediator.load() method once we have the page key, let's see how we compute it. This will depend on the LoadType.

LoadType.APPEND

When we need to load data at the end of the currently loaded data set, the load parameter is LoadType.APPEND. So now, based on the last item in the database we need to compute the network page key.

1. We need to get the remote key of the last Repo item loaded from the database—let's separate this in a function:

    private suspend fun getRemoteKeyForLastItem(state: PagingState<Int, Repo>): RemoteKeys? {
        // Get the last page that was retrieved, that contained items.
        // From that last page, get the last item
        return state.pages.lastOrNull() { it.data.isNotEmpty() }?.data?.lastOrNull()
                ?.let { repo ->
                    // Get the remote keys of the last item retrieved
repoDatabase.remoteKeysDao().remoteKeysRepoId(repo.id)
                }
    }

2. If remoteKeys is null, that means the refresh result is not in the database yet. We can return Success with endOfPaginationReached = false because Paging will call this method again if RemoteKeys becomes non-null. If remoteKeys is not null but its prevKey is null, that means we've reached the end of pagination for append.

val page = when (loadType) {
    LoadType.APPEND -> {
        val remoteKeys = getRemoteKeyForLastItem(state)
        // If remoteKeys is null, that means the refresh result is not in the database yet.
        // We can return Success with endOfPaginationReached = false because Paging
        // will call this method again if RemoteKeys becomes non-null.
        // If remoteKeys is NOT NULL but its prevKey is null, that means we've reached
        // the end of pagination for append.
        val nextKey = remoteKeys?.nextKey
        if (nextKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        nextKey
    }
      ...
  }

LoadType.PREPEND

When we need to load data at the beginning of the currently loaded data set, the load parameter is LoadType.PREPEND. Based on the first item in the database we need to compute the network page key.

1. We need to get the remote key of the first Repo item loaded from the database—let's separate this in a function:

private suspend fun getRemoteKeyForFirstItem(state: PagingState<Int, Repo>): RemoteKeys? {
    // Get the first page that was retrieved, that contained items.
    // From that first page, get the first item
    return state.pages.firstOrNull { it.data.isNotEmpty() }?.data?.firstOrNull()
            ?.let { repo ->
                // Get the remote keys of the first items retrieved
                repoDatabase.remoteKeysDao().remoteKeysRepoId(repo.id)
            }
}

2. If remoteKeys is null, that means the refresh result is not in the database yet. We can return Success with endOfPaginationReached = false because Paging will call this method again if RemoteKeys becomes non-null. If remoteKeys is not null but its prevKey is null, that means we've reached the end of pagination for append.

val page = when (loadType) {
    LoadType.PREPEND -> {
        val remoteKeys = getRemoteKeyForFirstItem(state)
        // If remoteKeys is null, that means the refresh result is not in the database yet.
        // We can return Success with `endOfPaginationReached = false` because Paging
        // will call this method again if RemoteKeys becomes non-null.
        // If remoteKeys is NOT NULL but its prevKey is null, that means we've reached
        // the end of pagination for prepend.
        val prevKey = remoteKeys?.prevKey
        if (prevKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        prevKey
    }

      ...
  }

LoadType.REFRESH

LoadType.REFRESH gets called when it's the first time we're loading data, or when PagingDataAdapter.refresh() is called; so now the point of reference for loading our data is the state.anchorPosition. If this is the first load, then the anchorPosition is null. When PagingDataAdapter.refresh() is called, the anchorPosition is the first visible position in the displayed list, so we will need to load the page that contains that specific item.

1. Based on the anchorPosition from the state, we can get the closest Repo item to that position by calling state.closestItemToPosition().
2. Based on the Repo item, we can get the RemoteKeys from the database.

private suspend fun getRemoteKeyClosestToCurrentPosition(
        state: PagingState<Int, Repo>
): RemoteKeys? {
    // The paging library is trying to load data after the anchor position
    // Get the item closest to the anchor position
    return state.anchorPosition?.let { position ->
        state.closestItemToPosition(position)?.id?.let { repoId -> 
   repoDatabase.remoteKeysDao().remoteKeysRepoId(repoId)
        }
    }
}

3. If remoteKey is not null, then we can get the nextKey from it. In the Github API the page keys are incremented sequentially. So to get the page that contains the current item, we just subtract 1 from remoteKey.nextKey.
4. If RemoteKey is null (because the anchorPosition was null), then the page we need to load is the initial one: GITHUB_STARTING_PAGE_INDEX

Now, the full page computation looks like this:

val page = when (loadType) {
    LoadType.REFRESH -> {
        val remoteKeys = getRemoteKeyClosestToCurrentPosition(state)
        remoteKeys?.nextKey?.minus(1) ?: GITHUB_STARTING_PAGE_INDEX
    }
    LoadType.PREPEND -> {
        val remoteKeys = getRemoteKeyForFirstItem(state)
        val prevKey = remoteKeys?.prevKey
        if (prevKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        prevKey
    }
    LoadType.APPEND -> {
        val remoteKeys = getRemoteKeyForLastItem(state)
        val nextKey = remoteKeys?.nextKey
        if (nextKey == null) {
            return MediatorResult.Success(endOfPaginationReached = remoteKeys != null)
        }
        nextKey
    }
}

Now that we have the GithubRemoteMediator and the PagingSource in our ReposDao implemented, we need to update GithubRepository.getSearchResultStream to use them.

In order to do this, GithubRepository needs access to the database. Let's pass the database as a parameter in the constructor. Also, since this class will use GithubRemoteMediator:

class GithubRepository(
        private val service: GithubService,
        private val database: RepoDatabase
) { ... }

Update the Injection file:

• provideGithubRepository method should get a context as a parameter and in the GithubRepository constructor invoke RepoDatabase.getInstance.
• provideViewModelFactory method should get a context as a parameter and pass it to provideGithubRepository.

object Injection {

    private fun provideGithubRepository(context: Context): GithubRepository {
        return GithubRepository(GithubService.create(), RepoDatabase.getInstance(context))
    }

    fun provideViewModelFactory(context: Context): ViewModelProvider.Factory {
        return ViewModelFactory(provideGithubRepository(context))
    }
}

Update the SearchRepositoriesActivity.onCreate() method and pass the context to Injection.provideViewModelFactory():

// get the view model
viewModel = ViewModelProvider(this, Injection.provideViewModelFactory(this))
        .get(SearchRepositoriesViewModel::class.java)

Let's get back to GithubRepository. First, to be able to search for repos by name, we'll have to add % to the beginning and end of the query string. Then, when calling the reposDao.reposByName, we get a PagingSource. Because the PagingSource is invalidated every time we make a change in the database, we need to tell Paging how to get a new instance of the PagingSource. To do this, we just create a function that calls the database query:

// appending '%' so we can allow other characters to be before and after the query string
val dbQuery = "%${query.replace(' ', '%')}%"
val pagingSourceFactory =  { database.reposDao().reposByName(dbQuery)}

Now we can change the Pager builder, to use a GithubRemoteMediator and the pagingSourceFactory. The Pager is an experimental API so we'll have to annotate it with @OptIn:

@OptIn(ExperimentalPagingApi::class)
return Pager(
        config = PagingConfig(
            pageSize = NETWORK_PAGE_SIZE,
            enablePlaceholders = false
         ),
        remoteMediator = GithubRemoteMediator(
                query,
                service,
                database
        ),
        pagingSourceFactory = pagingSourceFactory
).flow

That's it! Let's run the app!

Updating the load state source

Right now our app loads data from the network and saves it in the database but, when it comes to displaying a loading spinner during the initial page load (in SearchRepositoriesActivity.initAdapter) , the app still relies on LoadState.source. What we want right now is to show a loading spinner only for loads from RemoteMediator. For this, we need to change from LoadState.source to LoadState.mediator:

private fun initAdapter() {
         ...
        adapter.addLoadStateListener { loadState ->
            // Only show the list if refresh succeeds.
            binding.list.isVisible = loadState.mediator?.refresh is LoadState.NotLoading
            // Show loading spinner during initial load or refresh.
            binding.progressBar.isVisible = loadState.mediator?.refresh is LoadState.Loading
            // Show the retry state if initial load or refresh fails.
            binding.retryButton.isVisible = loadState.mediator?.refresh is LoadState.Error
            ... // everything else stays the same
    }

You can find the full code for the steps done so far in branch step13-19_network_and_database.

Now that we have added all the components, let's recap what we've learned!

• The PagingSource asynchronously loads the data from a source you define.
• The Pager.flow creates a Flow<PagingData> based on a configuration and a function that defines how to instantiate the PagingSource.
• The Flow emits a new PagingData whenever new data is loaded by the PagingSource.
• The UI observes the changed PagingData and uses a PagingDataAdapter to update the RecyclerView that presents the data.
• To retry a failed load from the UI, use the PagingDataAdapter.retry method. Under the hood, the Paging library will trigger the PagingSource.load() method.
• To add separators to your list, create a high-level type with separators as one of the supported types. Then use the PagingData.insertSeparators() method to implement your separator generation logic.
• To display the load state as header or footer, use PagingDataAdapter.withLoadStateHeaderAndFooter() method and implement a LoadStateAdapter. If you want to execute other actions based on the load state, use the PagingDataAdapter.addLoadStateListener() callback.
• To work with the network and database, implement a RemoteMediator.