Working with Preferences DataStore

1. Introduction

What is DataStore?

DataStore is a new and improved data storage solution aimed at replacing SharedPreferences. Built on Kotlin coroutines and Flow, DataStore provides two different implementations: Proto DataStore, that stores typed objects (backed by protocol buffers) and Preferences DataStore, that stores key-value pairs. Data is stored asynchronously, consistently, and transactionally, overcoming some of the drawbacks of SharedPreferences.

What you'll learn

• What DataStore is and why you should use it.
• How to add DataStore to your project.
• The differences between Preferences and Proto DataStore and the advantages of each.
• How to use Preferences DataStore.
• How to migrate from SharedPreferences to Preferences DataStore.

What you will build

In this codelab, you're going to start with a sample app that displays a list of tasks that can be filtered by their completed status and can be sorted by priority and deadline.

The boolean flag for the Show completed tasks filter is saved in memory. The sort order is persisted to disk using a SharedPreferences object.

In this codelab, you will learn how to use Preferences DataStore by completing the following tasks:

• Persist the completed status filter in DataStore.
• Migrate the sort order from SharedPreferences to DataStore.

We recommend working through the Proto DataStore codelab too, so you better understand the difference between the two.

What you'll need

• Android Studio Arctic Fox.
• Familiarity with the following Architecture Components: LiveData, ViewModel, View Binding and with the architecture suggested in the Guide to app architecture.
• Familiarity with coroutines and Kotlin Flow.

For an introduction to Architecture Components, check out the Room with a View codelab. For an introduction to Flow, check out the Advanced Coroutines with Kotlin Flow and LiveData codelab.

2. Getting set up

In this step, you will download the code for the entire codelab and then run a simple example app.

To get you started as quickly as possible, we have prepared a starter project for you to build on.

If you have git installed, you can simply run the command below. To check whether git is installed, type git --version in the terminal or command line and verify that it executes correctly.

 git clone https://github.com/android/codelab-android-datastore

The initial state is in the master branch. The solution code is located in the preferences_datastore branch.

If you do not have git, you can click the following button to download all of the code for this codelab:

Download source code

1. Unzip the code, and then open the project in Android Studio Arctic Fox.
2. Run the app run configuration on a device or emulator.

The app runs and displays the list of tasks:

3. Project overview

The app allows you to see a list of tasks. Each task has the following properties: name, completed status, priority, and deadline.

To simplify the code we need to work with, the app allows you to do only two actions:

• Toggle Show completed tasks visibility - by default the tasks are hidden
• Sort the tasks by priority, by deadline or by deadline and priority

The app follows the architecture recommended in the Guide to app architecture. Here's what you will find in each package:

data

• The Task model class.
• TasksRepository class - responsible for providing the tasks. For simplicity, it returns hardcoded data and exposes it via a Flow to represent a more realistic scenario.
• UserPreferencesRepository class - holds the SortOrder, defined as an enum. The current sort order is saved in SharedPreferences as a String, based on the enum value name. It exposes synchronous methods to save and get the sort order.

ui

• Classes related to displaying an Activity with a RecyclerView.
• The TasksViewModel class is responsible for the UI logic.

TasksViewModel - holds all the elements necessary to build the data that needs to be displayed in the UI: the list of tasks, the showCompleted and sortOrder flags, wrapped in a TasksUiModel object. Every time one of these values changes, we have to reconstruct a new TasksUiModel. To do this, we combine 3 elements:

• A Flow<List<Task>> is retrieved from the TasksRepository.
• A MutableStateFlow<Boolean> holding the latest showCompleted flag which is only kept in memory.
• A MutableStateFlow<SortOrder> holding the latest sortOrder value.

To ensure that we're updating the UI correctly, only when the Activity is started, we expose a LiveData<TasksUiModel>.

We have a couple of problems with our code:

• We block the UI thread on disk IO when initializing UserPreferencesRepository.sortOrder. This can result in UI jank.
• The showCompleted flag is only kept in memory, so this means it will be reset every time the user opens the app. Like the sortOrder, this should be persisted to survive closing the app.
• We're currently using SharedPreferences to persist data but we keep a MutableStateFlow in memory, that we modify manually, to be able to be notified of changes. This breaks easily if the value is modified somewhere else in the application.
• In UserPreferencesRepository we expose two methods for updating the sort order: enableSortByDeadline() and enableSortByPriority(). Both of these methods rely on the current sort order value but, if one is called before the other has finished, we would end up with the wrong final value. Even more, these methods can result in UI jank and Strict Mode violations as they're called on the UI thread.

Let's find out how to use DataStore to help us with these issues.

4. DataStore - the basics

Often you might find yourself needing to store small or simple data sets. For this, in the past, you might have used SharedPreferences but this API also has a series of drawbacks. Jetpack DataStore library aims at addressing those issues, creating a simple, safer and asynchronous API for storing data. It provides 2 different implementations:

• Preferences DataStore
• Proto DataStore

1 SharedPreferences has a synchronous API that can appear safe to call on the UI thread, but which actually does disk I/O operations. Furthermore, apply() blocks the UI thread on fsync(). Pending fsync() calls are triggered every time any service starts or stops, and every time an activity starts or stops anywhere in your application. The UI thread is blocked on pending fsync() calls scheduled by apply(), often becoming a source of ANRs.

2 SharedPreferences throws parsing errors as runtime exceptions.

Preferences vs Proto DataStore

While both Preferences and Proto DataStore allow saving data, they do this in different ways:

• Preference DataStore, like SharedPreferences, accesses data based on keys, without defining a schema upfront.
• Proto DataStore defines the schema using Protocol buffers. Using Protobufs allows persisting strongly typed data. They are faster, smaller, simpler, and less ambiguous than XML and other similar data formats. While Proto DataStore requires you to learn a new serialization mechanism, we believe that the strongly typed advantage brought by Proto DataStore is worth it.

Room vs DataStore

If you have a need for partial updates, referential integrity, or large/complex datasets, you should consider using Room instead of DataStore. DataStore is ideal for small or simple datasets and does not support partial updates or referential integrity.

5. Preferences DataStore overview

Preferences DataStore API is similar to SharedPreferences with several notable differences:

• Handles data updates transactionally
• Exposes a Flow representing the current state of data
• Does not have data persistent methods (apply(), commit())
• Does not return mutable references to its internal state
• Exposes an API similar to Map and MutableMap with typed keys

Let's see how to add it to the project and migrate SharedPreferences to DataStore.

Adding dependencies

Update the build.gradle file to add the following the Preference DataStore dependency:

implementation "androidx.datastore:datastore-preferences:1.0.0"

6. Persisting data in Preferences DataStore

Although both the showCompleted and sortOrder flags are user preferences, currently they're represented as two different objects. So one of our goals is to unify these two flags under a UserPreferences class and store it in UserPreferencesRepository using DataStore. Right now, the showCompleted flag is kept in memory, in TasksViewModel.

Let's start by creating a UserPreferences data class in UserPreferencesRepository. For now, it should just have one field: showCompleted. We'll add the sort order later.

data class UserPreferences(val showCompleted: Boolean)

Creating the DataStore

To create a DataStore instance we use the preferencesDataStore delegate, with the Context as receiver. For simplicity, in this codelab, let's do this in TasksActivity:

private const val USER_PREFERENCES_NAME = "user_preferences"

private val Context.dataStore by preferencesDataStore(
    name = USER_PREFERENCES_NAME
)

The preferencesDataStore delegate ensures that we have a single instance of DataStore with that name in our application. Currently, UserPreferencesRepository is implemented as a singleton, because it holds the sortOrderFlow and avoids having it tied to the lifecycle of the TasksActivity. Because UserPreferenceRepository will just work with the data from DataStore and it won't create and hold any new objects, we can already remove the singleton implementation:

• Remove the companion object
• Make the constructor public

The UserPreferencesRepository should get a DataStore instance as a constructor parameter. For now, we can leave the Context as a parameter as it's needed by SharedPreferences, but we'll remove it later on.

class UserPreferencesRepository(
    private val dataStore: DataStore<Preferences>,
    context: Context
) { ... }

Let's update the construction of UserPreferencesRepository in TasksActivity and pass in the dataStore:

viewModel = ViewModelProvider(
    this,
    TasksViewModelFactory(
        TasksRepository,
        UserPreferencesRepository(dataStore, this)
    )
).get(TasksViewModel::class.java)

Reading data from Preferences DataStore

Preferences DataStore exposes the data stored in a Flow<Preferences> that will emit every time a preference has changed. We don't want to expose the entire Preferences object but rather the UserPreferences object. To do this, we'll have to map the Flow<Preferences>, get the Boolean value we're interested in, based on a key and construct a UserPreferences object.

So, the first thing we need to do is define the show_completed key - this is a booleanPreferencesKey value that we can declare as a member in a private PreferencesKeys object.

private object PreferencesKeys {
  val SHOW_COMPLETED = booleanPreferencesKey("show_completed")
}

Let's expose a userPreferencesFlow: Flow<UserPreferences>, constructed based on dataStore.data: Flow<Preferences>, which is then mapped, to retrieve the right preference:

val userPreferencesFlow: Flow<UserPreferences> = dataStore.data
    .map { preferences ->
        // Get our show completed value, defaulting to false if not set:
        val showCompleted = preferences[PreferencesKeys.SHOW_COMPLETED]?: false
        UserPreferences(showCompleted)
    }

Handling exceptions while reading data

As DataStore reads data from a file, IOExceptions are thrown when an error occurs while reading data. We can handle these by using the catch() Flow operator before map() and emitting emptyPreferences() in case the exception thrown was an IOException. If a different type of exception was thrown, prefer re-throwing it.

val userPreferencesFlow: Flow<UserPreferences> = dataStore.data
    .catch { exception ->
        // dataStore.data throws an IOException when an error is encountered when reading data
        if (exception is IOException) {
            emit(emptyPreferences())
        } else {
            throw exception
        }
    }.map { preferences ->
        // Get our show completed value, defaulting to false if not set:
        val showCompleted = preferences[PreferencesKeys.SHOW_COMPLETED]?: false
        UserPreferences(showCompleted)
    }

Writing data to Preferences DataStore

To write data, DataStore offers a suspending DataStore.edit(transform: suspend (MutablePreferences) -> Unit) function, which accepts a transform block that allows us to transactionally update the state in DataStore.

The MutablePreferences passed to the transform block will be up-to-date with any previously run edits. All changes to MutablePreferences in the transform block will be applied to disk after transform completes and before edit completes. Setting one value in MutablePreferences will leave all other preferences unchanged.

Note: do not attempt to modify the MutablePreferences outside of the transform block.

Let's create a suspend function that allows us to update the showCompleted property of UserPreferences, called updateShowCompleted(), that calls dataStore.edit() and sets the new value:

suspend fun updateShowCompleted(showCompleted: Boolean) {
    dataStore.edit { preferences ->
        preferences[PreferencesKeys.SHOW_COMPLETED] = showCompleted
    }
}

edit() can throw an IOException if an error was encountered while reading or writing to disk. If any other error happens in the transform block, it will be thrown by edit().

At this point, the app should compile but the functionality we just created in UserPreferencesRepository is not used.

7. SharedPreferences to Preferences DataStore

The sort order is saved in SharedPreferences. Let's move it to DataStore. To do this, let's start by updating UserPreferences to also store the sort order:

data class UserPreferences(
    val showCompleted: Boolean,
    val sortOrder: SortOrder
)

Migrating from SharedPreferences

To be able to migrate it to DataStore, we need to update the dataStore builder to pass in a SharedPreferencesMigration to the list of migrations. DataStore will be able to migrate from SharedPreferences to DataStore automatically, for us. Migrations are run before any data access can occur in DataStore. This means that your migration must have succeeded before DataStore.data emits any values and before DataStore.edit() can update data.

Note: keys are only migrated from SharedPreferences once, so you should discontinue using the old SharedPreferences once the code is migrated to DataStore.

First, let's update the DataStore creation in TasksActivity:

private const val USER_PREFERENCES_NAME = "user_preferences"

private val Context.dataStore by preferencesDataStore(
    name = USER_PREFERENCES_NAME,
    produceMigrations = { context ->
        // Since we're migrating from SharedPreferences, add a migration based on the
        // SharedPreferences name
        listOf(SharedPreferencesMigration(context, USER_PREFERENCES_NAME))
    }
)

Then add the sort_order to our PreferencesKeys:

private object PreferencesKeys {
    ...
    // Note: this has the same name that we used with SharedPreferences.
    val SORT_ORDER = stringPreferencesKey("sort_order")
}

All keys will be migrated to our DataStore and deleted from the user preferences SharedPreferences. Now, from Preferences we will be able to get and update the SortOrder based on the SORT_ORDER key.

Reading the sort order from DataStore

Let's update the userPreferencesFlow to also retrieve the sort order in the map() transformation:

val userPreferencesFlow: Flow<UserPreferences> = dataStore.data
    .catch { exception ->
        if (exception is IOException) {
            emit(emptyPreferences())
        } else {
            throw exception
        }
    }.map { preferences ->
        // Get the sort order from preferences and convert it to a [SortOrder] object
        val sortOrder =
            SortOrder.valueOf(
                preferences[PreferencesKeys.SORT_ORDER] ?: SortOrder.NONE.name)

        // Get our show completed value, defaulting to false if not set:
        val showCompleted = preferences[PreferencesKeys.SHOW_COMPLETED] ?: false
        UserPreferences(showCompleted, sortOrder)
    }

Saving the sort order to DataStore

Currently UserPreferencesRepository only exposes a synchronous way to set the sort order flag and it has a concurrency problem. We expose two methods for updating the sort order: enableSortByDeadline() and enableSortByPriority(); both of these methods rely on the current sort order value but, if one is called before the other has finished, we would end up with the wrong final value.

As DataStore guarantees that data updates happen transactionally, we won't have this problem anymore. Let's do the following changes:

• Update enableSortByDeadline() and enableSortByPriority() to be suspend functions that use the dataStore.edit().
• In the transform block of edit(), we'll get the currentOrder from the Preferences parameter, instead of retrieving it from the _sortOrderFlow field.
• Instead of calling updateSortOrder(newSortOrder) we can directly update the sort order in the preferences.

Here's what the implementation looks like.

suspend fun enableSortByDeadline(enable: Boolean) {
    // edit handles data transactionally, ensuring that if the sort is updated at the same
    // time from another thread, we won't have conflicts
    dataStore.edit { preferences ->
        // Get the current SortOrder as an enum
        val currentOrder = SortOrder.valueOf(
            preferences[PreferencesKeys.SORT_ORDER] ?: SortOrder.NONE.name
        )

        val newSortOrder =
            if (enable) {
                if (currentOrder == SortOrder.BY_PRIORITY) {
                    SortOrder.BY_DEADLINE_AND_PRIORITY
                } else {
                    SortOrder.BY_DEADLINE
                }
            } else {
                if (currentOrder == SortOrder.BY_DEADLINE_AND_PRIORITY) {
                    SortOrder.BY_PRIORITY
                } else {
                    SortOrder.NONE
                }
            }
        preferences[PreferencesKeys.SORT_ORDER] = newSortOrder.name
    }
}

suspend fun enableSortByPriority(enable: Boolean) {
    // edit handles data transactionally, ensuring that if the sort is updated at the same
    // time from another thread, we won't have conflicts
    dataStore.edit { preferences ->
        // Get the current SortOrder as an enum
        val currentOrder = SortOrder.valueOf(
            preferences[PreferencesKeys.SORT_ORDER] ?: SortOrder.NONE.name
        )

        val newSortOrder =
            if (enable) {
                if (currentOrder == SortOrder.BY_DEADLINE) {
                    SortOrder.BY_DEADLINE_AND_PRIORITY
                } else {
                    SortOrder.BY_PRIORITY
                }
            } else {
                if (currentOrder == SortOrder.BY_DEADLINE_AND_PRIORITY) {
                    SortOrder.BY_DEADLINE
                } else {
                    SortOrder.NONE
                }
            }
        preferences[PreferencesKeys.SORT_ORDER] = newSortOrder.name
    }
}

Now you can remove the context constructor parameter and all the usages of SharedPreferences.

8. Update TasksViewModel to use UserPreferencesRepository

Now that UserPreferencesRepository stores both show_completed and sort_order flags in DataStore and exposes a Flow<UserPreferences>, let's update the TasksViewModel to use them.

Remove showCompletedFlow and sortOrderFlow and instead, create a value called userPreferencesFlow that gets initialised with userPreferencesRepository.userPreferencesFlow:

private val userPreferencesFlow = userPreferencesRepository.userPreferencesFlow

In the tasksUiModelFlow creation, replace showCompletedFlow and sortOrderFlow with userPreferencesFlow. Replace the parameters accordingly.

When calling filterSortTasks pass in the showCompleted and sortOrder of the userPreferences. Your code should look like this:

private val tasksUiModelFlow = combine(
        repository.tasks,
        userPreferencesFlow
    ) { tasks: List<Task>, userPreferences: UserPreferences ->
        return@combine TasksUiModel(
            tasks = filterSortTasks(
                tasks,
                userPreferences.showCompleted,
                userPreferences.sortOrder
            ),
            showCompleted = userPreferences.showCompleted,
            sortOrder = userPreferences.sortOrder
        )
    }

The showCompletedTasks() function should now be updated to call userPreferencesRepository.updateShowCompleted(). As this is a suspend function, create a new coroutine in the viewModelScope:

fun showCompletedTasks(show: Boolean) {
    viewModelScope.launch {
        userPreferencesRepository.updateShowCompleted(show)
    }
}

userPreferencesRepository functions enableSortByDeadline() and enableSortByPriority() are now suspend functions so they should also be called in a new coroutine, launched in the viewModelScope:

fun enableSortByDeadline(enable: Boolean) {
    viewModelScope.launch {
       userPreferencesRepository.enableSortByDeadline(enable)
    }
}

fun enableSortByPriority(enable: Boolean) {
    viewModelScope.launch {
        userPreferencesRepository.enableSortByPriority(enable)
    }
}

Clean up UserPreferencesRepository

Let's remove the fields and methods that are no longer needed. You should be able to delete the following:

• _sortOrderFlow
• sortOrderFlow
• updateSortOrder()
• private val sortOrder: SortOrder

Our app should now compile successfully. Let's run it to see if the show_completed and sort_order flags are saved correctly.

Check out the preferences_datastore branch of the codelab repo to compare your changes.

9. Wrap up

Now that you migrated to Preferences DataStore let's recap what we've learned:

• SharedPreferences comes with a series of drawbacks - from synchronous API that can appear safe to call on the UI thread, no mechanism of signaling errors, lack of transactional API and more.
• DataStore is a replacement for SharedPreferences addressing most of the shortcomings of the API.
• DataStore has a fully asynchronous API using Kotlin coroutines and Flow, handles data migration, guarantees data consistency and handles data corruption.