Transform data streams

When you work with paged data, you often need to transform the data stream as you load it. For example, you might need to filter a list of items, or convert items to a different type before you present them in the UI. Another common use case for data stream transformation is adding list separators.

More generally, applying transformations directly to the data stream allows you to keep your repository constructs and UI constructs separate.

This page assumes that you are familiar with basic use of the Paging library.

Apply basic transformations

Because PagingData is encapsulated in a reactive stream, you can apply transform operations on the data incrementally between loading the data and presenting it.

In order to apply transformations to each PagingData object in the stream, place the transformations inside a map() operation on the stream:

Kotlin

pager.flow // Type is Flow<PagingData<User>>.
  // Map the outer stream so that the transformations are applied to
  // each new generation of PagingData.
  .map { pagingData ->
    // Transformations in this block are applied to the items
    // in the paged data.
}

Java

PagingRx.getFlowable(pager) // Type is Flowable<PagingData<User>>.
  // Map the outer stream so that the transformations are applied to
  // each new generation of PagingData.
  .map(pagingData -> {
    // Transformations in this block are applied to the items
    // in the paged data.
  });

Java

// Map the outer stream so that the transformations are applied to
// each new generation of PagingData.
Transformations.map(
  // Type is LiveData<PagingData<User>>.
  PagingLiveData.getLiveData(pager),
  pagingData -> {
    // Transformations in this block are applied to the items
    // in the paged data.
  });

The example below demonstrates this pattern, applying the following transformations to the paged data:

The outer map() operation on the data stream applies all of the operations inside its block to each subsequent generation of the PagingData object in the stream.
A filter operation discards any list items that are not visible in the UI.
Another map operation converts each User object from the list into the UiModel type.

Kotlin

pager.flow // Type is Flow<PagingData<User>>.
    .map { pagingData ->
        pagingData.filter { user -> !user.hiddenFromUi }
            .map { user -> UiModel(user) }
    }
    .cachedIn(viewModelScope)

Java

// CoroutineScope helper provided by the lifecycle-viewmodel-ktx artifact.
CoroutineScope viewModelScope = ViewModelKt.getViewModelScope(viewModel);
PagingRx.cachedIn(
  // Type is Flowable<PagingData<User>>.
  PagingRx.getFlowable(pager)
    .map(pagingData -> pagingData
      .filter(user -> !user.isHiddenFromUi())
      .map(UiModel.UserModel::new)),
  viewModelScope);
}

Java

// CoroutineScope helper provided by the lifecycle-viewmodel-ktx artifact.
CoroutineScope viewModelScope = ViewModelKt.getViewModelScope(viewModel);
PagingLiveData.cachedIn(
  Transformations.map(
    // Type is LiveData<PagingData<User>>.
    PagingLiveData.getLiveData(pager),
    pagingData -> pagingData
      .filter(user -> !user.isHiddenFromUi())
      .map(UiModel.UserModel::new)),
  viewModelScope);

The cachedIn() operation caches the results of any transformations that occur before it. Therefore, cachedIn() should be the last call in your ViewModel. For more information on using cachedIn() with a stream of PagingData, see Set up a stream of PagingData.

Add list separators

The Paging library supports dynamic list separators. You can improve list readability by inserting separators directly into the data stream as RecyclerView list items. As a result, separators are fully-featured ViewHolder objects, enabling interactivity, accessibility focus, and all of the other features provided by a View.

There are three steps involved in inserting separators into your paged list:

Convert the UI model to accommodate the separator items.
Transform the data stream to dynamically add the separators between loading the data and presenting the data.
Update the UI to handle separator items.

Convert the UI model

The Paging library inserts list separators into the RecyclerView as actual list items, but the separator items must be distinguishable from the other items in the list because the separator UI will most likely be different from the UI of the other list items. The solution is to create a Kotlin sealed class with subclasses to represent your data and your separators. Alternatively, you can create a base class that is extended by your list item class and your separator class.

Suppose that you want to add separators to a paged list of User items. The following snippet shows how to create a base class where the instances can be either a UserModel or a SeparatorModel:

Kotlin

sealed class UiModel {
  class UserModel(val id: String, val label: String) : UiModel() {
    constructor(user: User) : this(user.id, user.label)
  }

  class SeparatorModel(val description: String) : UiModel()
}

Java

class UiModel {
  private UiModel() {}

  static class UserModel extends UiModel {
    @NonNull
    private String mId;
    @NonNull
    private String mLabel;

    UserModel(@NonNull String id, @NonNull String label) {
      mId = id;
      mLabel = label;
    }

    UserModel(@NonNull User user) {
      mId = user.id;
      mLabel = user.label;
    }

    @NonNull
    public String getId() {
      return mId;
    }

    @NonNull
    public String getLabel() {
      return mLabel;
      }
    }

    static class SeparatorModel extends UiModel {
    @NonNull
    private String mDescription;

    SeparatorModel(@NonNull String description) {
      mDescription = description;
    }

    @NonNull
    public String getDescription() {
      return mDescription;
    }
  }
}

Java

class UiModel {
  private UiModel() {}

  static class UserModel extends UiModel {
    @NonNull
    private String mId;
    @NonNull
    private String mLabel;

    UserModel(@NonNull String id, @NonNull String label) {
      mId = id;
      mLabel = label;
    }

    UserModel(@NonNull User user) {
      mId = user.id;
      mLabel = user.label;
    }

    @NonNull
    public String getId() {
      return mId;
    }

    @NonNull
    public String getLabel() {
      return mLabel;
      }
    }

    static class SeparatorModel extends UiModel {
    @NonNull
    private String mDescription;

    SeparatorModel(@NonNull String description) {
      mDescription = description;
    }

    @NonNull
    public String getDescription() {
      return mDescription;
    }
  }
}

Transform the data stream

You must apply transformations to the data stream after loading it and before you present it. The transformations should do the following:

Convert the loaded list items to reflect the new base item type.
Use the PagingData.insertSeparators() method to add the separators.

To learn more about transformation operations, see Apply basic transformations.

The following example shows transformation operations to update the PagingData<User> stream to a PagingData<UiModel> stream with separators added:

Kotlin

pager.flow.map { pagingData: PagingData<User> ->
  // Map outer stream, so you can perform transformations on
  // each paging generation.
  pagingData
  .map { user ->
    // Convert items in stream to UiModel.UserModel.
    UiModel.UserModel(user)
  }
  .insertSeparators<UiModel.UserModel, UiModel> { before, after ->
    when {
      before == null -> UiModel.SeparatorModel("HEADER")
      after == null -> UiModel.SeparatorModel("FOOTER")
      shouldSeparate(before, after) -> UiModel.SeparatorModel(
        "BETWEEN ITEMS $before AND $after"
      )
      // Return null to avoid adding a separator between two items.
      else -> null
    }
  }
}

Java

// Map outer stream, so you can perform transformations on each
// paging generation.
PagingRx.getFlowable(pager).map(pagingData -> {
  // First convert items in stream to UiModel.UserModel.
  PagingData<UiModel> uiModelPagingData = pagingData.map(
    UiModel.UserModel::new);

  // Insert UiModel.SeparatorModel, which produces PagingData of
  // generic type UiModel.
  return PagingData.insertSeparators(uiModelPagingData,
    (@Nullable UiModel before, @Nullable UiModel after) -> {
      if (before == null) {
        return new UiModel.SeparatorModel("HEADER");
      } else if (after == null) {
        return new UiModel.SeparatorModel("FOOTER");
      } else if (shouldSeparate(before, after)) {
        return new UiModel.SeparatorModel("BETWEEN ITEMS "
          + before.toString() + " AND " + after.toString());
      } else {
        // Return null to avoid adding a separator between two
        // items.
        return null;
      }
    });
});

Java

// Map outer stream, so you can perform transformations on each
// paging generation.
Transformations.map(PagingLiveData.getLiveData(pager),
  pagingData -> {
    // First convert items in stream to UiModel.UserModel.
    PagingData<UiModel> uiModelPagingData = pagingData.map(
      UiModel.UserModel::new);

    // Insert UiModel.SeparatorModel, which produces PagingData of
    // generic type UiModel.
    return PagingData.insertSeparators(uiModelPagingData,
      (@Nullable UiModel before, @Nullable UiModel after) -> {
        if (before == null) {
          return new UiModel.SeparatorModel("HEADER");
        } else if (after == null) {
          return new UiModel.SeparatorModel("FOOTER");
        } else if (shouldSeparate(before, after)) {
          return new UiModel.SeparatorModel("BETWEEN ITEMS "
            + before.toString() + " AND " + after.toString());
        } else {
          // Return null to avoid adding a separator between two
          // items.
          return null;
        }
      });
  });

Handle separators in the UI

The final step is to change your UI to accommodate the separator item type. Create a layout and a view holder for your separator items and change the list adapter to use RecyclerView.ViewHolder as its view holder type so that it can handle more than one type of view holder. Alternatively, you can define a common base class that both your item and separator view holder classes extend.

You must also make the following changes to your list adapter:

Add cases to the onCreateViewHolder() and onBindViewHolder() methods to account for separator list items.
Implement a new comparator.

Kotlin

class UiModelAdapter :
  PagingDataAdapter<UiModel, RecyclerView.ViewHolder>(UiModelComparator) {

  override fun onCreateViewHolder(
    parent: ViewGroup,
    viewType: Int
  ) = when (viewType) {
    R.layout.item -> UserModelViewHolder(parent)
    else -> SeparatorModelViewHolder(parent)
  }

  override fun getItemViewType(position: Int) = when (getItem(position)) {
    is UiModel.UserModel -> R.layout.item
    is UiModel.SeparatorModel -> R.layout.separator_item
    null -> throw IllegalStateException("Unknown view")
  }

  override fun onBindViewHolder(
    holder: RecyclerView.ViewHolder,
    position: Int
  ) {
    val item = getItem(position)
    if (holder is UserModelViewHolder) {
      holder.bind(item as UserModel)
    } else if (holder is SeparatorModelViewHolder) {
      holder.bind(item as SeparatorModel)
    }
  }
}

object UiModelComparator : DiffUtil.ItemCallback<UiModel>() {
  override fun areItemsTheSame(
    oldItem: UiModel,
    newItem: UiModel
  ): Boolean {
    val isSameRepoItem = oldItem is UiModel.UserModel
      && newItem is UiModel.UserModel
      && oldItem.id == newItem.id

    val isSameSeparatorItem = oldItem is UiModel.SeparatorModel
      && newItem is UiModel.SeparatorModel
      && oldItem.description == newItem.description

    return isSameRepoItem || isSameSeparatorItem
  }

  override fun areContentsTheSame(
    oldItem: UiModel,
    newItem: UiModel
  ) = oldItem == newItem
}

Java

class UiModelAdapter extends PagingDataAdapter<UiModel, RecyclerView.ViewHolder> {
  UiModelAdapter() {
    super(new UiModelComparator(), Dispatchers.getMain(),
      Dispatchers.getDefault());
  }

  @NonNull
  @Override
  public RecyclerView.ViewHolder onCreateViewHolder(@NonNull ViewGroup parent,
    int viewType) {
    if (viewType == R.layout.item) {
      return new UserModelViewHolder(parent);
    } else {
      return new SeparatorModelViewHolder(parent);
    }
  }

  @Override
  public int getItemViewType(int position) {
    UiModel item = getItem(position);
    if (item instanceof UiModel.UserModel) {
      return R.layout.item;
    } else if (item instanceof UiModel.SeparatorModel) {
      return R.layout.separator_item;
    } else {
      throw new IllegalStateException("Unknown view");
    }
  }

  @Override
  public void onBindViewHolder(@NonNull RecyclerView.ViewHolder holder,
    int position) {
    if (holder instanceOf UserModelViewHolder) {
      UserModel userModel = (UserModel) getItem(position);
      ((UserModelViewHolder) holder).bind(userModel);
    } else {
      SeparatorModel separatorModel = (SeparatorModel) getItem(position);
      ((SeparatorModelViewHolder) holder).bind(separatorModel);
    }
  }
}

class UiModelComparator extends DiffUtil.ItemCallback<UiModel> {
  @Override
  public boolean areItemsTheSame(@NonNull UiModel oldItem,
    @NonNull UiModel newItem) {
    boolean isSameRepoItem = oldItem instanceof UserModel
      && newItem instanceof UserModel
      && ((UserModel) oldItem).getId().equals(((UserModel) newItem).getId());

    boolean isSameSeparatorItem = oldItem instanceof SeparatorModel
      && newItem instanceof SeparatorModel
      && ((SeparatorModel) oldItem).getDescription().equals(
      ((SeparatorModel) newItem).getDescription());

    return isSameRepoItem || isSameSeparatorItem;
  }

  @Override
  public boolean areContentsTheSame(@NonNull UiModel oldItem,
    @NonNull UiModel newItem) {
    return oldItem.equals(newItem);
  }
}

Java

class UiModelAdapter extends PagingDataAdapter<UiModel, RecyclerView.ViewHolder> {
  UiModelAdapter() {
    super(new UiModelComparator(), Dispatchers.getMain(),
      Dispatchers.getDefault());
  }

  @NonNull
  @Override
  public RecyclerView.ViewHolder onCreateViewHolder(@NonNull ViewGroup parent,
    int viewType) {
    if (viewType == R.layout.item) {
      return new UserModelViewHolder(parent);
    } else {
      return new SeparatorModelViewHolder(parent);
    }
  }

  @Override
  public int getItemViewType(int position) {
    UiModel item = getItem(position);
    if (item instanceof UiModel.UserModel) {
      return R.layout.item;
    } else if (item instanceof UiModel.SeparatorModel) {
      return R.layout.separator_item;
    } else {
      throw new IllegalStateException("Unknown view");
    }
  }

  @Override
  public void onBindViewHolder(@NonNull RecyclerView.ViewHolder holder,
    int position) {
    if (holder instanceOf UserModelViewHolder) {
      UserModel userModel = (UserModel) getItem(position);
      ((UserModelViewHolder) holder).bind(userModel);
    } else {
      SeparatorModel separatorModel = (SeparatorModel) getItem(position);
      ((SeparatorModelViewHolder) holder).bind(separatorModel);
    }
  }
}

class UiModelComparator extends DiffUtil.ItemCallback<UiModel> {
  @Override
  public boolean areItemsTheSame(@NonNull UiModel oldItem,
    @NonNull UiModel newItem) {
    boolean isSameRepoItem = oldItem instanceof UserModel
      && newItem instanceof UserModel
      && ((UserModel) oldItem).getId().equals(((UserModel) newItem).getId());

    boolean isSameSeparatorItem = oldItem instanceof SeparatorModel
      && newItem instanceof SeparatorModel
      && ((SeparatorModel) oldItem).getDescription().equals(
      ((SeparatorModel) newItem).getDescription());

    return isSameRepoItem || isSameSeparatorItem;
  }

  @Override
  public boolean areContentsTheSame(@NonNull UiModel oldItem,
    @NonNull UiModel newItem) {
    return oldItem.equals(newItem);
  }
}

Transform data streams

Apply basic transformations

Kotlin

Java

Java

Kotlin

Java

Java

Add list separators

Convert the UI model

Kotlin

Java

Java

Transform the data stream

Kotlin

Java

Java

Handle separators in the UI

Kotlin

Java

Java

Additional resources

Codelabs

Samples