A Composition describes the UI of your app and is produced by running composables. The Composition is a tree-structure that consists of the composables that describe your UI.
Next to the Composition, there exists a parallel tree, called the Semantics tree. This tree describes your UI in an alternative manner that is understandable for Accessibility services and for the Testing framework. Accessibility services use the tree to describe the app to users with a specific need. The Testing framework uses it to interact with your app and make assertions about it. The Semantics tree does not contain the information on how to draw your composables, but it contains information about the semantic meaning of your composables.
Figure 1. A typical UI hierarchy and its semantics tree.
If your app consists of composables and modifiers from the Compose foundation and material library, the Semantics tree is automatically filled and generated for you. However when you’re adding custom low-level composables, you will have to manually provide its semantics. There might also be situations where your tree does not correctly or fully represent the meaning of the elements on the screen, in which case you can adapt the tree.
Consider for example this custom calendar composable:
Figure 2. A custom calendar composable with selectable day elements.
In this example, the entire calendar is implemented as a single low-level
composable, using the Layout
composable and drawing directly to the Canvas
.
If you do not do anything else, accessibility services will not receive enough
information about the content of the composable and the user's selection within
the calendar. For example, if a user clicks on the day containing 17, the
accessibility framework only receives the description information for the whole
calendar control. In this case, the TalkBack accessibility service would simply
announce "Calendar" or, only slightly better, "April Calendar" and the user
would be left to wonder what day was selected. To make this composable more
accessible, you’ll need to add semantic information manually.
Semantics properties
All nodes in the UI tree with some semantic meaning have a parallel node in the
Semantics tree. The node in the Semantics tree contains those properties that
convey the meaning of the corresponding composable. For example, the Text
composable contains a semantic property text
, because that’s the meaning of
that composable. An Icon
contains a contentDescription
property (if set by
the developer) that conveys in text what the meaning of the Icon
is.
Composables and modifiers that are built on top of the Compose
foundation library already set
the relevant properties for you. Optionally, you can set or override the
properties yourself with the semantics
and
clearAndSetSemantics
modifiers. For example, you can add custom accessibility actions to a node, provide an
alternative state description for a
toggleable element, or indicate that a certain text composable should be
considered as a heading.
To visualize the Semantics tree, we can use the Layout Inspector Tool or use the printToLog()
method inside our tests. This will print the current Semantics tree inside
Logcat.
class MyComposeTest { @get:Rule val composeTestRule = createComposeRule() @Test fun MyTest() { // Start the app composeTestRule.setContent { MyTheme { Text("Hello world!") } } // Log the full semantics tree composeTestRule.onRoot().printToLog("MY TAG") } }
The output of this test would be:
Printing with useUnmergedTree = 'false'
Node #1 at (l=0.0, t=63.0, r=221.0, b=120.0)px
|-Node #2 at (l=0.0, t=63.0, r=221.0, b=120.0)px
Text = '[Hello world!]'
Actions = [GetTextLayoutResult]
Let’s look at an example to see how semantics properties are used to convey the meaning of a composable. Let’s think about a Switch
. This is how it looks to the user:
Figure 3. A Switch in its “On” and “Off” state.
To describe the meaning of this element, you could say the following: “This is a Switch, which is a toggleable element, currently in its 'On' state. You can click it to interact with it.”
This is exactly what the semantics properties are used for. The semantics node of this Switch element contains the following properties, as visualized with the Layout Inspector:
Figure 4. Layout Inspector showing the Semantics properties of a Switch composable.
The Role
indicates what type of element we’re looking at. The
StateDescription
describes how the “On” state should be referenced. By
default this is simply a localized version of the word “On”, but this can be
made more specific (for example, “Enabled”) based on the context. The
ToggleableState
is the current state of the Switch. The OnClick
property
references the method used to interact with this element. For a full list of
semantics properties, check out the SemanticsProperties
object.
For a full list of possible Accessibility Actions, check out the
SemanticsActions
object.
Keeping track of the semantics properties of each composable in your app unlocks a lot of powerful possibilities. Some examples:
- Talkback uses the properties to read aloud what’s shown on the screen and lets the user smoothly interact with it. For our Switch, it might say: “On; Switch; double tap to toggle”. The user can double tap their screen to toggle the Switch off.
-
The Testing framework uses the properties to find nodes, interact with
them, and make assertions. A sample test for our Switch could be:
val mySwitch = SemanticsMatcher.expectValue( SemanticsProperties.Role, Role.Switch ) composeTestRule.onNode(mySwitch) .performClick() .assertIsOff()
Merged and unmerged Semantics tree
As mentioned before, each composable in the UI tree might have zero or more semantics properties set. When a composable has no semantics properties set, it isn’t included as part of the Semantics tree. That way, the Semantics tree contains only the nodes that actually contain semantic meaning. However, sometimes to convey the correct meaning of what is shown on the screen, it is also useful to merge certain sub-trees of nodes and treat them as one. That way we can reason about a set of nodes as a whole, instead of dealing with each descendant node individually. As a rule of thumb, each node in this tree represents a focusable element when using Accessibility services.
An example of such a composable is Button. We’d like to reason about the Button as a single element, even though it may contain multiple child nodes:
Button(onClick = { /*TODO*/ }) { Icon( imageVector = Icons.Filled.Favorite, contentDescription = null ) Spacer(Modifier.size(ButtonDefaults.IconSpacing)) Text("Like") }
In our Semantics tree, the properties of the Button’s descendants are merged, and the Button is presented as a single leaf node in the tree:
Composables and modifiers can indicate that they want to merge their
descendants' semantics properties by calling Modifier.semantics
(mergeDescendants = true) {}
. Setting this property to true
indicates that
the semantics properties should be merged. In our Button
example, the
Button
composable uses the clickable
modifier internally that includes this
semantics
modifier. Therefore, the descendant nodes of the Button will be
merged. Read the accessibility documentation to learn more about when you
should change merging behavior in your composable.
Several modifiers and composables in the Foundation and Material Compose
libraries have this property set. For example, the clickable
and toggleable
modifiers will automatically merge their descendants. Also, the ListItem
composable will merge its descendants.
Inspecting the trees
When talking about the Semantics tree, we are actually talking about two
different trees. There’s a merged Semantics tree, which merges descendant
nodes when mergeDescendants
is set to true
. There’s also an unmerged
Semantics tree, which does not apply the merging, but keeps every node intact.
Accessibility services use the unmerged tree and apply their own merging
algorithms, taking into consideration the mergeDescendants
property. The
Testing framework uses the merged tree by default.
You can inspect both trees with the printToLog()
method. By default, and as in
the earlier examples, the merged tree will be logged. To print the unmerged
tree instead, set the useUnmergedTree
parameter of the onRoot()
matcher to
true
:
composeTestRule.onRoot(useUnmergedTree = true).printToLog("MY TAG")
The Layout Inspector allows you to display both the merged and the unmerged Semantics tree, by selecting the preferred one in the view filter:
Figure 5. Layout Inspector view options, allowing both the display of the merged and the unmerged Semantics tree.
For each node in your tree, the Layout Inspector shows both the Merged Semantics and the Semantics set on that node in the properties panel:
By default, matchers in the Testing Framework use the merged Semantics tree. That’s why you can interact with a Button by matching the text shown inside it:
composeTestRule.onNodeWithText("Like").performClick()
You can override this behavior by setting the useUnmergedTree
parameter of the
matchers to true
, as we did before with the onRoot
matcher.
Merging behavior
When a composable indicates that its descendants should be merged, how does this merging happen exactly?
Each semantics property has a defined merging strategy. For example, the
ContentDescription
property adds all descendant ContentDescription values to
a list. You can check the merging strategy of a semantics property by checking
its mergePolicy
implementation in SemanticsProperties.kt
.
Properties can choose to always pick the parent or child value, merge the
values into a list or string, not allow merging at all and throw an exception
instead, or any other custom merging strategy.
An important note is that descendants that themselves have set
mergeDescendants = true
are not included in the merge. Let’s take a look at an example:
Figure 6. List item with image, some text, and a bookmark icon.
Here we have a clickable list item. When the user presses the row, the app navigates to the article detail page, where the user can read the article. Inside the list item, there is a button to bookmark this article. In this case we have a nested clickable element, so the button will show up separately in the merged tree. The rest of the content in the row is merged:
Figure 7. The merged tree contains multiple texts in a list inside the Row node. The unmerged tree contains separate nodes for each Text composable.
Adapting the Semantics tree
As mentioned before, you can override or clear certain semantics properties, or change the merging behavior of the tree. This is particularly relevant when you’re creating your own custom components. Without setting the correct properties and merge behavior, your app might not be accessible, and tests might behave differently than you expect. To read more about some common use cases where you should adapt the Semantics tree, read the accessibility documentation. If you want to learn more about testing, check the Testing Guide.