While many Android TV apps are built with Android native components, it's equally important to give careful consideration to accessibility of third party frameworks or components, in particular where custom views are used.
Such components may be interfacing directly with OpenGL or Canvas and consequently, accessibility services such as Talkback and SwitchAccess may not work well.
Consider some of the following issues that might occur with Talkback switched on:
- The accessibility focus (a green rectangle) disappears in your app.
- The accessibility focus selected the boundary of the whole screen.
- The accessibility focus cannot be moved.
- Four direction DPAD keys have no effect, even if your code is handling them.
If you observe any of these issues in your app, you may need to ensure that your app exposes its AccessibilityNodeInfo tree to the accessibility services.
Root Cause: DPAD Events consumed by accessibility services
The main reason for the issue is that the DPAD key events have been consumed by accessibility services. And those will not pass over to your App. Yes, the issue is raised by the Android Accessibility Suite, itself.
As illustrated in the figure above, when Talkback is switched on, the DPAD event will not be passed to the DPAD handler defined by developer. Instead, accessibility services receive the DPAD events in order to move the accessibility focus. Because custom Android components do not by default expose information to accessibility services about their position on the screen, accessibility services cannot move the accessibility focus to highlight them.
This is the reason your app might not be able to receive the DPAD events..
In conclusion, the reasons why apps developed with custom view do not work with Talkback are:
- DPAD key events are consumed by accessibility services.
- Accessibility services do not know what and where the UI components are on the screen.
Similarly, this issue also impacts the SwitchAccess service; in much the same way as Talkback, the SwitchAccess navigation utility also depends on the AccessibilityNodeInfo tree.
To solve the issue, we should focus on those two parts.
Why the DPAD key events are being consumed
If Accessibility Services such as TalkBack are not enabled, Android TV only moves focus on necessary focusable elements, such as buttons, links and icons skipping all elements that the sighted user can read. But when an Accessibility Service such as TalkBack is enabled, accessibility focus should also move to text only elements and announce them. To solve this, the Accessibility Services on TV need to intercept the keypad events to move both input and accessibility focus in sync.
Exposing information to accessibility services
As we mentioned in the last section, to the accessibility services, the UI components, like button, link, list, text description drawed by non-native framework, are unknown components. The services does not know their location and anything else information. Thus to solve the issue, we should tell the accessibility services all it need to know.
The AccessibilityNodeInfo is the class to store the information for each component. And then we can use ExploreByTouchHelper to define and expose all components information to the services. And then use setAccessibilityDetegate to set the ExploreByTouchHelper object.
What you should do is to create a new class to inherit the ExploreByTouchHelper. And then override its 4 methods in the ExploreByTouchHelper here:
// Return the virtual view ID whose view is covered by the input point (x, y). protected fun getVirtualViewAt(x:Float, y:Float):Int // Fill the virtual view ID list into the input parameter virutalViewIds. protected fun getVisibleVirtualViews(virtualViewIds:List<Int>) // For the view whose virtualViewId is the input virtualViewId, populate the // accessibility node information into the AccessibilityNodeInfoCompat parameter. protected fun onPopulateNodeForVirtualView(virtualViewId:Int, @NonNull node:AccessibilityNodeInfoCompat) // Set the accessibility handling when perform action. protected fun onPerformActionForVirtualView(virtualViewId:Int, action:Int, @Nullable arguments:Bundle):Boolean
// Return the virtual view ID whose view is covered by the input point (x, y). protected int getVirtualViewAt(float x, float y) // Fill the virtual view ID list into the input parameter virutalViewIds. protected void getVisibleVirtualViews(List<Integer> virtualViewIds) // For the view whose virtualViewId is the input virtualViewId, populate the // accessibility node information into the AccessibilityNodeInfoCompat parameter. protected void onPopulateNodeForVirtualView(int virtualViewId, @NonNull AccessibilityNodeInfoCompat node) // Set the accessibility handling when perform action. protected boolean onPerformActionForVirtualView(int virtualViewId, int action, @Nullable Bundle arguments)
What information you need to expose
To make sure the component visible to accessibility services, you have to build its own AccessibilityNodeInfo for each component. And make sure below items:
Required AccessibilityNodeInfo.getBoundsInScreen() to set the position of the component.
Required AccessibilityNodeInfo.setVisibleToUser() should be true to make the virtual node visible.
Required AccessibilityNodeInfo.getContentDescription() should set the content description for the Talkback to announce.
AccessibilityNodeInfo.setClassName() should be set to allow services distinguish the component type.
If we want to implement more ACTION types, like ACTION_CLICK, invoke AccessibilityNodeInfo.addAction(ACTION_CLICK) to add the action. And also add handling logic in performAction() method.
The more methods you set in AccessibilityNodeInfo, the more information the accessibility services know your components more.
Also consult the custom view accessibility sample for Android TV with best practices for adding accessibility support for apps using custom views.