Retrieve depth information in your app with ARCore for Jetpack XR

Your app can retrieve depth information through ARCore for Jetpack XR to determine how close physical objects are to the device.

Create an ARCore for Jetpack XR session

Obtain head pose information through an ARCore for Jetpack XR session. See Understand a Session's lifecycle to obtain a Session.

Configure the session

Depth map retrieval is not enabled by default on XR sessions. To enable depth map retrieval, configure the session and set a DepthEstimationMode:

val newConfig = session.config.copy(
    depthEstimation = Config.DepthEstimationMode.SMOOTH_ONLY,
)
when (val result = session.configure(newConfig)) {
    is SessionConfigureSuccess -> TODO(/* Success! */)
    is SessionConfigureConfigurationNotSupported ->
        TODO(/* Some combinations of configurations are not valid. Handle this failure case. */)
    else ->
        TODO(/* The session could not be configured. See SessionConfigureResult for possible causes. */)
}
DepthMaps.kt

The following values of DepthEstimationMode are available:

• DISABLED: No information about scene depth is provided.
• RAW_ONLY: Depth estimation is enabled with raw depth and confidence values.
• SMOOTH_ONLY: Depth estimation is enabled with smooth depth and confidence values.
• SMOOTH_AND_RAW: Depth estimation is enabled with both raw and smooth depth and confidence values.

Raw depth maps provide depth estimates with higher accuracy, but raw depth images might not include depth estimates for all pixels in the camera image. In contrast, the smooth depth maps provide estimated depth for every pixel, but per-pixel depth data might be less accurate due to smoothing and interpolation of depth estimates.

Retrieve depth data

To obtain depth data for a given camera, use DepthMap:

val depthMap = DepthMap.left(session) ?: return
DepthMaps.kt

Different devices have different capabilities. Devices with a stereo camera configuration return non-null depth maps for the left and right cameras. Likewise, devices with a singular camera return a non-null depth map using mono.

Calculate depth values

You can obtain depth and confidence values from the resulting depth map:

val depthMap = DepthMap.left(session) ?: return
DepthMaps.kt

Depending on the configuration setting used, access the corresponding depth map using smoothDepthMap or rawDepthMap. Measurements contained in these maps are expressed in meters. You can also access the confidence values using smoothConfidenceMap and rawConfidenceMap. These values range from 0 to 255, where 255 represents the highest confidence.

To render a depth map for debug or visualization purposes, see the Depth part of the ARCore test app.