使用 ARCore for Jetpack XR 在应用中加入面部跟踪功能

适用的 XR 设备

本指南可帮助您为这些类型的 XR 设备打造体验。

了解 XR 设备类型 →

XR 头戴设备

有线 XR 眼镜

了解 XR 设备类型 →

用户授予面部追踪权限后，您的应用可以通过 ARCore for Jetpack XR 检索面部形状信息。面部形状信息可帮助您的应用在虚拟世界中显示用户，例如用于虚拟眼镜试戴。

创建 ARCore for Jetpack XR 会话

通过 Jetpack XR 运行时 Session 获取面部信息，应用可以创建该运行时。

配置会话

默认情况下，XR 会话中未启用人脸跟踪功能。如需启用人脸跟踪，请配置会话并设置 FaceTrackingMode.USER 模式：

val newConfig = session.config.copy(
    faceTracking = Config.FaceTrackingMode.USER,
)
when (val result = session.configure(newConfig)) {
    is SessionConfigureSuccess -> TODO(/* Success! */)
    is SessionConfigureConfigurationNotSupported ->
        TODO(/* Some combinations of configurations are not valid. Handle this failure case. */)
    else ->
        TODO(/* The session could not be configured. See SessionConfigureResult for possible causes. */)
}FaceTracking.kt

检索人脸数据

使用 Face.getUserFace(session) 可检索用户面部数据，其中包含以下信息：

面部混合形状值：面部的可能表情是 68 个混合形状值的组合。每个混合形状值都表示面部运动或面部变形，该值表示其强度，范围为 0.0 到 1.0。如需了解每种混合形状类型的说明，请参阅了解面部混合形状类型。
各区域的置信度值：面部有三个区域。置信度值表示给定姿势的准确度确定程度，范围为 0.0 到 1.0，其中 1.0 表示置信度最高。

val face = Face.getUserFace(session) ?: return
face.state.collect { state ->
    if (state.trackingState != TrackingState.TRACKING) return@collect

    val confidence = state.getConfidence(FaceConfidenceRegion.FACE_CONFIDENCE_REGION_LOWER)
    val blendShapeValue = state.blendShapes[FaceBlendShapeType.FACE_BLEND_SHAPE_TYPE_LIPS_TOWARD]
}FaceTracking.kt

了解面部融合变形类型

下表列出了每种面部混合形状：

上部区域混合形状

名称	参考图片
`BROW_LOWERER_L`	`BROW_LOWERER_L = 0.0`	`BROW_LOWERER_L = 1.0`
`BROW_LOWERER_R`	`BROW_LOWERER_R = 0.0`	`BROW_LOWERER_R = 1.0`
`EYES_CLOSED_L`	`EYES_CLOSED_L = 0.0`	`EYES_CLOSED_L = 1.0`
`EYES_CLOSED_R`	`EYES_CLOSED_R = 0.0`	`EYES_CLOSED_R = 1.0`
`EYES_LOOK_DOWN_L`	`EYES_LOOK_DOWN_L = 0.0`	`EYES_LOOK_DOWN_L = 1.0`
`EYES_LOOK_DOWN_R`	`EYES_LOOK_DOWN_R = 0.0`	`EYES_LOOK_DOWN_R = 1.0`
`EYES_LOOK_LEFT_L`	`EYES_LOOK_LEFT_L = 0.0`	`EYES_LOOK_LEFT_L = 1.0`
`EYES_LOOK_LEFT_R`	`EYES_LOOK_LEFT_R = 0.0`	`EYES_LOOK_LEFT_R = 1.0`
`EYES_LOOK_RIGHT_L`	`EYES_LOOK_RIGHT_L = 0.0`	`EYES_LOOK_RIGHT_L = 1.0`
`EYES_LOOK_RIGHT_R`	`EYES_LOOK_RIGHT_R = 0.0`	`EYES_LOOK_RIGHT_R = 1.0`
`EYES_LOOK_UP_L`	`EYES_LOOK_UP_L = 0.0`	`EYES_LOOK_UP_L = 1.0`
`EYES_LOOK_UP_R`	`EYES_LOOK_UP_R = 0.0`	`EYES_LOOK_UP_R = 1.0`
`INNER_BROW_RAISER_L`	`INNER_BROW_RAISER_L = 0.0`	`INNER_BROW_RAISER_L = 1.0`
`INNER_BROW_RAISER_R`	`INNER_BROW_RAISER_R = 0.0`	`INNER_BROW_RAISER_R = 1.0`
`LID_TIGHTENER_L`	`LID_TIGHTENER_L = 0.0`	`LID_TIGHTENER_L = 1.0`
`LID_TIGHTENER_R`	`LID_TIGHTENER_R = 0.0`	`LID_TIGHTENER_R = 1.0`
`OUTER_BROW_RAISER_L`	`OUTER_BROW_RAISER_L = 0.0`	`OUTER_BROW_RAISER_L = 1.0`
`OUTER_BROW_RAISER_R`	`OUTER_BROW_RAISER_R = 0.0`	`OUTER_BROW_RAISER_R = 1.0`
`UPPER_LID_RAISER_L`	`UPPER_LID_RAISER_L = 0.0`	`UPPER_LID_RAISER_L = 1.0`
`UPPER_LID_RAISER_R`	`UPPER_LID_RAISER_R = 0.0`	`UPPER_LID_RAISER_R = 1.0`

下部区域混合形状

名称	参考图片
`CHEEK_PUFF_L`	`CHEEK_PUFF_L = 0.0`	`CHEEK_PUFF_L = 1.0`
`CHEEK_PUFF_R`	`CHEEK_PUFF_R = 0.0`	`CHEEK_PUFF_R = 1.0`
`CHEEK_RAISER_L`	`CHEEK_RAISER_L = 0.0`	`CHEEK_RAISER_L = 1.0`
`CHEEK_RAISER_R`	`CHEEK_RAISER_R = 0.0`	`CHEEK_RAISER_R = 1.0`
`CHEEK_SUCK_L`	`CHEEK_SUCK_L = 0.0`	`CHEEK_SUCK_L = 1.0`
`CHEEK_SUCK_R`	`CHEEK_SUCK_R = 0.0`	`CHEEK_SUCK_R = 1.0`
`CHIN_RAISER_B`	`CHIN_RAISER_B = 0.0`	`CHIN_RAISER_B = 1.0`
`CHIN_RAISER_T`	`CHIN_RAISER_T = 0.0`	`CHIN_RAISER_T = 1.0`
`DIMPLER_L`	`DIMPLER_L = 0.0`	`DIMPLER_L = 1.0`
`DIMPLER_R`	`DIMPLER_R = 0.0`	`DIMPLER_R = 1.0`
`JAW_DROP`	`JAW_DROP = 0.0`	`JAW_DROP = 1.0`
`JAW_SIDEWAYS_LEFT`	`JAW_SIDEWAYS_LEFT = 0.0`	`JAW_SIDEWAYS_LEFT = 1.0`
`JAW_SIDEWAYS_RIGHT`	`JAW_SIDEWAYS_RIGHT = 0.0`	`JAW_SIDEWAYS_RIGHT = 1.0`
`JAW_THRUST`	`JAW_THRUST = 0.0`	`JAW_THRUST = 1.0`
`LIP_CORNER_DEPRESSOR_L`	`LIP_CORNER_DEPRESSOR_L = 0.0`	`LIP_CORNER_DEPRESSOR_L = 1.0`
`LIP_CORNER_DEPRESSOR_R`	`LIP_CORNER_DEPRESSOR_R = 0.0`	`LIP_CORNER_DEPRESSOR_R = 1.0`
`LIP_CORNER_PULLER_L`	`LIP_CORNER_PULLER_L = 0.0`	`LIP_CORNER_PULLER_L = 1.0`
`LIP_CORNER_PULLER_R`	`LIP_CORNER_PULLER_R = 0.0`	`LIP_CORNER_PULLER_R = 1.0`
`LIP_FUNNELER_LB`	`LIP_FUNNELER_LB = 0.0`	`LIP_FUNNELER_LB = 1.0`
`LIP_FUNNELER_LT`	`LIP_FUNNELER_LT = 0.0`	`LIP_FUNNELER_LT = 1.0`
`LIP_FUNNELER_RB`	`LIP_FUNNELER_RB = 0.0`	`LIP_FUNNELER_RB = 1.0`
`LIP_FUNNELER_RT`	`LIP_FUNNELER_RT = 0.0`	`LIP_FUNNELER_RT = 1.0`
`LIP_PRESSOR_L`	`LIP_PRESSOR_L = 0.0`	`LIP_PRESSOR_L = 1.0`
`LIP_PRESSOR_R`	`LIP_PRESSOR_R = 0.0`	`LIP_PRESSOR_R = 1.0`
`LIP_PUCKER_L`	`LIP_PUCKER_L = 0.0`	`LIP_PUCKER_L = 1.0`
`LIP_PUCKER_R`	`LIP_PUCKER_R = 0.0`	`LIP_PUCKER_R = 1.0`
`LIP_STRETCHER_L`	`LIP_STRETCHER_L = 0.0`	`LIP_STRETCHER_L = 1.0`
`LIP_STRETCHER_R`	`LIP_STRETCHER_R = 0.0`	`LIP_STRETCHER_R = 1.0`
`LIP_SUCK_LB`	`LIP_SUCK_LB = 0.0`	`LIP_SUCK_LB = 1.0`
`LIP_SUCK_LT`	`LIP_SUCK_LT = 0.0`	`LIP_SUCK_LT = 1.0`
`LIP_SUCK_RB`	`LIP_SUCK_RB = 0.0`	`LIP_SUCK_RB = 1.0`
`LIP_SUCK_RT`	`LIP_SUCK_RT = 0.0`	`LIP_SUCK_RT = 1.0`
`LIP_TIGHTENER_L`	`LIP_TIGHTENER_L = 0.0`	`LIP_TIGHTENER_L = 1.0`
`LIP_TIGHTENER_R`	`LIP_TIGHTENER_R = 0.0`	`LIP_TIGHTENER_R = 1.0`
`LIPS_TOWARD`	`LIPS_TOWARD = 0.0`	`JAW_DROP = 1.0 and LIPS_TOWARD = 1.0`
`LOWER_LIP_DEPRESSOR_L`	`LOWER_LIP_DEPRESSOR_L = 0.0`	`LOWER_LIP_DEPRESSOR_L = 1.0`
`LOWER_LIP_DEPRESSOR_R`	`LOWER_LIP_DEPRESSOR_R = 0.0`	`LOWER_LIP_DEPRESSOR_R = 1.0`
`MOUTH_LEFT`	`MOUTH_LEFT = 0.0`	`MOUTH_LEFT = 1.0`
`MOUTH_RIGHT`	`MOUTH_RIGHT = 0.0`	`MOUTH_RIGHT = 1.0`
`NOSE_WRINKLER_L`	`NOSE_WRINKLER_L = 0.0`	`NOSE_WRINKLER_L = 1.0`
`NOSE_WRINKLER_R`	`NOSE_WRINKLER_R = 0.0`	`NOSE_WRINKLER_R = 1.0`
`UPPER_LIP_RAISER_L`	`UPPER_LIP_RAISER_L = 0.0`	`UPPER_LIP_RAISER_L = 1.0`
`UPPER_LIP_RAISER_R`	`UPPER_LIP_RAISER_R = 0.0`	`UPPER_LIP_RAISER_R = 1.0`
`TONGUE_OUT`	TONGUE_OUT = 0.0	TONGUE_OUT = 1.0
`TONGUE_LEFT`	TONGUE_LEFT = 0.0	TONGUE_LEFT = 1.0
`TONGUE_RIGHT`	TONGUE_RIGHT = 0.0	TONGUE_RIGHT = 1.0
`TONGUE_UP`	TONGUE_UP = 0.0	TONGUE_UP = 1.0
`TONGUE_DOWN`	TONGUE_DOWN = 0.0	TONGUE_DOWN = 1.0