XR_EXT_spatial_marker_tracking

Name String

XR_EXT_spatial_marker_tracking

Extension Type

Instance extension

Registered Extension Number

744

Revision

1

Ratification Status

Ratified

Extension and Version Dependencies

XR_EXT_spatial_entity

Contributors

Ron Bessems, Meta
Nihav Jain, Google
Natalie Fleury, Meta
Yuichi Taguchi, Meta
Yin Li, Microsoft
Jimmy Alamparambil, ByteDance
Zhipeng Liu, ByteDance
Jun Yan, ByteDance

Overview

This extension builds on XR_EXT_spatial_entity and allows applications to detect and track markers in their environment. Markers are 2D codes which may include QR Codes, Micro QR Codes, ArUco markers, or AprilTags.

A tracked marker is represented as a spatial entity with (or "that has") the following components:

  • XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT
  • XR_SPATIAL_COMPONENT_TYPE_BOUNDED_2D_EXT

Runtime support

A runtime must advertise its support for the various marker tracking capabilities using xrEnumerateSpatialCapabilitiesEXT by listing any of the following capabilities:

  • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT
  • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_MICRO_QR_CODE_EXT
  • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_ARUCO_MARKER_EXT
  • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_APRIL_TAG_EXT

Configuration

To enable detection of a marker type the application must pass the corresponding configuration structure to xrCreateSpatialContextAsyncEXT .

Marker Type Configurations

QR codes

The XrSpatialCapabilityConfigurationQrCodeEXT structure is defined as:

typedef struct XrSpatialCapabilityConfigurationQrCodeEXT {
    XrStructureType                     type;
    const void*                         next;
    XrSpatialCapabilityEXT              capability;
    uint32_t                            enabledComponentCount;
    const XrSpatialComponentTypeEXT*    enabledComponents;
} XrSpatialCapabilityConfigurationQrCodeEXT;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain.
  • capability must be XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT .
  • enabledComponentCount is a uint32_t with the number of elements in enabledComponents .
  • enabledComponents is a pointer to an array of components to enable for this capability.

If QR codes are supported, the runtime must enable QR Code tracking when an XrSpatialCapabilityConfigurationQrCodeEXT structure is passed in XrSpatialContextCreateInfoEXT :: capabilityConfigs when calling xrCreateSpatialContextAsyncEXT .

The runtime must return XR_ERROR_SPATIAL_CAPABILITY_UNSUPPORTED_EXT if XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT is not enumerated by xrEnumerateSpatialCapabilitiesEXT .

Valid Usage (Implicit)

Micro QR codes

The XrSpatialCapabilityConfigurationMicroQrCodeEXT structure is defined as:

typedef struct XrSpatialCapabilityConfigurationMicroQrCodeEXT {
    XrStructureType                     type;
    const void*                         next;
    XrSpatialCapabilityEXT              capability;
    uint32_t                            enabledComponentCount;
    const XrSpatialComponentTypeEXT*    enabledComponents;
} XrSpatialCapabilityConfigurationMicroQrCodeEXT;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain.
  • capability must be XR_SPATIAL_CAPABILITY_MARKER_TRACKING_MICRO_QR_CODE_EXT .
  • enabledComponentCount is a uint32_t with the number of elements in enabledComponents .
  • enabledComponents is a pointer to an array of components to enable for this capability.

If Micro QR codes are supported, the runtime must enable Micro QR Code tracking when an XrSpatialCapabilityConfigurationMicroQrCodeEXT structure is passed in XrSpatialContextCreateInfoEXT :: capabilityConfigs when calling xrCreateSpatialContextAsyncEXT .

The runtime must return XR_ERROR_SPATIAL_CAPABILITY_UNSUPPORTED_EXT if XR_SPATIAL_CAPABILITY_MARKER_TRACKING_MICRO_QR_CODE_EXT is not enumerated by xrEnumerateSpatialCapabilitiesEXT .

Valid Usage (Implicit)

ArUco Markers

The XrSpatialCapabilityConfigurationArucoMarkerEXT structure is defined as:

typedef struct XrSpatialCapabilityConfigurationArucoMarkerEXT {
    XrStructureType                     type;
    const void*                         next;
    XrSpatialCapabilityEXT              capability;
    uint32_t                            enabledComponentCount;
    const XrSpatialComponentTypeEXT*    enabledComponents;
    XrSpatialMarkerArucoDictEXT         arUcoDict;
} XrSpatialCapabilityConfigurationArucoMarkerEXT;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain.
  • capability must be XR_SPATIAL_CAPABILITY_MARKER_TRACKING_ARUCO_MARKER_EXT .
  • enabledComponentCount is a uint32_t with the number of elements in enabledComponents .
  • enabledComponents is a pointer to an array of components to enable for this capability.
  • arUcoDict is the marker dictionary to detect.

If ArUco markers are supported, the runtime must enable ArUco marker tracking when an XrSpatialCapabilityConfigurationArucoMarkerEXT structure is passed in XrSpatialContextCreateInfoEXT :: capabilityConfigs when calling xrCreateSpatialContextAsyncEXT .

The runtime must return XR_ERROR_SPATIAL_CAPABILITY_UNSUPPORTED_EXT from xrCreateSpatialContextAsyncEXT if an XrSpatialCapabilityConfigurationArucoMarkerEXT structure is in XrSpatialContextCreateInfoEXT :: capabilityConfigs but XR_SPATIAL_CAPABILITY_MARKER_TRACKING_ARUCO_MARKER_EXT is not enumerated by xrEnumerateSpatialCapabilitiesEXT .

Valid Usage (Implicit)

The XrSpatialMarkerArucoDictEXT enumeration is defined as:

typedef enum XrSpatialMarkerArucoDictEXT {
    XR_SPATIAL_MARKER_ARUCO_DICT_4X4_50_EXT = 1,
    XR_SPATIAL_MARKER_ARUCO_DICT_4X4_100_EXT = 2,
    XR_SPATIAL_MARKER_ARUCO_DICT_4X4_250_EXT = 3,
    XR_SPATIAL_MARKER_ARUCO_DICT_4X4_1000_EXT = 4,
    XR_SPATIAL_MARKER_ARUCO_DICT_5X5_50_EXT = 5,
    XR_SPATIAL_MARKER_ARUCO_DICT_5X5_100_EXT = 6,
    XR_SPATIAL_MARKER_ARUCO_DICT_5X5_250_EXT = 7,
    XR_SPATIAL_MARKER_ARUCO_DICT_5X5_1000_EXT = 8,
    XR_SPATIAL_MARKER_ARUCO_DICT_6X6_50_EXT = 9,
    XR_SPATIAL_MARKER_ARUCO_DICT_6X6_100_EXT = 10,
    XR_SPATIAL_MARKER_ARUCO_DICT_6X6_250_EXT = 11,
    XR_SPATIAL_MARKER_ARUCO_DICT_6X6_1000_EXT = 12,
    XR_SPATIAL_MARKER_ARUCO_DICT_7X7_50_EXT = 13,
    XR_SPATIAL_MARKER_ARUCO_DICT_7X7_100_EXT = 14,
    XR_SPATIAL_MARKER_ARUCO_DICT_7X7_250_EXT = 15,
    XR_SPATIAL_MARKER_ARUCO_DICT_7X7_1000_EXT = 16,
    XR_SPATIAL_MARKER_ARUCO_DICT_MAX_ENUM_EXT = 0x7FFFFFFF
} XrSpatialMarkerArucoDictEXT;

Supported predefined ArUco dictionary:

Enumerant Descriptions

  • XR_SPATIAL_MARKER_ARUCO_DICT_4X4_50_EXT — 4 by 4 pixel Aruco marker dictionary with 50 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_4X4_100_EXT — 4 by 4 pixel Aruco marker dictionary with 100 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_4X4_250_EXT — 4 by 4 pixel Aruco marker dictionary with 250 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_4X4_1000_EXT — 4 by 4 pixel Aruco marker dictionary with 1000 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_5X5_50_EXT — 5 by 5 pixel Aruco marker dictionary with 50 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_5X5_100_EXT — 5 by 5 pixel Aruco marker dictionary with 100 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_5X5_250_EXT — 5 by 5 pixel Aruco marker dictionary with 250 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_5X5_1000_EXT — 5 by 5 pixel Aruco marker dictionary with 1000 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_6X6_50_EXT — 6 by 6 pixel Aruco marker dictionary with 50 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_6X6_100_EXT — 6 by 6 pixel Aruco marker dictionary with 100 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_6X6_250_EXT — 6 by 6 pixel Aruco marker dictionary with 250 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_6X6_1000_EXT — 6 by 6 pixel Aruco marker dictionary with 1000 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_7X7_50_EXT — 7 by 7 pixel Aruco marker dictionary with 50 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_7X7_100_EXT — 7 by 7 pixel Aruco marker dictionary with 100 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_7X7_250_EXT — 7 by 7 pixel Aruco marker dictionary with 250 IDs.
  • XR_SPATIAL_MARKER_ARUCO_DICT_7X7_1000_EXT — 7 by 7 pixel Aruco marker dictionary with 1000 IDs.
AprilTags

The XrSpatialCapabilityConfigurationAprilTagEXT structure is defined as:

typedef struct XrSpatialCapabilityConfigurationAprilTagEXT {
    XrStructureType                     type;
    const void*                         next;
    XrSpatialCapabilityEXT              capability;
    uint32_t                            enabledComponentCount;
    const XrSpatialComponentTypeEXT*    enabledComponents;
    XrSpatialMarkerAprilTagDictEXT      aprilDict;
} XrSpatialCapabilityConfigurationAprilTagEXT;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain.
  • capability must be XR_SPATIAL_CAPABILITY_MARKER_TRACKING_APRIL_TAG_EXT .
  • enabledComponentCount is a uint32_t with the number of elements in enabledComponents .
  • enabledComponents is a pointer to an array of components to enable for this capability.
  • aprilDict is the marker dictionary to detect.

If AprilTags are supported, the runtime must enable AprilTag tracking when an XrSpatialCapabilityConfigurationAprilTagEXT structure is passed in XrSpatialContextCreateInfoEXT :: capabilityConfigs when calling xrCreateSpatialContextAsyncEXT .

The runtime must return XR_ERROR_SPATIAL_CAPABILITY_UNSUPPORTED_EXT from xrCreateSpatialContextAsyncEXT if an XrSpatialCapabilityConfigurationAprilTagEXT structure is in XrSpatialContextCreateInfoEXT :: capabilityConfigs but XR_SPATIAL_CAPABILITY_MARKER_TRACKING_APRIL_TAG_EXT is not enumerated by xrEnumerateSpatialCapabilitiesEXT .

Valid Usage (Implicit)

The XrSpatialMarkerAprilTagDictEXT enumeration is defined as:

typedef enum XrSpatialMarkerAprilTagDictEXT {
    XR_SPATIAL_MARKER_APRIL_TAG_DICT_16H5_EXT = 1,
    XR_SPATIAL_MARKER_APRIL_TAG_DICT_25H9_EXT = 2,
    XR_SPATIAL_MARKER_APRIL_TAG_DICT_36H10_EXT = 3,
    XR_SPATIAL_MARKER_APRIL_TAG_DICT_36H11_EXT = 4,
    XR_SPATIAL_MARKER_APRIL_TAG_DICT_MAX_ENUM_EXT = 0x7FFFFFFF
} XrSpatialMarkerAprilTagDictEXT;

Supported predefined AprilTag dictionary:

Enumerant Descriptions

  • XR_SPATIAL_MARKER_APRIL_TAG_DICT_16H5_EXT — 4 by 4 bits, minimum Hamming distance between any two codes = 5, 30 codes.
  • XR_SPATIAL_MARKER_APRIL_TAG_DICT_25H9_EXT — 5 by 5 bits, minimum Hamming distance between any two codes = 9, 35 codes.
  • XR_SPATIAL_MARKER_APRIL_TAG_DICT_36H10_EXT — 6 by 6 bits, minimum Hamming distance between any two codes = 10, 2320 codes.
  • XR_SPATIAL_MARKER_APRIL_TAG_DICT_36H11_EXT — 6 by 6 bits, minimum Hamming distance between any two codes = 11, 587 codes.

Optional Marker Configurations

Applications should call xrEnumerateSpatialCapabilityFeaturesEXT to get the list of supported optional features.

See XrSpatialCapabilityFeatureEXT for a complete list of all spatial capability features supported by any extension.

Marker Size

The XrSpatialMarkerSizeEXT structure is defined as:

typedef struct XrSpatialMarkerSizeEXT {
    XrStructureType    type;
    const void*        next;
    float              markerSideLength;
} XrSpatialMarkerSizeEXT;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain.
  • markerSideLength is the size in meters of all markers.

If XR_SPATIAL_CAPABILITY_FEATURE_MARKER_TRACKING_FIXED_SIZE_MARKERS_EXT is enumerated by xrEnumerateSpatialCapabilityFeaturesEXT for a certain capability, and if the application chains XrSpatialMarkerSizeEXT to the corresponding configuration structure of that capability, the runtime must assume that all markers detected have width and height of markerSideLength . Providing this information to the runtime allows the runtime to return a more accurate pose and size. This structure must be linked into the next chain of XrSpatialCapabilityConfigurationQrCodeEXT , XrSpatialCapabilityConfigurationMicroQrCodeEXT , XrSpatialCapabilityConfigurationArucoMarkerEXT , or XrSpatialCapabilityConfigurationAprilTagEXT .

Valid Usage (Implicit)

Static Marker Optimization

The XrSpatialMarkerStaticOptimizationEXT structure is defined as:

typedef struct XrSpatialMarkerStaticOptimizationEXT {
    XrStructureType    type;
    const void*        next;
    XrBool32           optimizeForStaticMarker;
} XrSpatialMarkerStaticOptimizationEXT;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain.
  • optimizeForStaticMarker indicates if all markers in the space are expected to not move.

If XR_SPATIAL_CAPABILITY_FEATURE_MARKER_TRACKING_STATIC_MARKERS_EXT is enumerated by xrEnumerateSpatialCapabilityFeaturesEXT for a certain capability, and if the application chains XrSpatialMarkerStaticOptimizationEXT to the corresponding configuration structure of that capability, the runtime must assume that all markers detected are static if optimizeForStaticMarker is set to XR_TRUE . This allows the runtime to generate a more accurate pose and size. This structure must be linked into the next chain of XrSpatialCapabilityConfigurationQrCodeEXT , XrSpatialCapabilityConfigurationMicroQrCodeEXT , XrSpatialCapabilityConfigurationArucoMarkerEXT , or XrSpatialCapabilityConfigurationAprilTagEXT .

Valid Usage (Implicit)

Guaranteed Components

A runtime that supports XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT , XR_SPATIAL_CAPABILITY_MARKER_TRACKING_MICRO_QR_CODE_EXT , XR_SPATIAL_CAPABILITY_MARKER_TRACKING_ARUCO_MARKER_EXT , or XR_SPATIAL_CAPABILITY_MARKER_TRACKING_APRIL_TAG_EXT must provide the following spatial components as guaranteed components of all entities discovered by those capabilities, and must enumerate them in xrEnumerateSpatialCapabilityComponentTypesEXT :

  • XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT
  • XR_SPATIAL_COMPONENT_TYPE_BOUNDED_2D_EXT

Marker Component

Component data

The XrSpatialMarkerDataEXT structure is defined as:

typedef struct XrSpatialMarkerDataEXT {
    XrSpatialCapabilityEXT    capability;
    uint32_t                  markerId;
    XrSpatialBufferEXT        data;
} XrSpatialMarkerDataEXT;

Member Descriptions

  • capability is the XrSpatialCapabilityEXT that detected the marker.
  • markerId is the encoded identifier from the marker. For ArUco markers and AprilTag this field must be valid and filled with the encoded ID. For QR codes this field must be zero.
  • data is the buffer ID and type of additional information contained in the marker.

XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT and XR_SPATIAL_CAPABILITY_MARKER_TRACKING_MICRO_QR_CODE_EXT support extra data. If capability is one of these -

  • If the runtime has successfully decoded the data for the marker, it must set the data buffer type to either XR_SPATIAL_BUFFER_TYPE_UINT8_EXT or XR_SPATIAL_BUFFER_TYPE_STRING_EXT , depending on the data in the marker. The runtime must also set a valid buffer ID in data which the application can use with the appropriate xrGetSpatialBuffer* function to get the data.
  • If the runtime has not yet decoded the data of the marker, it must set data buffer ID to XR_NULL_SPATIAL_BUFFER_ID_EXT and the buffer type to XR_SPATIAL_BUFFER_TYPE_UNKNOWN_EXT .

XR_SPATIAL_CAPABILITY_MARKER_TRACKING_ARUCO_MARKER_EXT and XR_SPATIAL_CAPABILITY_MARKER_TRACKING_APRIL_TAG_EXT do not support extra data and the runtime must set the buffer ID of data to XR_NULL_SPATIAL_BUFFER_ID_EXT .

Valid Usage (Implicit)

Component list structure to query data

The XrSpatialComponentMarkerListEXT structure is defined as:

typedef struct XrSpatialComponentMarkerListEXT {
    XrStructureType            type;
    void*                      next;
    uint32_t                   markerCount;
    XrSpatialMarkerDataEXT*    markers;
} XrSpatialComponentMarkerListEXT;

Member Descriptions

  • type is the XrStructureType of this structure.
  • next is NULL or a pointer to the next structure in a structure chain. No such structures are defined in core OpenXR or this extension.
  • markerCount is the number of elements in the markers member.
  • markers is an array of XrSpatialMarkerDataEXT .

The application can query the marker component of the spatial entities in an XrSpatialSnapshotEXT by adding XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT in XrSpatialComponentDataQueryConditionEXT :: componentTypes and adding XrSpatialComponentMarkerListEXT to the next pointer chain of XrSpatialComponentDataQueryResultEXT .

The runtime must return XR_ERROR_VALIDATION_FAILURE from xrQuerySpatialComponentDataEXT if XrSpatialComponentMarkerListEXT is in the next chain of XrSpatialComponentDataQueryResultEXT :: next but XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT is not included in XrSpatialComponentDataQueryConditionEXT :: componentTypes .

The runtime must return XR_ERROR_SIZE_INSUFFICIENT from xrQuerySpatialComponentDataEXT if markerCount is less than XrSpatialComponentDataQueryResultEXT :: entityIdCountOutput .

Valid Usage (Implicit)

Configuration

If XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT is enumerated in XrSpatialCapabilityComponentTypesEXT :: componentTypes for some capability, an application can enable it by including the enumerant in the XrSpatialCapabilityConfigurationBaseHeaderEXT :: enabledComponents list of the XrSpatialCapabilityConfigurationBaseHeaderEXT derived structure of the capability that supports this component.

This component does not require any special configuration to be included in the XrSpatialCapabilityConfigurationBaseHeaderEXT :: next chain.

Bounded 2D Component

The bounded 2D component provides the center and extents of the marker represented by the entity it is on. See Bounded 2D for more details about the bounded 2D component.

The XrSpatialBounded2DDataEXT :: center must point to the center of the marker. When looking at the front face of the marker, the X-axis must point to the right, and the Y-axis must point to the top of the marker. The runtime must follow the right-handed coordinate system convention thus the Z-axis comes out of the front face of the marker. This means that a marker with a position of {0, 0, 0}, rotation of {0, 0, 0, 1} (no rotation), and an extent of {1, 1} refers to a 1 meter x 1 meter marker centered at {0, 0, 0} with its front face normal vector pointing towards the +Z direction in the component’s space.

A representation of the orientation of the marker is shown below.

xr ml marker understanding axis

Figure 25. QR code marker with axis

Test Codes

The following codes must have their X-Y plane inside the document and the Z-axis pointing at the viewer. The axis origin must appear at the center of each marker. The X-axis must point to the right, the Y-axis must point to the top of the document.

ext marker tracking qr

Figure 26. QR code with text 'OpenXR'

ext marker tracking apriltag

Figure 27. AprilTag XR_SPATIAL_MARKER_APRIL_TAG_DICT_36H11_EXT with ID 42

ext marker tracking aruco

Figure 28. ArUco marker XR_SPATIAL_MARKER_ARUCO_DICT_5X5_50_EXT with ID 43

Example Code

Configure QR Code Tracking Capability

The following example code demonstrates how to configure the QR code tracking capability when creating a spatial context.

// Check if marker tracking capability is supported
uint32_t capabilityCount;
CHK_XR(xrEnumerateSpatialCapabilitiesEXT(instance, systemId, 0, &capabilityCount, nullptr));
std::vector<XrSpatialCapabilityEXT> capabilities(capabilityCount);
CHK_XR(xrEnumerateSpatialCapabilitiesEXT(instance, systemId, capabilityCount, &capabilityCount, capabilities.data()));

if (std::find(capabilities.begin(), capabilities.end(), XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT) == capabilities.end()) {
  return;
}

uint32_t featureCount = 0;
CHK_XR(xrEnumerateSpatialCapabilityFeaturesEXT(instance, systemId, XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT, 0, &featureCount, nullptr));
std::vector<XrSpatialCapabilityFeatureEXT> capabilityFeatures(featureCount);
CHK_XR(xrEnumerateSpatialCapabilityFeaturesEXT(instance, systemId, XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT, featureCount, &featureCount, capabilityFeatures.data()));

bool supportsFixedMarkerSize = std::find(capabilityFeatures.begin(), capabilityFeatures.end(), XR_SPATIAL_CAPABILITY_FEATURE_MARKER_TRACKING_FIXED_SIZE_MARKERS_EXT) != capabilityFeatures.end();

// Create a spatial context
XrSpatialContextEXT spatialContext{};

// Enable the 2 guaranteed components of the qr code tracking capability
std::vector<XrSpatialComponentTypeEXT> enabledComponents = {
  XR_SPATIAL_COMPONENT_TYPE_BOUNDED_2D_EXT,
  XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT,
};
XrSpatialCapabilityConfigurationQrCodeEXT markerConfiguration{XR_TYPE_SPATIAL_CAPABILITY_CONFIGURATION_QR_CODE_EXT};
markerConfiguration.capability = XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT;
markerConfiguration.enabledComponentCount = static_cast<uint32_t>(enabledComponents.size());
markerConfiguration.enabledComponents = enabledComponents.data();

// only chained if features.markerSideLength is true.
XrSpatialMarkerSizeEXT markerSize{XR_TYPE_SPATIAL_MARKER_SIZE_EXT};
markerSize.markerSideLength = 0.10f;
if (supportsFixedMarkerSize) {
  markerConfiguration.next = &markerSize;
}


std::array<XrSpatialCapabilityConfigurationBaseHeaderEXT*, 1> capabilityConfigs = {
  reinterpret_cast<XrSpatialCapabilityConfigurationBaseHeaderEXT*>(&markerConfiguration),
};

XrSpatialContextCreateInfoEXT spatialContextCreateInfo{XR_TYPE_SPATIAL_CONTEXT_CREATE_INFO_EXT};
spatialContextCreateInfo.capabilityConfigCount = capabilityConfigs.size();
spatialContextCreateInfo.capabilityConfigs = capabilityConfigs.data();
XrFutureEXT createContextFuture;
CHK_XR(xrCreateSpatialContextAsyncEXT(session, &spatialContextCreateInfo, &createContextFuture));

waitUntilReady(createContextFuture);

XrCreateSpatialContextCompletionEXT completion{XR_TYPE_CREATE_SPATIAL_CONTEXT_COMPLETION_EXT};
CHK_XR(xrCreateSpatialContextCompleteEXT(session, createContextFuture, &completion));
if (completion.futureResult != XR_SUCCESS) {
  return;
}

spatialContext = completion.spatialContext;

// ...
// Discovery entities with the spatial context
// ...

CHK_XR(xrDestroySpatialContextEXT(spatialContext));

Discover Spatial Entities & Query Component Data

The following example code demonstrates how to discover spatial entities for a context configured with XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT and query its component data.

XrFutureEXT future = XR_NULL_FUTURE_EXT;

// We want to look for entities that have the following components.
std::vector<XrSpatialComponentTypeEXT> snapshotComponents = {
  XR_SPATIAL_COMPONENT_TYPE_BOUNDED_2D_EXT,
  XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT,
};

auto discoverSpatialEntities = [&](XrSpatialContextEXT spatialContext, XrTime time) {
  XrSpatialDiscoverySnapshotCreateInfoEXT snapshotCreateInfo{XR_TYPE_SPATIAL_DISCOVERY_SNAPSHOT_CREATE_INFO_EXT};
  snapshotCreateInfo.componentTypeCount = snapshotComponents.size();
  snapshotCreateInfo.componentTypes = snapshotComponents.data();
  CHK_XR(xrCreateSpatialDiscoverySnapshotAsyncEXT(spatialContext, &snapshotCreateInfo, &future));

  waitUntilReady(future);

  XrCreateSpatialDiscoverySnapshotCompletionInfoEXT completionInfo{XR_TYPE_CREATE_SPATIAL_DISCOVERY_SNAPSHOT_COMPLETION_INFO_EXT};
  completionInfo.baseSpace = localSpace;
  completionInfo.time = time;
  completionInfo.future = future;

  XrCreateSpatialDiscoverySnapshotCompletionEXT completion{XR_TYPE_CREATE_SPATIAL_DISCOVERY_SNAPSHOT_COMPLETION_EXT};
  CHK_XR(xrCreateSpatialDiscoverySnapshotCompleteEXT(spatialContext, &completionInfo, &completion));
  if (completion.futureResult == XR_SUCCESS) {

    // Query for the bounded2D and marker component data
    XrSpatialComponentDataQueryConditionEXT queryCond{XR_TYPE_SPATIAL_COMPONENT_DATA_QUERY_CONDITION_EXT};
    queryCond.componentTypeCount = snapshotComponents.size();
    queryCond.componentTypes = snapshotComponents.data();

    XrSpatialComponentDataQueryResultEXT queryResult{XR_TYPE_SPATIAL_COMPONENT_DATA_QUERY_RESULT_EXT};
    CHK_XR(xrQuerySpatialComponentDataEXT(completion.snapshot, &queryCond, &queryResult));

    std::vector<XrSpatialEntityIdEXT> entityIds(queryResult.entityIdCountOutput);
    std::vector<XrSpatialEntityTrackingStateEXT> entityStates(queryResult.entityIdCountOutput);
    queryResult.entityIdCapacityInput = entityIds.size();
    queryResult.entityIds = entityIds.data();
    queryResult.entityStateCapacityInput = entityStates.size();
    queryResult.entityStates = entityStates.data();

    std::vector<XrSpatialBounded2DDataEXT> bounded2D(queryResult.entityIdCountOutput);
    XrSpatialComponentBounded2DListEXT bounded2DList{XR_TYPE_SPATIAL_COMPONENT_BOUNDED_2D_LIST_EXT};
    bounded2DList.boundCount = bounded2D.size();
    bounded2DList.bounds = bounded2D.data();
    queryResult.next = &bounded2DList;

    std::vector<XrSpatialMarkerDataEXT> markers;
    XrSpatialComponentMarkerListEXT markerList{XR_TYPE_SPATIAL_COMPONENT_MARKER_LIST_EXT};
    markers.resize(queryResult.entityIdCountOutput);
    markerList.markerCount = markers.size();
    markerList.markers = markers.data();
    bounded2DList.next = &markerList;

    CHK_XR(xrQuerySpatialComponentDataEXT(completion.snapshot, &queryCond, &queryResult));

    for (int32_t i = 0; i < queryResult.entityIdCountOutput; ++i) {
      if (entityStates[i] != XR_SPATIAL_ENTITY_TRACKING_STATE_TRACKING_EXT) {
        continue;
      }

      // 2D bounds for entity entityIds[i] is bounded2D[i].extents centered on bounded2D[i].center.

      if (markers[i].capability == XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT) {
        // Check if marker data has been decoded.
        if (markers[i].data.bufferId != XR_NULL_SPATIAL_BUFFER_ID_EXT) {
          if (markers[i].data.bufferType == XR_SPATIAL_BUFFER_TYPE_STRING_EXT) {
            // Qr Code data can be queried using
            // XrSpatialBufferGetInfoEXT getInfo{XR_TYPE_SPATIAL_BUFFER_GET_INFO_EXT};
            // info.bufferId = markers[i].data.bufferId;
            // xrGetSpatialBufferStringEXT(completion.snapshot, &getInfo, ...)
          } else if (markers[i].data.bufferType == XR_SPATIAL_BUFFER_TYPE_UINT8_EXT) {
            // Qr Code data can be queried using
            // XrSpatialBufferGetInfoEXT getInfo{XR_TYPE_SPATIAL_BUFFER_GET_INFO_EXT};
            // info.bufferId = markers[i].data.bufferId;
            // xrGetSpatialBufferUint8(completion.snapshot, &getInfo, ...)
          }
        }
      }
    }

    CHK_XR(xrDestroySpatialSnapshotEXT(completion.snapshot));
  }
};

while (1) {
  // ...
  // For every frame in frame loop
  // ...

  XrFrameState frameState;  // previously returned from xrWaitFrame
  const XrTime time = frameState.predictedDisplayTime;

  // Poll for the XR_TYPE_EVENT_DATA_SPATIAL_DISCOVERY_RECOMMENDED_EXT event
  XrEventDataBuffer event = {XR_TYPE_EVENT_DATA_BUFFER};
  XrResult result = xrPollEvent(instance, &event);
  if (result == XR_SUCCESS) {
      switch (event.type) {
          case XR_TYPE_EVENT_DATA_SPATIAL_DISCOVERY_RECOMMENDED_EXT: {
              const XrEventDataSpatialDiscoveryRecommendedEXT& eventdata =
                  *reinterpret_cast<XrEventDataSpatialDiscoveryRecommendedEXT*>(&event);
              // Discover spatial entities for the context that we received the "discovery
              // recommended" event for.
              discoverSpatialEntities(eventdata.spatialContext, time);
              break;
          }
      }
  }

  // ...
  // Finish frame loop
  // ...
}

New Structures

New Enums

New Enum Constants

  • XR_EXT_SPATIAL_MARKER_TRACKING_EXTENSION_NAME
  • XR_EXT_spatial_marker_tracking_SPEC_VERSION

  • Extending XrSpatialCapabilityEXT :

    • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_APRIL_TAG_EXT
    • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_ARUCO_MARKER_EXT
    • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_MICRO_QR_CODE_EXT
    • XR_SPATIAL_CAPABILITY_MARKER_TRACKING_QR_CODE_EXT

  • Extending XrSpatialCapabilityFeatureEXT :

    • XR_SPATIAL_CAPABILITY_FEATURE_MARKER_TRACKING_FIXED_SIZE_MARKERS_EXT
    • XR_SPATIAL_CAPABILITY_FEATURE_MARKER_TRACKING_STATIC_MARKERS_EXT

  • Extending XrSpatialComponentTypeEXT :

    • XR_SPATIAL_COMPONENT_TYPE_MARKER_EXT

  • Extending XrStructureType :

    • XR_TYPE_SPATIAL_CAPABILITY_CONFIGURATION_APRIL_TAG_EXT
    • XR_TYPE_SPATIAL_CAPABILITY_CONFIGURATION_ARUCO_MARKER_EXT
    • XR_TYPE_SPATIAL_CAPABILITY_CONFIGURATION_MICRO_QR_CODE_EXT
    • XR_TYPE_SPATIAL_CAPABILITY_CONFIGURATION_QR_CODE_EXT
    • XR_TYPE_SPATIAL_COMPONENT_MARKER_LIST_EXT
    • XR_TYPE_SPATIAL_MARKER_SIZE_EXT
    • XR_TYPE_SPATIAL_MARKER_STATIC_OPTIMIZATION_EXT

Version History

  • Revision 1, 2024-07-29 (Ron Bessems, Meta)

    • Initial extension description