SurfaceOutput

abstract void close()

Call this method to mark the Surface as no longer in use.

Once the SurfaceProcessor implementation receives a request to close the Surface, it should call this method to acknowledge after stop writing to the Surface. Writing to the Surface after calling this method might cause errors.

default @NonNull Matrix getSensorToBufferTransform()

Returns the sensor to image buffer transform matrix.

The value is a mapping from sensor coordinates to buffer coordinates, which is, from the rect of SENSOR_INFO_ACTIVE_ARRAY_SIZE to the rect defined by (0, 0, #getSize()#getWidth(), #getSize()#getHeight()). The matrix can be used to map the coordinates from one UseCase to another. For example, detecting face with ImageAnalysis, and then highlighting the face in Preview.

Code sample

 // Get the transformation from sensor to effect output.
 Matrix sensorToEffect = surfaceOutput.getSensorToBufferTransform();
 // Get the transformation from sensor to ImageAnalysis.
 Matrix sensorToAnalysis = imageProxy.getSensorToBufferTransform();
 // Concatenate the two matrices to get the transformation from ImageAnalysis to effect.
 Matrix analysisToEffect = Matrix()
 sensorToAnalysis.invert(analysisToEffect);
 analysisToEffect.postConcat(sensorToEffect);

abstract @NonNull Size getSize()

Gets the size of the Surface.

abstract @NonNull Surface getSurface(
    @NonNull Executor executor,
    @NonNull Consumer<SurfaceOutput.Event> listener
)

Gets the Surface for drawing processed frames.

If there are multiple calls to the method, only the < from the last call will be triggered.

abstract int getTargets()

This field indicates that what purpose the Surface will be used for.

PREVIEW if the Surface will be used for Preview.

abstract void updateTransformMatrix(@NonNull float[] updated, @NonNull float[] original)

Applies an additional 4x4 transformation on the original matrix.

When the input Surface of SurfaceProcessor is backed by a SurfaceTexture, use this method to update the texture transform matrix.

Typically, after retrieving the transform matrix from getTransformMatrix, the SurfaceProcessor implementation should always call this method to update the value. The result is a matrix of the same format, which is a transform matrix maps 2D homogeneous texture coordinates of the form (s, t, 0, 1) with s and t in the inclusive range [0, 1] to the texture coordinate that should be used to sample that location from the texture. The matrix is stored in column-major order so that it may be passed directly to OpenGL ES via the glLoadMatrixf or glUniformMatrix4fv functions.

The additional transformation is calculated based on the target rotation, target resolution and the ViewPort associated with the target UseCase. The value could also include workarounds for device specific bugs. For example, correcting a stretched camera output stream.

Code sample:

float[] transform = new float[16];
float[] updatedTransform = new float[16];

surfaceTexture.setOnFrameAvailableListener(surfaceTexture -> {
    surfaceTexture.getTransformMatrix(transform);
    outputSurface.updateTransformMatrix(updatedTransform, transform);
    // Use the value of updatedTransform for OpenGL rendering.
});

To get the value of the additional transformation, pass in an identity matrix as the original value. This is useful when getTransformMatrix is not applied by the implementation.

Code sample:

float[] identity = new float[16];
Matrix.setIdentityM(identity, 0);
float[] updatedTransform = new float[16];

surfaceTexture.setOnFrameAvailableListener(surfaceTexture -> {
    outputSurface.updateTransformMatrix(updatedTransform, identity);
    // Use the value of updatedTransform for OpenGL rendering.
});