# OverlayEffect

Artifact: [androidx.camera:camera-effects](https://developer.android.com/jetpack/androidx/releases/camera) [View Source](https://cs.android.com/search?q=file:androidx/camera/effects/OverlayEffect.java+class:androidx.camera.effects.OverlayEffect) Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

*** ** * ** ***


```
public class OverlayEffect extends CameraEffect implements AutoCloseable
```

<br />

|---|---|---|
| [java.lang.Object](https://developer.android.com/reference/java/lang/Object.html) |||
| ↳ | [androidx.camera.core.CameraEffect](https://developer.android.com/reference/androidx/camera/core/CameraEffect) ||
|   | ↳ | [androidx.camera.effects.OverlayEffect](https://developer.android.com/reference/androidx/camera/effects/OverlayEffect) |

*** ** * ** ***

A `https://developer.android.com/reference/androidx/camera/core/CameraEffect` for drawing overlay on top of the camera frames.

This class manages and processes camera frames with OpenGL. Upon arrival, frames are enqueued into an array of GL textures for deferred rendering. Calling `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` dequeues frames and renders them to the output. Additionally, when the texture queue reaches its capacity, the oldest frame is automatically dequeued and rendered. The size of the texture queue can be defined in the constructor.

The queuing mechanism provides the flexibility to postpone frame rendering until analysis results are available. For instance, to highlight on a QR code in a preview, one can apply a QR code detection algorithm using `https://developer.android.com/reference/androidx/camera/core/ImageAnalysis`. Once the frame's analysis result is ready, invoke `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` and pass in the `https://developer.android.com/reference/androidx/camera/core/ImageInfo#getTimestamp()` to release the frame and draw overlay. If the app doesn't render real-time analysis results, set the queue depth to 0 to avoid unnecessary buffer copies. For example, when laying over a static watermark.

Prior to rendering a frame, the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` invokes the listener set in `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#setOnDrawListener(androidx.arch.core.util.Function<androidx.camera.effects.Frame,java.lang.Boolean>)`. This listener provides a `https://developer.android.com/reference/androidx/camera/effects/Frame` object, which contains both a `https://developer.android.com/reference/android/graphics/Canvas.html` object for drawing the overlay and the metadata like crop rect, rotation degrees, etc to calculate how the overlay should be positioned. Once the listener returns, `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` updates the `https://developer.android.com/reference/android/graphics/SurfaceTexture.html` behind the `https://developer.android.com/reference/android/graphics/Canvas.html` and blends it with the camera frame before rendering to the output.

This class is thread-safe. The methods can be invoked on any thread. The app provides a `https://developer.android.com/reference/android/os/Handler.html` object in the constructor which is used for listening for Surface updates, performing OpenGL operations and invoking app provided listeners.

## Summary

| ### Constants |
|---|---|
| `static final int` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_CANCELLED_BY_CALLER() = 4` The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call failed because the caller cancelled the drawing. |
| `static final int` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_FRAME_NOT_FOUND() = 2` The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call failed because the frame with the exact timestamp was not found in the queue. |
| `static final int` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_INVALID_SURFACE() = 3` The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call failed because the output surface is missing, or the output surface no longer matches the frame. |
| `static final int` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_SUCCESS() = 1` The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call was successful. |

| ### Public constructors |
|---|
| `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#OverlayEffect(int,int,android.os.Handler,androidx.core.util.Consumer<java.lang.Throwable>)( int targets, int queueDepth, @https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/android/os/Handler.html handler, @https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/androidx/core/util/Consumer<https://developer.android.com/reference/java/lang/Throwable.html> errorListener )` Creates an `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect`. |

| ### Public methods |
|---|---|
| `void` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#clearOnDrawListener()()` Clears the listener set in `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#setOnDrawListener(androidx.arch.core.util.Function<androidx.camera.effects.Frame,java.lang.Boolean>)`. |
| `void` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#close()()` Closes the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect`. |
| `@https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://guava.dev/releases/snapshot-android/api/docs/com/google/common/util/concurrent/ListenableFuture.html<https://developer.android.com/reference/java/lang/Integer.html>` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)(long timestampNs)` Draws the queued frame with the given timestamp. |
| `@https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/android/os/Handler.html` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#getHandler()()` Gets the `https://developer.android.com/reference/android/os/Handler.html` set in the constructor. |
| `int` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#getQueueDepth()()` Gets the depth of the queue set in the constructor. |
| `void` | `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#setOnDrawListener(androidx.arch.core.util.Function<androidx.camera.effects.Frame,java.lang.Boolean>)(@https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/androidx/arch/core/util/Function<https://developer.android.com/reference/androidx/camera/effects/Frame, https://developer.android.com/reference/java/lang/Boolean.html> onDrawListener)` Sets the listener for drawing overlay. |

### Inherited Constants

From [androidx.camera.core.CameraEffect](https://developer.android.com/reference/androidx/camera/core/CameraEffect) `static final int` `https://developer.android.com/reference/androidx/camera/core/CameraEffect#IMAGE_CAPTURE() = 4`

Bitmask option to indicate that CameraX should apply this effect to `https://developer.android.com/reference/androidx/camera/core/ImageCapture`.
`static final int` `https://developer.android.com/reference/androidx/camera/core/CameraEffect#PREVIEW() = 1`

Bitmask option to indicate that CameraX should apply this effect to `https://developer.android.com/reference/androidx/camera/core/Preview`.
`static final int` `https://developer.android.com/reference/androidx/camera/core/CameraEffect#VIDEO_CAPTURE() = 2`

Bitmask option to indicate that CameraX should apply this effect to `VideoCapture`.

### Inherited methods

From [androidx.camera.core.CameraEffect](https://developer.android.com/reference/androidx/camera/core/CameraEffect) `@https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/androidx/core/util/Consumer<https://developer.android.com/reference/java/lang/Throwable.html>` `https://developer.android.com/reference/androidx/camera/core/CameraEffect#getErrorListener()()`

Gets the Error listener associated with this effect.
`@https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/java/util/concurrent/Executor.html` `https://developer.android.com/reference/androidx/camera/core/CameraEffect#getExecutor()()`

Gets the `https://developer.android.com/reference/java/util/concurrent/Executor.html` associated with this effect.
`@https://jspecify.dev/docs/api/org/jspecify/annotations/Nullable.html https://developer.android.com/reference/androidx/camera/core/SurfaceProcessor` `https://developer.android.com/reference/androidx/camera/core/CameraEffect#getSurfaceProcessor()()`

Gets the `https://developer.android.com/reference/androidx/camera/core/SurfaceProcessor` associated with this effect.
`int` `https://developer.android.com/reference/androidx/camera/core/CameraEffect#getTargets()()`

Ges the target `https://developer.android.com/reference/androidx/camera/core/UseCase`s of this effect.

## Constants

### RESULT_CANCELLED_BY_CALLER

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public static final int RESULT_CANCELLED_BY_CALLER = 4
```

The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call failed because the caller cancelled the drawing. This happens when the listener in `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#setOnDrawListener(androidx.arch.core.util.Function<androidx.camera.effects.Frame,java.lang.Boolean>)` returned false.

### RESULT_FRAME_NOT_FOUND

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public static final int RESULT_FRAME_NOT_FOUND = 2
```

The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call failed because the frame with the exact timestamp was not found in the queue. It could be one of the following reasons:

- the timestamp provided was incorrect, or
- the frame was removed because `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` had been called with a newer timestamp, or
- the frame was removed due to the queue being full.
If it's the last case, the caller may avoid this issue by increasing the queue depth.

### RESULT_INVALID_SURFACE

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public static final int RESULT_INVALID_SURFACE = 3
```

The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call failed because the output surface is missing, or the output surface no longer matches the frame. It can happen when the output Surface is replaced or disabled. For example, when the `https://developer.android.com/reference/androidx/camera/core/UseCase` was unbound.

### RESULT_SUCCESS

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public static final int RESULT_SUCCESS = 1
```

The `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#drawFrameAsync(long)` call was successful. The frame with the exact timestamp was drawn to the output surface.

## Public constructors

### OverlayEffect

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public OverlayEffect(
    int targets,
    int queueDepth,
    @NonNull Handler handler,
    @NonNull Consumer<Throwable> errorListener
)
```

Creates an `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect`.

| Parameters |
|---|---|
| `int targets` | The targets the effect applies to. For example, `https://developer.android.com/reference/androidx/camera/core/CameraEffect#PREVIEW()` \| `https://developer.android.com/reference/androidx/camera/core/CameraEffect#VIDEO_CAPTURE()`. See `https://developer.android.com/reference/androidx/camera/core/UseCaseGroup.Builder#addEffect(androidx.camera.core.CameraEffect)` for supported targets combinations. |
| `int queueDepth` | The depth of the queue. This value indicates how many frames can be queued before the oldest frame being automatically released. `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` allocates an array of OpenGL 2D textures that matches this size. The maximum value of the queueDepth depends on the size of the image and the device capabilities. Set a larger value if an ImageAnalysis processing takes a long time to produce a result to be used for overlay, so the frame is not auto-released before the result is ready. If the queue depth is 0, the input frames are rendered immediately without queuing. |
| `@https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/android/os/Handler.html handler` | The Handler for listening for the input Surface updates and for performing OpenGL operations. |
| `@https://jspecify.dev/docs/api/org/jspecify/annotations/NonNull.html https://developer.android.com/reference/androidx/core/util/Consumer<https://developer.android.com/reference/java/lang/Throwable.html> errorListener` | invoked if the effect runs into unrecoverable errors. The `https://developer.android.com/reference/java/lang/Throwable.html` will be the error thrown by this `https://developer.android.com/reference/androidx/camera/core/CameraEffect`. For example, `https://developer.android.com/reference/androidx/camera/core/ProcessingException`. This is invoked on the provided {@param Handler}. |

## Public methods

### clearOnDrawListener

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public void clearOnDrawListener()
```

Clears the listener set in `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#setOnDrawListener(androidx.arch.core.util.Function<androidx.camera.effects.Frame,java.lang.Boolean>)`.

### close

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public void close()
```

Closes the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect`.

Once closed, the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` can no longer be used.

### drawFrameAsync

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public @NonNull ListenableFuture<Integer> drawFrameAsync(long timestampNs)
```

Draws the queued frame with the given timestamp.

Once invoked, `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` retrieves the queued frame with the given timestamp and draws it to the output Surface. If the frame is successfully drawn, `https://guava.dev/releases/snapshot-android/api/docs/com/google/common/util/concurrent/ListenableFuture.html` completes with `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_SUCCESS()`. Otherwise, it completes with one of the following results: `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_FRAME_NOT_FOUND()`, `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_INVALID_SURFACE()` or `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#RESULT_CANCELLED_BY_CALLER()`. If this method is called after the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` is released, the `https://guava.dev/releases/snapshot-android/api/docs/com/google/common/util/concurrent/ListenableFuture.html` completes with an `https://developer.android.com/reference/java/lang/IllegalStateException.html`.

This method is thread safe. When calling from the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#getHandler()` thread, it's executed right away; otherwise, it posts the execution on the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#getHandler()`. It's recommended to call this method from the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#getHandler()` thread to avoid thread hopping.

### getHandler

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public @NonNull Handler getHandler()
```

Gets the `https://developer.android.com/reference/android/os/Handler.html` set in the constructor.

### getQueueDepth

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public int getQueueDepth()
```

Gets the depth of the queue set in the constructor.

### setOnDrawListener

Added in [1.4.0](https://developer.android.com/jetpack/androidx/releases/camera#1.4.0)

```
public void setOnDrawListener(@NonNull Function<Frame, Boolean> onDrawListener)
```

Sets the listener for drawing overlay.

Each time before `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` draws a frame to the output, the listener receives a `https://developer.android.com/reference/androidx/camera/effects/Frame` object, which contains the necessary APIs for drawing overlay.

To draw an overlay, first call `https://developer.android.com/reference/androidx/camera/effects/Frame#getOverlayCanvas()` ()} to get a `https://developer.android.com/reference/android/graphics/Canvas.html` object. The `https://developer.android.com/reference/android/graphics/Canvas.html` object is backed by a `https://developer.android.com/reference/android/graphics/SurfaceTexture.html` with the size of `https://developer.android.com/reference/androidx/camera/effects/Frame#getSize()`. `https://developer.android.com/reference/androidx/camera/effects/Frame#getSensorToBufferTransform()` represents the mapping from camera sensor coordinates to the frame's coordinates. To draw objects in the sensor coordinates, call `https://developer.android.com/reference/android/graphics/Canvas.html#setMatrix(android.graphics.Matrix)` with the value of `https://developer.android.com/reference/androidx/camera/effects/Frame#getSensorToBufferTransform()`.

Once the drawing is done, the listener should return true for the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` to draw it to the output Surface. If it returns false, the frame will be dropped.

`https://developer.android.com/reference/androidx/camera/effects/OverlayEffect` invokes the listener on the `https://developer.android.com/reference/androidx/camera/effects/OverlayEffect#getHandler()` thread.

| See also |
|---|---|
| `https://developer.android.com/reference/androidx/camera/effects/Frame` |   |