Analyze images

The image analysis use case provides your app with a CPU-accessible image to perform image processing, computer vision, or machine learning inference on. The application implements an analyze method that is run on each frame.

Implementation

Images are processed by passing an executor in which the image analysis is run and an ImageAnalysis.Analyzer parameter to the setAnalyzer() method.

The code example in this topic shows how to do this, as well as how to bind the image analysis use case and a preview use case to the LifecycleOwner. To read about creating preview use cases, see Implement a preview.

Image analysis can work in two modes: blocking and non-blocking. Blocking mode is enabled by calling setBackpressureStrategy() with STRATEGY_BLOCK_PRODUCER. In this mode, the executor receives frames from the camera in sequential order; this means that, if the analyze() method takes longer than the latency of a single frame at the current framerate, the frames may no longer be current since new frames are blocked from entering the pipeline until the method returns.

Non-blocking mode is enabled by calling setBackpressureStrategy() with STRATEGY_KEEP_ONLY_LATEST. In this mode, the executor receives the last available frame from the camera at the time that the analyze() method is called. If the method takes longer than the latency of a single frame at the current framerate, some frames might be skipped so that the next time analyze() receives data, it gets the last frame available in the camera pipeline.

Before returning from analyze(), close the image reference by calling image.close() to avoid blocking the production of further images (causing the preview to stall) and to avoid potentially dropping images. The method must complete analysis or make a copy instead of passing the image reference beyond the analysis method.

val imageAnalysis = ImageAnalysis.Builder()
    .setTargetResolution(Size(1280, 720))
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .build()

imageAnalysis.setAnalyzer(executor, ImageAnalysis.Analyzer { image ->
    val rotationDegrees = image.imageInfo.rotationDegrees
    // insert your code here.
})

cameraProvider.bindToLifecycle(this as LifecycleOwner, cameraSelector, imageAnalysis, preview)

ImageAnalysis imageAnalysis =
    new ImageAnalysis.Builder()
        .setTargetResolution(new Size(1280, 720))
        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
        .build();

imageAnalysis.setAnalyzer(executor, new ImageAnalysis.Analyzer() {
    @Override
    public void analyze(@NonNull ImageProxy image) {
        int rotationDegrees = image.getImageInfo().getRotationDegrees();
            // insert your code here.
        }
    });


cameraProvider.bindToLifecycle((LifecycleOwner) this, cameraSelector, imageAnalysis, preview);

CameraX produces images in YUV_420_888 format.

Additional resources

To learn more about CameraX, consult the following additional resources.

Codelab

Code sample