Analyze images

The image analysis use case provides your app with a CPU-accessible image to perform image processing, computer vision, or machine learning inference on. The application implements an analyze method that is run on each frame.

Implementation

Images are processed by passing a method that takes an ImageProxy parameter and a rotation parameter to the setAnalyzer() method.

The following code example shows how to do this, as well as how to bind the image analysis use case and a preview use case to the LifecycleOwner. To read about creating preview use cases, see Implement a preview.

Upon return from this method, the image reference is closed. Therefore, the method should complete analysis or make a copy instead of passing the image reference beyond the analysis method.

Image analysis can work in two modes: blocking and non-blocking. Blocking mode is set using ImageAnalysis.ImageReaderMode.ACQUIRE_NEXT_IMAGE. In this mode, the Analyzer receives frames from the camera in sequential order; this means that, if the analyze method takes longer than the latency of a single frame at the current framerate, the frames may no longer be current since new frames are blocked from entering the pipeline until the method returns.

Non-blocking mode is set using ImageAnalysis.ImageReaderMode.ACQUIRE_LATEST_IMAGE. In this mode, the Analyzer receives the last available frame from the camera at the time that the analyze method is called. If the method takes longer than the latency of a single frame at the current framerate, some frames might be skipped so that the next time analyze receives data, it gets the last frame available in the camera pipeline. You can set the analyze method to run on a callback handler using ImageAnalysisConfig.Builder.setCallbackHandler, which allows the rest of the pipeline to run concurrently while the analyzer function runs. If you do not set a handler, the analyze method will run on the main thread.

val imageAnalysisConfig = ImageAnalysisConfig.Builder()
    .setTargetResolution(Size(1280, 720))
    .setImageReaderMode(ImageAnalysis.ImageReaderMode.ACQUIRE_LATEST_IMAGE)
    .build()
val imageAnalysis = ImageAnalysis(imageAnalysisConfig)

imageAnalysis.setAnalyzer({ image: ImageProxy, rotationDegrees: Int ->
    // insert your code here.
})

CameraX.bindToLifecycle(this as LifecycleOwner, imageAnalysis, preview)

ImageAnalysisConfig config =
    new ImageAnalysisConfig.Builder()
        .setTargetResolution(new Size(1280, 720))
        .setImageReaderMode(ImageAnalysis.ImageReaderMode.ACQUIRE_LATEST_IMAGE)
        .build();

ImageAnalysis imageAnalysis = new ImageAnalysis(config);

imageAnalysis.setAnalyzer(
    new ImageAnalysis.Analyzer() {
        @Override
        public void analyze(ImageProxy image, int rotationDegrees) {
            // insert your code here.
        }
});

CameraX.bindToLifecycle((LifecycleOwner) this, imageAnalysis, preview);

CameraX produces images in YUV_420_888 format.

Additional resources

To learn more about CameraX, consult the following additional resources.

Codelab

Code sample