CameraX video capturing architecture

A capturing system generally records video and audio streams, compresses them, muxes the two streams, then writes the resultant stream to disk.

In CameraX, the solution for video capturing is the VideoCapture use case:

As shown in figure 2, CameraX video capture includes a few high-level architectural components:

• SurfaceProvider for the video source.
• AudioSource for audio source.
• Two encoders to encode and compress video/audio.
• A media muxer to mux the two streams.
• A file saver to write out the result.

The VideoCapture API abstracts the complex capturing engine and provides applications with a much simpler and straightforward API.

VideoCapture API overview

VideoCapture is a CameraX use case that works well on its own or when combined with other use cases. Specific supported combinations depend on the camera hardware capabilities, but Preview and VideoCapture is a valid use case combination on all devices.

The VideoCapture API consists of the following objects that communicate with applications:

• VideoCapture is the top-level use case class. VideoCapture binds to a LifecycleOwner with a CameraSelector and other CameraX UseCases. For more information about these concepts and usages, see CameraX Architecture.
• A Recorder is an implementation of VideoOutput that is tightly coupled with VideoCapture. Recorder is used to perform the video and audio capturing. An application creates recordings from a Recorder.
• A PendingRecording configures a recording, providing options like enabling audio and setting an event listener. You must use a Recorder to create a PendingRecording. A PendingRecording does not record anything.
• A Recording performs the actual recording. You must use a PendingRecording to create a Recording.

Figure 3 shows the relationships between these objects:

Legend:

1. Create a Recorder with QualitySelector.
2. Configure the Recorder with one of the OutputOptions.
3. Enable audio with withAudioEnabled() if needed.
4. Call start() with a VideoRecordEvent listener to begin recording.
5. Use pause()/resume()/stop() on the Recording to control the recording.
6. Respond to VideoRecordEvents inside your event listener.

The detailed API list is in the current.txt inside the source code.

Using the VideoCapture API

To integrate the CameraX VideoCapture use case into your app, do the following:

1. Bind VideoCapture.
2. Prepare and configure recording.
3. Start and control the runtime recording.

The following sections outline what you can do at each step to get an end-to-end recording session.

Bind VideoCapture

To bind the VideoCapture use case, do the following:

1. Create a Recorder object.
2. Create VideoCapture object.
3. Bind to a Lifecycle.

CameraX VideoCapture API follows the builder design pattern. Applications use Recorder.Builder to create a Recorder. You can also configure the video resolution for the Recorder through a QualitySelector object.

CameraX Recorder supports the following pre-defined Qualities for video resolutions:

• Quality.UHD for 4K ultra HD video size (2160p)
• Quality.QHD for quad HD video size (1440p), introduced in CameraX 1.7.0-alpha02
• Quality.FHD for full HD video size (1080p)
• Quality.HD for HD video size (720p)
• Quality.SD for SD video size (480p)

Note that CameraX can also choose other resolutions when authorized by the app.

The exact video size of each selection depends on the camera and encoder's capabilities. For more information, see the documentation for CamcorderProfile.

Applications can configure resolution by creating a QualitySelector. You can create a QualitySelector using one of the following methods:

• Provide a few preferred resolutions by using fromOrderedList(), and include a fallback strategy to use in case none of the preferred resolutions is supported.

  CameraX can decide the best fallback match based on the selected camera's capability, refer to QualitySelector's FallbackStrategy specification for more details. For example, the following code requests the highest supported resolution for recording, and if none of the request resolutions can be supported, authorize CameraX to choose one that is the closest to the Quality.SD resolution:

  val qualitySelector = QualitySelector.fromOrderedList(
           listOf(Quality.UHD, Quality.QHD, Quality.FHD, Quality.HD, Quality.SD),
           FallbackStrategy.lowerQualityOrHigherThan(Quality.SD))
  

• Query the camera capabilities first, and choose from the supported resolutions using QualitySelector::from():

  val cameraInfo = cameraProvider.availableCameraInfos.filter {
      Camera2CameraInfo
      .from(it)
      .getCameraCharacteristic(CameraCharacteristics.LENS\_FACING) == CameraMetadata.LENS_FACING_BACK
  }
  
  val supportedQualities = QualitySelector.getSupportedQualities(cameraInfo[0])
  val filteredQualities = arrayListOf (Quality.UHD, Quality.QHD, Quality.FHD, Quality.HD, Quality.SD)
                         .filter { supportedQualities.contains(it) }
  
  // Use a simple ListView with the id of simple_quality_list_view
  viewBinding.simpleQualityListView.apply {
      adapter = ArrayAdapter(context,
                             android.R.layout.simple_list_item_1,
                             filteredQualities.map { it.qualityToString() })
  
      // Set up the user interaction to manually show or hide the system UI.
      setOnItemClickListener { _, _, position, _ ->
          // Inside View.OnClickListener,
          // convert Quality.* constant to QualitySelector
          val qualitySelector = QualitySelector.from(filteredQualities[position])
  
          // Create a new Recorder/VideoCapture for the new quality
          // and bind to lifecycle
          val recorder = Recorder.Builder()
              .setQualitySelector(qualitySelector).build()
  
           // ...
      }
  }
  
  // A helper function to translate Quality to a string
  fun Quality.qualityToString() : String {
      return when (this) {
          Quality.UHD -> "UHD"
          Quality.QHD -> "QHD"
          Quality.FHD -> "FHD"
          Quality.HD -> "HD"
          Quality.SD -> "SD"
          else -> throw IllegalArgumentException()
      }
  }
  
  

  Note that the returned capability from QualitySelector.getSupportedQualities() is guaranteed to work for either the VideoCapture use case or the combination of VideoCapture and Preview use cases. When binding together with ImageCapture or ImageAnalysis use case, CameraX might still fail the binding when the required combination is not supported on the requested camera.

Once you have a QualitySelector, the application can create a VideoCapture object and perform the binding. Note that this binding is the same as with other use cases:

val recorder = Recorder.Builder()
    .setExecutor(cameraExecutor).setQualitySelector(qualitySelector)
    .build()
val videoCapture = VideoCapture.withOutput(recorder)

try {
    // Bind use cases to camera
    cameraProvider.bindToLifecycle(
            this, CameraSelector.DEFAULT_BACK_CAMERA, preview, videoCapture)
} catch(exc: Exception) {
    Log.e(TAG, "Use case binding failed", exc)
}

Note that bindToLifecycle() returns a Camera object. See this guide for more information about controlling camera output, such as zoom and exposure.

The Recorder selects the most suitable format for the system. The most common video codec is H.264 AVC) with container format MPEG-4.

Configure and create recording

From a Recorder, the application can create recording objects to perform the video and audio capturing. Applications create recordings by doing the following:

1. Configure OutputOptions with the prepareRecording().
2. (Optional) Enable audio recording.
3. Use start() to register a VideoRecordEvent listener, and begin video capturing.

TheRecorder returns a Recording object when you call the start() function. Your application can use this Recording object to finish capturing or to perform other actions, such as pausing or resuming.

A Recorder supports one Recording object at a time. You can start a new recording once you've called Recording.stop() or Recording.close() on the previous Recording object.

Let's look at these steps in more detail. First, the application configures the OutputOptions for a Recorder with Recorder.prepareRecording(). A Recorder supports the following types of OutputOptions:

• FileDescriptorOutputOptions for capturing into a FileDescriptor.
• FileOutputOptions for capturing into a File.
• MediaStoreOutputOptions for capturing into a MediaStore.

All OutputOptions types enable you to set a maximum file size with setFileSizeLimit(). Other options are specific to the individual output type, such as ParcelFileDescriptor for the FileDescriptorOutputOptions.

prepareRecording() returns a PendingRecording object, which is an intermediate object that is used to create the corresponding Recording object. PendingRecording is a transient class that should be invisible in most cases and is rarely cached by the app.

Applications can further configure the recording, such as:

• Enable audio with withAudioEnabled().
• Register a listener to receive video recording events with start(Executor, Consumer<VideoRecordEvent>).
• Allow a recording to continuously record while the VideoCapture it's attached to is rebound to another camera, with PendingRecording.asPersistentRecording().

To start recording, call PendingRecording.start(). CameraX turns the PendingRecording into a Recording, queues the recording request, and returns the newly created Recording object to the application. Once recording begins on the corresponding Camera device, CameraX sends a VideoRecordEvent.EVENT_TYPE_START event.

The following example shows how to record video and audio into a MediaStore file:

// Create MediaStoreOutputOptions for our recorder
val name = "CameraX-recording-" +
        SimpleDateFormat(FILENAME_FORMAT, Locale.US)
                .format(System.currentTimeMillis()) + ".mp4"
val contentValues = ContentValues().apply {
   put(MediaStore.Video.Media.DISPLAY_NAME, name)
}
val mediaStoreOutput = MediaStoreOutputOptions.Builder(this.contentResolver,
                              MediaStore.Video.Media.EXTERNAL_CONTENT_URI)
                              .setContentValues(contentValues)
                              .build()

// 2. Configure Recorder and Start recording to the mediaStoreOutput.
val recording = videoCapture.output
                .prepareRecording(context, mediaStoreOutput)
                .withAudioEnabled()
                .start(ContextCompat.getMainExecutor(this), captureListener)

While the camera preview is mirrored on the front camera by default, videos recorded by VideoCapture are not mirrored by default. With CameraX 1.3, it's now possible to mirror video recordings so that the front camera preview and the recorded video match.

There are three MirrorMode options: MIRROR_MODE_OFF, MIRROR_MODE_ON, and MIRROR_MODE_ON_FRONT_ONLY. To align to the camera preview, Google recommends using MIROR_MODE_ON_FRONT_ONLY, which means that mirroring is not enabled for the rear camera, but is enabled for the front camera. For more information about MirrorMode, see MirrorMode constants.

This code snippet shows how to call VideoCapture.Builder.setMirrorMode() using MIRROR_MODE_ON_FRONT_ONLY. For more information, see setMirrorMode().

val recorder = Recorder.Builder().build()

val videoCapture = VideoCapture.Builder(recorder)
    .setMirrorMode(MIRROR_MODE_ON_FRONT_ONLY)
    .build()

useCases.add(videoCapture);

Recorder.Builder builder = new Recorder.Builder();
if (mVideoQuality != QUALITY_AUTO) {
    builder.setQualitySelector(
        QualitySelector.from(mVideoQuality));
}
  VideoCapture<Recorder> videoCapture = new VideoCapture.Builder<>(builder.build())
      .setMirrorMode(MIRROR_MODE_ON_FRONT_ONLY)
      .build();
    useCases.add(videoCapture);

Dynamic range (HDR video)

CameraX supports 10-bit high dynamic range (HDR) video recording, such as HLG10, HDR10, HDR10+, or Dolby Vision, using DynamicRange.

To use HDR video recording, complete the following steps:

1. Query the camera's supported dynamic ranges using VideoCapabilities.getSupportedDynamicRanges().
2. Optional: If you configure a custom QualitySelector, check which qualities are supported for that dynamic range using getSupportedQualities(DynamicRange).
3. Configure VideoCapture.Builder with setDynamicRange().

The following example enables HLG 10-bit HDR recording when the camera supports it:

val videoCapabilities = Recorder.getVideoCapabilities(cameraInfo)
val supportedHdrRanges = videoCapabilities.supportedDynamicRanges
    .filter { it != DynamicRange.SDR }

val recorder = Recorder.Builder().build()
val videoCapture = VideoCapture.Builder(recorder)
    .apply {
        if (supportedHdrRanges.contains(DynamicRange.HLG_10_BIT)) {
            setDynamicRange(DynamicRange.HLG_10_BIT)
        }
    }
    .build()

VideoCapabilities videoCapabilities = Recorder.getVideoCapabilities(cameraInfo);
Set<DynamicRange> supportedRanges = videoCapabilities.getSupportedDynamicRanges();

Recorder recorder = new Recorder.Builder().build();
VideoCapture.Builder<Recorder> builder = new VideoCapture.Builder<>(recorder);
if (supportedRanges.contains(DynamicRange.HLG_10_BIT)) {
    builder.setDynamicRange(DynamicRange.HLG_10_BIT);
}
VideoCapture<Recorder> videoCapture = builder.build();

Video stabilization

CameraX supports video stabilization to reduce handheld shaking and jitter during video recording.

To enable video stabilization, complete the following steps:

1. Check whether the camera device supports stabilization using VideoCapabilities.isStabilizationSupported().
2. Enable it on VideoCapture.Builder using setVideoStabilizationEnabled(true).

The following example enables stabilization when the camera supports it:

val videoCapabilities = Recorder.getVideoCapabilities(cameraInfo)
val videoCapture = VideoCapture.Builder(recorder)
    .setVideoStabilizationEnabled(videoCapabilities.isStabilizationSupported)
    .build()

VideoCapabilities videoCapabilities = Recorder.getVideoCapabilities(cameraInfo);
VideoCapture<Recorder> videoCapture = new VideoCapture.Builder<>(recorder)
    .setVideoStabilizationEnabled(videoCapabilities.isStabilizationSupported())
    .build();

Control an active recording

You can pause, resume, and stop an ongoing Recording by using the following methods:

• pause to pause the current active recording.
• resume() to resume a paused active recording.
• stop() to finish recording and flush any associated recording objects.
• mute() to mute or un-mute the current recording.

Note that you can call stop() to terminate a Recording regardless of whether the recording is in a paused or active recording state.

If you've registered an EventListener with PendingRecording.start(), the Recording communicates by using a VideoRecordEvent.

• VideoRecordEvent.EVENT_TYPE_STATUS is used for recording statistics such as current file size and recorded time span.
• VideoRecordEvent.EVENT_TYPE_FINALIZE is used for the recording result and includes information such as the URI of the final file along with any related errors.

Once your app receives a EVENT_TYPE_FINALIZE that indicates a successful recording session, you can then access the captured video from the location specified in OutputOptions.

Real-time audio amplitude and state

During recording, VideoRecordEvent.Status events provide real-time audio statistics through AudioStats:

• AudioStats.getAudioAmplitude() returns the current audio peak amplitude, ranging from 0.0 to 1.0. You can use this value to drive UI meters or audio waveform visualizers.
• AudioStats.getAudioState() indicates the current status of the audio stream, such as AUDIO_STATE_ACTIVE, AUDIO_STATE_MUTED, or AUDIO_STATE_SOURCE_SILENCED.

The following example reads the audio amplitude from a status event:

when (event) {
    is VideoRecordEvent.Status -> {
        val audioStats = event.recordingStats.audioStats
        if (audioStats.hasAudio()) {
            val amplitude = audioStats.audioAmplitude
            updateAudioWaveform(amplitude)
        }
    }
    // ...
}

if (event instanceof VideoRecordEvent.Status) {
    VideoRecordEvent.Status status = (VideoRecordEvent.Status) event;
    AudioStats audioStats = status.getRecordingStats().getAudioStats();
    if (audioStats.hasAudio()) {
        double amplitude = audioStats.getAudioAmplitude();
        updateAudioWaveform(amplitude);
    }
}

High-speed and slow-motion video

CameraX supports high-speed video recording, such as 120, 240, or 480 fps, which enables two distinct use cases:

• High-frame-rate video (player-handled slow motion): CameraX captures and saves frames at the high frame rate (for example, 120 fps). The video player controls playback speed, allowing users to toggle between normal speed and slow motion or perform frame-by-frame analysis.
• Ready-to-play slow-motion video: CameraX processes the high-speed stream and outputs a standard 30 fps video file with adjusted frame timestamps. The slow-motion effect is "baked in", so the video plays in slow motion automatically in any standard player without special player support. For example, a 120 fps capture played back at 30 fps produces a 1/4x slow-motion video.

Check high-speed capabilities

High-speed video capture requires hardware support and corresponding CamcorderProfile definitions from device manufacturers, such as CamcorderProfile.QUALITY_HIGH_SPEED_1080P or QUALITY_HIGH_SPEED_720P.

Before configuring high-speed recording, query high-speed capabilities using Recorder.getHighSpeedVideoCapabilities(CameraInfo):

val highSpeedCapabilities = Recorder.getHighSpeedVideoCapabilities(cameraInfo)
if (highSpeedCapabilities == null) {
    // Camera does not support high-speed video capture
    return
}
val supportedQualities = highSpeedCapabilities.getSupportedQualities(DynamicRange.SDR)

VideoCapabilities highSpeedCapabilities = Recorder.getHighSpeedVideoCapabilities(cameraInfo);
if (highSpeedCapabilities == null) {
    // Camera does not support high-speed video capture
    return;
}
List<Quality> supportedQualities = highSpeedCapabilities.getSupportedQualities(DynamicRange.SDR);

Configure HighSpeedVideoSessionConfig

To use high-speed capture, combine your VideoCapture and Preview use cases in a HighSpeedVideoSessionConfig. To configure the session, complete the following steps:

1. Create a VideoCapture use case and a Preview use case.
2. Initialize HighSpeedVideoSessionConfig.Builder with the VideoCapture use case and attach the Preview.
3. Query the supported frame rate ranges using CameraInfo.getSupportedFrameRateRanges(SessionConfig) and set the selected range.
4. Choose between high-frame-rate video or ready-to-play slow-motion video:
   • To create a high-frame-rate video, leave setSlowMotionEnabled(false) (default).
   • To create a ready-to-play slow-motion video, call setSlowMotionEnabled(true).
5. Bind the session config using cameraProvider.bindToLifecycle().

The following example configures and binds a ready-to-play slow-motion session:

val preview = Preview.Builder().build()
val recorder = Recorder.Builder()
    .setQualitySelector(QualitySelector.from(supportedQualities.first()))
    .build()
val videoCapture = VideoCapture.withOutput(recorder)

// Build HighSpeedVideoSessionConfig
val sessionConfigBuilder = HighSpeedVideoSessionConfig.Builder(videoCapture)
    .setPreview(preview)
    // Enable automatic slow-motion playback at 30 fps
    .setSlowMotionEnabled(true)

// Query and set a supported high-speed frame rate range (such as 120 or 240 fps).
// The set is non-empty because getHighSpeedVideoCapabilities() returned non-null earlier.
val supportedFpsRanges = cameraInfo.getSupportedFrameRateRanges(sessionConfigBuilder.build())
sessionConfigBuilder.setFrameRateRange(supportedFpsRanges.first())

val sessionConfig = sessionConfigBuilder.build()

// Bind session config to lifecycle
cameraProvider.bindToLifecycle(
    lifecycleOwner,
    CameraSelector.DEFAULT_BACK_CAMERA,
    sessionConfig
)

Preview preview = new Preview.Builder().build();
Recorder recorder = new Recorder.Builder()
    .setQualitySelector(QualitySelector.from(supportedQualities.get(0)))
    .build();
VideoCapture<Recorder> videoCapture = VideoCapture.withOutput(recorder);

HighSpeedVideoSessionConfig.Builder sessionConfigBuilder =
    new HighSpeedVideoSessionConfig.Builder(videoCapture)
        .setPreview(preview)
        .setSlowMotionEnabled(true);

// The set is non-empty because getHighSpeedVideoCapabilities() returned non-null earlier.
Set<Range<Integer>> supportedFpsRanges =
    cameraInfo.getSupportedFrameRateRanges(sessionConfigBuilder.build());
sessionConfigBuilder.setFrameRateRange(supportedFpsRanges.iterator().next());

HighSpeedVideoSessionConfig sessionConfig = sessionConfigBuilder.build();

cameraProvider.bindToLifecycle(
    lifecycleOwner,
    CameraSelector.DEFAULT_BACK_CAMERA,
    sessionConfig
);

You can also use the Kotlin DSL, introduced in CameraX 1.7.0-alpha02, to configure high-speed sessions concisely:

val sessionConfig = highSpeedVideoSessionConfig(videoCapture) {
    this.preview = preview
    isSlowMotionEnabled = true
    frameRateRange = supportedFpsRanges.first()
}
cameraProvider.bindToLifecycle(
    lifecycleOwner,
    CameraSelector.DEFAULT_BACK_CAMERA,
    sessionConfig
)

Record high-speed or slow-motion video

After you bind the HighSpeedVideoSessionConfig, start recording using VideoCapture in the same way as standard video recordings:

val recording = videoCapture.output
    .prepareRecording(context, mediaStoreOutput)
    .withAudioEnabled()
    .start(ContextCompat.getMainExecutor(context)) { event ->
        when (event) {
            is VideoRecordEvent.Start -> Log.d(TAG, "Recording started")
            is VideoRecordEvent.Finalize -> {
                if (!event.hasError()) {
                    Log.d(TAG, "Video saved: ${event.outputResults.outputUri}")
                }
            }
        }
    }

Recording recording = videoCapture.getOutput()
    .prepareRecording(context, mediaStoreOutput)
    .withAudioEnabled()
    .start(ContextCompat.getMainExecutor(context), event -> {
        if (event instanceof VideoRecordEvent.Start) {
            Log.d(TAG, "Recording started");
        } else if (event instanceof VideoRecordEvent.Finalize) {
            VideoRecordEvent.Finalize finalize = (VideoRecordEvent.Finalize) event;
            if (!finalize.hasError()) {
                Log.d(TAG, "Video saved: " + finalize.getOutputResults().getOutputUri());
            }
        }
    });

Additional resources

To learn more about CameraX, see the following additional resources:

• Getting Started with CameraX Codelab
• Official CameraX sample app
• Latest CameraX Video Capture API List
• CameraX release notes
• CameraX source code
• High-Speed Capture and Slow-Motion Video with CameraX 1.5