同時使用多個相機串流畫面

注意：本頁面所述是指 Camera2 套件。除非應用程式需要 Camera2 的特定低階功能，否則建議使用 CameraX。CameraX 和 Camera2 均支援 Android 5.0 (API 級別 21) 以上版本。

相機應用程式可以同時使用多個影格串流。於 在某些情況中，不同串流甚至需要不同的影格解析度或像素 格式。常見的用途包括：

  • 錄影：一個用於預覽的串流，另一個則進行編碼並儲存 複製到檔案中
  • 條碼掃描：一個用於預覽的串流，另一個用於偵測條碼。
  • 計算攝影：一個用於預覽影像的串流，另一個用於臉部/場景拍攝 偵測。

處理影格時產生的效能相當低廉，但成本會降低 是平行串流或管道處理時的負荷倍數。

CPU、GPU 和 DSP 等 架構的重新處理 功能，但記憶體等資源會呈現線性成長。

每項要求多個目標

多個相機串流可合併為單一畫面 CameraCaptureRequest。 下列程式碼片段說明如何設定攝影機工作階段 用於相機預覽的串流，以及另一個用於處理影像的串流：

Kotlin

val session: CameraCaptureSession = ...  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
val combinedRequest = session.device.createCaptureRequest(requestTemplate)

// Link the Surface targets with the combined request
combinedRequest.addTarget(previewSurface)
combinedRequest.addTarget(imReaderSurface)

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
session.setRepeatingRequest(combinedRequest.build(), null, null)

Java

CameraCaptureSession session = …;  // from CameraCaptureSession.StateCallback

// You will use the preview capture template for the combined streams
// because it is optimized for low latency; for high-quality images, use
// TEMPLATE_STILL_CAPTURE, and for a steady frame rate use TEMPLATE_RECORD
        CaptureRequest.Builder combinedRequest = session.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);

// Link the Surface targets with the combined request
        combinedRequest.addTarget(previewSurface);
        combinedRequest.addTarget(imReaderSurface);

// In this simple case, the SurfaceView gets updated automatically. ImageReader
// has its own callback that you have to listen to in order to retrieve the
// frames so there is no need to set up a callback for the capture request
        session.setRepeatingRequest(combinedRequest.build(), null, null);

如果您正確設定目標介面，則此程式碼只會產生 符合最低 FPS 的標準 StreamComfigurationMap.GetOutputMinFrameDuration(int, Size)敬上 和 StreamComfigurationMap.GetOutputStallDuration(int, Size)。 實際效能會因裝置而異，但 Android 提供 保證能夠根據以下三種變數支援特定組合： 「output type」、「output size」和「Hardware level」。

使用不支援的變數組合可能會導致影格速率低落；如果 否則會觸發其中一個失敗回呼。 createCaptureSession 說明文件 說明保證能運作的內容

輸出類型

「輸出類型」是指影格編碼的格式。 可能為 PRIV、YUV、JPEG 和 RAW。 createCaptureSession敬上 可以說明這些情境

選擇應用程式的輸出類型時，如果的目標是 相容性，然後使用 ImageFormat.YUV_420_888敬上 影格分析和 ImageFormat.JPEG 仍待機 所以映像檔較小在預覽和錄製情況下，您可能會用到 SurfaceView、 TextureView、 MediaRecorder、 MediaCodec，或 RenderScript.Allocation。於 在這種情況下，請勿指定圖片格式基於相容性考量，系統會以 ImageFormat.PRIVATE、 無論內部使用的實際格式為何查詢支援的格式 特定裝置提供的 CameraCharacteristics、 請使用以下程式碼：

Kotlin

val characteristics: CameraCharacteristics = ...
val supportedFormats = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).outputFormats

Java

CameraCharacteristics characteristics = …;
        int[] supportedFormats = characteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP).getOutputFormats();

輸出大小

系統會列出所有可用的輸出大小 StreamConfigurationMap.getOutputSizes()、 但只有兩個與相容性有關：PREVIEW 和 MAXIMUM。大小 做為上限如果 PREVIEW 大小有效，則只要具備 值小於 PREVIEW 也能解決問題MAXIMUM 也是如此。 說明文件 CameraDevice敬上 說明這些大小

可用的輸出大小取決於所選格式。由於 CameraCharacteristics敬上 和格式，您就能查詢可用的輸出大小，如下所示：

Kotlin

val characteristics: CameraCharacteristics = ...
val outputFormat: Int = ...  // such as ImageFormat.JPEG
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(outputFormat)

Java

CameraCharacteristics characteristics = …;
        int outputFormat = …;  // such as ImageFormat.JPEG
Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

在相機預覽和記錄用途中，請使用目標類別來判斷 支援的大小。格式會由相機架構自行處理：

Kotlin

val characteristics: CameraCharacteristics = ...
val targetClass: Class <T> = ...  // such as SurfaceView::class.java
val sizes = characteristics.get(
    CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
    .getOutputSizes(targetClass)

Java

CameraCharacteristics characteristics = …;
   int outputFormat = …;  // such as ImageFormat.JPEG
   Size[] sizes = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
                .getOutputSizes(outputFormat);

如要取得 MAXIMUM 大小，請依面積將輸出大小排序，並傳回最大的 一：

Kotlin

fun <T>getMaximumOutputSize(
    characteristics: CameraCharacteristics, targetClass: Class <T>, format: Int? = null):
    Size {
  val config = characteristics.get(
      CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

  // If image format is provided, use it to determine supported sizes; or else use target class
  val allSizes = if (format == null)
    config.getOutputSizes(targetClass) else config.getOutputSizes(format)
  return allSizes.maxBy { it.height * it.width }
}

Java

 @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getMaximumOutputSize(CameraCharacteristics characteristics,
                                            Class <T> targetClass,
                                            Integer format) {
        StreamConfigurationMap config = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);

        // If image format is provided, use it to determine supported sizes; else use target class
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }
        return Arrays.stream(allSizes).max(Comparator.comparing(s -> s.getHeight() * s.getWidth())).get();
    }

PREVIEW 是指與裝置螢幕解析度或 1080p (1920x1080)，以較小者為準。長寬比可能與 選擇適當的螢幕大小，因此您可能需要套用上下黑邊 裁剪成直播影片，以全螢幕模式顯示。為了正確處理 預覽大小，比較可用的輸出大小與顯示大小 以考慮到螢幕可以旋轉的情況

以下程式碼定義了一個會調整尺寸的輔助類別 SmartSize 方便您比較：

Kotlin

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
class SmartSize(width: Int, height: Int) {
    var size = Size(width, height)
    var long = max(size.width, size.height)
    var short = min(size.width, size.height)
    override fun toString() = "SmartSize(${long}x${short})"
}

/** Standard High Definition size for pictures and video */
val SIZE_1080P: SmartSize = SmartSize(1920, 1080)

/** Returns a [SmartSize] object for the given [Display] */
fun getDisplaySmartSize(display: Display): SmartSize {
    val outPoint = Point()
    display.getRealSize(outPoint)
    return SmartSize(outPoint.x, outPoint.y)
}

/**
 * Returns the largest available PREVIEW size. For more information, see:
 * https://d.android.com/reference/android/hardware/camera2/CameraDevice
 */
fun <T>getPreviewOutputSize(
        display: Display,
        characteristics: CameraCharacteristics,
        targetClass: Class <T>,
        format: Int? = null
): Size {

    // Find which is smaller: screen or 1080p
    val screenSize = getDisplaySmartSize(display)
    val hdScreen = screenSize.long >= SIZE_1080P.long || screenSize.short >= SIZE_1080P.short
    val maxSize = if (hdScreen) SIZE_1080P else screenSize

    // If image format is provided, use it to determine supported sizes; else use target class
    val config = characteristics.get(
            CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)!!
    if (format == null)
        assert(StreamConfigurationMap.isOutputSupportedFor(targetClass))
    else
        assert(config.isOutputSupportedFor(format))
    val allSizes = if (format == null)
        config.getOutputSizes(targetClass) else config.getOutputSizes(format)

    // Get available sizes and sort them by area from largest to smallest
    val validSizes = allSizes
            .sortedWith(compareBy { it.height * it.width })
            .map { SmartSize(it.width, it.height) }.reversed()

    // Then, get the largest output size that is smaller or equal than our max size
    return validSizes.first { it.long <= maxSize.long && it.short <= maxSize.short }.size
}

Java

/** Helper class used to pre-compute shortest and longest sides of a [Size] */
    class SmartSize {
        Size size;
        double longSize;
        double shortSize;

        public SmartSize(Integer width, Integer height) {
            size = new Size(width, height);
            longSize = max(size.getWidth(), size.getHeight());
            shortSize = min(size.getWidth(), size.getHeight());
        }

        @Override
        public String toString() {
            return String.format("SmartSize(%sx%s)", longSize, shortSize);
        }
    }

    /** Standard High Definition size for pictures and video */
    SmartSize SIZE_1080P = new SmartSize(1920, 1080);

    /** Returns a [SmartSize] object for the given [Display] */
    SmartSize getDisplaySmartSize(Display display) {
        Point outPoint = new Point();
        display.getRealSize(outPoint);
        return new SmartSize(outPoint.x, outPoint.y);
    }

    /**
     * Returns the largest available PREVIEW size. For more information, see:
     * https://d.android.com/reference/android/hardware/camera2/CameraDevice
     */
    @RequiresApi(api = Build.VERSION_CODES.N)
    <T> Size getPreviewOutputSize(
            Display display,
            CameraCharacteristics characteristics,
            Class <T> targetClass,
            Integer format
    ){

        // Find which is smaller: screen or 1080p
        SmartSize screenSize = getDisplaySmartSize(display);
        boolean hdScreen = screenSize.longSize >= SIZE_1080P.longSize || screenSize.shortSize >= SIZE_1080P.shortSize;
        SmartSize maxSize;
        if (hdScreen) {
            maxSize = SIZE_1080P;
        } else {
            maxSize = screenSize;
        }

        // If image format is provided, use it to determine supported sizes; else use target class
        StreamConfigurationMap config = characteristics.get(
                CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
        if (format == null)
            assert(StreamConfigurationMap.isOutputSupportedFor(targetClass));
        else
            assert(config.isOutputSupportedFor(format));
        Size[] allSizes;
        if (format == null) {
            allSizes = config.getOutputSizes(targetClass);
        } else {
            allSizes = config.getOutputSizes(format);
        }

        // Get available sizes and sort them by area from largest to smallest
        List <Size> sortedSizes = Arrays.asList(allSizes);
        List <SmartSize> validSizes =
                sortedSizes.stream()
                        .sorted(Comparator.comparing(s -> s.getHeight() * s.getWidth()))
                        .map(s -> new SmartSize(s.getWidth(), s.getHeight()))
                        .sorted(Collections.reverseOrder()).collect(Collectors.toList());

        // Then, get the largest output size that is smaller or equal than our max size
        return validSizes.stream()
                .filter(s -> s.longSize <= maxSize.longSize && s.shortSize <= maxSize.shortSize)
                .findFirst().get().size;
    }

查看支援的硬體等級

如要判斷執行階段的可用功能，請查看支援的硬體 級別 CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL。

內建 CameraCharacteristics敬上 物件，便可使用單一陳述式擷取硬體層級：

Kotlin

val characteristics: CameraCharacteristics = ...

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
val hardwareLevel = characteristics.get(
        CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL)

Java

CameraCharacteristics characteristics = ...;

// Hardware level will be one of:
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LEGACY,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_EXTERNAL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_FULL,
// - CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL_3
Integer hardwareLevel = characteristics.get(
                CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);

將所有環節放在一起

有了輸出類型、輸出大小和硬體等級，就可以決定 都是有效的組合。下圖是 設定由 CameraDevice 支援的 LEGACY 硬體等級

目標 1 目標 2 目標 3 用途範例
類型 大小上限 類型 大小上限 類型 大小上限
PRIV MAXIMUM 簡易預覽、GPU 影片處理，或無預覽影片錄製功能。
JPEG MAXIMUM 沒有觀景窗靜態相片拍攝。
YUV MAXIMUM 應用程式內的影片/圖片處理。
PRIV PREVIEW JPEG MAXIMUM 標準靜止影像。
YUV PREVIEW JPEG MAXIMUM 應用程式內處理和靜態擷取。
PRIV PREVIEW PRIV PREVIEW 標準錄製功能。
PRIV PREVIEW YUV PREVIEW 預先發布版和應用程式內處理作業。
PRIV PREVIEW YUV PREVIEW 預先發布版和應用程式內處理作業。
PRIV PREVIEW YUV PREVIEW JPEG MAXIMUM 仍可擷取加上應用程式內處理作業。

LEGACY 是最低的硬體級別。這個表格顯示 支援 Camera2 (API 級別 21 以上) 的裝置最多可輸出三則 同時串流，且不需要太多設定 會對效能造成限制，例如記憶體、CPU 或熱力限制。

您的應用程式也需要設定目標輸出緩衝區。舉例來說 指定具備 LEGACY 硬體等級的裝置，您可以設定兩個目標輸出裝置 途徑，一個使用 ImageFormat.PRIVATE，另一個使用 ImageFormat.PRIVATE ImageFormat.YUV_420_888。在使用 PREVIEW大小。使用本主題先前定義的函式，取得 相機 ID 所規定的預覽大小需要下列程式碼：

Kotlin

val characteristics: CameraCharacteristics = ...
val context = this as Context  // assuming you are inside of an activity

val surfaceViewSize = getPreviewOutputSize(
    context, characteristics, SurfaceView::class.java)
val imageReaderSize = getPreviewOutputSize(
    context, characteristics, ImageReader::class.java, format = ImageFormat.YUV_420_888)

Java

CameraCharacteristics characteristics = ...;
        Context context = this; // assuming you are inside of an activity

        Size surfaceViewSize = getPreviewOutputSize(
                context, characteristics, SurfaceView.class);
        Size imageReaderSize = getPreviewOutputSize(
                context, characteristics, ImageReader.class, format = ImageFormat.YUV_420_888);

此函式需要等待 SurfaceView 使用提供的回呼準備就緒：

Kotlin

val surfaceView = findViewById <SurfaceView>(...)
surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
  override fun surfaceCreated(holder: SurfaceHolder) {
    // You do not need to specify image format, and it will be considered of type PRIV
    // Surface is now ready and you could use it as an output target for CameraSession
  }
  ...
})

Java

SurfaceView surfaceView = findViewById <SurfaceView>(...);

surfaceView.getHolder().addCallback(new SurfaceHolder.Callback() {
            @Override
            public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
                // You do not need to specify image format, and it will be considered of type PRIV
                // Surface is now ready and you could use it as an output target for CameraSession
            }
            ...
        });

您可以呼叫，強制 SurfaceView 與相機輸出大小一致 SurfaceHolder.setFixedSize() 或者採取類似於 AutoFitSurfaceView常見常見 模組 當中會設定絕對大小 會將顯示比例和可用空間一併納入考量 調整觸發活動變更的時機

設定另一個介面 ImageReader 為所需格式的 更容易，因為不必等待以下回呼：

Kotlin

val frameBufferCount = 3  // just an example, depends on your usage of ImageReader
val imageReader = ImageReader.newInstance(
    imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
    frameBufferCount)

Java

int frameBufferCount = 3;  // just an example, depends on your usage of ImageReader
ImageReader imageReader = ImageReader.newInstance(
                imageReaderSize.width, imageReaderSize.height, ImageFormat.YUV_420_888,
                frameBufferCount);

使用 ImageReader 等封鎖目標緩衝區時，請捨棄之後的影格 使用這些服務：

Kotlin

imageReader.setOnImageAvailableListener({
  val frame =  it.acquireNextImage()
  // Do something with "frame" here
  it.close()
}, null)

Java

imageReader.setOnImageAvailableListener(listener -> {
            Image frame = listener.acquireNextImage();
            // Do something with "frame" here
            listener.close();
        }, null);

LEGACY 硬體層級指定最小共同分母的裝置。你可以 新增條件式分支，並將 RECORD 大小用於其中一個輸出目標 裝置在硬體級別為 LIMITED 的裝置時， MAXIMUM 大小 (適用於硬體等級 FULL 的裝置)。