MediaExtractor

Kotlin |Java

class MediaExtractor

kotlin.Any
↳	android.media.MediaExtractor

MediaExtractor facilitates extraction of demuxed, typically encoded, media data from a data source.

It is generally used like this:

MediaExtractor extractor = new MediaExtractor();
  extractor.setDataSource(...);
  int numTracks = extractor.getTrackCount();
  for (int i = 0; i < numTracks; ++i) {
    MediaFormat format = extractor.getTrackFormat(i);
    String mime = format.getString(MediaFormat.KEY_MIME);
    if (weAreInterestedInThisTrack) {
      extractor.selectTrack(i);
    }
  }
  ByteBuffer inputBuffer = ByteBuffer.allocate(...)
  while (extractor.readSampleData(inputBuffer, ...) >= 0) {
    int trackIndex = extractor.getSampleTrackIndex();
    long presentationTimeUs = extractor.getSampleTime();
    ...
    extractor.advance();
  }
 
  extractor.release();
  extractor = null;

This class requires the android.Manifest.permission#INTERNET permission when used with network-based content.

Summary

Nested classes
	`CasInfo` Describes the conditional access system used to scramble a track.
	`MetricsConstants`

Constants
static Int	`SAMPLE_FLAG_ENCRYPTED` The sample is (at least partially) encrypted, see also the documentation for `android.media.MediaCodec#queueSecureInputBuffer`
static Int	`SAMPLE_FLAG_PARTIAL_FRAME` This indicates that the buffer only contains part of a frame, and the decoder should batch the data until a buffer without this flag appears before decoding the frame.
static Int	`SAMPLE_FLAG_SYNC` The sample is a sync sample (or in `MediaCodec`'s terminology it is a key frame.
static Int	`SEEK_TO_CLOSEST_SYNC` If possible, seek to the sync sample closest to the specified time
static Int	`SEEK_TO_NEXT_SYNC` If possible, seek to a sync sample at or after the specified time
static Int	`SEEK_TO_PREVIOUS_SYNC` If possible, seek to a sync sample at or before the specified time

Public constructors
`MediaExtractor()`

Public methods
Boolean	`advance()` Advance to the next sample.
MutableList<AudioPresentation!>	`getAudioPresentations(trackIndex: Int)` Get the list of available audio presentations for the track.
Long	`getCachedDuration()` Returns an estimate of how much data is presently cached in memory expressed in microseconds.
MediaExtractor.CasInfo!	`getCasInfo(index: Int)` Retrieves the information about the conditional access system used to scramble a track.
DrmInitData!	`getDrmInitData()` Extract DRM initialization data if it exists
LogSessionId	`getLogSessionId()` Returns the `LogSessionId` for MediaExtractor.
PersistableBundle!	`getMetrics()` Return Metrics data about the current media container.
MutableMap<UUID!, ByteArray!>?	`getPsshInfo()` Get the PSSH info if present.
Boolean	`getSampleCryptoInfo(info: MediaCodec.CryptoInfo)` If the sample flags indicate that the current sample is at least partially encrypted, this call returns relevant information about the structure of the sample data required for decryption.
Int	`getSampleFlags()` Returns the current sample's flags.
Long	`getSampleSize()`
Long	`getSampleTime()` Returns the current sample's presentation time in microseconds.
Int	`getSampleTrackIndex()` Returns the track index the current sample originates from (or -1 if no more samples are available)
Int	`getTrackCount()` Count the number of tracks found in the data source.
MediaFormat	`getTrackFormat(index: Int)` Get the track format at the specified index.
Boolean	`hasCacheReachedEndOfStream()` Returns true iff we are caching data and the cache has reached the end of the data stream (for now, a future seek may of course restart the fetching of data).
Int	`readSampleData(byteBuf: ByteBuffer, offset: Int)` Retrieve the current encoded sample and store it in the byte buffer starting at the given offset.
Unit	`release()` Make sure you call this when you're done to free up any resources instead of relying on the garbage collector to do this for you at some point in the future.
Unit	`seekTo(timeUs: Long, mode: Int)` All selected tracks seek near the requested time according to the specified mode.
Unit	`selectTrack(index: Int)` Subsequent calls to `readSampleData`, `getSampleTrackIndex` and `getSampleTime` only retrieve information for the subset of tracks selected.
Unit	`setDataSource(dataSource: MediaDataSource)` Sets the data source (MediaDataSource) to use.
Unit	`setDataSource(context: Context, uri: Uri, headers: MutableMap<String!, String!>?)` Sets the data source as a content Uri.
Unit	`setDataSource(path: String, headers: MutableMap<String!, String!>?)` Sets the data source (file-path or http URL) to use.
Unit	`setDataSource(path: String)` Sets the data source (file-path or http URL) to use.
Unit	`setDataSource(afd: AssetFileDescriptor)` Sets the data source (AssetFileDescriptor) to use.
Unit	`setDataSource(fd: FileDescriptor)` Sets the data source (FileDescriptor) to use.
Unit	`setDataSource(fd: FileDescriptor, offset: Long, length: Long)` Sets the data source (FileDescriptor) to use.
Unit	`setLogSessionId(logSessionId: LogSessionId)` Sets the `LogSessionId` for MediaExtractor.
Unit	`setMediaCas(mediaCas: MediaCas)` Sets the MediaCas instance to use.
Unit	`unselectTrack(index: Int)` Subsequent calls to `readSampleData`, `getSampleTrackIndex` and `getSampleTime` only retrieve information for the subset of tracks selected.

Protected methods
Unit	`finalize()`

Constants

SAMPLE_FLAG_ENCRYPTED

Added in API level 16

static val SAMPLE_FLAG_ENCRYPTED: Int

The sample is (at least partially) encrypted, see also the documentation for android.media.MediaCodec#queueSecureInputBuffer

Value: 2

SAMPLE_FLAG_PARTIAL_FRAME

Added in API level 26

static val SAMPLE_FLAG_PARTIAL_FRAME: Int

This indicates that the buffer only contains part of a frame, and the decoder should batch the data until a buffer without this flag appears before decoding the frame.

Value: 4

See Also

android.media.MediaCodec#BUFFER_FLAG_PARTIAL_FRAME

SAMPLE_FLAG_SYNC

Added in API level 16

static val SAMPLE_FLAG_SYNC: Int

The sample is a sync sample (or in MediaCodec's terminology it is a key frame.)

Value: 1

See Also

android.media.MediaCodec#BUFFER_FLAG_KEY_FRAME

SEEK_TO_CLOSEST_SYNC

Added in API level 16

static val SEEK_TO_CLOSEST_SYNC: Int

If possible, seek to the sync sample closest to the specified time

Value: 2

SEEK_TO_NEXT_SYNC

Added in API level 16

static val SEEK_TO_NEXT_SYNC: Int

If possible, seek to a sync sample at or after the specified time

Value: 1

SEEK_TO_PREVIOUS_SYNC

Added in API level 16

static val SEEK_TO_PREVIOUS_SYNC: Int

If possible, seek to a sync sample at or before the specified time

Value: 0

Public constructors

MediaExtractor

Added in API level 16

MediaExtractor()

Public methods

advance

Added in API level 16

fun advance(): Boolean

Advance to the next sample. Returns false if no more sample data is available (end of stream). When extracting a local file, the behaviors of advance and readSampleData are undefined in presence of concurrent writes to the same local file; more specifically, end of stream could be signalled earlier than expected.

getAudioPresentations

Added in API level 28

fun getAudioPresentations(trackIndex: Int): MutableList<AudioPresentation!>

Get the list of available audio presentations for the track.

Parameters
`trackIndex`	Int: index of the track.

Return
`MutableList<AudioPresentation!>`	a list of available audio presentations for a given valid audio track index. The list will be empty if the source does not contain any audio presentations. This value cannot be `null`.

getCachedDuration

Added in API level 16

fun getCachedDuration(): Long

Returns an estimate of how much data is presently cached in memory expressed in microseconds. Returns -1 if that information is unavailable or not applicable (no cache).

getCasInfo

Added in API level 26

fun getCasInfo(index: Int): MediaExtractor.CasInfo!

Retrieves the information about the conditional access system used to scramble a track.

Parameters
`index`	Int: of the track.

Return
`MediaExtractor.CasInfo!`	an `CasInfo` object describing the conditional access system.

getDrmInitData

Added in API level 24

fun getDrmInitData(): DrmInitData!

Extract DRM initialization data if it exists

Return
`DrmInitData!`	DRM initialization data in the content, or `null` if no recognizable DRM format is found;

See Also

android.media.DrmInitData

getLogSessionId

Added in API level 31

fun getLogSessionId(): LogSessionId

Returns the LogSessionId for MediaExtractor.

Return
`LogSessionId`	This value cannot be `null`.

getMetrics

Added in API level 26

fun getMetrics(): PersistableBundle!

Return Metrics data about the current media container.

Return
`PersistableBundle!`	a `PersistableBundle` containing the set of attributes and values available for the media container being handled by this instance of MediaExtractor. The attributes are descibed in `MetricsConstants`. Additional vendor-specific fields may also be present in the return value.

getPsshInfo

Added in API level 18

fun getPsshInfo(): MutableMap<UUID!, ByteArray!>?

Get the PSSH info if present.

Return
`MutableMap<UUID!, ByteArray!>?`	a map of uuid-to-bytes, with the uuid specifying the crypto scheme, and the bytes being the data specific to that scheme. This can be `null` if the source does not contain PSSH info.

getSampleCryptoInfo

Added in API level 16

fun getSampleCryptoInfo(info: MediaCodec.CryptoInfo): Boolean

If the sample flags indicate that the current sample is at least partially encrypted, this call returns relevant information about the structure of the sample data required for decryption.

Parameters
`info`	MediaCodec.CryptoInfo: The android.media.MediaCodec.CryptoInfo structure to be filled in. This value cannot be `null`.

Return
`Boolean`	true iff the sample flags contain `SAMPLE_FLAG_ENCRYPTED`

getSampleFlags

Added in API level 16

fun getSampleFlags(): Int

Returns the current sample's flags.

Return
`Int`	Value is either `0` or a combination of `android.media.MediaExtractor#SAMPLE_FLAG_SYNC`, `android.media.MediaExtractor#SAMPLE_FLAG_ENCRYPTED`, and `android.media.MediaExtractor#SAMPLE_FLAG_PARTIAL_FRAME`

getSampleSize

Added in API level 28

fun getSampleSize(): Long

Return
`Long`	size of the current sample in bytes or -1 if no more samples are available.

getSampleTime

Added in API level 16

fun getSampleTime(): Long

Returns the current sample's presentation time in microseconds. or -1 if no more samples are available.

getSampleTrackIndex

Added in API level 16

fun getSampleTrackIndex(): Int

Returns the track index the current sample originates from (or -1 if no more samples are available)

getTrackCount

Added in API level 16

fun getTrackCount(): Int

Count the number of tracks found in the data source.

getTrackFormat

Added in API level 16

fun getTrackFormat(index: Int): MediaFormat

Get the track format at the specified index. More detail on the representation can be found at android.media.MediaCodec

The following table summarizes support for format keys across android releases:

OS Version(s)	All Tracks	Audio Tracks	Video Tracks
OS Version(s)	`MediaFormat` keys used for
`android.os.Build.VERSION_CODES#JELLY_BEAN`	`MediaFormat#KEY_MIME`, `MediaFormat#KEY_DURATION`, `MediaFormat#KEY_MAX_INPUT_SIZE`	`MediaFormat#KEY_SAMPLE_RATE`, `MediaFormat#KEY_CHANNEL_COUNT`, `MediaFormat#KEY_CHANNEL_MASK`, gapless playback information.mp3, .mp4, `MediaFormat#KEY_IS_ADTS`AAC if streaming, codec-specific dataAAC, Vorbis	`MediaFormat#KEY_WIDTH`, `MediaFormat#KEY_HEIGHT`, codec-specific dataAVC, MPEG4
`android.os.Build.VERSION_CODES#JELLY_BEAN_MR1`
`android.os.Build.VERSION_CODES#JELLY_BEAN_MR2`			as above, plus Pixel aspect ratio informationAVC, *
`android.os.Build.VERSION_CODES#KITKAT`
`android.os.Build.VERSION_CODES#KITKAT_WATCH`
`android.os.Build.VERSION_CODES#LOLLIPOP`		as above, plus `MediaFormat#KEY_BIT_RATE`AAC, codec-specific dataOpus	as above, plus `MediaFormat#KEY_ROTATION`.mp4, `MediaFormat#KEY_BIT_RATE`MPEG4, codec-specific dataHEVC
`android.os.Build.VERSION_CODES#LOLLIPOP_MR1`
`android.os.Build.VERSION_CODES#M`		as above, plus gapless playback informationOpus	as above, plus `MediaFormat#KEY_FRAME_RATE` (integer)
`android.os.Build.VERSION_CODES#N`	as above, plus `MediaFormat#KEY_TRACK_ID`, `MediaFormat#KEY_BIT_RATE`, .mp4	as above, plus `MediaFormat#KEY_PCM_ENCODING`, `MediaFormat#KEY_PROFILE`AAC	as above, plus `MediaFormat#KEY_HDR_STATIC_INFO`, .webm, `MediaFormat#KEY_COLOR_STANDARD`, `MediaFormat#KEY_COLOR_TRANSFER`, `MediaFormat#KEY_COLOR_RANGE`, `MediaFormat#KEY_PROFILE`MPEG2, H.263, MPEG4, AVC, HEVC, VP9, `MediaFormat#KEY_LEVEL`H.263, MPEG4, AVC, HEVC, VP9, codec-specific dataVP9
Notes: #: container-specified value only. .mp4, .webm…: for listed containers MPEG4, AAC…: for listed codecs
Note that that level information contained in the container many times does not match the level of the actual bitstream. You may want to clear the level using `MediaFormat.setString(KEY_LEVEL, null)` before using the track format to find a decoder that can play back a particular track.
Pixel (sample) aspect ratio is returned in the following keys. The display width can be calculated for example as: display-width = display-height crop-width / crop-height * sar-width / sar-height
Format Key	Value Type	Description
`"sar-width"`	Integer	Pixel aspect ratio width
`"sar-height"`	Integer	Pixel aspect ratio height

hasCacheReachedEndOfStream

Added in API level 16

fun hasCacheReachedEndOfStream(): Boolean

Returns true iff we are caching data and the cache has reached the end of the data stream (for now, a future seek may of course restart the fetching of data). This API only returns a meaningful result if getCachedDuration indicates the presence of a cache, i.e. does NOT return -1.

readSampleData

Added in API level 16

fun readSampleData(
    byteBuf: ByteBuffer, 
    offset: Int
): Int

Retrieve the current encoded sample and store it in the byte buffer starting at the given offset.

Note:As of API 21, on success the position and limit of byteBuf is updated to point to the data just read.

Parameters
`byteBuf`	ByteBuffer: the destination byte buffer This value cannot be `null`.

Return
`Int`	the sample size (or -1 if no more samples are available).

release

Added in API level 16

fun release(): Unit

Make sure you call this when you're done to free up any resources instead of relying on the garbage collector to do this for you at some point in the future.

seekTo

Added in API level 16

fun seekTo(
    timeUs: Long, 
    mode: Int
): Unit

All selected tracks seek near the requested time according to the specified mode.

Parameters
`mode`	Int: Value is `android.media.MediaExtractor#SEEK_TO_PREVIOUS_SYNC`, `android.media.MediaExtractor#SEEK_TO_NEXT_SYNC`, or `android.media.MediaExtractor#SEEK_TO_CLOSEST_SYNC`

selectTrack

Added in API level 16

fun selectTrack(index: Int): Unit

Subsequent calls to readSampleData, getSampleTrackIndex and getSampleTime only retrieve information for the subset of tracks selected. Selecting the same track multiple times has no effect, the track is only selected once.

setDataSource

Added in API level 23

fun setDataSource(dataSource: MediaDataSource): Unit

Sets the data source (MediaDataSource) to use.

Parameters
`dataSource`	MediaDataSource: the MediaDataSource for the media you want to extract from This value cannot be `null`.

Exceptions
`java.lang.IllegalArgumentException`	if dataSource is invalid.

setDataSource

Added in API level 16

fun setDataSource(
    context: Context, 
    uri: Uri, 
    headers: MutableMap<String!, String!>?
): Unit

Sets the data source as a content Uri.

Parameters
`context`	Context: the Context to use when resolving the Uri This value cannot be `null`.
`uri`	Uri: the Content URI of the data you want to extract from. When `uri` refers to a network file the `android.Manifest.permission#INTERNET` permission is required. This value cannot be `null`.
`headers`	MutableMap<String!, String!>?: the headers to be sent together with the request for the data. This can be `null` if no specific headers are to be sent with the request.

setDataSource

Added in API level 16

fun setDataSource(
    path: String, 
    headers: MutableMap<String!, String!>?
): Unit

Sets the data source (file-path or http URL) to use.

Parameters
`path`	String: the path of the file, or the http URL When `path` refers to a network file the `android.Manifest.permission#INTERNET` permission is required. This value cannot be `null`.
`headers`	MutableMap<String!, String!>?: the headers associated with the http request for the stream you want to play. This can be `null` if no specific headers are to be sent with the request.

setDataSource

Added in API level 16

fun setDataSource(path: String): Unit

Sets the data source (file-path or http URL) to use.

Parameters

Parameters
`path`	String: the path of the file, or the http URL of the stream When `path` refers to a local file, the file may actually be opened by a process other than the calling application. This implies that the pathname should be an absolute path (as any other process runs with unspecified current working directory), and that the pathname should reference a world-readable file. As an alternative, the application could first open the file for reading, and then use the file descriptor form `setDataSource(java.io.FileDescriptor)`. When `path` refers to a network file the `android.Manifest.permission#INTERNET` permission is required. This value cannot be `null`.

path

String: the path of the file, or the http URL of the stream

When path refers to a local file, the file may actually be opened by a process other than the calling application. This implies that the pathname should be an absolute path (as any other process runs with unspecified current working directory), and that the pathname should reference a world-readable file. As an alternative, the application could first open the file for reading, and then use the file descriptor form setDataSource(java.io.FileDescriptor).

When path refers to a network file the android.Manifest.permission#INTERNET permission is required. This value cannot be null.

setDataSource

Added in API level 24

fun setDataSource(afd: AssetFileDescriptor): Unit

Sets the data source (AssetFileDescriptor) to use. It is the caller's responsibility to close the file descriptor. It is safe to do so as soon as this call returns.

Parameters
`afd`	AssetFileDescriptor: the AssetFileDescriptor for the file you want to extract from. This value cannot be `null`.

setDataSource

Added in API level 16

fun setDataSource(fd: FileDescriptor): Unit

Sets the data source (FileDescriptor) to use. It is the caller's responsibility to close the file descriptor. It is safe to do so as soon as this call returns.

Parameters
`fd`	FileDescriptor: the FileDescriptor for the file you want to extract from. This value cannot be `null`.

setDataSource

Added in API level 16

fun setDataSource(
    fd: FileDescriptor, 
    offset: Long, 
    length: Long
): Unit

Sets the data source (FileDescriptor) to use. The FileDescriptor must be seekable (N.B. a LocalSocket is not seekable). It is the caller's responsibility to close the file descriptor. It is safe to do so as soon as this call returns.

Parameters
`fd`	FileDescriptor: the FileDescriptor for the file you want to extract from. This value cannot be `null`.
`offset`	Long: the offset into the file where the data to be extracted starts, in bytes
`length`	Long: the length in bytes of the data to be extracted

setLogSessionId

Added in API level 31

fun setLogSessionId(logSessionId: LogSessionId): Unit

Sets the LogSessionId for MediaExtractor.

Parameters
`logSessionId`	LogSessionId: This value cannot be `null`.

setMediaCas

Added in API level 26
Deprecated in API level 33

fun setMediaCas(mediaCas: MediaCas): Unit

Deprecated: Use the Descrambler system API instead, or DRM public APIs like MediaDrm.

Sets the MediaCas instance to use. This should be called after a successful setDataSource() if at least one track reports mime type of android.media.MediaFormat#MIMETYPE_AUDIO_SCRAMBLED or android.media.MediaFormat#MIMETYPE_VIDEO_SCRAMBLED. Stream parsing will not proceed until a valid MediaCas object is provided.

Parameters
`mediaCas`	MediaCas: the MediaCas object to use. This value cannot be `null`.

unselectTrack

Added in API level 16

fun unselectTrack(index: Int): Unit

Subsequent calls to readSampleData, getSampleTrackIndex and getSampleTime only retrieve information for the subset of tracks selected.

Protected methods

finalize

Added in API level 16

protected fun finalize(): Unit

Exceptions
`java.lang.Throwable`	the `Exception` raised by this method