MediaExtractor


public final class MediaExtractor
extends Object

java.lang.Object
   ↳ android.media.MediaExtractor


MediaExtractor facilitates extraction of demuxed, typically encoded, media data from a data source.

It is generally used like this:

 MediaExtractor extractor = new MediaExtractor();
 extractor.setDataSource(...);
 int numTracks = extractor.getTrackCount();
 for (int i = 0; i < numTracks; ++i) {
   MediaFormat format = extractor.getTrackFormat(i);
   String mime = format.getString(MediaFormat.KEY_MIME);
   if (weAreInterestedInThisTrack) {
     extractor.selectTrack(i);
   }
 }
 ByteBuffer inputBuffer = ByteBuffer.allocate(...)
 while (extractor.readSampleData(inputBuffer, ...) >= 0) {
   int trackIndex = extractor.getSampleTrackIndex();
   long presentationTimeUs = extractor.getSampleTime();
   ...
   extractor.advance();
 }

 extractor.release();
 extractor = null;
 

This class requires the Manifest.permission.INTERNET permission when used with network-based content.

Summary

Nested classes

class MediaExtractor.CasInfo

Describes the conditional access system used to scramble a track. 

class MediaExtractor.MetricsConstants

 

Constants

int SAMPLE_FLAG_ENCRYPTED

The sample is (at least partially) encrypted, see also the documentation for MediaCodec.queueSecureInputBuffer(int, int, CryptoInfo, long, int)

int SAMPLE_FLAG_PARTIAL_FRAME

This indicates that the buffer only contains part of a frame, and the decoder should batch the data until a buffer without this flag appears before decoding the frame.

int SAMPLE_FLAG_SYNC

The sample is a sync sample (or in MediaCodec's terminology it is a key frame.)

int SEEK_TO_CLOSEST_SYNC

If possible, seek to the sync sample closest to the specified time

int SEEK_TO_NEXT_SYNC

If possible, seek to a sync sample at or after the specified time

int SEEK_TO_PREVIOUS_SYNC

If possible, seek to a sync sample at or before the specified time

Public constructors

MediaExtractor()

Public methods

boolean advance()

Advance to the next sample.

List<AudioPresentation> getAudioPresentations(int trackIndex)

Get the list of available audio presentations for the track.

long getCachedDuration()

Returns an estimate of how much data is presently cached in memory expressed in microseconds.

MediaExtractor.CasInfo getCasInfo(int index)

Retrieves the information about the conditional access system used to scramble a track.

DrmInitData getDrmInitData()

Extract DRM initialization data if it exists

LogSessionId getLogSessionId()

Returns the LogSessionId for MediaExtractor.

PersistableBundle getMetrics()

Return Metrics data about the current media container.

Map<UUID, byte[]> getPsshInfo()

Get the PSSH info if present.

boolean getSampleCryptoInfo(MediaCodec.CryptoInfo info)

If the sample flags indicate that the current sample is at least partially encrypted, this call returns relevant information about the structure of the sample data required for decryption.

int getSampleFlags()

Returns the current sample's flags.

long getSampleSize()
long getSampleTime()

Returns the current sample's presentation time in microseconds.

int getSampleTrackIndex()

Returns the track index the current sample originates from (or -1 if no more samples are available)

int getTrackCount()

Count the number of tracks found in the data source.

MediaFormat getTrackFormat(int index)

Get the track format at the specified index.

boolean hasCacheReachedEndOfStream()

Returns true iff we are caching data and the cache has reached the end of the data stream (for now, a future seek may of course restart the fetching of data).

int readSampleData(ByteBuffer byteBuf, int offset)

Retrieve the current encoded sample and store it in the byte buffer starting at the given offset.

void release()

Make sure you call this when you're done to free up any resources instead of relying on the garbage collector to do this for you at some point in the future.

void seekTo(long timeUs, int mode)

All selected tracks seek near the requested time according to the specified mode.

void selectTrack(int index)

Subsequent calls to readSampleData(ByteBuffer, int), getSampleTrackIndex() and getSampleTime() only retrieve information for the subset of tracks selected.

void setDataSource(AssetFileDescriptor afd)

Sets the data source (AssetFileDescriptor) to use.

void setDataSource(FileDescriptor fd)

Sets the data source (FileDescriptor) to use.

void setDataSource(MediaDataSource dataSource)

Sets the data source (MediaDataSource) to use.

void setDataSource(Context context, Uri uri, Map<StringString> headers)

Sets the data source as a content Uri.

void setDataSource(FileDescriptor fd, long offset, long length)

Sets the data source (FileDescriptor) to use.

void setDataSource(String path, Map<StringString> headers)

Sets the data source (file-path or http URL) to use.

void setDataSource(String path)

Sets the data source (file-path or http URL) to use.

void setLogSessionId(LogSessionId logSessionId)

Sets the LogSessionId for MediaExtractor.

void setMediaCas(MediaCas mediaCas)

This method was deprecated in API level 33. Use the Descrambler system API instead, or DRM public APIs like MediaDrm.

void unselectTrack(int index)

Subsequent calls to readSampleData(ByteBuffer, int), getSampleTrackIndex() and getSampleTime() only retrieve information for the subset of tracks selected.

Protected methods

void finalize()

Called by the garbage collector on an object when garbage collection determines that there are no more references to the object.

Inherited methods

Constants

SAMPLE_FLAG_ENCRYPTED

Added in API level 16
public static final int SAMPLE_FLAG_ENCRYPTED

The sample is (at least partially) encrypted, see also the documentation for MediaCodec.queueSecureInputBuffer(int, int, CryptoInfo, long, int)

Constant Value: 2 (0x00000002)

SAMPLE_FLAG_PARTIAL_FRAME

Added in API level 26
public static final int SAMPLE_FLAG_PARTIAL_FRAME

This indicates that the buffer only contains part of a frame, and the decoder should batch the data until a buffer without this flag appears before decoding the frame.

Constant Value: 4 (0x00000004)

SAMPLE_FLAG_SYNC

Added in API level 16
public static final int SAMPLE_FLAG_SYNC

The sample is a sync sample (or in MediaCodec's terminology it is a key frame.)

Constant Value: 1 (0x00000001)

SEEK_TO_CLOSEST_SYNC

Added in API level 16
public static final int SEEK_TO_CLOSEST_SYNC

If possible, seek to the sync sample closest to the specified time

Constant Value: 2 (0x00000002)

SEEK_TO_NEXT_SYNC

Added in API level 16
public static final int SEEK_TO_NEXT_SYNC

If possible, seek to a sync sample at or after the specified time

Constant Value: 1 (0x00000001)

SEEK_TO_PREVIOUS_SYNC

Added in API level 16
public static final int SEEK_TO_PREVIOUS_SYNC

If possible, seek to a sync sample at or before the specified time

Constant Value: 0 (0x00000000)

Public constructors

MediaExtractor

Added in API level 16
public MediaExtractor ()

Public methods

advance

Added in API level 16
public boolean advance ()

Advance to the next sample. Returns false if no more sample data is available (end of stream). When extracting a local file, the behaviors of advance() and readSampleData(ByteBuffer, int) are undefined in presence of concurrent writes to the same local file; more specifically, end of stream could be signalled earlier than expected.

Returns
boolean

getAudioPresentations

Added in API level 28
public List<AudioPresentation> getAudioPresentations (int trackIndex)

Get the list of available audio presentations for the track.

Parameters
trackIndex int: index of the track.

Returns
List<AudioPresentation> a list of available audio presentations for a given valid audio track index. The list will be empty if the source does not contain any audio presentations. This value cannot be null.

getCachedDuration

Added in API level 16
public long getCachedDuration ()

Returns an estimate of how much data is presently cached in memory expressed in microseconds. Returns -1 if that information is unavailable or not applicable (no cache).

Returns
long

getCasInfo

Added in API level 26
public MediaExtractor.CasInfo getCasInfo (int index)

Retrieves the information about the conditional access system used to scramble a track.

Parameters
index int: of the track.

Returns
MediaExtractor.CasInfo an CasInfo object describing the conditional access system.

getDrmInitData

Added in API level 24
public DrmInitData getDrmInitData ()

Extract DRM initialization data if it exists

Returns
DrmInitData DRM initialization data in the content, or null if no recognizable DRM format is found;

See also:

getLogSessionId

Added in API level 31
public LogSessionId getLogSessionId ()

Returns the LogSessionId for MediaExtractor.

Returns
LogSessionId This value cannot be null.

getMetrics

Added in API level 26
public PersistableBundle getMetrics ()

Return Metrics data about the current media container.

Returns
PersistableBundle a PersistableBundle containing the set of attributes and values available for the media container being handled by this instance of MediaExtractor. The attributes are descibed in MetricsConstants. Additional vendor-specific fields may also be present in the return value.

getPsshInfo

Added in API level 18
public Map<UUID, byte[]> getPsshInfo ()

Get the PSSH info if present.

Returns
Map<UUID, byte[]> a map of uuid-to-bytes, with the uuid specifying the crypto scheme, and the bytes being the data specific to that scheme. This can be null if the source does not contain PSSH info.

getSampleCryptoInfo

Added in API level 16
public boolean getSampleCryptoInfo (MediaCodec.CryptoInfo info)

If the sample flags indicate that the current sample is at least partially encrypted, this call returns relevant information about the structure of the sample data required for decryption.

Parameters
info MediaCodec.CryptoInfo: The android.media.MediaCodec.CryptoInfo structure to be filled in. This value cannot be null.

Returns
boolean true iff the sample flags contain SAMPLE_FLAG_ENCRYPTED

getSampleFlags

Added in API level 16
public int getSampleFlags ()

Returns the current sample's flags.

Returns
int Value is either 0 or a combination of SAMPLE_FLAG_SYNC, SAMPLE_FLAG_ENCRYPTED, and SAMPLE_FLAG_PARTIAL_FRAME

getSampleSize

Added in API level 28
public long getSampleSize ()

Returns
long size of the current sample in bytes or -1 if no more samples are available.

getSampleTime

Added in API level 16
public long getSampleTime ()

Returns the current sample's presentation time in microseconds. or -1 if no more samples are available.

Returns
long

getSampleTrackIndex

Added in API level 16
public int getSampleTrackIndex ()

Returns the track index the current sample originates from (or -1 if no more samples are available)

Returns
int

getTrackCount

Added in API level 16
public int getTrackCount ()

Count the number of tracks found in the data source.

Returns
int

getTrackFormat

Added in API level 16
public MediaFormat getTrackFormat (int index)

Get the track format at the specified index. More detail on the representation can be found at MediaCodec

The following table summarizes support for format keys across android releases:

OS Version(s) MediaFormat keys used for
All Tracks Audio Tracks Video Tracks
Build.VERSION_CODES.JELLY_BEAN MediaFormat#KEY_MIME,
MediaFormat#KEY_DURATION,
MediaFormat#KEY_MAX_INPUT_SIZE
MediaFormat#KEY_SAMPLE_RATE,
MediaFormat#KEY_CHANNEL_COUNT,
MediaFormat#KEY_CHANNEL_MASK,
gapless playback information.mp3, .mp4,
MediaFormat#KEY_IS_ADTSAAC if streaming,
codec-specific dataAAC, Vorbis
MediaFormat#KEY_WIDTH,
MediaFormat#KEY_HEIGHT,
codec-specific dataAVC, MPEG4
Build.VERSION_CODES.JELLY_BEAN_MR1
Build.VERSION_CODES.JELLY_BEAN_MR2 as above, plus
Pixel aspect ratio informationAVC, *
Build.VERSION_CODES.KITKAT
Build.VERSION_CODES.KITKAT_WATCH
Build.VERSION_CODES.LOLLIPOP as above, plus
MediaFormat#KEY_BIT_RATEAAC,
codec-specific dataOpus
as above, plus
MediaFormat#KEY_ROTATION.mp4,
MediaFormat#KEY_BIT_RATEMPEG4,
codec-specific dataHEVC
Build.VERSION_CODES.LOLLIPOP_MR1
Build.VERSION_CODES.M as above, plus
gapless playback informationOpus
as above, plus
MediaFormat#KEY_FRAME_RATE (integer)
Build.VERSION_CODES.N as above, plus
MediaFormat#KEY_TRACK_ID,
MediaFormat#KEY_BIT_RATE#, .mp4
as above, plus
MediaFormat#KEY_PCM_ENCODING,
MediaFormat#KEY_PROFILEAAC
as above, plus
MediaFormat#KEY_HDR_STATIC_INFO#, .webm,
MediaFormat#KEY_COLOR_STANDARD#,
MediaFormat#KEY_COLOR_TRANSFER#,
MediaFormat#KEY_COLOR_RANGE#,
MediaFormat#KEY_PROFILEMPEG2, H.263, MPEG4, AVC, HEVC, VP9,
MediaFormat#KEY_LEVELH.263, MPEG4, AVC, HEVC, VP9,
codec-specific dataVP9

Notes:
#: container-specified value only.
.mp4, .webm…: for listed containers
MPEG4, AAC…: for listed codecs

Note that that level information contained in the container many times does not match the level of the actual bitstream. You may want to clear the level using MediaFormat.setString(KEY_LEVEL, null) before using the track format to find a decoder that can play back a particular track.

*Pixel (sample) aspect ratio is returned in the following keys. The display width can be calculated for example as:

display-width = display-height * crop-width / crop-height * sar-width / sar-height

Format KeyValue TypeDescription
"sar-width"IntegerPixel aspect ratio width
"sar-height"IntegerPixel aspect ratio height

Parameters
index int

Returns
MediaFormat

hasCacheReachedEndOfStream

Added in API level 16
public boolean hasCacheReachedEndOfStream ()

Returns true iff we are caching data and the cache has reached the end of the data stream (for now, a future seek may of course restart the fetching of data). This API only returns a meaningful result if getCachedDuration() indicates the presence of a cache, i.e. does NOT return -1.

Returns
boolean

readSampleData

Added in API level 16
public int readSampleData (ByteBuffer byteBuf, 
                int offset)

Retrieve the current encoded sample and store it in the byte buffer starting at the given offset.

Note:As of API 21, on success the position and limit of byteBuf is updated to point to the data just read.

Parameters
byteBuf ByteBuffer: the destination byte buffer This value cannot be null.

offset int

Returns
int the sample size (or -1 if no more samples are available).

release

Added in API level 16
public void release ()

Make sure you call this when you're done to free up any resources instead of relying on the garbage collector to do this for you at some point in the future.

seekTo

Added in API level 16
public void seekTo (long timeUs, 
                int mode)

All selected tracks seek near the requested time according to the specified mode.

Parameters
timeUs long

mode int: Value is SEEK_TO_PREVIOUS_SYNC, SEEK_TO_NEXT_SYNC, or SEEK_TO_CLOSEST_SYNC

selectTrack

Added in API level 16
public void selectTrack (int index)

Subsequent calls to readSampleData(ByteBuffer, int), getSampleTrackIndex() and getSampleTime() only retrieve information for the subset of tracks selected. Selecting the same track multiple times has no effect, the track is only selected once.

Parameters
index int

setDataSource

Added in API level 24
public void setDataSource (AssetFileDescriptor afd)

Sets the data source (AssetFileDescriptor) to use. It is the caller's responsibility to close the file descriptor. It is safe to do so as soon as this call returns.

Parameters
afd AssetFileDescriptor: the AssetFileDescriptor for the file you want to extract from. This value cannot be null.

Throws
IOException
IllegalArgumentException
IllegalStateException

setDataSource

Added in API level 16
public void setDataSource (FileDescriptor fd)

Sets the data source (FileDescriptor) to use. It is the caller's responsibility to close the file descriptor. It is safe to do so as soon as this call returns.

Parameters
fd FileDescriptor: the FileDescriptor for the file you want to extract from. This value cannot be null.

Throws
IOException

setDataSource

Added in API level 23
public void setDataSource (MediaDataSource dataSource)

Sets the data source (MediaDataSource) to use.

Parameters
dataSource MediaDataSource: the MediaDataSource for the media you want to extract from This value cannot be null.

Throws
IllegalArgumentException if dataSource is invalid.
IOException

setDataSource

Added in API level 16
public void setDataSource (Context context, 
                Uri uri, 
                Map<StringString> headers)

Sets the data source as a content Uri.

Parameters
context Context: the Context to use when resolving the Uri This value cannot be null.

uri Uri: the Content URI of the data you want to extract from.

When uri refers to a network file the Manifest.permission.INTERNET permission is required. This value cannot be null.

headers Map: the headers to be sent together with the request for the data. This can be null if no specific headers are to be sent with the request.

Throws
IOException

setDataSource

Added in API level 16
public void setDataSource (FileDescriptor fd, 
                long offset, 
                long length)

Sets the data source (FileDescriptor) to use. The FileDescriptor must be seekable (N.B. a LocalSocket is not seekable). It is the caller's responsibility to close the file descriptor. It is safe to do so as soon as this call returns.

Parameters
fd FileDescriptor: the FileDescriptor for the file you want to extract from. This value cannot be null.

offset long: the offset into the file where the data to be extracted starts, in bytes

length long: the length in bytes of the data to be extracted

Throws
IOException

setDataSource

Added in API level 16
public void setDataSource (String path, 
                Map<StringString> headers)

Sets the data source (file-path or http URL) to use.

Parameters
path String: the path of the file, or the http URL

When path refers to a network file the Manifest.permission.INTERNET permission is required. This value cannot be null.

headers Map: the headers associated with the http request for the stream you want to play. This can be null if no specific headers are to be sent with the request.

Throws
IOException

setDataSource

Added in API level 16
public void setDataSource (String path)

Sets the data source (file-path or http URL) to use.

Parameters
path String: the path of the file, or the http URL of the stream

When path refers to a local file, the file may actually be opened by a process other than the calling application. This implies that the pathname should be an absolute path (as any other process runs with unspecified current working directory), and that the pathname should reference a world-readable file. As an alternative, the application could first open the file for reading, and then use the file descriptor form setDataSource(java.io.FileDescriptor).

When path refers to a network file the Manifest.permission.INTERNET permission is required. This value cannot be null.

Throws
IOException

setLogSessionId

Added in API level 31
public void setLogSessionId (LogSessionId logSessionId)

Sets the LogSessionId for MediaExtractor.

Parameters
logSessionId LogSessionId: This value cannot be null.

setMediaCas

Added in API level 26
Deprecated in API level 33
public void setMediaCas (MediaCas mediaCas)

This method was deprecated in API level 33.
Use the Descrambler system API instead, or DRM public APIs like MediaDrm.

Sets the MediaCas instance to use. This should be called after a successful setDataSource() if at least one track reports mime type of MediaFormat.MIMETYPE_AUDIO_SCRAMBLED or MediaFormat.MIMETYPE_VIDEO_SCRAMBLED. Stream parsing will not proceed until a valid MediaCas object is provided.

Parameters
mediaCas MediaCas: the MediaCas object to use. This value cannot be null.

unselectTrack

Added in API level 16
public void unselectTrack (int index)

Subsequent calls to readSampleData(ByteBuffer, int), getSampleTrackIndex() and getSampleTime() only retrieve information for the subset of tracks selected.

Parameters
index int

Protected methods

finalize

Added in API level 16
protected void finalize ()

Called by the garbage collector on an object when garbage collection determines that there are no more references to the object. A subclass overrides the finalize method to dispose of system resources or to perform other cleanup.

The general contract of finalize is that it is invoked if and when the Java virtual machine has determined that there is no longer any means by which this object can be accessed by any thread that has not yet died, except as a result of an action taken by the finalization of some other object or class which is ready to be finalized. The finalize method may take any action, including making this object available again to other threads; the usual purpose of finalize, however, is to perform cleanup actions before the object is irrevocably discarded. For example, the finalize method for an object that represents an input/output connection might perform explicit I/O transactions to break the connection before the object is permanently discarded.

The finalize method of class Object performs no special action; it simply returns normally. Subclasses of Object may override this definition.

The Java programming language does not guarantee which thread will invoke the finalize method for any given object. It is guaranteed, however, that the thread that invokes finalize will not be holding any user-visible synchronization locks when finalize is invoked. If an uncaught exception is thrown by the finalize method, the exception is ignored and finalization of that object terminates.

After the finalize method has been invoked for an object, no further action is taken until the Java virtual machine has again determined that there is no longer any means by which this object can be accessed by any thread that has not yet died, including possible actions by other objects or classes which are ready to be finalized, at which point the object may be discarded.

The finalize method is never invoked more than once by a Java virtual machine for any given object.

Any exception thrown by the finalize method causes the finalization of this object to be halted, but is otherwise ignored.