StudioSoundAudioProcessor

@RequiresApi(value = 27)
@ExperimentalApi
public final class StudioSoundAudioProcessor extends BaseAudioProcessor

java.lang.Object
   ↳ androidx.media3.common.audio.BaseAudioProcessor
     ↳ androidx.media3.effect.playservices.StudioSoundAudioProcessor

AudioProcessor implementation for audio source separation using the StudioSound Google Play services module.

This AudioProcessor outputs 8 or 16 interleaved channels containing the separated sources from the received input stream, for mono or stereo input respectively. getResults provides information about each resulting source, including what label it was classified as and what channel indices in the output stream that source corresponds to.

The StudioSound module only supports up to 2 minutes of input. queueInput will throw if the caller attempts to process an input stream beyond 2 minutes.

Each instance only supports processing one input stream. Any attempt to reuse the processor after processing a stream will throw.

This AudioProcessor holds an internal ExecutorService for handling asynchronous events. If using this class as a standalone effect, call reset to release this ExecutorService and other resources.

Summary

Nested types

Type of separated source.

A record containing metadata about a separated source.

Constants
static final int

See CROWD.
static final int

See MUSIC.
static final int

See NATURE.
static final int

See NOISE.
static final int

See SPEECH.
static final int

See UNSPECIFIED.
static final int

See WIND.

Public methods
static ListenableFuture<StudioSoundAudioProcessor>
create(Context context)

Returns a future that provides a new StudioSoundAudioProcessor instance.
ByteBuffer

Returns a buffer containing processed output data between its position and limit.
@Nullable ImmutableList<StudioSoundAudioProcessor.SeparationResult>

Returns a list of SeparationResult that holds information about the channels and labels corresponding to each separated source, or null if processing has not completed.
boolean

Returns whether this processor will return no more output from getOutput until flush has been called and more input has been queued.
void
queueInput(ByteBuffer inputBuffer)

Queues audio data between the position and limit of the inputBuffer for processing.

Protected methods
AudioProcessor.AudioFormat

Called when the processor is configured for a new input format.
void

Called when the processor is flushed.
void

Called when the end-of-stream is queued to the processor.
void

Called when the processor is reset.

Inherited Constants
From androidx.media3.common.audio.AudioProcessor
static final ByteBuffer

An empty, direct ByteBuffer.

Inherited fields
From androidx.media3.common.audio.BaseAudioProcessor
AudioProcessor.AudioFormat

The current input audio format.
AudioProcessor.AudioFormat

The current output audio format.

Inherited methods
From androidx.media3.common.audio.AudioProcessor
long

Returns the expected duration of the output stream when the processor is applied given a input durationUs.
From androidx.media3.common.audio.BaseAudioProcessor
final AudioProcessor.AudioFormat

Configures the processor to process input audio with the specified format.
final void

This method is deprecated.

Use flush instead.
final void

Prepares the audio processor to receive a new stream of input in the last configured format and clears any buffered data and pending output.
final boolean

Returns whether the current output buffer has any data remaining.
boolean

Returns whether the processor is configured and will process input buffers.
void

This method is deprecated.

Use onFlush instead.
final void

Queues an end of stream signal.
final ByteBuffer

Replaces the current output buffer with a buffer of at least size bytes and returns it.
final void

Resets the processor to its unconfigured state, releasing any resources.

Constants

AUDIO_TYPE_CROWD

public static final int AUDIO_TYPE_CROWD = 2

See CROWD.

AUDIO_TYPE_MUSIC

public static final int AUDIO_TYPE_MUSIC = 3

See MUSIC.

AUDIO_TYPE_NATURE

public static final int AUDIO_TYPE_NATURE = 4

See NATURE.

AUDIO_TYPE_NOISE

public static final int AUDIO_TYPE_NOISE = 6

See NOISE.

AUDIO_TYPE_SPEECH

public static final int AUDIO_TYPE_SPEECH = 1

See SPEECH.

AUDIO_TYPE_UNSPECIFIED

public static final int AUDIO_TYPE_UNSPECIFIED = 0

See UNSPECIFIED.

AUDIO_TYPE_WIND

public static final int AUDIO_TYPE_WIND = 5

See WIND.

Public methods

create

public static ListenableFuture<StudioSoundAudioProcessor> create(Context context)

Returns a future that provides a new StudioSoundAudioProcessor instance.

This method verify that the current device supports the Google Play services StudioSound module, attempt to download and install it if needed, and then return a new instance of StudioSoundAudioProcessor.

Parameters
Context context

The Context.

getOutput

public ByteBuffer getOutput()

Returns a buffer containing processed output data between its position and limit. The buffer will always be a direct byte buffer with native byte order. Calling this method invalidates any previously returned buffer. The buffer will be empty if no output is available.

Returns
ByteBuffer

A buffer containing processed output data between its position and limit.

getResults

public @Nullable ImmutableList<StudioSoundAudioProcessor.SeparationResult> getResults()

Returns a list of SeparationResult that holds information about the channels and labels corresponding to each separated source, or null if processing has not completed.

Any channels that do not map to a SeparationResult are empty and should be ignored.

isEnded

public boolean isEnded()

Returns whether this processor will return no more output from getOutput until flush has been called and more input has been queued.

queueInput

public void queueInput(ByteBuffer inputBuffer)

Queues audio data between the position and limit of the inputBuffer for processing. After calling this method, processed output may be available via getOutput. Calling queueInput(ByteBuffer) again invalidates any pending output.

Parameters
ByteBuffer inputBuffer

The input buffer to process. It must be a direct byte buffer with native byte order. Its contents are treated as read-only. Its position will be advanced by the number of bytes consumed (which may be zero). The caller retains ownership of the provided buffer.

Protected methods

onConfigure

protected AudioProcessor.AudioFormat onConfigure(AudioProcessor.AudioFormat inputAudioFormat)

Called when the processor is configured for a new input format.

Throws
androidx.media3.common.audio.AudioProcessor.UnhandledAudioFormatException

onFlush

protected void onFlush(AudioProcessor.StreamMetadata streamMetadata)

Called when the processor is flushed.

onQueueEndOfStream

protected void onQueueEndOfStream()

Called when the end-of-stream is queued to the processor.

onReset

protected void onReset()

Called when the processor is reset.