Visualizer

public class Visualizer
extends Object

java.lang.Object
   ↳ android.media.audiofx.Visualizer


The Visualizer class enables application to retrieve part of the currently playing audio for visualization purpose. It is not an audio recording interface and only returns partial and low quality audio content. However, to protect privacy of certain audio data (e.g voice mail) the use of the visualizer requires the permission android.permission.RECORD_AUDIO.

The audio session ID passed to the constructor indicates which audio content should be visualized:

  • If the session is 0, the audio output mix is visualized
  • If the session is not 0, the audio from a particular MediaPlayer or AudioTrack using this audio session is visualized

Two types of representation of audio content can be captured:

  • Waveform data: consecutive 8-bit (unsigned) mono samples by using the getWaveForm(byte[]) method
  • Frequency data: 8-bit magnitude FFT by using the getFft(byte[]) method

The length of the capture can be retrieved or specified by calling respectively getCaptureSize() and setCaptureSize(int) methods. The capture size must be a power of 2 in the range returned by getCaptureSizeRange().

In addition to the polling capture mode described above with getWaveForm(byte[]) and getFft(byte[]) methods, a callback mode is also available by installing a listener by use of the setDataCaptureListener(android.media.audiofx.Visualizer.OnDataCaptureListener, int, boolean, boolean) method. The rate at which the listener capture method is called as well as the type of data returned is specified.

Before capturing data, the Visualizer must be enabled by calling the setEnabled(boolean) method. When data capture is not needed any more, the Visualizer should be disabled.

It is good practice to call the release() method when the Visualizer is not used anymore to free up native resources associated to the Visualizer instance.

Creating a Visualizer on the output mix (audio session 0) requires permission Manifest.permission.MODIFY_AUDIO_SETTINGS

The Visualizer class can also be used to perform measurements on the audio being played back. The measurements to perform are defined by setting a mask of the requested measurement modes with setMeasurementMode(int). Supported values are MEASUREMENT_MODE_NONE to cancel any measurement, and MEASUREMENT_MODE_PEAK_RMS for peak and RMS monitoring. Measurements can be retrieved through getMeasurementPeakRms(android.media.audiofx.Visualizer.MeasurementPeakRms).

Summary

Nested classes

class Visualizer.MeasurementPeakRms

A class to store peak and RMS values. 

interface Visualizer.OnDataCaptureListener

The OnDataCaptureListener interface defines methods called by the Visualizer to periodically update the audio visualization capture. 

Constants

int ALREADY_EXISTS

Internal operation status.

int ERROR

Unspecified error.

int ERROR_BAD_VALUE

Operation failed due to bad parameter value.

int ERROR_DEAD_OBJECT

Operation failed due to dead remote object.

int ERROR_INVALID_OPERATION

Operation failed because it was requested in wrong state.

int ERROR_NO_INIT

Operation failed due to bad object initialization.

int ERROR_NO_MEMORY

Operation failed due to lack of memory.

int MEASUREMENT_MODE_NONE

Defines a measurement mode in which no measurements are performed.

int MEASUREMENT_MODE_PEAK_RMS

Defines a measurement mode which computes the peak and RMS value in mB below the "full scale", where 0mB is normally the maximum sample value (but see the note below).

int SCALING_MODE_AS_PLAYED

Defines a capture mode where the playback volume will affect (scale) the range of the captured data.

int SCALING_MODE_NORMALIZED

Defines a capture mode where amplification is applied based on the content of the captured data.

int STATE_ENABLED

State of a Visualizer object that is active.

int STATE_INITIALIZED

State of a Visualizer object that is ready to be used.

int STATE_UNINITIALIZED

State of a Visualizer object that was not successfully initialized upon creation

int SUCCESS

Successful operation.

Public constructors

Visualizer(int audioSession)

Class constructor.

Public methods

int getCaptureSize()

Returns current capture size.

static int[] getCaptureSizeRange()

Returns the capture size range.

boolean getEnabled()

Get current activation state of the visualizer.

int getFft(byte[] fft)

Returns a frequency capture of currently playing audio content.

static int getMaxCaptureRate()

Returns the maximum capture rate for the callback capture method.

int getMeasurementMode()

Returns the current measurement modes performed by this audio effect

int getMeasurementPeakRms(Visualizer.MeasurementPeakRms measurement)

Retrieves the latest peak and RMS measurement.

int getSamplingRate()

Returns the sampling rate of the captured audio.

int getScalingMode()

Returns the current scaling mode on the captured visualization data.

int getWaveForm(byte[] waveform)

Returns a waveform capture of currently playing audio content.

void release()

Releases the native Visualizer resources.

int setCaptureSize(int size)

Sets the capture size, i.e.

int setDataCaptureListener(Visualizer.OnDataCaptureListener listener, int rate, boolean waveform, boolean fft)

Registers an OnDataCaptureListener interface and specifies the rate at which the capture should be updated as well as the type of capture requested.

int setEnabled(boolean enabled)

Enable or disable the visualization engine.

int setMeasurementMode(int mode)

Sets the combination of measurement modes to be performed by this audio effect.

int setScalingMode(int mode)

Set the type of scaling applied on the captured visualization data.

Protected methods

void finalize()

Called by the garbage collector on an object when garbage collection determines that there are no more references to the object.

Inherited methods

Constants

ALREADY_EXISTS

Added in API level 9
public static final int ALREADY_EXISTS

Internal operation status. Not returned by any method.

Constant Value: -2 (0xfffffffe)

ERROR

Added in API level 9
public static final int ERROR

Unspecified error.

Constant Value: -1 (0xffffffff)

ERROR_BAD_VALUE

Added in API level 9
public static final int ERROR_BAD_VALUE

Operation failed due to bad parameter value.

Constant Value: -4 (0xfffffffc)

ERROR_DEAD_OBJECT

Added in API level 9
public static final int ERROR_DEAD_OBJECT

Operation failed due to dead remote object.

Constant Value: -7 (0xfffffff9)

ERROR_INVALID_OPERATION

Added in API level 9
public static final int ERROR_INVALID_OPERATION

Operation failed because it was requested in wrong state.

Constant Value: -5 (0xfffffffb)

ERROR_NO_INIT

Added in API level 9
public static final int ERROR_NO_INIT

Operation failed due to bad object initialization.

Constant Value: -3 (0xfffffffd)

ERROR_NO_MEMORY

Added in API level 9
public static final int ERROR_NO_MEMORY

Operation failed due to lack of memory.

Constant Value: -6 (0xfffffffa)

MEASUREMENT_MODE_NONE

Added in API level 19
public static final int MEASUREMENT_MODE_NONE

Defines a measurement mode in which no measurements are performed.

Constant Value: 0 (0x00000000)

MEASUREMENT_MODE_PEAK_RMS

Added in API level 19
public static final int MEASUREMENT_MODE_PEAK_RMS

Defines a measurement mode which computes the peak and RMS value in mB below the "full scale", where 0mB is normally the maximum sample value (but see the note below). Minimum value depends on the resolution of audio samples used by the audio framework. The value of -9600mB is the minimum value for 16-bit audio systems and -14400mB or below for "high resolution" systems. Values for peak and RMS can be retrieved with getMeasurementPeakRms(android.media.audiofx.Visualizer.MeasurementPeakRms).

Note: when Visualizer effect is attached to the global session (with session ID 0), it is possible to observe RMS peaks higher than 0 dBFS, for example in the case when there are multiple audio sources playing simultaneously. In this case getMeasurementPeakRms(android.media.audiofx.Visualizer.MeasurementPeakRms) method can return a positive value.

Constant Value: 1 (0x00000001)

SCALING_MODE_AS_PLAYED

Added in API level 16
public static final int SCALING_MODE_AS_PLAYED

Defines a capture mode where the playback volume will affect (scale) the range of the captured data. A low playback volume will lead to low sample and fft values, and vice-versa.

Constant Value: 1 (0x00000001)

SCALING_MODE_NORMALIZED

Added in API level 16
public static final int SCALING_MODE_NORMALIZED

Defines a capture mode where amplification is applied based on the content of the captured data. This is the default Visualizer mode, and is suitable for music visualization.

Constant Value: 0 (0x00000000)

STATE_ENABLED

Added in API level 9
public static final int STATE_ENABLED

State of a Visualizer object that is active.

Constant Value: 2 (0x00000002)

STATE_INITIALIZED

Added in API level 9
public static final int STATE_INITIALIZED

State of a Visualizer object that is ready to be used.

Constant Value: 1 (0x00000001)

STATE_UNINITIALIZED

Added in API level 9
public static final int STATE_UNINITIALIZED

State of a Visualizer object that was not successfully initialized upon creation

Constant Value: 0 (0x00000000)

SUCCESS

Added in API level 9
public static final int SUCCESS

Successful operation.

Constant Value: 0 (0x00000000)

Public constructors

Visualizer

Added in API level 9
public Visualizer (int audioSession)

Class constructor.

Parameters
audioSession int: system wide unique audio session identifier. If audioSession is not 0, the visualizer will be attached to the MediaPlayer or AudioTrack in the same audio session. Otherwise, the Visualizer will apply to the output mix.

Throws
java.lang.UnsupportedOperationException
java.lang.RuntimeException
RuntimeException
UnsupportedOperationException

Public methods

getCaptureSize

Added in API level 9
public int getCaptureSize ()

Returns current capture size.

Returns
int the capture size in bytes.

Throws
IllegalStateException

getCaptureSizeRange

Added in API level 9
public static int[] getCaptureSizeRange ()

Returns the capture size range.

Returns
int[] the mininum capture size is returned in first array element and the maximum in second array element.

getEnabled

Added in API level 9
public boolean getEnabled ()

Get current activation state of the visualizer.

Returns
boolean true if the visualizer is active, false otherwise

getFft

Added in API level 9
public int getFft (byte[] fft)

Returns a frequency capture of currently playing audio content.

This method must be called when the Visualizer is enabled.

The capture is an 8-bit magnitude FFT, the frequency range covered being 0 (DC) to half of the sampling rate returned by getSamplingRate(). The capture returns the real and imaginary parts of a number of frequency points equal to half of the capture size plus one.

Note: only the real part is returned for the first point (DC) and the last point (sampling frequency / 2).

The layout in the returned byte array is as follows:

  • n is the capture size returned by getCaptureSize()
  • Rfk, Ifk are respectively the real and imaginary parts of the kth frequency component
  • If Fs is the sampling frequency retuned by getSamplingRate() the kth frequency is: k * Fs / n
Index

0

1

2

3

4

5

...

n - 2

n - 1

Data

Rf0

Rf(n/2)

Rf1

If1

Rf2

If2

...

Rf(n/2-1)

If(n/2-1)

In order to obtain magnitude and phase values the following code can be used:

       int n = fft.size();
       float[] magnitudes = new float[n / 2 + 1];
       float[] phases = new float[n / 2 + 1];
       magnitudes[0] = (float)Math.abs(fft[0]);      // DC
       magnitudes[n / 2] = (float)Math.abs(fft[1]);  // Nyquist
       phases[0] = phases[n / 2] = 0;
       for (int k = 1; k < n / 2; k++) {
           int i = k * 2;
           magnitudes[k] = (float)Math.hypot(fft[i], fft[i + 1]);
           phases[k] = (float)Math.atan2(fft[i + 1], fft[i]);
       }

Parameters
fft byte: array of bytes where the FFT should be returned

Returns
int SUCCESS in case of success, ERROR_NO_MEMORY, ERROR_INVALID_OPERATION or ERROR_DEAD_OBJECT in case of failure.

Throws
java.lang.IllegalStateException
IllegalStateException

getMaxCaptureRate

Added in API level 9
public static int getMaxCaptureRate ()

Returns the maximum capture rate for the callback capture method. This is the maximum value for the rate parameter of the setDataCaptureListener(android.media.audiofx.Visualizer.OnDataCaptureListener, int, boolean, boolean) method.

Returns
int the maximum capture rate expressed in milliHertz

getMeasurementMode

Added in API level 19
public int getMeasurementMode ()

Returns the current measurement modes performed by this audio effect

Returns
int the mask of the measurements, MEASUREMENT_MODE_NONE (when no measurements are performed) or MEASUREMENT_MODE_PEAK_RMS.

Throws
java.lang.IllegalStateException
IllegalStateException

getMeasurementPeakRms

Added in API level 19
public int getMeasurementPeakRms (Visualizer.MeasurementPeakRms measurement)

Retrieves the latest peak and RMS measurement. Sets the peak and RMS fields of the supplied Visualizer.MeasurementPeakRms to the latest measured values.

Parameters
measurement Visualizer.MeasurementPeakRms: a non-null Visualizer.MeasurementPeakRms instance to store the measurement values.

Returns
int SUCCESS in case of success, ERROR_BAD_VALUE, ERROR_NO_MEMORY, ERROR_INVALID_OPERATION or ERROR_DEAD_OBJECT in case of failure.

getSamplingRate

Added in API level 9
public int getSamplingRate ()

Returns the sampling rate of the captured audio.

Returns
int the sampling rate in milliHertz.

Throws
IllegalStateException

getScalingMode

Added in API level 16
public int getScalingMode ()

Returns the current scaling mode on the captured visualization data.

Returns
int the scaling mode, see SCALING_MODE_NORMALIZED and SCALING_MODE_AS_PLAYED.

Throws
java.lang.IllegalStateException
IllegalStateException

getWaveForm

Added in API level 9
public int getWaveForm (byte[] waveform)

Returns a waveform capture of currently playing audio content. The capture consists in a number of consecutive 8-bit (unsigned) mono PCM samples equal to the capture size returned by getCaptureSize().

This method must be called when the Visualizer is enabled.

Parameters
waveform byte: array of bytes where the waveform should be returned, array length must be at least equals to the capture size returned by getCaptureSize().

Returns
int SUCCESS in case of success, ERROR_NO_MEMORY, ERROR_INVALID_OPERATION or ERROR_DEAD_OBJECT in case of failure.

Throws
java.lang.IllegalStateException
java.lang.IllegalArgumentException
IllegalStateException

release

Added in API level 9
public void release ()

Releases the native Visualizer resources. It is a good practice to release the visualization engine when not in use.

setCaptureSize

Added in API level 9
public int setCaptureSize (int size)

Sets the capture size, i.e. the number of bytes returned by getWaveForm(byte[]) and getFft(byte[]) methods. The capture size must be a power of 2 in the range returned by getCaptureSizeRange(). This method must not be called when the Visualizer is enabled.

Parameters
size int: requested capture size

Returns
int SUCCESS in case of success, ERROR_BAD_VALUE in case of failure.

Throws
java.lang.IllegalStateException
IllegalStateException

setDataCaptureListener

Added in API level 9
public int setDataCaptureListener (Visualizer.OnDataCaptureListener listener, 
                int rate, 
                boolean waveform, 
                boolean fft)

Registers an OnDataCaptureListener interface and specifies the rate at which the capture should be updated as well as the type of capture requested.

Call this method with a null listener to stop receiving the capture updates.

Parameters
listener Visualizer.OnDataCaptureListener: OnDataCaptureListener registered This value may be null.

rate int: rate in milliHertz at which the capture should be updated

waveform boolean: true if a waveform capture is requested: the onWaveFormDataCapture() method will be called on the OnDataCaptureListener interface.

fft boolean: true if a frequency capture is requested: the onFftDataCapture() method will be called on the OnDataCaptureListener interface.

Returns
int SUCCESS in case of success, ERROR_NO_INIT or ERROR_BAD_VALUE in case of failure.

setEnabled

Added in API level 9
public int setEnabled (boolean enabled)

Enable or disable the visualization engine.

Parameters
enabled boolean: requested enable state

Returns
int SUCCESS in case of success, ERROR_INVALID_OPERATION or ERROR_DEAD_OBJECT in case of failure.

Throws
java.lang.IllegalStateException
IllegalStateException

setMeasurementMode

Added in API level 19
public int setMeasurementMode (int mode)

Sets the combination of measurement modes to be performed by this audio effect.

Parameters
mode int: a mask of the measurements to perform. The valid values are MEASUREMENT_MODE_NONE (to cancel any measurement) or MEASUREMENT_MODE_PEAK_RMS.

Returns
int SUCCESS in case of success, ERROR_BAD_VALUE in case of failure.

Throws
java.lang.IllegalStateException
IllegalStateException

setScalingMode

Added in API level 16
public int setScalingMode (int mode)

Set the type of scaling applied on the captured visualization data.

Parameters
mode int: see SCALING_MODE_NORMALIZED and SCALING_MODE_AS_PLAYED

Returns
int SUCCESS in case of success, ERROR_BAD_VALUE in case of failure.

Throws
java.lang.IllegalStateException
IllegalStateException

Protected methods

finalize

Added in API level 9
protected void finalize ()

Called by the garbage collector on an object when garbage collection determines that there are no more references to the object. A subclass overrides the finalize method to dispose of system resources or to perform other cleanup.

The general contract of finalize is that it is invoked if and when the Java virtual machine has determined that there is no longer any means by which this object can be accessed by any thread that has not yet died, except as a result of an action taken by the finalization of some other object or class which is ready to be finalized. The finalize method may take any action, including making this object available again to other threads; the usual purpose of finalize, however, is to perform cleanup actions before the object is irrevocably discarded. For example, the finalize method for an object that represents an input/output connection might perform explicit I/O transactions to break the connection before the object is permanently discarded.

The finalize method of class Object performs no special action; it simply returns normally. Subclasses of Object may override this definition.

The Java programming language does not guarantee which thread will invoke the finalize method for any given object. It is guaranteed, however, that the thread that invokes finalize will not be holding any user-visible synchronization locks when finalize is invoked. If an uncaught exception is thrown by the finalize method, the exception is ignored and finalization of that object terminates.

After the finalize method has been invoked for an object, no further action is taken until the Java virtual machine has again determined that there is no longer any means by which this object can be accessed by any thread that has not yet died, including possible actions by other objects or classes which are ready to be finalized, at which point the object may be discarded.

The finalize method is never invoked more than once by a Java virtual machine for any given object.

Any exception thrown by the finalize method causes the finalization of this object to be halted, but is otherwise ignored.