1. Introduction
In this codelab we'll build an audio sampler. The app records audio from the phone's built in microphone and plays it back.
The app records up to 10 seconds of audio while the Record button is pressed. When you press Play the recorded audio plays back once (while you hold down the button). Alternatively, you can turn on Loop which replays the recording over and over until you release the Play button. Each time you press Record the previous audio recording is overwritten.
What you'll learn
- Basic concepts for creating a low latency recording stream
- How to store and playback audio data recorded from a microphone
Prerequisites
Before starting this codelab you should consider completing the WaveMaker Part 1 codelab. It covers some basic concepts for creating audio streams which are not discussed here.
What you'll need
- Android Studio 3.0.0 or above
- Android 8.0 (API level 26) SDK
- NDK and Build Tools installed
- A simulator or Android device running Android 8.0 (API level 26) or above for testing
- Some knowledge of C++ is helpful but not required
2. Architecture overview
Our sampler app has four components:
- UI - Written in Java, the MainActivity class is responsible for receiving touch events and forwarding them to the JNI bridge
- JNI bridge - This C++ file uses JNI to provide a communication mechanism between the UI and C++ objects. It forwards events from the UI to the Audio Engine.
- Audio engine - This C++ class creates the recording and playback audio streams.
- Sound recording - This C++ class stores the audio data in memory.
Here's the architecture:
3. Getting started
Clone project
Clone the codelab repository on github.
git clone https://github.com/googlecodelabs/android-wavemaker2
Import project into Android Studio
Open Android Studio and import the project:
- File -> New -> Import project...
- Choose the "android-wavemaker2" folder
Run the project
Choose the base run configuration.
Then press CTRL+R to build and run the template app - it should compile and run but is non-functional. You will add functionality to it during this codelab.
Open the base module
The files you'll work on for this codelab are stored in the base
module. Expand this module in the Project window, making sure that the Android view is selected.
Note: The finished source code for the WaveMaker2 app can be found in the final
module.
4. Build the SoundRecording class
The SoundRecording
object represents recorded audio data in memory. It lets the app write data from the microphone into memory and read data out for playback.
Let's start by figuring out how to store this audio data.
Choose an audio format
First we need to choose an audio format for our samples. AAudio supports two formats:
float
: Single precision floating point (4 bytes per sample)int16_t
: 16-bit integers (2 bytes per sample)
For good sound quality at low volume and other reasons we use float
samples. If memory capacity is an issue, you can sacrifice fidelity and gain space by using 16-bit integers.
Choose how much storage we need
Let's assume we want to store 10 seconds of audio data. At a sample rate of 48,000 samples per second, which is the most common sample rate on modern Android devices, we need to allocate memory for 480,000 samples.
Open base/cpp/SoundRecording.h and define this constant at the top of the file.
constexpr int kMaxSamples = 480000; // 10s of audio data @ 48kHz
Define the storage array
Now we have all the information we need to define an array of float
s. Add the following declaration to SoundRecording.h:
private:
std::array<float,kMaxSamples> mData { 0 };
The { 0 }
uses aggregate initialization to set all values in the array to 0.
Implement write
The write
method has this signature:
int32_t write(const float *sourceData, int32_t numFrames);
This method receives an array of audio samples in sourceData
. The size of the array is specified by numFrames
. The method should return the number of samples that it actually writes out.
This can be implemented by storing the next available write index. Initially it is 0:
After each sample is written the next write index moves along by one:
This can be easily implemented as a for
loop. Add the following code to the write
method in SoundRecording.cpp
for (int i = 0; i < numSamples; ++i) {
mData[mWriteIndex++] = sourceData[i];
}
return numSamples;
But wait, what if we try to write more samples than we have space for? Bad things will happen! We'll get a segmentation fault caused by attempting to access an out-of-bounds array index.
Let's add a check which changes numSamples
if mData
doesn't have enough space left. Add the following above the existing code.
if (mWriteIndex + numSamples > kMaxSamples) {
numSamples = kMaxSamples - mWriteIndex;
}
Implement read
The read
method is similar to write
. We store the next read index.
And increment it after a sample is read.
We repeat this until we've read the requested number of samples. What happens when we reach the end of the available data? We have two behaviors:
- If looping is enabled: Set the read index to zero - the start of the data array
- If looping is disabled: Do nothing - don't increment the read index
For both these behaviors we need to know when we have reached the end of the available data. Conveniently, mWriteIndex
tells us this. It contains the total number of samples which have been written to the data array.
With this in mind we can now implement the read
method in SoundRecording.cpp
int32_t framesRead = 0;
while (framesRead < numSamples && mReadIndex < mWriteIndex){
targetData[framesRead++] = mData[mReadIndex++];
if (mIsLooping && mReadIndex == mWriteIndex) mReadIndex = 0;
}
return framesRead;
5. Build the AudioEngine class
AudioEngine
performs these main tasks:
- Create an instance of
SoundRecording
- Create a recording stream to record data from the microphone
- Write the recorded data into the
SoundRecording
instance in the recording stream callback - Create a playback stream to play back the recorded data
- Read the recorded data from the
SoundRecording
instance in the playback stream callback - Respond to UI events for recording, playback and looping
Let's get started with creating the SoundRecording
instance.
6. Create the SoundRecording instance
Start with something easy. Create an instance of SoundRecording
in AudioEngine.h:
private:
SoundRecording mSoundRecording;
7. Create the streams
We have two streams to create: playback and recording. Which one should we create first?
We should create the playback stream first because it has only one sample rate that provides the lowest latency. Once we've created it, we can then supply its sample rate to the recording stream builder. This ensures that the playback and recording streams have the same sample rate, which means we don't have to do any extra work resampling between the streams.
Playback stream properties
Use these properties to populate the StreamBuilder
which will create the playback stream:
- Direction: not specified, defaults to output
- Sample rate: not specified, defaults to the rate with the lowest latency
- Format: float
- Channel count: 2 (stereo)
- Performance mode: Low latency
- Sharing mode: Exclusive
Create the playback stream
We have everything we need now to create and open the playback stream. Add the following code to the top of the start
method in AudioEngine.cpp.
// Create the playback stream.
StreamBuilder playbackBuilder = makeStreamBuilder();
AAudioStreamBuilder_setFormat(playbackBuilder.get(), AAUDIO_FORMAT_PCM_FLOAT);
AAudioStreamBuilder_setChannelCount(playbackBuilder.get(), kChannelCountStereo);
AAudioStreamBuilder_setPerformanceMode(playbackBuilder.get(), AAUDIO_PERFORMANCE_MODE_LOW_LATENCY);
AAudioStreamBuilder_setSharingMode(playbackBuilder.get(), AAUDIO_SHARING_MODE_EXCLUSIVE);
AAudioStreamBuilder_setDataCallback(playbackBuilder.get(), ::playbackDataCallback, this);
AAudioStreamBuilder_setErrorCallback(playbackBuilder.get(), ::errorCallback, this);
aaudio_result_t result = AAudioStreamBuilder_openStream(playbackBuilder.get(), &mPlaybackStream);
if (result != AAUDIO_OK){
__android_log_print(ANDROID_LOG_DEBUG, __func__,
"Error opening playback stream %s",
AAudio_convertResultToText(result));
return;
}
// Obtain the sample rate from the playback stream so we can request the same sample rate from
// the recording stream.
int32_t sampleRate = AAudioStream_getSampleRate(mPlaybackStream);
result = AAudioStream_requestStart(mPlaybackStream);
if (result != AAUDIO_OK){
__android_log_print(ANDROID_LOG_DEBUG, __func__,
"Error starting playback stream %s",
AAudio_convertResultToText(result));
closeStream(&mPlaybackStream);
return;
}
Note that template methods for the data and error callbacks have been created for you. If you need to recap on how these work please refer back to the first codelab.
Recording stream properties
Use these properties to create the recording stream:
- Direction: input (recording is an input, whereas playback is an output)
- Sample rate: same as output stream
- Format: float
- Channel count: 1 (mono)
- Performance mode: Low latency
- Sharing mode: Exclusive
Create the recording stream
Now add the following code underneath the previously added code in start
.
// Create the recording stream.
StreamBuilder recordingBuilder = makeStreamBuilder();
AAudioStreamBuilder_setDirection(recordingBuilder.get(), AAUDIO_DIRECTION_INPUT);
AAudioStreamBuilder_setPerformanceMode(recordingBuilder.get(), AAUDIO_PERFORMANCE_MODE_LOW_LATENCY);
AAudioStreamBuilder_setSharingMode(recordingBuilder.get(), AAUDIO_SHARING_MODE_EXCLUSIVE);
AAudioStreamBuilder_setFormat(recordingBuilder.get(), AAUDIO_FORMAT_PCM_FLOAT);
AAudioStreamBuilder_setSampleRate(recordingBuilder.get(), sampleRate);
AAudioStreamBuilder_setChannelCount(recordingBuilder.get(), kChannelCountMono);
AAudioStreamBuilder_setDataCallback(recordingBuilder.get(), ::recordingDataCallback, this);
AAudioStreamBuilder_setErrorCallback(recordingBuilder.get(), ::errorCallback, this);
result = AAudioStreamBuilder_openStream(recordingBuilder.get(), &mRecordingStream);
if (result != AAUDIO_OK){
__android_log_print(ANDROID_LOG_DEBUG, __func__,
"Error opening recording stream %s",
AAudio_convertResultToText(result));
closeStream(&mRecordingStream);
return;
}
result = AAudioStream_requestStart(mRecordingStream);
if (result != AAUDIO_OK){
__android_log_print(ANDROID_LOG_DEBUG, __func__,
"Error starting recording stream %s",
AAudio_convertResultToText(result));
return;
}
8. Store the recorded data
Now we're getting into the fun part: actually storing the recorded data from the microphone into the SoundRecording
object.
When we built the recording stream we specified the data callback as ::recordingDataCallback
. This method calls AudioEngine::recordingCallback
which has the following signature:
aaudio_data_callback_result_t AudioEngine::recordingCallback(float *audioData,
int32_t numFrames)
The audio data is supplied in audioData.
Its size (in samples) is numFrames
because there is only one sample per frame, since we're recording in mono.
All we need to do is:
- Check
mIsRecording
to see whether we should be recording - If not, ignore the incoming data
- If we are recording:
- Supply
audioData
toSoundRecording
using itswrite
method - Check the return value of
write
, if it's zero it means thatSoundRecording
is full and we should stop recording - Return
AAUDIO_CALLBACK_RESULT_CONTINUE
so that the callbacks continue
Add the following code to recordingCallback
:
if (mIsRecording) {
int32_t framesWritten = mSoundRecording.write(audioData, numFrames);
if (framesWritten == 0) mIsRecording = false;
}
return AAUDIO_CALLBACK_RESULT_CONTINUE;
9. Play the recorded data
Similar to the recording stream, the playback stream calls playbackDataCallback
when it needs new data. This method calls AudioEngine::playbackCallback,
which has the following signature:
aaudio_data_callback_result_t AudioEngine::playbackCallback(float *audioData, int32_t numFrames)
Inside this method we need to:
- Fill the array with zeros using
fillArrayWithZeros
- If we are playing the recorded data, indicated by
mIsPlaying
then: - Read
numFrames
of data frommSoundRecording
- Convert
audioData
from mono to stereo usingconvertArrayMonoToStereo
- If the number of frames read was less than the number of frames requested we have reached the end of the recorded data. Stop playback by setting
mIsPlaying
tofalse
Add the following code to playbackCallback
:
fillArrayWithZeros(audioData, numFrames * kChannelCountStereo);
if (mIsPlaying) {
int32_t framesRead = mSoundRecording.read(audioData, numFrames);
convertArrayMonoToStereo(audioData, framesRead);
if (framesRead < numFrames) mIsPlaying = false;
}
return AAUDIO_CALLBACK_RESULT_CONTINUE;
10. Respond to UI events
Start and stop recording
The method setRecording
is used to start and stop recording. Here's its signature:
void setRecording(bool isRecording)
When the record button is pressed isRecording
is true; when the button is released isRecording
is false.
A member variable mIsRecording
is used to toggle data storage inside recordingCallback
. All we need to do is set it to the value of isRecording
.
There's one more behaviour we need to add. When recording starts any existing data in mSoundRecording
should be overwritten. This can be achieved using clear
which will reset mSoundRecording
's write index to zero.
Here's the code for setRecording
:
if (isRecording) mSoundRecording.clear();
mIsRecording = isRecording;
Start and stop playback
Playback control is similar recording. setPlaying
is called when the play button is pressed or released. The member variable mIsPlaying
toggles playback inside the playbackCallback
.
When the play button is pressed we want playback to start at the beginning of the recorded audio data. This can be achieved using setReadPositionToStart
which reset's mSoundRecording
's read head to zero.
Here's the code for setPlaying
:
if (isPlaying) mSoundRecording.setReadPositionToStart();
mIsPlaying = isPlaying;
Toggle looped playback
Lastly, when the LOOP switch is toggled setLooping
is called. Handling this is simple, we just pass the isOn
argument to mSoundRecording
.setLooping
:
Here's the code for setLooping
:
mSoundRecording.setLooping(isOn);
11. Add the recording permission
Apps which record audio must request the RECORD_AUDIO
permission from the user. Much of the permission handling code is already written, however, we still need to declare that our app uses this permission.
Add the following line to manifests/AndroidManifest.xml inside the <manifest>
section:
<uses-permission android:name="android.permission.RECORD_AUDIO" />
12. Build and run
Time to see whether all your hard work has paid off. Build and run the app, you should see the following UI.
Tap the red button to start recording. Recording continues while you keep the button pressed, up to a maximum of 10 seconds. Tap the green button to play the recorded data. Playback continues while you keep the button pressed until the end of the audio data. If LOOP is enabled the audio data will loop forever.
Further reading
High-performance audio samples
High-performance audio guide on the Android NDK documentation