Live Audio Recording and Playing in Android and Thread & callback handling

Question

I want to record the live audio and play it.As far as UI is concerned the app just has three buttons:one for start recording and streaming it, one for playing a pre recorded file and the last one for stopping the current task(recording / playing). For that purpose I have used AudioRecord and AudioTrack classes for recording and playing respectively. My Program looks like....

/** * @author amit * */

public class AudioRecorder extends Activity {
    private String LOG_TAG = null;

    /* variables which are required to generate and manage the UI of the App */
    // private RecordButton mRecordButton = null;
    private Button recordBtn, stopBtn, playBtn;

    /*
     * variables which are required for the actual functioning of the recording
     * and playing
     */
    private AudioRecord recorder = null;
    private AudioTrack player = null;
    private AudioManager audioManager = null;
    private int recorderBufSize, recordingSampleRate;
    private int trackBufSize;
    private short[] audioData;
    private boolean isRecording = false, isPlaying = false;
    private Thread startRecThread;

    private AudioRecord.OnRecordPositionUpdateListener posUpdateListener;

    /**
     * constructor method for initializing the variables
     */
    public AudioRecorder() {
        super();
        LOG_TAG = "Constructor";
        recorderBufSize = recordingSampleRate = trackBufSize = 0;

        // init function will initialize all the necessary variables ...
        init();

        if (recorder != null && player != null) {
            Log.e(LOG_TAG, "recorder and player initialized");
            audioData = new short[recorderBufSize / 2]; // since we r reading shorts

        } else {
            Log.e(LOG_TAG, "Problem inside init function ");
        }
        posUpdateListener = new AudioRecord.OnRecordPositionUpdateListener() {
            int numShortsRead = 0;

            @Override
            public void onPeriodicNotification(AudioRecord rec) {
                // TODO Auto-generated method stub
//              String LOG_TAG = Thread.currentThread().getName();
//               Log.e(LOG_TAG, "inside position listener");
                audioData = new short[recorderBufSize / 2]; // divide by 2 since now we are reading shorts 
                numShortsRead = rec.read(audioData, 0, audioData.length);
                player.write(audioData, 0, numShortsRead);

            }

            @Override
            public void onMarkerReached(AudioRecord recorder) {
                // TODO Auto-generated method stub
                Log.e(LOG_TAG, "Marker Reached");
            }
        };
        // listener will be called every time 160 frames are reached
        recorder.setPositionNotificationPeriod(160);
        recorder.setRecordPositionUpdateListener(posUpdateListener);

        Log.e(LOG_TAG, "inside constructor");
    }

    private void init() {
        LOG_TAG = "initFunc";
        // int[] mSampleRates = new int[] { 8000, 11025, 22050, 44100 };
        short audioFormat = AudioFormat.ENCODING_PCM_16BIT;
        // for (int rate : mSampleRates) {
        this.recordingSampleRate = AudioTrack.getNativeOutputSampleRate(AudioManager.STREAM_MUSIC);
        try {
            // Log.d(LOG_TAG, "Attempting rate " + rate + "Hz, bits: " +
            // audioFormat);
            int bufrSize = AudioRecord.getMinBufferSize(this.recordingSampleRate,
                    AudioFormat.CHANNEL_IN_MONO, audioFormat);

                // lets find out the minimum required size for AudioTrack
            int audioTrackBufSize = AudioTrack.getMinBufferSize(this.recordingSampleRate,
                    AudioFormat.CHANNEL_OUT_MONO, audioFormat);

            if (bufrSize != AudioRecord.ERROR_BAD_VALUE
                    && bufrSize != AudioRecord.ERROR) {
                // check if we can instantiate and have a success
                if(audioTrackBufSize >= bufrSize){
                    this.recorderBufSize = audioTrackBufSize;
                }else{
                    this.recorderBufSize = bufrSize;
                }

                AudioRecord rec = new AudioRecord(
                        MediaRecorder.AudioSource.DEFAULT, this.recordingSampleRate,
                        AudioFormat.CHANNEL_IN_MONO, audioFormat, this.recorderBufSize);

                if (rec != null
                        && rec.getState() == AudioRecord.STATE_INITIALIZED) {

                    // storing variables for future use . . .
//                  this.recordingSampleRate = rate;
//                  this.recorderBufSize = bufrSize;

                    Log.e(LOG_TAG,
                            "Returning..(rate:channelConfig:audioFormat:recorderBufSize)"
                                    + this.recordingSampleRate + ":" + AudioFormat.CHANNEL_IN_MONO
                                    + ":" + audioFormat + ":" + this.recorderBufSize);

                    // Now create an instance of the AudioTrack
//                  int audioTrackBufSize = AudioTrack.getMinBufferSize(rate,
//                          AudioFormat.CHANNEL_OUT_MONO, audioFormat);

                    Log.e(LOG_TAG, "Audio Record / Track / Final buf size :" + bufrSize + "/ " +audioTrackBufSize + "/ "+this.recorderBufSize);


                    this.player = new AudioTrack(AudioManager.STREAM_MUSIC,
                            this.recordingSampleRate, AudioFormat.CHANNEL_OUT_MONO, audioFormat,
                            this.recorderBufSize, AudioTrack.MODE_STREAM);

                    this.recorder = rec;
                    this.player.stop();
                    this.player.flush();
                    this.player.setPlaybackRate(this.recordingSampleRate);
                    return;
                }
            }
        } catch (IllegalArgumentException e) {
            Log.d(LOG_TAG, this.recordingSampleRate + "Exception, keep trying.", e);
        } catch (Exception e) {
            Log.e(LOG_TAG, this.recordingSampleRate + "Some Exception!!", e);
        }
        // for loop for channel config ended here. . . .
        // for loop for audioFormat ended here. . .
        // }// for loop for sampleRate
        return;
    }

    private void startPlaying() {
        LOG_TAG = "startPlaying";

        Log.e(LOG_TAG, "start Playing");
    }

    private void stopPlaying() {
        LOG_TAG = "stopPlaying";

        Log.e(LOG_TAG, "stop Playing");
    }

    private void startRecording() {
        LOG_TAG = "startRecording"; 

        /* start a separate recording thread from here . . . */
        startRecThread = new Thread() {
            @Override
            public void run() {
                // TODO Auto-generated method stub
                android.os.Process
                        .setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
//              String LOG_TAG = Thread.currentThread().getName();
                if(recorder.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING){
                    recorder.startRecording();
                }
//              Log.e(LOG_TAG, "running" +recorder.getRecordingState());
                while (recorder.getRecordingState() == AudioRecord.RECORDSTATE_RECORDING) {
                    recorder.read(audioData, 0, audioData.length);
                    try {

                         Thread.sleep(1000); // sleep for 2s
                    } catch (InterruptedException e) {
                        // TODO Auto-generated catch block
                        Log.e("run Method", "recorder thread is interrupted");
                        e.printStackTrace();
                    }
                }
            }
        };
        setVolumeControlStream(AudioManager.STREAM_MUSIC);
        audioManager.setSpeakerphoneOn(false);
        player.flush();     
        player.play();
        startRecThread.start();

        Log.e(LOG_TAG, "start Recording");
    }

    private void stopRecording() {
        LOG_TAG = "stopRecording";
        recorder.stop();

        if (startRecThread != null && startRecThread.isAlive()) {           
            startRecThread.destroy();
            startRecThread = null;
        }

        player.stop();
        player.flush();
        Log.e(LOG_TAG, "stop Recording");
    }

    private void stop() {
        if (isRecording) {
            isRecording = false;
            stopRecording();
        }
        if (isPlaying) {
            isPlaying = false;
            stopPlaying();
        }
        recordBtn.setEnabled(true);
        playBtn.setEnabled(true);
    }

    @Override
    public void onCreate(Bundle icicle) {
        super.onCreate(icicle);
        LOG_TAG = "onCreate";
//      Log.e(LOG_TAG, "Create Called");
        // getting the audio service
        audioManager = (AudioManager) getSystemService(AUDIO_SERVICE);
        LinearLayout ll = new LinearLayout(this);

        // creating Buttons one by one . . . .
        // button to start the recording process
        recordBtn = new Button(this);
        recordBtn.setText("Record");
        recordBtn.setOnClickListener(new OnClickListener() {
            @Override
            public void onClick(View v) {
                // TODO Auto-generated method stub
                recordBtn.setEnabled(false);
                playBtn.setEnabled(false);
                isRecording = true;
                startRecording();
            }
        });
        // single button to stop recording and playing as applicable
        stopBtn = new Button(this);
        stopBtn.setText("Stop");
        stopBtn.setOnClickListener(new OnClickListener() {
            @Override
            public void onClick(View v) {
                // TODO Auto-generated method stub
                stop();
            }
        });
        // button to play the recorded sound
        playBtn = new Button(this);
        playBtn.setText("Play");
        playBtn.setOnClickListener(new OnClickListener() {

            @Override
            public void onClick(View v) {
                // TODO Auto-generated method stub
                // reverse the isPlaying
                isPlaying = true;
                recordBtn.setEnabled(false);
                playBtn.setEnabled(false);
                startPlaying();
            }
        });

        ll.addView(recordBtn, new LinearLayout.LayoutParams(
                ViewGroup.LayoutParams.WRAP_CONTENT,
                ViewGroup.LayoutParams.WRAP_CONTENT, 1));

        ll.addView(playBtn, new LinearLayout.LayoutParams(
                ViewGroup.LayoutParams.WRAP_CONTENT,
                ViewGroup.LayoutParams.WRAP_CONTENT, 1));

        ll.addView(stopBtn, new LinearLayout.LayoutParams(
                ViewGroup.LayoutParams.WRAP_CONTENT,
                ViewGroup.LayoutParams.WRAP_CONTENT, 1));

        setContentView(ll);
    }

    @Override
    protected void onDestroy() {
        // Clean up code . ..
        super.onDestroy();
        if (recorder != null)
            recorder.release();
        if (startRecThread!=null && startRecThread.isAlive())
            startRecThread.destroy();
        if (recorder != null)
            recorder.release();
        if (player != null)
            player.release();
        startRecThread = null;
        recorder = null;
        player = null;
        recordBtn = null;
        stopBtn = null;
        playBtn = null;
        audioData = null;
        System.gc();
    }

}

As you might see that startPlaying() and stopPlaying() functions are not yet implemented so lets not talk about them. Currently I am just trying to record and play.When I run the program It plays the recorded audio but the audio appears coming from a distance. Another problem is that UI thread of the app hangs though I have a separate thread for reading the Audio . Please help....

Solution 1

If your requirement is while it is recording it should play(means looping back audio), In your while loop thread, you are storing the recorded data (audioData bufer), there itself you can copy it to player object with in the while loop (player.write(audioData, 0, numShortsRead);). You said like your UI thread is stuck, it might be because of you are giving more priority to Audio record thread.

Check the below the code which I used for above loop back requirement

boolean m_isRun=true;
public void loopback() {
        // Prepare the AudioRecord & AudioTrack
        try {
            buffersize = AudioRecord.getMinBufferSize(SAMPLE_RATE,
                    AudioFormat.CHANNEL_CONFIGURATION_MONO,
                    AudioFormat.ENCODING_PCM_16BIT);

            if (buffersize <= BUF_SIZE) {
                buffersize = BUF_SIZE;
            }
            Log.i(LOG_TAG,"Initializing Audio Record and Audio Playing objects");

            m_record = new AudioRecord(MediaRecorder.AudioSource.MIC,
                    SAMPLE_RATE, AudioFormat.CHANNEL_CONFIGURATION_MONO,
                    AudioFormat.ENCODING_PCM_16BIT, buffersize * 1);

            m_track = new AudioTrack(AudioManager.STREAM_ALARM,
                    SAMPLE_RATE, AudioFormat.CHANNEL_CONFIGURATION_MONO,
                    AudioFormat.ENCODING_PCM_16BIT, buffersize * 1,
                    AudioTrack.MODE_STREAM);

            m_track.setPlaybackRate(SAMPLE_RATE);
        } catch (Throwable t) {
            Log.e("Error", "Initializing Audio Record and Play objects Failed "+t.getLocalizedMessage());
        }

        m_record.startRecording();
        Log.i(LOG_TAG,"Audio Recording started");
        m_track.play();
        Log.i(LOG_TAG,"Audio Playing started");

        while (m_isRun) {
            m_record.read(buffer, 0, BUF_SIZE);
            m_track.write(buffer, 0, buffer.length);
        }

        Log.i(LOG_TAG, "loopback exit");
    }

    private void do_loopback() {
        m_thread = new Thread(new Runnable() {
            public void run() {
                loopback();
            }
        });

One more thing, If your requirement is record for few seconds and then play, while it is playing your record should start again, you can do that with a delay handler thread with a time out, In that thread you can stop recording copy the buffer, then start recording.

Solution 2

My suggestion is to use Async task other wise known as painless threading. That's the best way to leverage from the threading in Android. You can make use of background processing, pre-execute and post execute methods by dividing your existing program.

http://android-developers.blogspot.ca/2009/05/painless-threading.html

Comments

[L7ColWinters]

choose one rate, config, and format. Then stick with it or make your init function take in the property and switch through it.

[Amit]

@L7ColWinters thanks and plz see the edited question....

[L7ColWinters]

is your desired functionality to be able to say something while recording and it automatically plays it back without hitting play? Thats what your current code does do, although it does hang and play a very high pitch squeel, which might be feedback.

[Amit]

@L7ColWinters, Yes thats all I want (to record and play it back, the play button will be used for playing a file which is not yet implemented), You got my problems why my program hangs and plays such a bad audio.... do I need to use same buffer size for AudioRecord and AudioTrack both ? plz help...

[Amit]

@L7ColWinters, plz see the updated startRecording().... in which I call Thread.sleep(20) this sleeps the recorder thread for 20ms ( I read somewhere that 20ms is the perfect delay... for such type of apps..... Thanks once again...

[Amit]

It seems in somewhat working condition when I made both the Recorder and Player with equal bufferSize... However the UI thread still hangs (and I wonder why ?) though I have made a separate thread for recording.... can you help in it ?

[Rahul Baradia]

Can you please post your running code. I am unable to do this since 2days.. or provide me a link through which i can get help. Thank you. +100 for this question.

[Amit]

@Tech.Rahul, the code pasted in candy's answer also works, however I have a diff code since my app is quite big... I suggest first you try candy's code and if that doesn't work. I will paste my Code as a separate answer... Good luck buddy, and don't get nervous it takes time at the beginning level.

[Rahul Baradia]

@anDroider -- thanks a lot.. i ll try & let you know..

[Rahul Baradia]

@anDroider : please can you post your code. I am not able to solve it. Thanks..

[Amit]

have u set the necessary permission in the AndroidManifiest.xml, I have assigned following permissions <uses-permission android:name="android.permission.RECORD_AUDIO" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

[Rahul Baradia]

ya I am using that.. can u please post your code..

[Rahul Baradia]

stackoverflow.com/questions/10619675/… .... this is my question which i asked. I am getting some errors can you help me out. thanks

[Amit]

how u decide BUF_SIZE value ? I have also used positionUpdateListener() what are your views on its importance... and one more thing what is the stop condition of while loop ? and does you main thread hangs during the process ?

[candy]

for stop condition, I am using keyevent from adb commands as per my requirement. In that I am making "m_isRun" to false, you can use a time out thread, or a button event. I don't think that main thread stops, previously I was using button events instead of keyevents. if it is exactly a loopback requirement, I dont think position update is also required.

[candy]

And In BUF_SIZE case, when I was simply using getMinBufferSize(), it was giving buffer not sufficient issues. some where in net links it is mentioned like buffer size should larger than SAMPLE_RATE. I go on this. But I can not say that this is perfect solution for buffer size issue

[Sandra]

Can you just explain what this m_isRun variable is for? I also want to do audio recording and playing simultaneously, and I am a beginner in this area so I need a little help:) +1 for your answer

[candy]

m_isRun is a simple boolean variable which I used to start/stop record-play

[Sandra]

I followed your example and my app works for about 2 seconds, if I say something I hear my voice back, and suddenly the stream is finished, you hear nothing, and this message is displayed over and over int the log: obtainBuffer() track 0x16ab30 disabled, restarting. Did you have these kind of difficulties or know how to solve the problem? Thank you for your answer and trouble.

[candy]

try changing the buffer size.

[Arunraj Shanmugam]

I tried this code, but noise and echo gets build while using. How to avoid it?