Sound recognition in Android

Question

I want my Android app to recognize sound. For example I want to know if the sound from microphone is a clapping or knocking or something else.

Do I need to use math, or can I just use some library for that?

If there are any libraries for sound analysis please let me know. Thanks.

Solution 1

Musicg library is useful for whistle detection. Concerning claps, I wouldn't recommend use it, cause it reacts to every loud sound (even speech).

For clap and other percussive sounds detection I recommend TarsosDSP. It has a simple API with a rich functionality (pitch detection and so on). For clap detection you can use something like (if you use TarsosDSPAndroid-v3):

MicrophoneAudioDispatcher mDispatcher = new MicrophoneAudioDispatcher((int) SAMPLE_RATE, BUFFER_SIZE, BUFFER_OVERLAP);
double threshold = 8;
double sensitivity = 20;
mPercussionDetector = new PercussionOnsetDetector(22050, 1024, 
        new OnsetHandler() {

            @Override
            public void handleOnset(double time, double salience) {
                Log.d(TAG, "Clap detected!");
            }
        }, sensitivity, threshold);
mDispatcher.addAudioProcessor(mPercussionDetector);
new Thread(mDispatcher).start();

You can tune your detector by adjusting sensitivity (0-100) and threshold (0-20).

Good luck!

Solution 2

There is an Api that works very well for your needs in my opinion.

http://code.google.com/p/musicg/

Good Luck!!!

Solution 3

You don't need math and you don't need AudioRecord. Just check MediaRecorder.getMaxAmplitude() every 1000 milliseconds.

this code and this code might be helpful.

Here is some code you will need.

public class Clapper
{
    private static final String TAG = "Clapper";

    private static final long DEFAULT_CLIP_TIME = 1000;
    private long clipTime = DEFAULT_CLIP_TIME;
    private AmplitudeClipListener clipListener;

    private boolean continueRecording;

    /**
     * how much louder is required to hear a clap 10000, 18000, 25000 are good
     * values
     */
    private int amplitudeThreshold;

    /**
     * requires a little of noise by the user to trigger, background noise may
     * trigger it
     */
    public static final int AMPLITUDE_DIFF_LOW = 10000;
    public static final int AMPLITUDE_DIFF_MED = 18000;
    /**
     * requires a lot of noise by the user to trigger. background noise isn't
     * likely to be this loud
     */
    public static final int AMPLITUDE_DIFF_HIGH = 25000;

    private static final int DEFAULT_AMPLITUDE_DIFF = AMPLITUDE_DIFF_MED;

    private MediaRecorder recorder;

    private String tmpAudioFile;

    public Clapper() throws IOException
    {
        this(DEFAULT_CLIP_TIME, "/tmp.3gp", DEFAULT_AMPLITUDE_DIFF, null, null);
    }

    public Clapper(long snipTime, String tmpAudioFile,
            int amplitudeDifference, Context context, AmplitudeClipListener clipListener)
            throws IOException
    {
        this.clipTime = snipTime;
        this.clipListener = clipListener;
        this.amplitudeThreshold = amplitudeDifference;
        this.tmpAudioFile = tmpAudioFile;
    }

    public boolean recordClap()
    {
        Log.d(TAG, "record clap");
        boolean clapDetected = false;

        try
        {
            recorder = AudioUtil.prepareRecorder(tmpAudioFile);
        }
        catch (IOException io)
        {
            Log.d(TAG, "failed to prepare recorder ", io);
            throw new RecordingFailedException("failed to create recorder", io);
        }

        recorder.start();
        int startAmplitude = recorder.getMaxAmplitude();
        Log.d(TAG, "starting amplitude: " + startAmplitude);

        do
        {
            Log.d(TAG, "waiting while recording...");
            waitSome();
            int finishAmplitude = recorder.getMaxAmplitude();
            if (clipListener != null)
            {
                clipListener.heard(finishAmplitude);
            }

            int ampDifference = finishAmplitude - startAmplitude;
            if (ampDifference >= amplitudeThreshold)
            {
                Log.d(TAG, "heard a clap!");
                clapDetected = true;
            }
            Log.d(TAG, "finishing amplitude: " + finishAmplitude + " diff: "
                    + ampDifference);
        } while (continueRecording || !clapDetected);

        Log.d(TAG, "stopped recording");
        done();

        return clapDetected;
    }

    private void waitSome()
    {
        try
        {
            // wait a while
            Thread.sleep(clipTime);
        } catch (InterruptedException e)
        {
            Log.d(TAG, "interrupted");
        }
    }

    /**
     * need to call this when completely done with recording
     */
    public void done()
    {
        Log.d(TAG, "stop recording");
        if (recorder != null)
        {
            if (isRecording())
            {
                stopRecording();
            }
            //now stop the media player
            recorder.stop();
            recorder.release();
        }
    }

    public boolean isRecording()
    {
        return continueRecording;
    }

    public void stopRecording()
    {
        continueRecording = false;
    }
}

Solution 4

I realize this is a year old, but I stumbled across it. I'm pretty sure that general, open domain sound recognition is not a solved problem. So, no, you're not going to find any kind of library to do what you want on Android, because such code doesn't exist anywhere yet. If you pick some restricted domain, you could train a classifier to recognize the kinds of sounds your interested in, but that would require lots of math, and lots of examples of each of the potential sounds. It would be pretty cool if the library you wanted existed, but as far as I know, the technology just isn't there yet.

View more solutions

Comments

[coder]

Check out this post: stackoverflow.com/questions/2257075/…

[Elephant]

Yes, I've read about AudioRecord class. Method Read() of this class returns raw data, that needs to be analyzed using math. But I am asking if there is some third part APIs to analyze a sound without math?

[Elephant]

Code from your example will react to any loud sound (not only clapping). It cannot recognize the nature of sound. Am I right?

[gregm]

Correct This code will only recognize loud vs. not loud noises based on a threshold. It's very simple but useful for many applications

[Arpit Patel]

I am unable to detect clap sound using this. It will detect only whistles... Can you help me? I want to detect clap, whistle as well as finger snap sounds.

[Sagar Nayak]

@ArpitPatel did you successfully detected whistle sound in musicg api ?? i am getting error . please support me stackoverflow.com/questions/37925382/…

[KIRAN K J]

Can I get the library for the below requirement android.stackexchange.com/questions/237271/…