How to use web audio api to get raw pcm audio?

Question

How usergetmedia to use the microphone in chrome and then stream to get raw audio? I need need to get the audio in linear 16.

Solution 1

Unfortunately, the MediaRecorder doesn't support raw PCM capture. (A sad oversight, in my opinion.) Therefore, you'll need to get the raw samples and buffer/save them yourself.

You can do this with the ScriptProcessorNode. Normally, this Node is used to modify the audio data programmatically, for custom effects and what not. But, there's no reason you can't just use it as a capture point. Untested, but try something like this code:

const captureNode = audioContext.createScriptProcessor(8192, 1, 1);
captureNode.addEventListener('audioprocess', (e) => {
  const rawLeftChannelData = inputBuffer.getChannelData(0);
  // rawLeftChannelData is now a typed array with floating point samples
});

(You can find a more complete example on MDN.)

Those floating point samples are centered on zero 0 and will ideally be bound to -1 and 1. When converting to an integer range, you'll want to clamp values to this range, clipping anything beyond it. (The values can sometimes exceed -1 and 1 in the event loud sounds are mixed together in-browser. In theory, the browser can also record float32 samples from an external sound device which may also exceed that range, but I don't know of any browser/platform that does this.)

When converting to integer, it matters if the values are signed or unsigned. If signed, for 16-bit, the range is -32768 to 32767. For unsigned, it's 0 to 65535. Figure out what format you want to use and scale the -1 to 1 values up to that range.

One final note on this conversion... endianness can matter. See also: https://stackoverflow.com/a/7870190/362536

Solution 2

The only two examples I've found that are clear and make sense are the following:

AWS Labs: https://github.com/awslabs/aws-lex-browser-audio-capture/blob/master/lib/worker.js

The AWS resource is very good. It shows you how to export your recorded audio to "WAV format encoded as PCM". Amazon Lex, which is a transcription service offered by AWS requires the audio to be PCM encoded and wrapped in a WAV container. You can merely adapt some of the code to make it work for you! AWS has some additional features such as "downsampling" which allows you to change the sample rate without affecting the recording.

RecordRTC: https://github.com/muaz-khan/RecordRTC/blob/master/simple-demos/raw-pcm.html

RecordRTC is a complete library. You can, once again, adapt their code or find the snippet of code that encodes the audio to raw PCM. You could also implement their library and use the code as-is. Using the "desiredSampleRate" option for audio config with this library negatively affects the recording.

They are both excellent resources and you'll definitely be able to solve your question.

Solution 3

You should look into MediaTrackConstraints.sampleSize property for the MediaDevices.getUserMedia() API. Using the sampleSize constraint, if your audio hardware permits you can set the sample size to 16 bits.

As far as the implementation goes, well that's what the links are and google are for...

Comments

[Brad]

Welcome to StackOverflow. When giving an answer, it's good to put some relevant sample code in your answer. If you're just providing a link, you can leave that as a comment on the question. In any case, for this question, the capture bit depth doesn't matter as the Web Audio API always provides floating point samples. Also, you'll find that everything (except perhaps some specialty embedded hardware) supports 16-bit audio.

[Venryx]

Some libraries that support WebAudioAPI to Raw PCM audio are listed here: stackoverflow.com/a/57837816/2441655

[Gaurav Aggarwal]

after this conversion is there any way to convert it back to a audio media stream and play in audio tag??

[imbatman]

@Venryx very helpful stackoverflow link^^

[asnov]

I have "The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page." in Chrome 83.0.4103.97

[asnov]

removing () after function and adding ``` <button onclick="webaudio_tooling_obj()">Start</button> ``` helped me.