Playing PCM stream from Web Audio API on Node.js

Question

I'm streaming recorded PCM audio from a browser with web audio api.

I'm streaming it with binaryJS (websocket connection) to a nodejs server and I'm trying to play that stream on the server using the speaker npm module.

This is my client. The audio buffers are at first non-interleaved IEEE 32-bit linear PCM with a nominal range between -1 and +1. I take one of the two PCM channels to start off and stream it below.

var client = new BinaryClient('ws://localhost:9000');
var Stream = client.send();

recorder.onaudioprocess = function(AudioBuffer){
    var leftChannel = AudioBuffer.inputBuffer.getChannelData (0);
    Stream.write(leftChannel);
}

Now I receive the data as a buffer and try writing it to a speaker object from the npm package.

var Speaker = require('speaker');

var speaker = new Speaker({
  channels: 1,          // 1 channel
  bitDepth: 32,         // 32-bit samples
  sampleRate: 48000,     // 48,000 Hz sample rate
  signed:true
});

server.on('connection', function(client){
    client.on('stream', function(stream, meta){
        stream.on('data', function(data){
            speaker.write(leftchannel);
        });
    });
});

The result is a high pitch screech on my laptop's speakers, which is clearly not what's being recorded. It's not feedback either. I can confirm that the recording buffers on the client are valid since I tried writing them to a WAV file and it played back fine.

The docs for speaker and the docs for the AudioBuffer in question

I've been stumped on this for days. Can someone figure out what is wrong or perhaps offer a different approach?

Update with solution

First off, I was using the websocket API incorrectly. I updated above to use it correctly.

I needed to convert the audio buffers to an array buffer of integers. I choose to use Int16Array. Since the given audio buffer has a range in-between 1 and -1, it was as simple as multiplying by the range of the new ArrayBuffer (32767 to -32768).

recorder.onaudioprocess = function(AudioBuffer){

    var left = AudioBuffer.inputBuffer.getChannelData (0);

    var l = left.length;
    var buf = new Int16Array(l)

    while (l--) {
        buf[l] = left[l]*0xFFFF;    //convert to 16 bit
    }

    Stream.write(buf.buffer);

}

Solution

It looks like you're sending your stream through as the meta object.

According to the docs, BinaryClient.send takes a data object (the stream) and a meta object, in that order. The callback for the stream event receives the stream (as a BinaryStream object, not a Buffer) in the first parameter and the meta object in the second.

You're passing send() the string 'channel' as the stream and the Float32Array from getChannelData() as the meta object. Perhaps if you were to swap those two parameters (or just use client.send(leftChannel)) and then change the server code to pass stream to speaker.write instead of leftchannel (which should probably be renamed to meta, or dropped if you don't need it), it might work.

Note that since Float32Array isn't a stream or buffer object, BinaryJS might try to send it in one chunk. You may want to send leftChannel.buffer (the ArrayBuffer behind that object) instead.

Let me know if this works for you; I'm not able to test your exact setup right now.

Comments

[grant]

To either eliminate or identify the Speaker as the source of error, if you write the buffer on the receiving end to a wav in the same way that you did on the client, does it play back correctly (problem with the Speaker) or as a screeching sound (problem with the stream)? Alternatively, are you able to play a working AudioBuffer through the Speaker?

[TiansHUo]

Hi, can I find your code somewhere on github?

[Conor Patrick]

I indeed faulted with the websocket API. I updated my answer with the correct usage. This was part of the error and I figured out the other part which had to do with the audio buffer. Since you helped figure out part of the problem I'll award you the bounty.