How to stream MP3 data via WebSockets with node.js and socket.io?

javascript html node.js websocket socket.io

24,088

Solution 1

I've found a way to stream MP3 data via Websockets myself.

One problem was the chunk size of the MP3 data. It seems that the Web Audio API needs to be fed with valid MP3 chunks to be able to decode the data. Probably not surprising. In my demo app I provide a set of MP3 chunk files.

Also the quality of the audio is not perfect. I have some subtle glitches. I was able to improve that by sending larger chunks of MP3 data but there are still tiny crackles.

EDIT: I managed to improve the audio quality. It seems the Web Audio method decodeAudioData isn't really designed to decode continuos chunks of MP3 data.

Solution 2

In your case context.decodeAudioData expects an ArrayBuffer of the binary data, I would suggest converting your chunk to a base64 string, then to an ArrayBuffer client-side for the most predictable results. This script should be a good starting point for the client-side decode from base64 of the chunked data.

Adding a line with data = Base64Binary.decodeArrayBuffer(data); right after getting your data (base-64 encoded string) would do the trick...

24,088

Author by

Jan Deinhard

Updated on September 18, 2020

Comments

Jan Deinhard over 3 years

I have problems streaming MP3 data via WebSocket with node.js and socket.io. Everything seems to work but decodeAudioData doesn't play fair with me.

This is my toy server:

var app = require('http').createServer(handler)
  , io = require('socket.io').listen(app)
  , fs = require('fs')

app.listen(8081);

function handler (req, res) {
    res.writeHead(200, {
        'Content-Type': 'text/html',
    });
    res.end('Hello, world!');
}

io.configure('development', function() {
  io.set('log level', 1);

  io.set('transports', [ 'websocket' ]);
});

io.sockets.on('connection', function (socket) {
    console.log('connection established');

    var readStream = fs.createReadStream("test.mp3", 
                                         {'flags': 'r',
                                          'encoding': 'binary', 
                                          'mode': 0666, 
                                          'bufferSize': 64 * 1024});
    readStream.on('data', function(data) {
        console.log(typeof data);
        console.log('sending chunk of data')
        socket.send(data);
    });

    socket.on('disconnect', function () {
        console.log('connection droped');
    });
});

console.log('Server running at http://127.0.0.1:8081/');

The client receives the data as type string but I want to feed the data to decodeAudioData and it seems it doesn't like strings. The call to decodeAudioData results in the following error message:

Uncaught Error: SYNTAX_ERR: DOM Exception 12

I think decodeAudioData needs the data stored in an ArrayBuffer. Is there a way to convert the data?

This is the client code:

<script src="http://127.0.0.1:8081/socket.io/socket.io.js"></script>
<script>
    var audioBuffer = null;
    var context = null;
    window.addEventListener('load', init, false);
    function init() {
        try {
            context = new webkitAudioContext();
        } catch(e) {
            alert('Web Audio API is not supported in this browser');
        }
    }

    function decodeHandler(buffer) {
        console.log(data);
    }

    var socket = io.connect('http://127.0.0.1:8081');
    socket.on('message', function (data) {
            // HERE IS THE PROBLEM
        context.decodeAudioData(data, decodeHandler, function(e) { console.log(e); });
    });
</script>

Jan Deinhard over 12 years

Thanks for your answer but I can't get it working with the script. The resulting ArrayBuffer is longer than it should be. I'd like to avoid Base64 anyway because of it's overhead. I think I've found a solution myself. I'm going to post it here soon.
fyasar over 11 years

@Fair Dinkum Thinkum I would like to know your solution.
Jan Deinhard over 11 years

@fyasar My solution was to use the ws module instead of socket.io. At that time socket.io wasn't capable of handling binary data. Not sure if that is still the case.
Light almost 11 years

Can you please post your solution? I am also trying to stream MP3 data to websocket.
Light almost 11 years

@Fair Dinkum Thinkum can you please post your solution?
Brad.Smith over 10 years

Please supply your solution.
Casey Chow almost 10 years

For anyone still looking, the solution is on his Github page: github.com/polaris/audio-demo
cmarrero01 about 9 years

@FairDinkumThinkum Socket.io 1.0 have support for binary data, but I stil cant make it works.. socket.io/blog/introducing-socket-io-1-0/#binary you can share your solution ?
Jan Deinhard about 9 years

@cmarrero01 I'd love to share it but I can't. I lost the code. I know that it works with Socket.io now because a colleague implemented the solution with Socket.io to support older browsers some time ago. I remember that is was tricky because of sparse documentation.
lintu over 7 years

@digitxp github project does not have anything related to streaming
Reyan Tropia over 3 years

How did you solve this? We're doing this but the audio sound is distorted/static noises.
Chinh Nguyen almost 3 years

how did you improve the audio quality, i really want to know the solution