How to convert javascript array to binary data and back for websocket?

Question

I want send this example array

[{
    id: 1,
    name: "test",
    position: [1234,850], //random position on the map
    points: 100 //example points
}];

to my websocket server as binary data. At server side I want decode that binary data back into array, make changes and send back binary data to client. And finally at client side how to decode binary data back into array?

Example screenshot what I mean:

This is my actual code:

var connection = new WebSocket('wss://my_website.eu:1234');
connection.binaryType = "ArrayBuffer";
connection.onmessage = function (event) {

    // console.log(event);
    if (event.data instanceof window["ArrayBuffer"]) {
        var data3 = JSON.parse(String.fromCharCode.apply(null, new Uint16Array(event.data)));
        console.log(data3);
    } else {
        console.log(event.data); // Blob {size: 0, type: ""}
    }
};

$("body").mousemove(function( event ) {

    var data = {
        name: "lol",
        pos: [event.pageX, event.pageY]
    };

    //convert to binary frame
    var data2 = new Uint16Array(data);
    console.log(data2); // []

    //try to convert back to array
    var data3 = String.fromCharCode.apply(null, new Uint16Array(data2));
    console.log(data3); // empty

    connection.send(data2); // Binary Frame (Opcode 2, mask) | length: 0

});

Server side code:

connection.on('message', function(message) {
    for (var i = players.length - 1; i >= 0; i--) {
        players[i].connection.send(message.binaryData);
    }
});

LATEST EDIT READ FROM HERE

I now can send message as binary frame to websocket server. I found functions to convert string to binary type and send it to ws server.

Now I have problem. This functions (below) not working at server-side. Example code:

var data = {
    name: "value"
};

connection.send(JSON.stringify(data));

This code working good. Now, when I try to send as array buffer:

var data = {
    name: "value"
};

connection.send(StringToArrayBuffer(JSON.stringify(data)));

output is not Binary frame, is just string "[object ArrayBuffer]":

I also tried:

connection.send(JSON.stringify(data), {binary: true, mask: false});

but this sending message as normal string, not binary frame.

So, how I can send Binary frame from websocket server to client? When I send back received binary message:

connection.on('message', function(message) {
    for (var i = players.length - 1; i >= 0; i--) {
        playerConnection[i].send(message.binaryData);
    }
}

only this works.

Solution 1

First of all, Browsers treat binary data differently than NodeJS. In browser, binaries can be seen as Blob or ArrayBuffer, but in NodeJS, it is seen as Buffer doesn't understand ArrayBuffer. I won't go in too deep into this, but you need to handle data differently between browser and nodeJS.

When using WebSocket at the browser side, data are transmitted as either string or binary, if binary will be used, then you have to specify BinaryType, and in this particular case, I will use ArrayBuffer.

As to string to buffer, I suggest to use the standard UTF-8 as there are 2 ways of encoding UTF-16. For example '\u0024' in UTF-16 will be stored as 00 24 in UTF-16BE, and in UTF-16LE, it is stored as 24 00. That is, if you are going to use UTF-16, then you should use TextEncoder and TextDecoder. Otherwise you can simply do this

strToAB = str =>
  new Uint8Array(str.split('')
    .map(c => c.charCodeAt(0))).buffer;

ABToStr = ab => 
  new Uint8Array(ab).reduce((p, c) =>
  p + String.fromCharCode(c), '');

console.log(ABToStr(strToAB('hello world!')));

For UTF-16, the browser code should be something like:

const ENCODING = 'utf-16le';
var ws = new WebSocket('ws://localhost');

ws.binaryType = 'arraybuffer';
ws.onmessage = event => {
  let str = new TextDecoder(ENCODING).decode(event.data),
    json = JSON.parse(str);
    console.log('received', json);
};
ws.onopen = () => {
  let json = { client: 'hi server' },
    str = JSON.stringify(json);
  console.log('sent',json);

  //JSON.toString() returns "[object Object]" which isn't what you want,
  //so ws.send(json) will send wrong data.
  ws.send(new TextEncoder(ENCODING).encode(str));
}

At the server side, data is stored as Buffer and it more or less does everything natively. You however need to specify Encoding unless it is UTF-8.

const ENCODING = 'utf-16le';
//You may use a different websocket implementation, but the core
//logic reminds as they all build on top of Buffer.
var WebSocketServer = require('websocket').server,
  http = require('http'),
  //This is only here so webSocketServer can be initialize.
  wss = new WebSocketServer({
    httpServer: http.createServer()
      .listen({ port: 80 })});

wss.on('request', request => {
  var connection = request.accept(null, request.origin);
  connection.on('message', msg => {
    if (msg.type === 'binary') {
      //In NodeJS (Buffer), you can use toString(encoding) to get
      //the string representation of the buffer.
      let str = msg.binaryData.toString(ENCODING);
      console.log(`message : ${str}`);

      //send data back to browser.
      let json = JSON.parse(str);
      json.server = 'Go away!';
      str = JSON.stringify(json);

      //In NodeJS (Buffer), you can create a new Buffer with a
      //string+encoding, and the default encoding is UTF-8.
      let buf = new Buffer(str, ENCODING);
      connection.sendBytes(buf);
    }
  });
});

Solution 2

Try it:

Sending data example:

var data = [{
    id: 1,
    name: "test",
    position: [1234, 850], //random position on the map
    points: 100 //example points
}];

var data2 = new Uint16Array(data);
socket.send(data2);

In your event onMessage websocket try it:

function onMessage(event) {

    if (event.data instanceof window["ArrayBuffer"]){
        var data3 = JSON.parse(String.fromCharCode.apply(null, new Uint16Array(event.data)));
    };

};

Comments

[Dave]

Received { type: 'binary', binaryData: <Buffer 00> }. Now convert it back to array?

[Dave]

Still not working. I tried add else { console.log("test"); } and thats working. I checked console.log(event.data) - data.event is Blob.

[toto]

before send data configure the socket.binaryType = "arraybuffer" or socket.binaryType = "blob". in your case is "arraybuffer"

[toto]

still you are receiving as blob data?

[Dave]

Yes. This is event.data: Blob {size: 0, type: ""}. Before .onmessage function I have line: connection.binaryType = "ArrayBuffer";

[toto]

is connection.binaryType = "arraybuffer"; not "ArrayBuffer"

[toto]

look this link using BSON library for javascript: stackoverflow.com/questions/13028604/…

[Dave]

Thank you its working. Only for information, try console.log(JSON.parse(ABToStr(strToAB('{a: "b"}')))); you will see error...

[Hin Fan Chan]

a needs to be in a bracket console.log(JSON.parse(ABToStr(strToAB('{"a": "b"}')))); I did however forgot to convert ab to uInt8Array in ABToStr.

[Erick Wendel]

It also worked to me like JSON.parse(String.fromCharCode.apply(null, new Uint8Array(data)));