WebRTC Failed to Add a Third Peer: "Cannot set remote answer in state stable"

Question

I am writing a multi-peer WebRTC video chat. Two peers have no trouble connecting, no error or warning in console, and video works well, but I cannot add a third party to the chat successfully.

On the host (the first participant, Firefox), the error appear as "Cannot set remote answer in state stable" when trying to create an answer. At the second participant (Chrome), the error is "Failed to set remote answer sdp: Called in wrong state: STATE_INPROGRESS". At he third peer, the error is "the error is "Failed to set remote answer sdp: Called in wrong state: STATE_RECEIVEDINITIATE".

As it turn out, the first peer failed to have video with the third peer. Other two links appear fine.

Generally, my communication model is as below, self_id is a unique id per each peer in the session, and locate_peer_connection() will return the local peer_connection of the particular peer from which we receive message:

• a new client send "peer_arrival" to the session using signalling server

• all peers already in the session setlocaldescription, create offer and send to the new client

• new client create answers to all other peers and setremotedescription

• new client has video coming up

Signalling is done using WebSocket on a node.js server.

I have some of the core code below, some more note:

• self_id is an unique id per client in a session

• peer_connection stores peerConnection to other nodes, and peer_id store the respective user_id of these objects

• local_stream is the local video stream from getUserMedia (already considered different browser)

Any insights in to the issue? Is there something wrong with my model?

// locate a peer connection according to its id
function locate_peer_connection(id) {
   var index = peer_id.indexOf(id);
   // not seen before
   if (index == -1) {
    add_peer_connection();
     peer_id.push(id);
    index = peer_id.length - 1;
  }
   return index;
 }

// add a peer connection
function add_peer_connection() {
  console.log('add peer connection');
  // add another peer connection for use
  peer_connection.push(new rtc_peer_connection({ "iceServers": [{ "url": "stun:"+stun_server }]}));

  // generic handler that sends any ice candidate to the other peer
  peer_connection[peer_connection.length - 1].onicecandidate = function (ice_event) {
    if (ice_event.candidate) {
      signaling_server.send(
        JSON.stringify({
          type: "new_ice_candidate",
          candidate: ice_event.candidate,
          id: self_id,
          token:call_token
        })
      );
      console.log('send new ice candidate, from ' + self_id);
    }
  };

  // display remote video streams when they arrive using local <video> MediaElement
  peer_connection[peer_connection.length - 1].onaddstream = function (event) {
    video_src.push(event.stream); // store this src
    video_src_id.push(peer_connection.length - 1);
    if (video_src.length == 1) { // first peer
      connect_stream_to_src(event.stream, document.getElementById("remote_video"));
      // video rotating function
      setInterval(function() {
        // rorating video src
        var video_now = video_rotate;
        if (video_rotate == video_src.length - 1) {
          video_rotate = 0;
        } else {
          video_rotate++;
        }
        var status = peer_connection[video_src_id[video_rotate]].iceConnectionState;
        if (status == "disconnected" || status == "closed") { // connection lost, do not show video
          console.log('connection ' + video_rotate + ' liveness check failed');
        } else if (video_now != video_rotate) {
          connect_stream_to_src(video_src[video_rotate], document.getElementById("remote_video"));
        }
      }, 8000);
      // hide placeholder and show remote video
      console.log('first remote video');
      document.getElementById("loading_state").style.display = "none";
      document.getElementById("open_call_state").style.display = "block";
    }
    console.log('remote video');
  };
  peer_connection[peer_connection.length - 1].addStream(local_stream);
}

// handle new peer
function new_peer(signal) {
  // locate peer connection
  var id = locate_peer_connection(signal.id);
  console.log('new peer ' + id);
  // create offer
  peer_connection[id].createOffer(function(sdp) {
    peer_connection[id].setLocalDescription(sdp, 
    function() { // call back
      console.log('set local, send offer, connection '+ id);
      signaling_server.send(
        JSON.stringify({
          token: call_token,
          id: self_id,
          type:"new_offer",
          sdp: sdp
        })
      );
    }, log_error);
  }, log_error);
}

// handle offer
function new_offer_handler(signal) {
  var id = locate_peer_connection(signal.id);
  console.log('new offer ' + id);
  // set remote description
  peer_connection[id].setRemoteDescription(
    new rtc_session_description(signal.sdp), 
    function() { // call back
      peer_connection[id].createAnswer(function(sdp) {
        peer_connection[id].setLocalDescription(sdp, function () {
          console.log('set local, send answer, connection '+ id);
          signaling_server.send(
            JSON.stringify({
              token: call_token,
              id: self_id,
              type:"new_answer",
              sdp: sdp
            })
          );
        }, 
        log_error);
    }, log_error);
  }, log_error);
}

// handle answer
function new_answer_handler(signal) {
  var id = locate_peer_connection(signal.id);
  console.log('new answer ' + id);
  peer_connection[id].setRemoteDescription(new rtc_session_description(signal.sdp),
    function() {
      console.log('receive offer answer, set remote, connection '+ id);
    }
    , log_error);
}

// handle ice candidate
function ice_candidate_handler(signal) {
  var id = locate_peer_connection(signal.id);
  console.log('get new_ice_candidate from ' + id);
  if (typeof(RTCIceCandidate) != "undefined") {
    peer_connection[id].addIceCandidate(
        new RTCIceCandidate(signal.candidate)
    );
  } else { // firefox
    peer_connection[id].addIceCandidate(
      new mozRTCIceCandidate(signal.candidate)
    );
  }
}

function event_handler(event) {
  var signal = JSON.parse(event.data);
  if (signal.type === "peer_arrival") {
    new_peer(signal);
  } else if (signal.type === "new_ice_candidate") {
    ice_candidate_handler(signal);
  } else if (signal.type === "new_offer") { // get peer description offer
    new_offer_handler(signal);
  } else if (signal.type === "new_answer") { // get peer description answer
    new_answer_handler(signal);
  } else if (signal.type === "new_chat_message") { // chat message and file sharing info
    add_chat_message(signal);
  } else if (signal.type === "new_file_thumbnail_part") { // thumbnail
    store_file_part(signal.name, "thumbnail", signal.id, signal.part, signal.length, signal.data);
    if (file_store[signal.id].thumbnail.parts.length == signal.length) {
      document.getElementById("file_list").innerHTML = get_file_div(signal.id, signal.name)+document.getElementById("file_list").innerHTML;
      document.getElementById("file-img-"+signal.id).src = file_store[signal.id].thumbnail.parts.join(""); 
    }
  } else if (signal.type === "new_file_part") { // file
    console.log('get new_file_part ' + signal.id);
    store_file_part(signal.name, "file", signal.id, signal.part, signal.length, signal.data);
    update_file_progress(signal.name, signal.id, file_store[signal.id].file.parts.length, signal.length);
  }
}

// generic error handler
function log_error(error) {
  console.log(error);
}

Solution 1

I might be wrong, but these two must be your major problems:

1. signaling_server.send(... I am not seeing any target here, so guessing that the server just broadcasts this message to everyone. When you are sending an sdp to already established peer connection, you are bound to get the error which you are getting now. My suggesting would be to add a target id in the message, either the server can forward it to that particular peer or, server can just broadcast, but event_handler of the peer can check if the target id of message is same as it's own id, if not, just ignore the message.

2. onicecandidate event, you are broadcasting the ICE candidates to all remote peers, again this is meant for single peer, another issue might be, addIceCandidate on PeerConnection before setting it's local and remote description would throw error, you need to some sort of mechanism to handle this( add ICE candidates only after setting the connections descriptions).

finally a suggestion. I am guessing peer_connection is an Array, if you change it Object, you can remove the redundancy of locate_peer_connection,

you can do something like.

if(peer_connection[signal.id]){
    //do something...
}else{
    peer_connection[signal.id] = new PeerConnection(...
}

Solution 2

i had the same problem when i was implementing one-to-many rtc broadcast, and what mido22 said is right. you might be sending/resetting existing established peer object with other incoming client. you have to create new RTCPeerConenction object for every new incoming client. the general work flow would be as follow.

peerconnection=[];//empty array

then you initialize your media devices with getUserMedia and store media stream into global variable so that it can be added when creating offer.
once this is done you inform your singalling server with unique id
your signalling server may then broadcast this to all clients except from which it is received. each client will then check if that unique id does exist in there peerconnection array and like mido22 said you can do this as

if(peerconnection[signal.id])
{
//do something with existing peerconnections
}
else
{
peerconnection[signal.id]=new RTCPeerConnection({"stun_server_address"});
//register peerconnection[signal.id].onicecandidate callback
//register peerconnection[signal.id].onaddstream callback
//createoffer
//if local stream is ready then
peerconnection[signal.id].addStream(localStream);
//and rest of the stuff go as it is  like in one-to-one call..
//setLocalDescriptor setRemoteDescriptor
}