webRTC convert webm to mp4 with ffmpeg.js

Question

I am trying to convert webM files to mp4 with ffmpeg.js. I am recording a video from canvas(overlayer with some information) and recording the audio data from the video.

stream = new MediaStream();
var videoElem = document.getElementById('video');
var videoStream = videoElem.captureStream();
stream.addTrack(videoStream.getAudioTracks()[0]);
stream.addTrack(canvas.captureStream().getVideoTracks()[0]);
var options = {mimeType: 'video/webm'};
  recordedBlobs = [];
  mediaRecorder = new MediaRecorder(stream, options);
  mediaRecorder.onstop = handleStop;
  mediaRecorder.ondataavailable = handleDataAvailable;
  mediaRecorder.start(100); // collect 100ms of data

function handleDataAvailable(event) {
  if (event.data && event.data.size > 0) {
    recordedBlobs.push(event.data);
  }
}
mediaRecorder.stop();

This code works as expected and returns a webm video

var blob = new Blob(recordedBlobs, {type: 'video/webm'});

Now I want a mp4 file and checked the ffmpeg.js from muaz-khan. The examples just show how to convert to mp4 when you have 2 single streams (audio and video). But I have one stream with an additional audio track. Can I convert such a stream to mp4? How can that be done?

Solution

As per the provided code sample, your recorder stream is having only one audio & one video tracks.

If your input file is having both Audio & Video, then you need to specify output codec for both tracks here as following.

worker.postMessage({
    type: 'command',
    arguments: [
       '-i', 'audiovideo.webm',
       '-c:v', 'mpeg4',
       '-c:a', 'aac', // or vorbis
       '-b:v', '6400k',  // video bitrate
       '-b:a', '4800k',  // audio bitrate
       '-strict', 'experimental', 'audiovideo.mp4'
     ],
    files: [
        {
            data: new Uint8Array(fileReaderData),
            name: 'audiovideo.webm'
        }
     ]
    });

Trans-coding the video inside browser is not recommend, as it will consume more CPU Time & Memory. And ffmpeg_asm.js is heavy. May be ok for POC :)

What is your use case? webm(vp8/vp9) is widely using these days.

Chrome will support following mime types:

"video/webm"
"video/webm;codecs=vp8"
"video/webm;codecs=vp9"
"video/webm;codecs=h264"
"video/x-matroska;codecs=avc1"

So you can get mp4 recording directly from chrome MediaRecorder with following hack

var options = {mimeType: 'video/webm;codecs=h264'}; 
mediaRecorder = new MediaRecorder(stream, options);
.....
//Before merging blobs change output mime 
var blob = new Blob(recordedBlobs, {type: 'video/mp4'});
// And name your file as video.mp4

Comments

[q-jack]

this is an awesome hack. Now I am saving a lot of code and can get rid of the ffmpeg_asm.js. Thank you very much.

[guest271314]

@Ajay "So you can get mp4 recording directly from chrome MediaRecorder with following hack" $ ffmpeg -i file.mp3 outputs Input #0, matroska,webm, from 'file.mp3': Metadata: encoder : Chrome Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default) and $ ffprobe -v error -select_streams a:0 -show_entries stream=codec_name -of default=noprint_wrappers=1:nokey=1 file.mp3 outputs opus for using approach with .wav file input.

[John Doe]

@Ajay what do you mean by "get mp4 recording directly"? Maybe i made some mistake, but in my test i only get the original webm/h264 video but with a different mime type. So the video container doesnt change. Thus the binary data is exactly the same after the new Blob(...) line, right? Thats what i tried: jsfiddle.net/zqd8qf6e Isn't the goal here that the video container changes from webm to mp4 without ffmpeg.js?

[Rishabh]

BTW size will be almost same in both format, don't expect this will apply compression as well like ffmpeg does

[NME New Media Entertainment]

@John Doe What you are saying is correct. The hack might change the the type to video/mp4. This will play in any browser including Safari and iOS. But the problem here is that the video container still will be WEBM. This results in the problem that the seeker bar doesn't work while playing the video in the browser.

[Jayna Tanawala]

@NMENewMediaEntertainment, So how to resolve the seeker bar issue?