How to perfectly sync two or more html5 video tags?

Question

Is there any way to have two or more (preferably three) html5 < video > tags playing simultaneously and to be in perfect sync.

If I have let's say three tiles of one video and I want them to appear in browser as one big video. They need to be perfectly synchronized. Without even smallest visual/vertical hint that they are tiled.

Unfortunately I cannot use MediaController because it is not supported well enough.

I've tried some workouts, including canvases, but I still get visual differentiation. Has anyone had any similar problem/solution?

Solution 1

Disclaimer: I'm not a video guy, but here are some thoughts anyway.

If they need to be absolutely perfect...you are fighting several problems at once:

1. A device might not be powerful enough to acquire, synchronize and render 3 streams at once.

2. Even if #1 is solved, a device is never totally dedicated to your task. For example, it might pause for garbage collection between processing stream#1 and stream#2--resulting in dropped/unsynchronized frames.

So to give yourself the best chance at perfection, you should first merge your 3 videos into 1 vertical video in the studio (or using studio software).

Then you can use the extended clipping properties of canvas context.drawImage to break each single frame into 2-3 separate frames.

Additionally, buffer a few frames you acquire on the stream (this goes without saying!).

Use requestAnimationFrame (RAF) to control the drawing. RAF does a fairly good job of drawing frames when system resources are available and delaying frames when system resources are lacking.

Your result won't be perfect, but they will be synchronized. You will always have to make the decision whether to drop or delay frames when system resources are unavailable, but at least the frames you do present will be synchronized.

Solution 2

As far as I know it's currently impossible to play HTML5 video frame-by-frame, or seek to a frame accurate time-code. The nearest seek seems to be precise to roughly 1-second.

But you can still get pretty close using the some of the media frameworks:

1. Popcorn.js library made for synchronizing video with content.
2. mediagroup.js another library used to add support for mediagroup attributes on HTML5 media elements

Solution 3

The only feature that allowed that is named mediaGroup and it was removed from Chrome(apparently for not being popular enough). It's still present in WebKit. Relevant discussion here and here.

I think you can implement you own "mediagroup"-like tag using wasm though without DOM support it may be tricky.

Comments

[1934286]

Did you ever find anything better than the answers suggest?

[edin-m]

@fredsbend I found there's definitely no way to fully do this. I don't have updates past two years but I doubt it's possible now.

[themihai]

If you have synchronization primitives you can sync multiple streams for sure regardless the power of the device... if the device is not powerful enough it would buffer/pause the player. It's a solved problem. Of course you can't do that in JavaScript...maybe in WebAssembly(though lack of DOM may be a bumper) or a WebAPI (i.e. mediaGroup)

[1934286]

You seem to be suggesting that in production, all three videos are put into a single oblong video with irregular dimensions. Then in HTML, use css to clip three video elements to display them as separate items on the page. This approach does work (I've done it), but all browsers seem to treat each video element separately and will download the video source three times, even though it is the exact same file. Maybe that's changed in the three years since I've tried this.

[1934286]

This solution was a deal breaker for me. I opted for less than perfect sync with something like vid1.currentTime = vid2.currentTime on play, pause, and seek event listerners. I tried to work in an interval checker, but it just made the video choppy, as it was always seeking, even when I used parseInt(vid1.currentTime), so that it would round out to an even second. I thought to add some time to the following video, but then I figured that would add even more inconsistency among various user download speeds.