Save both audio and video to one file using GStreamer from camera and microphone

Question

I need a way to record video from my camera and audio from my mic simultaneously, such that they are saved in one file.

Currently, I now how to record video...

gst-launch-1.0 -e autovideosrc ! videoconvert ! matroskamux ! filesink location=recording.mkv

and audio...

gst-launch-1.0 -e autoaudiosrc ! audioconvert ! wavenc ! filesink location=recording.wav

...separately.

What I need is a file that contains video from recording.mkv and audio from recording.wav.

---

I have found something that may work if tweaked a bit.

gst-launch-1.0 -e autovideosrc ! queue ! videoconvert ! mkv. autoaudiosrc ! queue ! audioconvert ! mkv. matroskamux name=mkv ! filesink location=test.mkv sync=false

This records video but not audio, though, as I said, I think it just needs to be tweaked a little bit to record audio as well.

Solution

Here it is:

gst-launch-1.0 -e autovideosrc ! queue ! videoconvert ! mkv. autoaudiosrc ! queue ! audioconvert ! mkv. matroskamux name=mkv ! filesink location=test.mkv sync=false

This is how it works:

1. We get video feed from webcam using autovideosrc.
2. Next, we place that feed in it's own thread by outputing it into queue.
3. Video feed from queue goes to videoconvert where it is converted (I'm not sure is this step necessary).
4. Finally, video feed goes into element named mkv, which is of type matroskamux. This element converts video feed to Matroska.
5. After that we get audio feed using autoaudiosrc.
6. We place feed from autoaudiosrc in another thread using queue.
7. From queue audio goes to audioconvert where it is converted (also not sure if necessary).
8. Converted audio goes to the same matroskamux element mkv. Here audio feed is converted to Matroska.
9. Since both audio and video feed go to the same element, they are merged into one Matroska feed.
10. At last, that Matroska feed is saved to a file using filesink. You'll probably have to set sync=false if your computer isn't a beast.

That's it. Also, if someone's wondering why I wrote in the question that this command doesn't record audio, it's because I forgot to turn mic on.

Comments

[Ravindra Bawane]

stackoverflow.com/questions/37416015/…

[Hanlon]

@music2myer So I have to record video and audio separately and then concatenate them? Can I not just save them to a same file at once (while recording)?

[Ravindra Bawane]

I don't know the features of gstreamer, but the answer for the question I linked suggested to read documentation on gst-launch, which while offering concat features, I suppose may also offer real-time audio & video capture, as suggested by the answer you figured out below. Glad you were able to get it all working.

[DGoiko]

Hi there. I'm using your command inside python gstreammer, but it is not working when I plug some audio / video src to the pipeline insteqad autovideosrc. Have you ever used python for this?

[Hanlon]

@DGoiko Hello! I didn't try it using Python but I did try it in C and it worked.

[DGoiko]

I've got it connected to an appsrc (tomething similar to this: github.com/streamlink/streamlink/blob/master/examples/… , but using uridecodebin instead of playbin and some custom bins, included one based in your pipeline to record) and it stops at the begining. The player itself is fine: If I connect audio and video tees with queues to autoaudiosrc + autovideosrc + flac audio recording everything works fine, but if I put the video queue it freezes at the beginning. Debug 3 shows nothing... Its strange.

[Hanlon]

Could you post some code? Preferably you could open a new question, but copying code to Pastebin is fine too. Just be sure to give me link here.