Capturing multiple webcams (uvcvideo) with OpenCV on Linux

Question

I am trying to simultaneously stream the images from 3 Logitech Webcam Pro 900 devices using OpenCV 2.1 on Ubuntu 11.10. The uvcvideo driver gets loaded for these.

Capturing two devices works fine, however with three I run into the out of space error for the third:

libv4l2: error turning on stream: No space left on device

I seem to be running into this issue: http://renoirsrants.blogspot.com.au/2011/07/multiple-webcams-on-zoneminder.html and I have attempted to do the quirks=128 (or pretty much any other power-of-two value) trick but to no avail. I also tried on another machine with two USB 2.0 hubs and connecting two cameras to one and the third camera to the second, which resulted into the same problem. I am initializing roughly as follows (using N cameras so the result is actually put into an STL vector):

cv::VideoCapture cap0(0); //(0,1,2..)

and attempting to capture all the cameras in a loop as

cap0.retrieve(frame0);

This works fine for N=2 cameras. When I set N=3 the third window opens but no image appears and the console is spammed full of V4L2 errors. Similarly, when I set N=2, and attempt to open the third camera in say Cheese (simple webcam capture application), this doesn't work either.

Now comes the big but: After trying guvcview by starting three instances of that, I was able to view three cameras at once (with no problems in terms of frame rate or related), so it does not seem to be a hardware issue. I figure there is some property that I should set, but I'm not sure what that is. I have looked into MJPEG (which these cameras seem to support), but haven't succeeded into setting this property, or detect in which mode (yuyv?) they are running if I start them from OpenCV.

Thoughts?

Solution 1

I had this problem too and have a solution that lets me capture 2 cameras at 640x480 with mjpeg compression. I am using a Creative "Live Cam Sync HD VF0770" which incorrectly reports its bandwidth requirements. The quirks=128 fix works for 320x240 uncompressed video. But for compressed (mjpg) format the quirks=128 does not work (it does nothing for compressed formats).

To fix this I modified the uvc driver as follows:

download the kernel sources

mkdir -p ~/Software/kernel-git
cd ~/Software/kernel-git
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
git checkout v3.2
# NOTE: `uname -r`  shows me my current kernel is 3.2.0-60-generic
# For a different kernel use a different tag

copy uvc dir:

mkdir -p ~/Software/uvcvideo_driver
cd ~/Software/uvcvideo_driver
#cp -a ~/Software/kernel-git/linux/drivers/media/usb/uvc .
cp ~/Software/kernel-git/linux/drivers/media/video/uvc .

modify Makefile

cd ~/Software/uvcvideo_driver/uvc
vi Makefile

        obj-m += aauvcvideo.o
        aauvcvideo-objs  := uvc_driver.o uvc_queue.o uvc_v4l2.o uvc_video.o uvc_ctrl.o \
              uvc_status.o uvc_isight.o
        all:
          make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

        clean:
          make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Force bandwith to 0x400 when compressed.

cd ~/Software/uvcvideo_driver/uvc
vw uvc_video.c
Find the uvc_fixup_video_ctrl() function.  At the end of the function add:
      if (format->flags & UVC_FMT_FLAG_COMPRESSED) {
        ctrl->dwMaxPayloadTransferSize = 0x400;
      }

build the aauvcvideo module:

make

remove old module and insert new one:

sudo rmmod uvcvideo
sudo insmod ./aauvcvideo.ko quirks=128

run gucview twice with compression in 2 different windows to test

guvcview --device=/dev/video1 --format=mjpg --size=640x480
guvcview --device=/dev/video2 --format=mjpg --size=640x480

Good luck! -Acorn

Solution 2

I had this exact problem, using three logitech quickcam pro 9000 cameras (using ubuntu). I could read from two, but not three. In my case, I wasn't using opencv, but was accessing the cameras through V4L2 directly, using memory-mapped IO. Simply put, there was not enough USB bandwidth to allocate three buffers.

I was reading in the uncompressed frames, however. As soon as I switched the format to MJPEG, the data was small enough, and I could read from the three cameras. I used libjpeg to decode the MJPEG stream.

I haven't looked into how to change the image format using OpenCV, but I do know that it needs to be MJPEG to fit all that data.

Before I switched to MJPEG, I spent a lot of time trying to access each camera one at a time, streaming a single frame before switching to the next. Not recommended!

Solution 3

Most likely there is USB bandwidth contention reported by the driver of the video capture device. Check if the pixel format is YUYV, which happens to be uncompressed. On the contrary, if the pixel format is MJPG (compressed), it is possible to have multiple devices on the same USB channel.

v4l2-ctl -d /dev/video0 --list-formats

The output would be something like below:

ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : 16bpp YUY2, 4:2:2, packed

The following are the possible solutions:

1. Use capture devices from different manufacturers such that the drivers loaded are different. Generally the same driver handling multiple devices need to handle the bandwidth effectively.
2. Use a PCI USB extension card if available to attach the 2nd USB video capture device. This workaround worked excellently for me when I tried attaching AVerMedia DVD EZMaker 7 that loaded the driver cx231xx.

Solution 4

OpenCV can be built to use either v4l or libv4l, and only the v4l version supports compressed formats, while the libv4l version supports just one uncompressed format for OpenCV 2.4.11. (See autosetup_capture_mode_v4l2() for v4l and the code following line 692 for libv4l.) OpenCV 3.0.0 does not improve much over 2.4.11 here; it still supports only uncompressed formats for libv4l.

Since your error mentions libv4l2, you seem to have the libv4l version and OpenCV captured uncompressed in your case. To build a v4l version of OpenCV, your cmake command should contain

-D WITH_LIBV4L=OFF

(WITH_LIBV4L was enabled by default for me.)

A note on bandwidth and USB. USB 2.0 (which virtually all webcams use) has a bandwidth of 480 Mbit/s. 640x480 at 30 fps and 24 bits/pixel uncompressed is about 221 Mbit/s, so one can use up USB 2.0 bandwidth quickly with uncompressed webcams. One gets 480 Mbit/s for each USB host controller, see this answer on how to list them. (USB hubs do not add host controllers, and several USB ports on a motherboard are typically connected to the same host controller. All devices and hubs connected to a host controller share the bandwidth.)

For webcams that reserve more USB bandwidth than they need, e.g., those with footnote [13] on the UVC driver page, the FIX_BANDWIDTH quirk can help. But the quirk works only for uncompressed formats (unless you do the kernel hack in Acorn's answer here). In my case (Ubuntu 14.04, many Microsoft LifeCam Cinemas at 320x240), the quirk worked when I used the libv4l version of OpenCV (four LifeCams on an ASMedia USB host controller worked well) but for the v4l version -- which I confirmed to use MJPEG -- I got a VIDIOC_STREAMON: No space left on device error as soon as I tried to capture from a second LifeCam! (For the same machine, Intel and VIA host controllers did better and each worked with two LifeCams for v4l; the LifeCam reserves 48% of USB 2.0 bandwidth.)

Solution 5

With kernel 4.15.0-34-generic on Ubuntu 18.04 and OpenCV 3.4 compiled with gstreamer/v4l support I am able to stream 3x720p on a single USB port using a powered hub using MJPG compression with gstreamer in python (using 2xC922 and 1xC920 cameras - the 10fps framerate isn't necessary for this to work):

def open_cam_usb(dev, width, height):
    gst_str = (
        "v4l2src device=/dev/video{} ! "
        "image/jpeg,widh=(int){},height=(int){},framerate=10/1,"
        "format=(string)RGB ! "
        "jpegdec ! videoconvert ! appsink"
    ).format(dev, width, height)
    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

View more solutions

Comments

[Micka]

does EZMaker 7 work with openCV VideoCapture objects on windows?

[Ulrich Stern]

A kernel hack to solve a camera issue deserves an upvote. :) I read that the FIX_BANDWIDTH quirk affects only uncompressed formats, matching your answer. Interestingly, the quirk worked for me even though v4l2-ctl -d 0 -V claimed Pixel Format 'MJPG' while I used OpenCV-based code to capture from multiple cameras.

[Ulrich Stern]

Figured out why the quirk worked for me, see my answer here. In a nutshell, OpenCV was capturing uncompressed notwithstanding what v4l2-ctl -d 0 -V claimed.

[Ulrich Stern]

For OpenCV, a v4l build supports MJPEG, while a libv4l build supports only uncompressed formats, see my answer here.

[Ulrich Stern]

Since part of the purpose of libv4l seems to be to "convert from any (known) pixelformat to BGR24," libv4l could in principle receive a compressed format from the webcam and then pass BGR24 to OpenCV. I checked and libv4l does not receive a compressed format from the webcam (for v4l-utils 1.10.0), matching my observation above that the quirk worked for libv4l.

[Tarek Loubani]

Thank you for this. To add a few more parts of information, here is exactly how I did this on a debian-based device: # apt-get source linux-image # tar xf <file> # cd linux-image # cp /lib/modules/<appropriate dir>/build/Module.symvers . # make oldconfig # make prepare # make scripts # [EDIT AS ABOVE in drivers/media/usb/uvc] # make SUBDIRS=drivers/media/usb/uvc/ modules