how to make cv2.videoCapture.read() faster?

Question

My question :

I was working on my computer vision project. I use opencv(4.1.2) and python to implement it.

I need a faster way to pass the reading frame into image processing on my Computer(Ubuntu 18.04 8 cores i7 3.00GHz Memory 32GB). the cv2.VideoCapture.read() read frame (frame size : 720x1280) will take about 120~140ms. which is too slow. my processing module take about 40ms per run. And we desire 25~30 FPS.

here is my demo code so far:

import cv2
from collections import deque
from time import sleep, time
import threading


class camCapture:
    def __init__(self, camID, buffer_size):
        self.Frame = deque(maxlen=buffer_size)
        self.status = False
        self.isstop = False
        self.capture = cv2.VideoCapture(camID)


    def start(self):
        print('camera started!')
        t1 = threading.Thread(target=self.queryframe, daemon=True, args=())
        t1.start()

    def stop(self):
        self.isstop = True
        print('camera stopped!')

    def getframe(self):
        print('current buffers : ', len(self.Frame))
        return self.Frame.popleft()

    def queryframe(self):
        while (not self.isstop):
            start = time()
            self.status, tmp = self.capture.read()
            print('read frame processed : ', (time() - start) *1000, 'ms')
            self.Frame.append(tmp)

        self.capture.release()

cam = camCapture(camID=0, buffer_size=50)
W, H = 1280, 720
cam.capture.set(cv2.CAP_PROP_FRAME_WIDTH, W)
cam.capture.set(cv2.CAP_PROP_FRAME_HEIGHT, H)


# start the reading frame thread
cam.start()

# filling frames
sleep(5)

while True:
  frame = cam.getframe() # numpy array shape (720, 1280, 3)

  cv2.imshow('video',frame)
  sleep( 40 / 1000) # mimic the processing time

  if cv2.waitKey(1) == 27:
        cv2.destroyAllWindows()
        cam.stop()
        break

What I tried :

1. multiThread - one thread just reading the frame, the other do the image processing things. It's NOT what I want. because I could set a buffer deque saving 50 frames for example. but the frame-reading thread worked with the speed ~ frame/130ms. my image processing thread worked with the speed ~ frame/40ms. then the deque just running out. so I've been tried the solution. but not what I need.

2. this topic is the discussion I found out which is most closest to my question. but unfortunately, I tried the accepted solutions (both of two below the discussion).

One of the solution (6 six thumbs up) point out that he could reading and saving 100 frames at 1 sec intervals on his mac. why my machine cannot handle the frame reading work? Do I missing something? my installation used conda and pip conda install -c conda-forge opencv, pip install opencv-python(yes, I tried both.)

The other of the solution(1 thumb up) using ffmpeg solution. but it seem's work with video file but not camera device?

3. adjust c2.waitKey() : the parameter just controls the frequency when video display. not a solution.

Then, I know I just need some keywords to follow.

code above is my demo code so far, I want some method or guide to make me videoCapture.read() faster. maybe a way to use multithread inside videoCapture object or other camera reading module.

Any suggestions?

Solution

This comes a bit late, but I was wondering this with my Logitech C920 HD Pro USB-camera on Ubuntu 20.04 and OpenCV. I tried to command the capture session to run Full HD @ 30 FPS but the FPS was fluctuating between 4-5 FPS.

The capture format for my camera defaulted as "YUYV 4:2:2". No matter how I tried to alter the video capture settings, OpenCV did not magically change the video format to match e.g. the desired FPS setting.

When I listed the video formats for my Logitech C920, it revealed:

ubuntu:~$ v4l2-ctl --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture

    [0]: 'YUYV' (YUYV 4:2:2)
<clip>
        Size: Discrete 1600x896
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 2304x1296
            Interval: Discrete 0.500s (2.000 fps)
    [1]: 'MJPG' (Motion-JPEG, compressed)
<clip>
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)

The solution was to manually command the OpenCV capture device to use the compressed 'MJPG' format:

import numpy as np
import cv2

capture = cv2.VideoCapture(0)
W, H = 1920, 1080
capture.set(cv2.CAP_PROP_FRAME_WIDTH, W)
capture.set(cv2.CAP_PROP_FRAME_HEIGHT, H)
capture.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'))
capture.set(cv2.CAP_PROP_FPS, 30)

Comments

[Yu-Long Tsai]

ya, it is a wired situation. and yes, CAP_PROP_BUFFERSIZE won't help. I'm wondering that is it a compile issue? as you said : But by using other applications such as cheese, we still get a full 30fps at 1920x1080 resolution. maybe something went wrong when using conda install or pip install? How do you think about it?

[Vu Gia Truong]

I don't think so. You can print out the build information to check if you want. As in docs.opencv.org/3.4/d0/da7/videoio_overview.html, you can see that there're some video io backend there. And the most important note is: "Each backend supports devices properties (cv::VideoCaptureProperties) in a different way or might not support any property at all." . It mean that they won't guarantee that your options will always work as you want. So, the last shot is trying video capturing with other backend such as CAP_GSTREAMER or CAP_FFMPEG.

[Ramesh]

this code helps to show the camera window faster than direct load camera @brunoob

[Benoit]

my camera increases from 12 to 60! Tanks!