Python webcam stream over Udp socket

Question

im trying to make a webcam stream over two computers in the same network, so i did some research on the internet and i found this client and server code this is the Client

import socket
import numpy as np
import cv2
UDP_IP = '127.0.0.1'                  
UDP_PORT = 999        
cap = cv2.VideoCapture(0)
while(True):
   ret, frame = cap.read()
   cv2.imshow('frame',frame)
   sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
   d = frame.flatten ()
   s = d.tostring ()
   for i in xrange(20):
       sock.sendto (s[i*46080:(i+1)*46080],(UDP_IP, UDP_PORT))


        if cv2.waitKey(1) & 0xFF == ord('q'):
          break

cap.release()
cv2.destroyAllWindows()

and this is the Server:

#Server prog

import socket
import numpy
import time
import cv2

UDP_IP="127.0.0.1"
UDP_PORT = 999
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))

s=""

while True:
      data, addr = sock.recvfrom(46080)
      s+= data
      if len(s) == (46080*20):
          frame = numpy.fromstring (s, dtype=numpy.uint8)
          frame = frame.reshape(480,640,3)
          cv2.imshow("frame",frame)

          s=""
      if cv2.waitKey(1) & 0xFF == ord('q'):
          break

there are some lines that i can't understand like this for

for i in xrange(20):
   sock.sendto (s[i*46080:(i+1)*46080],(UDP_IP, UDP_PORT))

why is he using it? (i know that in UDP you can't send a huge file and that u need to send in parts but why like this? and why 46080? ) and in the server itself if len(s) == (46080*20):

+when you run the server and the client the video stream that the server received is lagging and the frames repeat themselves..

Solution

46080 because each frame consists of 640x480 RGB pixels, one byte per channel, hence three bytes per pixel, and sent in 20 pieces:

>>> 640 * 480 * 3 / 20
46080

The value should be different from 46080 if your cam captures frames of different resolution. Frames of size 46k might be somewhat large for your network, so you might want to try to send it in a more than 20 pieces.

s[i*46080:(i+1)*46080] means "i-th fragment of size 46080". It is an example of so called slice notation:

>>> a = [0, 1, 2, 3, 4, 5]
>>> a[0:2]  # take element #0 and #1
[0, 1]
>>> a[2:4]  # take element #2 and #3
[2, 3]

It's a simple but powerful tool so I'd advise to learn more about it. You also use numpy - numpy supports a lot of additional ways to slice an n-dimensional array. Somewhat complex but even more powerful.

Regarding the laggy stream.

First, in UPD frames could be lost, reordered and duplicated. In order to avoid duplicates and reordered datagrams you may attach a sequence number to each datagram you send and have a priority queue on the receiver.

Second, I doubt cv2.imshow is a good choice for displaying smooth video. You might want to try ffmpeg instead.

Third, make sure your sender is fast enough to capture and send the frames at least at 20-30 fps. If not, try to split image acquisition + processing and transmitting into separate threads. My blind guess is that you're spending much time in sock.sendto. Don't hesitate to profile your code.

Fourth, sending video in a picture by picture way may be grossly inefficient even for rather small desired resolution and fps. You'll need to add a compression/decompression stage to your streaming code. Sending uncompressed 768x1024x3 @ 30 fps requires 70 MBytes/s of bandwidth. Again ffmpeg might help you here.

Comments

[Ziroff]

first of all, ty for your answer! but i dont understand why s[i*46080:(i+1)*46080] what does : means?

[u354356007]

@Ziroff this is called a slice - I've updated my asnwer

[Ziroff]

Ok, my webcam is recording at 768x1024 so how am i gonna transfer the data? its a very large number

[u354356007]

@Ziroff at this point UDP shouldn't be your top priority. Start with compressing your video stream and then think about how to transmit it. Updated the answer one more time.