Download TS files from video stream

Question

Videos on most sites make use of progressive downloading, which means that the video is downloaded to my computer, and easy to trace. There are lots of extensions out there to do this, and even in the dev-tools this is easily done.

On certain websites videos are streamed. which means that we do no just download 1 file, we download lots of small packages. In the dev-tools these packages can be traced. The website I'm interested in is: http://www.rtlxl.nl/#!/goede-tijden-slechte-tijden-10821/c8e2bff7-5a5c-45cb-be2b-4b3b3e866ffb.

-The packages have a .TS extension.

-Packages can be saved by copying the url of the request

-I can not play these files.

I must have done something wrong, or I'm missing something. I want to know what I am doing wrong. I want to create a chrome extension for personal use which captures the urls of all the packages. when I have all the urls I want to pass them on to a php scripts which downloads them and uses ffmpeg to paste them into a mp4 file.

Please help me get the packages.

Solution 1

Addition to @aalhanane and @Micheal Espinola Jr

As m3u8x is only available for windows. Once you have identified the m3u8 url you can also use Jdownloader2 or VLC Media Player to download and concatenate the stream.

Jdownloader2: Just copy the m3u8 url when it the Jdownloader is open. It will recognize the stream in Linkgrabber tab.

VLC 3:

Open Network -> Paste m3u8 url -> Checkmark Streamoutput -> Select Settings. Choose output file, container , video and audio encoding. (e.g output.mp4, container: mpeg4, video: h264, audio: mp4a) Start Stream. It will not play the video, but encode it, showing the encoding progress by moving the video play back progress bar.

WARNING: Previously suggesteed chrome extension Stream Video Downloader contains malware. See reddit post

Solution 2

Easy youtube-dl example on macOS (in the command line Terminal; Windows supported too):

# List variants (resolutions/bitrates)
$ youtube-dl -F https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8
[generic] f08e80da-bf1d-4e3d-8899-f0f6155f6efa: Requesting header
[generic] f08e80da-bf1d-4e3d-8899-f0f6155f6efa: Downloading m3u8 information
[info] Available formats for f08e80da-bf1d-4e3d-8899-f0f6155f6efa:
format code           extension  resolution note
audio-English_stereo  mp4        audio only [en] 
628                   mp4        320x180     628k , avc1.42c00d, video only
928                   mp4        480x270     928k , avc1.42c00d, video only
1728                  mp4        640x360    1728k , avc1.42c00d, video only
2528                  mp4        960x540    2528k , avc1.42c00d, video only
4928                  mp4        1280x720   4928k , avc1.42c00d, video only
9728                  mp4        1920x1080  9728k , avc1.42c00d, video only (best)

# Choose a variant to download, and use its format code below
$ youtube-dl --format 628 https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8
...
frame= 5257 fps=193 q=-1.0 Lsize=    6746kB time=00:03:30.16 bitrate= 263.0kbits/s speed=7.73x    
video:6679kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.998669%
[ffmpeg] Downloaded 6907810 bytes
[download] 100% of 6.59MiB in 00:29

$ open f08e80da-bf1d-4e3d-8899-f0f6155f6efa-f08e80da-bf1d-4e3d-8899-f0f6155f6efa.mp4

Use the browser's Developer Tools > Network to get the m3u8 (HLS manifest) URL when starting a streaming video.

Solution 3

---> Open Firefox

---> open page the video

---> Play Video

Click ---> Open menu

Click ---> open web developer tools

Click ---> Developer Toolbar

Click ---> Network

---> Go to Filter URLs ---> Write "M3u8" --> for Find "m3u8"

---> Copy URL ".m3u8"

========================

Now Download software "m3u8x" ----> https://tajaribsoft-en.blogspot.com/2016/06/m3u8x.html#downloadx12

---> open software "m3u8x"

---> paste URL "m3u8"

---> chose option "One...One"

---> Click Download

---> Start Download

========================

image "Open menu" ===>

image "Developer Toolbar" ===>

image "m3u8x" ===>

Solution 4

using this post

• Open Firefox / chrome

• open page the video

• Play Video

• click F12 on keyboard -> network

• in Filter URLs ts

• copy link of ts

• remove index and ts extension from link

  for example:

  http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000_454.ts
  

  will be copied as

   http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000
  

insert in below script under LINK

#!/bin/bash

# insert here urls
LINK=(

'http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000' # replace this with your url 

)

mkdir my-videos
cd mkdir my-videos

CNT=0

for URL in ${LINK[@]}
do
  # create folder for streaming media
  CNT=$((CNT + 1))
  mkdir $CNT
  cd $CNT

  (

   DIR="${URL##*/}"

   # download all videos
   wget $URL'_'{0..1200}.ts

   # link videos
   echo $DIR'_'{0..1200}.ts | tr " " "\n" > tslist
   while read line; do cat $line >> $CNT.mp4; done < tslist

   rm -rf media* tslist
   ) &
   cd ..

done

wait

EDIT

adding script in python - runs on windows and linux

import urllib.request
import os
import shutil

my_lessons = [
   #  http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000_454.ts
    "http://vid.com/vod/mp4:vod/PRV/Yg0WGN_6.mp4/media_b180000" # replace this with your url 


]

lesson_dir = "my_vids"
try:
    shutil.rmtree(lesson_dir)
except:
    print "ok"

os.makedirs(lesson_dir)
os.chdir(lesson_dir)

for lesson, dwn_link in enumerate(my_lessons):
    print ("downloading lesson  %d.. " % (lesson), dwn_link)
    file_name = '%04d.mp4' % lesson
    f = open(file_name, 'ab')
    for x in range(0, 1200):
        try:
            rsp = urllib.request.urlopen(dwn_link + "_%04d.ts" % (x) )
        except:
            break
        file_name = '%d.mp4' % lesson
        print "downloading  %d.ts" % (x)
        f.write(rsp.read())
    f.close()



print "done good luck!! ==================  "

if the script fails, or downloads empty file, try removing the try wrap to see what fails

Solution 5

You would need to download all of the transport stream (.ts) files, and concatenate them into a single mpeg for playback. Transport streams such as this have associated playlist files (.m3u8) that list all of the .ts files that you need to download and concatenate. If available, there may be a secondary .m3u8 playlist that will separately list subtitle steam files (.vtt).

View more solutions

Comments

[Vadzim]

forum.videohelp.com/threads/… also mentions livestreamer alternative which was recently forked as streamlink, but it doesn't handle authentication.

[Ariel]

There is no .m3u8 file in the network tab, there are only .ts files.

[Pastuh]

Thanks, Downloaded full video with program jDownloader2 , even host used Wowza Streaming Engine

[Admin]

Very nice. Result was not very good for my videos if I concatenated like that though. There were pretty annoying visual artefacts at junctions when I watched with VLC. The best result I got with ffmpeg "concat protocol". It was not perfect either, as VLC had some trouble moving backward and forward in big steps.

[Admin]

... that forward/backward thing was not a problem for all videos though.

[Developer]

That's awesome but you did not mention that after "paste URL "m3u8"" you have to type name for the file, for example "video" then click on icon of hand next to "quality" and only after that you can select "one on one" and "download". As for the rest, works good! Thanks

[Val Martinez]

Worked for me. I had to install also ffmpeg. As linux debian user had to exec "apt-get install ffmpeg" Just a claritication. On the 2nd command, the value of paramenter--format, (628) means resolution and should be chosen from the previous list

[vogash]

script works good, but not always, some site put some restrictions. If I run wget $URL, i get 403. If try to download from chrome it works. Looks like a site does some validation who and how the file has been accessed...

[IamVISH]

I am trying to download video from elearning website. I coped m3u8 file and tried all methods like VLC, youtube-dl, Jdownloader2 etc. Still unable to download but I can stream. For this site I have to login to stream videos. Even Internet Download Manager IDM fails to download saying "cannot download this protected stream". So, is there anyway I can download protected/encrypted streams without video recording. Please help.

[dre-hh]

pls post the website. if you have to login to watch it, the site can take additional measures to protect the content only to a logged in user. The methods above wont work then. Also there are DRM measures like widevine which make sure, only your browser gets proper decryption key, but not any other program. In this case, the only way is screen and oudio output capture

[Tiana987642]

2020 vid helper 7.3.9. and ff 80 and this works one click :)

[Frank Fu]

This is awesome! I wonder if this can be automated to find the "best"?

[jaimet]

@frank-fu Just omit the "format" parameter, and youtube-dl will automatically select the "best" i.e. youtube-dl https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8‌​s/f08e80da-bf1d-4e3d‌​-8899-f0f6155f6efa.m‌​3u8

[Gabriel Fair]

If you only see .ts files, you need to refresh the page to get the .m3u8 link which should be the first url to load

[Marcos Mendes]

Ho yeah! Like a charm!

[vr_driver]

Here's more of what is happening from my GIST - gist.github.com/vrdriver/bf9746af1b1ca8a544826a2be87d33ae

[Faizan]

Have to brew install ffmpeg because of this error on macos ERROR: m3u8 download detected but ffmpeg or avconv could not be found. Please install one.