Download everything from a YouTube video using youtube-dl

Question

I want to download everything (subtitles, cards, tags, annotations, thumbnails, etc. (and obviously the video w/ audio)) from a YouTube video (for example, this one).

I'm using youtube-dl and, with the overwhelming amount of switches to use, I can't seem to find anything to fit my needs. I also want to download the video/audio in the highest quality possible, with no quality loss. Even though it's not YouTube, I did also want to download some of my udemy courses I've bought so I can watch them while I'm on a long road trip.

I have ffmpeg installed, and I heard that you need this for lossless downloads from YouTube, but I'm not sure how to link it with youtube-dl.

This is my first time using youtube-dl, so any help is appreciated.

Solution 1

The relevant options you need:

# Filesystem
--write-annotations
--write-description
--write-info-json

# Thumbnail images
--write-all-thumbnails

# Video format
--format bestvideo+bestaudio/best
--merge-output-format mkv

# Subtitle
--all-subs
--write-auto-sub
--write-sub

# Post-processing
--add-metadata
--embed-subs

Copy that to your config file (either /etc/youtube-dl.conf or ~/.config/youtube-dl/config).

Using that configuration, I downloaded this video and youtube-dl wrote the following files:

$ ls
The Problem with Time & Timezones - Computerphile.annotations.xml
The Problem with Time & Timezones - Computerphile.description
The Problem with Time & Timezones - Computerphile.info.json
The Problem with Time & Timezones - Computerphile.jpg
The Problem with Time & Timezones - Computerphile.mkv

I can't post mediainfo's output here because it exceeds the characters limit, but you can see it in github.

---

I also want to download the video/audio in the highest quality possible, with no quality loss.

youtube-dl downloads videos at the highest quality possible by default, but you can force this behavior using --format bestvideo+bestaudio/best.

---

Even though it's not YouTube, I did also want to download some of my udemy courses I've bought so I can watch them while I'm on a long road trip.

Please read the article Can I Download a Course to my Computer?:

By default, complete courses are not downloadable from a computer. We do this out of concerns for piracy.

Please note that you can always save courses for offline viewing on the Udemy mobile app. For information on how to download video lectures to your iOS mobile device, please click here. For steps on how to download videos for offline viewing on an Android device, click here.

---

I have ffmpeg installed, and I heard that you need this for lossless downloads from YouTube, but I'm not sure how to link it with youtube-dl.

youtube-dl uses avconv by default, but --prefer-ffmpeg let you use ffmpeg instead. Anyway, this option is not needed for "lossless downloads". In the example I provided, youtube-dl only used ffmpeg to merge all the downloaded formats into a single mkv file.

Solution 2

To simply download the video, you can use this:

youtube-dl --all-subs "https://www.youtube.com/watch?v=KYBok-XGsKM"

To select the video quality, you should first use the -F option to list the available formats.

youtube-dl -F "https://www.youtube.com/watch?v=KYBok-XGsKM"

And this is the output for your video:

[youtube] KYBok-XGsKM: Downloading webpage
[youtube] KYBok-XGsKM: Downloading video info webpage
[youtube] KYBok-XGsKM: Extracting video information
WARNING: unable to extract uploader nickname
[info] Available formats for KYBok-XGsKM:
format code  extension  resolution note
249          webm       audio only DASH audio   52k , opus @ 50k,  4.19MiB
250          webm       audio only DASH audio   75k , opus @ 70k, 5.42MiB
140          m4a        audio only DASH audio  129k , m4a_dash container, mp4a.40.2@128k, 10.59MiB
171          webm       audio only DASH audio  131k , vorbis@128k, 7.66MiB
251          webm       audio only DASH audio  135k , opus @160k, 9.58MiB
278          webm       256x144    144p   98k , webm container, vp9, 30fps, video only, 6.59MiB
160          mp4        256x144    144p  114k , avc1.4d400c, 15fps, video only, 9.04MiB
242          webm       426x240    240p  205k , vp9, 30fps, video only, 11.47MiB
133          mp4        426x240    240p  265k , avc1.4d4015, 30fps, video only, 20.11MiB
243          webm       640x360    360p  362k , vp9, 30fps, video only, 21.93MiB
134          mp4        640x360    360p  602k , avc1.4d401e, 30fps, video only, 20.42MiB
244          webm       854x480    480p  662k , vp9, 30fps, video only, 37.82MiB
135          mp4        854x480    480p 1020k , avc1.4d401f, 30fps, video only, 42.62MiB
247          webm       1280x720   720p 1353k , vp9, 30fps, video only, 81.53MiB
136          mp4        1280x720   720p 2013k , avc1.4d401f, 30fps, video only, 84.69MiB
137          mp4        1920x1080  1080p 2438k , avc1.640028, 30fps, video only, 157.71MiB
248          webm       1920x1080  1080p 2593k , vp9, 30fps, video only, 162.48MiB
264          mp4        2560x1440  1440p 6973k , avc1.640032, 30fps, video only, 459.58MiB
271          webm       2560x1440  1440p 7523k , vp9, 30fps, video only, 485.66MiB
313          webm       3840x2160  2160p 19991k , vp9, 30fps, video only, 1.34GiB
36           3gp        320x?      small , mp4v.20.3,  mp4a.40.2
17           3gp        176x144    small , mp4v.20.3,  mp4a.40.2@ 24k
43           webm       640x360    medium , vp8.0,  vorbis@128k
18           mp4        640x360    medium , avc1.42001E,  mp4a.40.2@ 96k
22           mp4        1280x720   hd720 , avc1.64001F,  mp4a.40.2@192k (best)

Here are some useful options for subtitles:

   --write-sub
          Write subtitle file

   --write-auto-sub
          Write automatically generated subtitle file (YouTube only)

   --all-subs
          Download all the available subtitles of the video

   --list-subs
          List all available subtitles for the video

And for thumbnails:

   --write-thumbnail
          Write thumbnail image to disk

   --write-all-thumbnails
          Write all thumbnail image formats to disk

   --list-thumbnails
          Simulate and list all available thumbnail formats

You can also find more information about the quality of video here

Solution 3

@leetbacoon see comment response at very bottom. (low rep still sry)

Note: I will be using the longer switch names to avoid confusion. Many of the switches used have shorter versions. See Docs: Options.

Note: - for visual convenience \ is used to display switches separately; all \ can be omitted; if using \ make sure there is no space character after \.

---

Synopsis: A ytdl command to download a single video file of highest available video and audio quality. Saves the video into its own directory (specified by you). Downloads all available metadata, writes each to respective file formats and saves in same directory. Adds video metadata to video output file.

Substitute with your own:

• replace /archive/videos/TED_Archive/ in the switches --download-archive and --output with a directory path of your choice.
• example storage path /archive/videos
• example video link: https://www.youtube.com/watch?v=0bFs6ZiynSU
• example channel directory /TED_Archive

====

youtube-dl \
--limit-rate '0.25M' \
--retries '3' \
--no-overwrites \
--call-home \
--write-info-json \
--write-description \
--write-thumbnail \
--all-subs \
--convert-subs 'srt' \
--write-annotations \
--add-metadata \
--embed-subs \
--download-archive '/archive/videos/TED_Archive/TED_Archive.ytdlarchive' \
--format 'bestvideo+bestaudio/best' \
--merge-output-format 'mkv' \
--output '/archive/videos/TED_Archive/%(upload_date)s_%(id)s/TED_Archive_%(upload_date)s_%(id)s_%(title)s.%(ext)s' \
'https://www.youtube.com/watch?v=0bFs6ZiynSU' ;

Note: Enclose all arguments with ' ' for parsing safety.

Explanation of recommended switches:

• --limit-rate '0.25M' is used to limit the available maximum download speed of every youtube-dl instance in B/s (bytes per second).
  Arguments: any positive integer with a capitalized unit suffix.
  Unit Suffixes: K for Kilobyte; M for Megabyte; Unclear if Gigabyte is supported.
  Addition:
  1. avoid internet connection blockage.
  2. helpful when managing download speeds of multiple concurrent ytdl instances.
  Ommission:
  1. clogs up internet; unable to browse, lagging youtube video play.
  2. will download video as fast as possible.
  Note: You can run multiple youtube-dl instances concurrently.

• --retries '5' is used to auto-retry the download of a video if it fails. Useful when downloading a playlist or an entire channel (which is, in Youtube terms, also a playlist (of that specific channel)).
  Values: Choose any positive integer. Default is 10. Max is infinite (enter as literal string).

• --continue
  Arguments: none
  Addition:
  1. will force resume downalod of partial files
  Note: ytdl downloads videos in chunks; chunks are preserved if ytdl unexpectedly terminates during download.
  Ommission:
  1. By default ytdl will resume downloads, if possible.

• --no-overwrites
  Arguments: none
  Addition: will not overwrite existing metadata files with same names.
  Ommission: will overwrite existing metadata files with same names, but not the video file.

  • --call-home used to contact the ytdl server for debugging.

    "If your server has multiple IPs or you suspect censorship, adding --call-home may be a good idea to get more diagnostics." – Docs: Bugs, retrieved 20180408,

• --write-info-json
  Arguments: none
  Output Format: json;
  Information Content: view count, likes, upload date, video and audio quality information, etc. (but not video text description (posted below video by uploader (2018 youtube layout)))
  Addition: writes video metadata into a separate .info.json file. Saved in same folder as video file.
  Ommission: does not affect --add-metadata which will still write video metadata to the video output file.

• --write-description
  Arguments: none
  Output Format: plain text;
  Addition: writes text posted by video uploader in the description section of a video (below the video (2018 youtube layout)) into a separate Write .description file. Saved in same folder as video file.

• --write-thumbnail writes highest resolution thumbnail available (called maxresdefault.jpg by youtube) to .jpg file. Renamed according to template specified with --output. Saved in same directory as video file.
  Arguments: none
  Output Format: only JPEG;
  Note: Use youtube-dl --list-thumbnails '<video_url>' to investigate available thumbnails of a specific video (does not work on playlist links); Ex.: youtube-dl --list-thumbnails 'https://www.youtube.com/watch?v=odwfHu6MDuU'

• --write-annotations writes video annotations to .annotations.xml file. Renamed according to template specified with --output. Saved in same directory as video file.
  Arguments: none
  Output Format: .xml;

• --all-subs writes all custom created available subtitles for a video to their respective files (ex.: subtitles.en.vtt, subtitles.fr.vtt, subtitles.es.vtt)
  Arguments: none
  Output Format: Default .vtt; can specify other formats with --sub-format
  Error Messages: If there are no subtitles available a WARNING message will be printed to stdout - format: WARNING: video doesn't have subtitles

Useful Command:

Note: Use youtube-dl --list-subs '<video_url>' to investigate available thumbnails of a specific video (does not work on playlist links);
Ex.: youtube-dl --list-subs 'https://www.youtube.com/watch?v=odwfHu6MDuU'

• --convert-subs 'srt'
  Arguments: String : format extension ass, srt or best
  Output Format: Default .vtt; or according to specification
  Addition: writes subtitles to specified file format (ex.: .srt). Saved in same folder as video file.
  Ommission: subtitles will be written as .vtt
  Error Messages: If there are no subtitles available an INFO message will be printed to stdout - format: [ffmpeg] There aren't any subtitles to convert
  Note: conversion will be executed as a post-processing step on the .vtt file after download. If ytdl execution is terminated unexpectedly some subtitle files might not be converted (in this case execute command again).

• --add-metadata
  Arguments: none
  Behavior: writes metadata to video output file.
  Note: does not require a .info.json generated by --write-info-json

• --embed-subs
  Arguments: none
  Behavior:
  1. writes subtitles to video output file;
  2. only for mp4, webm and mkv videos
  Error Messages: If there are no subtitles available a INFO message will be printed to stdout - format: [ffmpeg] There aren't any subtitles to embed
  Note: does not require a .vtt generated by --write-sub, --write-auto-sub or --all-sub

• --download-archive '<archive_path>/<channel_name>/<channel_name>.ytdlarchive'
  Behavior: is used to create a text file containing links of previously downloaded videos. Allows skipping these videos when downloading an entire channel. Not necessary when downloading single videos only.
  Arguments: String : composite of the path to your video archive, directory named after channel. File name consists of channel name with .ytdlarchive file extension. (could also be .txt; but unique format allows for easy differentiation)
  Output Format: plain text; one videoID per line
  Addition:
  1. ytdl reads this file, ignores and skips all videos listed with their videoID in it, even if no downloaded yet.
  2. automatically adds the videoID of the last downloaded video to this file
  Ommission: if video file and metadata files exist ytdl will download them again.

• --format ''
  Arguments: Recommended String : bestvideo+bestaudio/best to ensure highest video and audio quality combination.
  Addition: full control over video quality and audio quality and their combinations, as well as fallback qualities
  Ommission: Default youtube-dl will guess it for you by default – Docs
  Note: See Docs: Format Selection Examples

• --merge-output-format '<video_format>'
  Arguments: video file format. (ex.: mp4, mkv, 3gp, etc.)
  Addition: can specify video file format
  Ommission: Default youtube-dl will guess it for you by default – Docs
  Note: See Docs: Format Selection

Useful Command:

Note: Use youtube-dl --list-formats '<video_url>' to investigate available video and audio qualities and formats of a specific video (does not work on playlist links);
Ex.: youtube-dl --list-formats 'https://www.youtube.com/watch?v=odwfHu6MDuU'

• --output '<template>'
  Arguments:
  1. video file format. (ex.: mp4, mkv, 3gp, etc.);
  2. Recommended: %(upload_date)s_%(id)s_%(title)s.%(ext)s
  Addition: influences file naming convention for video files and all metadata files
  Ommission: By default the video title will be the file name. Default template: %(id)s.%(ext)s will output ex.: testvideo.mkv
  Note: Do not forget the % prefix and s suffix (other suffices exist too.); Thee _ between template elements is just a separator and can be any other char. ex.: %(upload_date)s-%(id)s
  Note: See Docs: Output Template

• '<video_url>'
  Arguments: String : url to specific video
  Error Message:
  1. If no video url submitted ytdl will not execute and display help.
  2. If invalid video url submitted ERROR message will be printed to stdout - format: ERROR: u'' is not a valid URL. Set --default-search "ytsearch" (or run youtube-dl "ytsearch:" ) to search YouTube

---

Storage Recommendations:

1. make a misc directory in your video archive and store all single videos in there.

2. when you download one or more videos from a channel make a dedicated directory for that channel. In this case, the channel name is TED Archive and the directory name will be TED_Archive. (I preemptively substitute all space characters with underscores for easier directory navigation and file I/O actions when using bash (or python) scripts).

---

Responding to your comment @leetbacoon

And just wondering, why is a 1080p copy considered the best if 1440p is available? And why is 2160p displayed? That option doesn't appear on my end (it caps at 1440p) – leetbacoon

From the Docs
You can also use a file extension (currently 3gp, aac, flv, m4a, mp3, mp4, ogg, wav, webm are supported) to download the best quality format of a particular file extension served as a single file

best: Select the best quality format represented by a single file with video and audio.

bestvideo: Select the best quality video-only format

I think when you select just best and you specify an output format youtube sends the format it thinks is the best video+audio combo for that format. This might be subject to some youtube internal balancing decisions (speculation).

In my usage experience, I have had to specifically specify bestvideo+bestaudio to get the HIGHEST quality available of video and audio respectively and then combine them in a .mkv container.

I use /best as a fallback argument just to be sure.

---

Comments

[David Foerster]

What do you hope to achieve with manual format selection? youtube-dl uses the best available video and audio formats by default. If you want to override a different setting in the configuration file you can use -f bestvideo+bestaudio/best (which is the same as the default) or ignore the configuration file (--ignore-config).

[leetbacoon]

Will this embed the information from YouTube into the video? I don't exactly want that, I just want it to download it, but keep that all in separate files, kinda like what it looks like :)

[leetbacoon]

Don't quote me on this, but I thought I read earlier that to get lossless video w/ audio, you just use youtube-dl https://www.youtube.com/watch?v=KYBok-XGsKM

[leetbacoon]

And is .mkv the default container for lossless videos using youtube-dl?

[leetbacoon]

And just wondering, why is a 1080p copy considered the best if 1440p is available? And why is 2160p displayed? That option doesn't appear on my end (it caps at 1440p)

[leetbacoon]

Oh, and how did you download the video? youtube-dl [youtube-link]?

[nxnev]

@leetbacoon [1] It will only embed the subtitles and some metadata (description, uploader, date and url), but the other files will be kept apart. If you don't want to embed that info and only preserve the separated files, you can safely omit the --write-auto-sub, --write-sub and --add-metadata options.

[nxnev]

@leetbacoon [2] I used .mkv (matroska) instead of another container because it doesn't need to recode the formats (it accepts virtually any format), so your video won't lose quality.

[nxnev]

@leetbacoon [3] Yes, I downloaded the video using just youtube-dl [youtube-link]. That's the magic of configuration files, you don't need to write all those options at runtime in the command line.

[leetbacoon]

Will omitting those flags still download the subtitle files?

[leetbacoon]

And also, the thumbnail it downloaded isn't the largest available. Is is possible for me to force youtube-dl to download the highest resolution thumbnail, too?

[nxnev]

@leetbacoon Oh, sorry, the options you need to omit are --embed-subs and --add-metadata, not the --write ones. It's supposed that youtube-dl downloads the highest resolution thumbnail by default, in my example it downloads this thumbnail. What video are you trying to download?

[leetbacoon]

I tried this video and got a 480x360 image using the settings you described, vs a 640x480 image I manually got myself.

[nxnev]

@leetbacoon I see. I'll add a bug report.

[leetbacoon]

Thanks. Also, what about information on videos (like these) where they have text at the bottom that say credits? (in the example video that would be Caption author (French) Mr. Apricot, Category Entertainment, etc.) Also stuff like video uploader, publish date, tags, and user watermark image?

[nxnev]

@leetbacoon The tags, category, date and uploader are included in the .info.json file, but it seems that youtube-dl doesn't support the credits, watermark and profile picture. Please bear in mind that youtube-dl is not a fully featured scraping utility, it is intended to be used just as a video downloader utility. If you need even more functionality, you should ask for it in its GitHub site or look for another program.

[nxnev]

@leetbacoon I don't know which program could fulfill your needs, you'll need to search on the net. And building a scraper in Bash from scratch would be a very difficult task, so you could try doing that but I don't recommend it.

[leetbacoon]

What is the purpose of the --youtube-skip-dash-manifest option? For some reason I get 404s on youtube videos when this option is added -- removed it works fine (yes everything's up to date)

[nxnev]

@leetbacoon Back when I wrote this answer it was the opposite, youtube-dl wouldn't have been able to download certain videos if the --youtube-skip-dash-manifest option were not present. Just edited out that option, thanks for telling me.

[leetbacoon]

Thank you for updating me on this. Is there any benefit to including this versus leaving it out?

[Hassan Ahmed]

Seems like --prefer-ffmpeg flag is the default now. That is, youtube-dl uses FFmpeg by default rather than avconv