Youtube-dl library and ERROR 403: Forbidden when using generated direct link by youtube-dl from different locations

Question

I have a problem with youtube-dl library (https://rg3.github.io/youtube-dl/). I use youtube-dl not for directly downloading videos by this library, instead I first generate link using command like this:

youtube-dl --simulate --dump-json --prefer-ffmpeg -- -CmadmM5cOk

I get needed format link from json answer, I use default format 1280x720 mp4. Links look like this "https://r4---sn-nto52nx-px8e.googlevideo.com/videoplayback?lmt=1432339993780507&ip=x.x.x.x&key=yt6&fexp=9408710%2C9409069%2C9413137%2C9415365%2C9415485%2C9416023%2C9416126%2C941...".

Then I use this link to download video from different PC in different locations. For most links it works ok, but there are some videos, for example https://www.youtube.com/watch?v=-CmadmM5cOk, that I can't download from different server or PC, it returns ERROR 403: Forbidden.

So what I tried so far, I have 2 server and my local PC, all 3 of them are located in different countries, USA, England and Ukraine.

For example, I run youtube-dl and get link on Server 1 in the USA, I get link, I can it download in this Sever 1, but on Server 2 in London and my local PC in Ukraine it returns ERROR 403: Forbidden.

Another example, I generate link on Server 2 in London, I get different link, I can download it on the same server, but when I try on Server 1 or my local PC I get the same ERROR 403: Forbidden. The same when I generate link on local PC and try on servers.

This happens only with only some exact videos, but there are a lot of them. Some of videos that returns 403: https://www.youtube.com/watch?v=09R8_2nJtjg, https://www.youtube.com/watch?v=7PCkvCPvDXk.

Thank you, I appreciate any help!

Update:

I'm trying to use suggested by you option "-o -" to output video file to stdout and then I retranslate it to user. But I have another problem, it works if I login with ssh and try command with my user, but it throws 403 error when I launch the same command under "www-data" user. Here is verbose output:

Here is verbose report:

[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--prefer-ffmpeg', u'--no-warnings', u'-f', u'22', u'-o', u'-', u'-v', u'--', u'QcIy9NiNbmo']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.08.16.1
[debug] Python version 2.7.3 - Linux-4.1.0-x86_64-linode59-x86_64-with-debian-7.8
[debug] exe versions: avconv 0.8.17-6, avprobe 0.8.17-6, ffmpeg 0.8.17-6, ffprobe 0.8.17-6
[debug] Proxy map: {}
[youtube] QcIy9NiNbmo: Downloading webpage
[youtube] QcIy9NiNbmo: Downloading video info webpage
[youtube] QcIy9NiNbmo: Extracting video information
[youtube] {22} signature length 40.42, html5 player new
[youtube] {43} signature length 40.42, html5 player new
[youtube] {18} signature length 40.42, html5 player new
[youtube] {5} signature length 40.42, html5 player new
[youtube] {36} signature length 40.42, html5 player new
[youtube] {17} signature length 40.42, html5 player new
[youtube] {137} signature length 40.42, html5 player new
[youtube] {248} signature length 40.42, html5 player new
[youtube] {136} signature length 40.42, html5 player new
[youtube] {247} signature length 40.42, html5 player new
[youtube] {135} signature length 40.42, html5 player new
[youtube] {244} signature length 40.42, html5 player new
[youtube] {134} signature length 40.42, html5 player new
[youtube] {243} signature length 40.42, html5 player new
[youtube] {133} signature length 40.42, html5 player new
[youtube] {242} signature length 40.42, html5 player new
[youtube] {160} signature length 40.42, html5 player new
[youtube] {140} signature length 40.42, html5 player new
[youtube] {171} signature length 40.42, html5 player new
[youtube] QcIy9NiNbmo: Downloading DASH manifest
[debug] Invoking downloader on u'https://r6---sn-q4f7sn7s.googlevideo.com/videoplayback?dur=244.924&ipbits=0&sver=3&expire=1440076742&itag=22&lmt=1432153223392108&ratebypass=yes&fexp=9406010%2C9407118%2C9407156%2C9408710%2C9409069%2C9412515%2C9412774%2C9412779%2C9415365%2C9415485%2C9416023%2C9416126%2C9416326%2C9416916%2C9417707%2C9418153%2C9418449%2C9418954&key=yt5&mime=video%2Fmp4&initcwndbps=19078750&source=youtube&upn=H3j2i1TBZjA&mn=sn-q4f7sn7s&mm=31&pl=32&id=o-ADA7nHOi-VHvDpG35GbK7zvbnjmrppbPKN4PlVS25gcx&requiressl=yes&pcm2=no&ip=2600%3A3c00%3A%3Af03c%3A91ff%3Afe84%3A4cc7&mv=m&sparams=dur%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&mt=1440055050&ms=au&signature=FCB04C87410298BCDA619B8DB06221417292500B.669E8F187A5885C08D057F924A9551A657FF584B'
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1585, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1527, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 342, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 60, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1860, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 407, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 520, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 445, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 528, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

Solution 1

Final download URLs are only guaranteed to work on the same machine/IP where extracted. More on this.

UPD: To get rid of 403 remove the cache by running youtube-dl --rm-cache-dir on behalf of www-data.

Solution 2

youtube-dl --rm-cache-dir

it works perfectly :)

Solution 3

If dstftw's answer doesn't work for you, try with root: su youtube-dl --rm-cache-dir

Comments

[Vedmant]

Thank you, I'm trying to implement downloading through server with -o - option, and I have another problem, it works normally under one user, but fails with 403 error under "www-data". I'm updating question.

[AHmedRef]

what did you do to bypass this issue ?

[Paul Rougieux]

As explained in the answer and in this issue entering youtube-dl --rm-cache-dir at the command line fixes it.

[Toma]

perfect solution

[aggregate1166877]

If this worked for you, but the same command without su did not, then it means your user profile permissions are broken. Likely you ran youtube-dl as root - there is never a good reason to do this. I recommend you change your cache dir permissions (/home/<USER>/.cache/youtube-dl on Ubuntu) to the permissions of your actual user, recursively.

[babipsylon]

worked, but after 8 tracks I had the same error 403 again...

[babipsylon]

it seems like it's caused by the same track rather then by something on the local pc, rather some youtube triggered error

[Philipp Gächter]

this happened to me right after fresh install (of macOS even). I'm now using this alias: mp3='youtube-dl --rm-cache-dir && youtube-dl -x --audio-quality 0 --audio-format mp3 --no-playlist '