How to download video ads from Youtube?

Question

I'd like to download some (I mean a lot of) video ads from YouTube (not the videos, but the ads that are played before them) for research purposes.

Any suggestions for tools or ways this can be achieved? I won't mind some programming on my hack but I currently have no idea how. Also, YouTube is not an requirement; video ads from other video sites work for me too.

Solution 1

When I found your question, I thought it will be fun to play with it, as I recently developed site (http://savedeo.com) allowing you to download a video from many sites including YouTube.

I looked only on the Youtube's ads system, this will not work for other systems (most likely). The good thing about YouTube is, that you can get all necessary information directly from a video page, so it's easier to crawl it really fast (I downloaded almost 22M video pages in a single day on a very small server). So the part you are looking for is ;ytplayer.config = {(.*?)}; which contains inline json object (easy to deal with). You will not need anything else.

If you don't want to parse it from the HTML, you can get directly just the json object by adding a parameter &spf=prefetch at the end of any Youtube video link e.g. https://www.youtube.com/watch?v=bbEoRnaOIbs&spf=prefetch

Not every YouTube video is showing Ads (from my statistics, only 18% of videos eactually does). You can verify if ads are enabled for the video just by looking for ad_tag key inside of the json object.

YouTube is using three different Ad systems:

• AdSense
• AdSense/ADX
• 3rd party system

to serve Ads from two different sources: - video uploaded on YouTube used as ad (mostly only part of the video is shown) - video from external source (3rd party ads server)

The starting point for all of them is the same. Locate dynamic_allocation_ad_tag key inside of the json object. It contains an url leading to doubleclick server. This url will not work until you will change this part sz=WIDTHxHEIGHT; with a real AR values e.g. sz=480x70,480x360,480x361;.

You want to locate three other keys tpas_partner_id, tpas_video_id and video_id (is the video_id from the url) in the same json file, as these will be used for the 3rd party ad system.

Now you can hit the doubleclick url, which will return a XML file containg information about the ad which will be served for this video. The whole file is interesting and full of important information (so you should probably store it with the video). Look for these three keys AdSystem, AdTitle and Description.

• AdSystem: tells you what system is used for serving the ad. There are three options (as I've mentioned before)
  • AdSense
  • AdSense/ADX
  • GDFP
• AdTitle: tells you what type of the ad is it (e.g. video, or banner)
• Description: tells you mostly the same as AdTitle but sometimes it contains a description of the ad

If the ad is served from adsense system (both adsense or adx) this xml contains all information for the ad, including duration and the direct link for the ad. The link is exactly what are you looking for and you can find it under a key MediaFile.

The link mostly look like this http://www.youtube.com/get_video?video_id=LCeDi-d5CRg&ts=1391921207&t=CyJEI0XYwJVJEYE5CVhqY-DF3KQ&gad=1 and it redirects you to the real file in the mp4 format. If the Ad system is ADX, you will get a direct link to mostly a flv file e.g. http://playtime.tubemogul.com/ad_promoted_videos/4799351_dhxsYlMYHmLMmxL0oBem_1390593897.flv

If the ad is server from 3rd party system, GDFP, you have to call different server. For 3rd party ads Youtube uses FreeWheel service. To obtain the ad data, you have to prepare a XML request, which looks like this:

    <?xml version="1.0" encoding="UTF-8"?>
    <adRequest profile="{profile}" networkId="10613" version="1">
       <capabilities>
          <expectMultipleCreativeRenditions />
          <supportsAdUnitInMultipleSlots />
          <supportsSlotCallback />
          <supportNullCreative />
          <supportAdBundle />
          <supportsFallbackAds />
          <autoEventTracking />
          <requiresRendererManifest />
          <requiresVideoCallbackUrl />
       </capabilities>
       <visitor caller="AS3-5.6.0-r9954-1305270957">
          <httpHeaders>
             <httpHeader value="https://www.youtube.com/watch?v={video_id}" name="referer" />
             <httpHeader value="12,0,0,38" name="x-flash-version" />
          </httpHeaders>
       </visitor>
       <keyValues>
          <keyValue key="_fw_distributorvideoassetid" value="{video_id}" />
          <keyValue key="_fw_yt_type" value="short" />
          <keyValue key="_fwu:10613:lang" value="eng" />
       </keyValues>
       <siteSection pageViewRandom="{random}" customId="youtube_watch" siteSectionNetworkId="{tpas_partner_id}">
          <videoPlayer>
             <videoAsset autoPlay="true" duration="318" videoPlayRandom="{random}" customId="{tpas_video_id}" videoAssetNetworkId="{tpas_partner_id}">
                <adSlots height="390" defaultSlotProfile="{profile}" width="699" compatibleDimensions="2560,1440">
                   <temporalAdSlot height="390" adUnit="preroll" timePosition="0" customId="0_1" width="699" />
                   <temporalAdSlot height="390" adUnit="overlay" timePosition="0" customId="0_2" width="699" />
                </adSlots>
             </videoAsset>
             <adSlots>
                <nonTemporalAdSlot height="60" customId="0_5" width="300" acceptCompanion="true" />
                <nonTemporalAdSlot height="250" customId="0_6" width="300" acceptCompanion="true" />
             </adSlots>
          </videoPlayer>
          <adSlots />
       </siteSection>
    </adRequest>

You probably spoted multiple variables in {}. You have to replace them with custom data, mostly with the data you've obtained from the json object.

• tpas_partner_id: Obtained from the json object
• tpas_video_id: Obtained from the json object
• random: time since epoch in microseconds
• video_id: Obtained from the json object (or from the youtube url)
• profile: I found two different profiles. Sometimes one works and another doesn't and vice versa. Try everytime both 10613:10613_youtube_as3_player and 10613:youtube2.

Now, you send this XML file as a POST request to https://2975c.v.fwmrm.net/ad/p/1? (don't forget to send with the content type application/xml).

The response contains another XML file where you have all necessary data for the ad, including direct links in various formants and dimensions. You can find them under key asset. Again, you should probably store the whole file with the video as it contains additional data for the ad.

That's it, happy hunting.

Solution 2

A far simpler solution is to right click on the video window and select Stats for nerds.

A list of stats will appear, look for Video ID and copy it - only copy the part of the string before the / character, as you only need the Video ID, not the sCPN string. (For example I was looking for a Sam Ovens video, but the advert video had no clickable link, so I used the Stats for nerds feature and found the ID: a-C4m7teAAU).

Then simply append the video ID onto the standard YouTube watch URL like so:

https://www.youtube.com/watch?v=a-C4m7teAAU

Update

Some people are saying that this no longer works. I just tried it on a Samsung video and it still works fine: https://www.youtube.com/watch?v=aw-qPNT7E0k

Solution 3

2020 May, these methods worked for me

Copy paste mode

• Right click on the video ad while it is playing.
• Select [Stats for nerds].
• Open your developer browser console and paste this. Check the Explanation section if you worried about this javascript:

alert("https://www.youtube.com/watch?v="+$(".html5-video-info-panel-content").children[0].children[1].textContent.replace(" ","").split("/")[0])

• You will be prompted with the url ready to copy and paste!!

• The you can use http://savefrom.net to download the video.

---

Click mode

In the console of web browser, paste this:

var a = document.createElement('a');
var linkText = document.createTextNode("--> donwload ads");
a.appendChild(linkText);
a.title = "donwload ads";
a.target = "_blank";
a.href = "https://www.youtube.com/watch?v="+$(".html5-video-info-panel-content").children[0].children[1].textContent.replace(" ","").split("/")[0];
document.getElementsByClassName("html5-video-info-panel-content")[0].appendChild(a);

You will see a clickable link which will open the video in another tab

---

Javascript explanation

At the moment of write this, html of [Stats for nerds] was:

<div class="html5-video-info-panel" data-layer="4">
   <button class="html5-video-info-panel-close ytp-button" title="close">[x]</button>
   <div class="html5-video-info-panel-content">
      <div>
         <div>Video ID / sCPN</div>
         <span>I4VzIvGdTxI / GD6Y 2G9R N3W5</span>
      </div>
      <div>
         <div>Viewport / Frames</div>
         <span>640x360 / 0 dropped of 1706</span>
      </div>   
      <!-- a lot of divs    -->
   </div>
</div>

As you can see, our id is inside of span which is the second child of first child of div with class html5-video-info-panel-content.

Jquery help us to get the span value:

$(".html5-video-info-panel-content").children[0].children[1]

Then we need to delete empty spaces and split to get our video id:

.textContent.replace(" ","").split("/")[0]

Finally we concatenate this value to

"https://www.youtube.com/watch?v="

---

Manual mode

• Right click on the video ad while it is playing.
• Select [Copy Debug Info].
• Then open any text editing program like Notepad in Windows and paste it there.
• Search the pasted text for the line that says "ad_debug_videoId": "XXXXXXXXX"
• The video url will be:

https://www.youtube.com/watch?v=XXXXXXXXX

• The you can use http://savefrom.net to download the video.

---

References

• https://www.quora.com/Why-can-we-not-replay-pre-roll-ads-on-YouTube/answer/Fernando-Juliao

Solution 4

OK. So, I've also been trying to do this...

After I mostly gave up, I right-clicked on the ad - and it offered to copy the 'video id.' For some reason - I just pasted that clipboard content into the url bar - and arrived at the youtube video of the advertisement. It might have been a fluke / but it worked that time. From there - I used one of those 'download from youtube' type sites (that are clearly trying to get you to install software - so be careful) - https://www.y2mate.com/ etc ---- and then I got the video.

( I haven't been able to do this since...) - but maybe it'll connect some dots for someone else - and you can explain why that happened.

UPDATE: It seems like it works sometimes... maybe certain ads are set up to be found. Here's some proof: https://vimeo.com/372764735

View more solutions

Comments

[cosinepi]

thank you for the detailed anatomy of a YouTube page and the ads in it. That's a lot of information and it seems that my question is well answered! I will take a deeper study when I get some time. BTW, your site savedeo.com looks cool!

[Reno]

nice solution Rastislav.

[Guttemberg]

Your answer was very helpful

[Jon]

How did you crawl 22M video pages in a single day? I'm using scrapy and it'll stop crawling after 1000 pages or so, leaving most pages uncrawled. Any hints to achieve higher coverage?

[Rasty Turek]

@Jon wrote my own scrapper. Actually 22M is very low number as at Pexeso we crawl beyond 200M an hour. You will have to figure out how to go around Youtube's anti-crawl protection though

[miguel.martin]

Just tried this and dynamic_allocation_ad_tag doesn't appear to exist on the ytplayer.config object anymore? I can find ad_tag and some others but none of them appear to be the starting point for the pre-roll video... Currently attempting to decipher it again, but any updated information would be fantastic if you have it. Thanks.

[Jake232]

@miguel.martin Did you ever get this figured out?

[Rodrigo]

That's by far the best answer! Thank you!

[user1461607]

that doesn't work anymore, now youtube says "Video unavailable"

[Supertecnoboff]

@user1461607 I just tried it on a Toyota Advert, it's working fine. Perhaps it doesn't work for all adverts. Watch here: youtube.com/watch?v=qtapl-gE5kA

[sheriffderek]

AHHHHH!!!! I really want a copy of this one... Any advice??? Video ID / sCPNL9rQ3SJzuWQ / G5F6 P6NE 093D Happy to pay for it! (and that's an alternate edit - anyone who could find the original edit would be worth more $)

[Supertecnoboff]

@sheriffderek Here you go: youtube.com/watch?v=L9rQ3SJzuWQ

[sheriffderek]

HA! Thanks! I got it a while back by right-clicking - and in that case, it had the 'id' (url) - what was your technique?

[Umar Hassan]

For me the link it extracts is good, it is just that I have to copy it into a new window manually because it won't open on its own when I press OK.

[JRichardsz]

I will try to make the url clickeable :D

[paul]

It worked for me, I found this music youtube.com/watch?v=iNMSaytOihI so amazing. Is this a song originally?