How to decode google gclids

Question

Now, I realise the initial response to this is likely to be "you can't" or "use analytics", but I'll continue in the hope that someone has more insight than that.

Google adwords with "autotagging" appends a "gclid" (presumably "google click id") to link that sends you to the advertised site. It appears in the web log since it's a query parameter, and it's used by analytics to tie that visit to the ad/campaign.

What I would like to do is to extract any useful information from the gclid in order to do our own analysis on our traffic. The reasons for this are:

• Stats are imperfect, but if we are collating them, we know exactly what assumptions we have made, and how they were calculated.
• We can tie the data to the rest of our data and produce far more accurate stats wrt conversion rate.
• We don't have to rely on javascript for conversions.

Now it is clear that the gclid is base64 encoded (or some close variant), and some parts of it vary more than others. Beyond that, I haven't been able to determine what any of it relates to.

Does anybody have any insight into how I might approach decoding this, or has anybody already related gclids back to compaigns or even accounts?

I have spoken to a couple of people at google, and despite their "don't be evil" motto, they were completely unwilling to discuss the possibility of divulging this information, even under an NDA. It seems they like the monopoly they have over our web stats.

Solution 1

By far the easiest solution is to manually tag your links with Google Analytics campaign tracking parameters (utm_source, utm_campaign, utm_medium, etc.) and then pull out that data.

The gclid is dependent on more than just the adwords account/campaign/etc. If you click on the same adwords ad twice, it could give you different gclids, because there's all sorts of session and cost data associated with that particular click as well.

Gclid is probably not 100% random, true, but I'd be very surprised and concerned if it were possible to extract all your Adwords data from that number. That would be a HUGE security flaw (i.e. an arbitrary user could view your Adwords data). More likely, a pseudo-random gclid is generated with every impression, and if that ad is clicked on, the gclid is logged in Adwords (otherwise it's thrown out). Analytics then uses that number to reconcile the data with Adwords after the fact. Other than that, there's no intrinsic value in the gclid number itself.

In regards to your last point, attempting to crack or reverse-engineer this information is explicitly forbidden in both the Google Analytics and Google Adwords Terms of Service, and is grounds for a permanent ban. Additionally, the TOS that you agreed to when signing up for these services says that it is not your data to use in any way you feel like. Google is providing a free service, so there are strings attached. If you don't like not having complete control over your data, then there are plenty of other solutions out there. However, you will pay a premium for that kind of control.

Google makes nearly all their money from selling ads. Adwords is their biggest money-making product. They're not going to give you confidential information about how it works. They don't know who you are, or what you're going to do with that information. It doesn't matter if you sign an NDA and they have legal recourse to sue you; if you give away that information to a competitor, your life isn't worth enough to pay back the money you will have lost them.

Sorry to break it to you, but "Don't be Evil" or not, Google is a business, not a charity. They didn't become one of the most successful companies in the world by giving away their search algorithm to the first guy who asked for it.

Solution 2

The gclid parameter is encoded in Protocol Buffers, and then in a variant of Base64.

See this guide to decoding the gclid and interpreting it, including an (Apache-licensed) PHP function you can use.

There are basically 3 parameters encoded inside it, one of which is a timestamp. The other 2 as yet are not known.

As far as understanding what these other parameters mean—it may be helpful to compare it to the ei parameter, which is encoded in an extremely similar way (basically Protocol Buffers with the keys stripped out). The ei parameter also has a timestamp, with what seem to be microseconds, and 2 other integers.

Solution 3

I've been working on this problem at our company as well. We'd like to be able to get a better sense of what our AdWords are doing but we're frustrated with limitations in Analytics.

Our current solution is to look in the Apache access logs for GET requests using the regex:

.*[?&]gclid=([^$&]*)

If that exists, then we look at the referer string to get the keyword:

.*[?&]q=([^$&]*).*

An alternative option is to change your Apache web log to start logging the __utmz cookie that google sets, which should have a piece for the keyword in utmctr. Google __utmz cookie and you should be able to find plenty of information.

How accurate is the referer string? Not 100%. Firewalls and security appliances will strip it out. But parsing it out yourself does give you more flexibility than Google Analytics. It would be a great feature to send the gclid to AdWords and get data back, but that feature does not look like it's available.

EDIT: Since I wrote this we've also created our own tags that are appended to each destination url as a request parameter. Each tag is just an md5 hash of the text, ad group, and campaign name. We grab it using regex from the access log and look it up in a SQL database.

Solution 4

I think you can get all the goodies linked to the gclid via google's adword api. Specifically, you can query the click performance report.

https://developers.google.com/adwords/api/docs/appendix/reports#click

Solution 5

This is a non-programmatic way to decode the GCLID parameter. Chances are you are simply trying to figure out the campaign, ad group, keyword, placement, ad that drove the click and conversion. To do this, you can upload the GCLID into AdWords as a separate conversion type and then segment by conversion type to drill down to the criteria that triggered the conversion. These steps:

1. In AdWords UI, go to Tools->Conversions->Add conversion with source "Import from clicks"
2. Visit the AdWords help topic about importing conversions https://support.google.com/adwords/answer/7014069 and create a bulk load file with your GCLID values, assigning the conversions to you new "Import from clicks" conversion type
3. Upload the conversions into AdWords in Tools->Conversions->Conversion actions (Uploads) on left navigation
4. Go to campaigns tab, Segment->Conversions->Conversion name
5. Find your new conversion name in the segment list, this is where the conversion came from. Continue this same process on the ad groups and keywords tab until you know the GCLID originating criteria

View more solutions

Comments

[Draemon]

Re 1 - I'm pretty convinced they're not random. Our gclids are similar, other people's are similar too, but dissimilar to ours. They're definitely not a simple incrementing id. 2.1 - This is hard since there are a lot of gclids you don't see (if they don't click on them). ...

[Draemon]

... I have collected a large list from the logs, and I have identified which bytes change more than others, and my brain shouts "this isn't random" but beyond that nothing has lead anywhere. 2.2/2.3 - I'd love a link to any techniques or tools - instinct hasn't got me very far.

[Draemon]

tbh I don't hold out much hope either, but it would be very cool - and I really don't think this is something google should have a monopoly over. I just have a niggling feeling it's "easy if you know how"

[Sakie]

blog.merjis.com/2007/07/16/click-fraud-google-adwords-and-gc‌​lid seems to have a lot of discussion about the role of the gclid and googling seems to yield a lot of basic insight. They're new to me, to I'll poke in if I learn more.

[Sakie]

Since Google Analytics can understand the gclid, it's likely to be a two-way hash, which is a plus. Work on referrer_id.

[Draemon]

referrer data is sent by the client (user agent) and is unreliable.

[Draemon]

No, the gclid definitely isn't a cryptographic hash, as it has terrible entropy (gclids from the same account will be quite similar). . Something is encoded in these gclids, but finding out what or how is another matter altogether.

[mozz100]

@Jeff Wu I read your answer and, specifically your edit. I wonder how you appended your own request parameters to destination URLs - we have many campaigns already running so, it'd be mighty laborious. For the benefit of others who might wonder the same thing, Google AdWords scripts are one way to do it. Here's a couple of helpful links: Google documentation and Example script adding params to URLs

[Jeff Wu]

That's pretty cool, I wasn't aware of that Javascript library when we implemented this. I wrote a Python script to take a CSV of ads and add the tag to each one of the links. Our campaign manager would then take the CSV and upload it to AdWords using the desktop utility.

[Draemon]

+1. 5 Years later and someone thinks to try protobuf. Hopefully someone will figure out how to relate this back to adwords at some point!

[Bani]

Yes, this is a new option that Google provides. You can't get very old historical data from this API endpoint though, so it's just good for newer data.