How to fight off Google Analytics referrer spammers?

Question

Last months I have lots of referrer spammers in my GA statistics. Their count is ~10x higher than count of legit visitors (my site is not very popular yet). I've turned on an option to hide known spammers in GA settings, but it didn't help at all. It seems these spammers are using scripts to spam directly to GA (i.e. they are not logged in my IIS).

Is there anything I can do to stop these spammers?

UPD 10 months later, and they started spamming using fake target page names... and Google is still doing nothing about it.

Solution 1

The Spam is getting out of control. The list it's growing and it's time-consuming and not even efficient to add a filter for each of the spammers since most of them shows up for a few days and then disappear and a new one comes.

There is a lot of misinformation, the most common mistake is recommending to use the .htaccess, this file blocks the access to the Website, although there are a few crawlers(5 or 6) than can be block, the vast majority of the spam never access your site is Ghost Spam.

The best way to stop this type of spam (Ghosts) is by creating a valid hostname filter, the ghost spam use either a fake or not set hostname, so with this filter you don't have to add endless filters, one filter will take care of the old and new spam.. Been using this solution successfully for 3 months

More information about this method here:

https://stackoverflow.com/a/28354319/3197362

Solution 2

Okay. Without knowing the sites in question, I will try and explain a bit of what is going on and I will provide just a few links.

From: http://www.cradlecloud.com/ban-block-blackhatworth-com-spam-referrals/

I get the following domain names associated with the new method of referrer spam that people are seeing of late.

• BlackHatWorth.com
• Iskalko.ru
• Lomb.co
• Lombia.co
• Econom.co
• Darodar.com
• ILoveVitaly.Com
• Priceg.com
• Hulfingtonpost.com (New- added Jan 16 2015)
• Bestwebsitesawards.com (New- added Feb 3 2015)
• Ranksonic.info (New- added Feb 3 2015)
• Cenoval.ru (New- added Feb 6 2015)
• o-o-6-o-o.com (New- added Feb 25 2015)
• Humanorightswatch.org (New- added Mar 4 2015)
• S.click.aliexpress.com (New- added Mar 17 2015 - Suspected)
• www1.social-buttons.com (New- added Mar 23 2015 - Suspected)
• 4webmasters.org (New- added Mar 26 2015 - Suspected)
• Googlsucks.com (New- added Apr 07 2015)
• Addons.mozilla.org (New- added Apr 07 2015 - Suspected)
• Smallseotools.com (New- added Apr 13 2015 - Suspected)
• Theguardlan.com (New- added Apr 14 2015)
• Buy-cheap-online.info (New- added Apr 16 2015 - Suspected)
• Site1.free-share-buttons.com (New- added Apr 29 2015 - Suspected)
• Sanjosestartups.com (New- added May 25 2015)
• Trafficmonetize.org (New- added June 03 2015 - Suspected)
• Howtostopreferralspam.eu (New- added June 09 2015 - Suspected)
• Www10.free-social-buttons.com (New- added June 16 2015 - Suspected)
• Getitfree.us (New - added June 18 2015 Ownership cannot be determined. Thank You - Trey Copeland)
• Www6.free-social-buttons.com (New- added June 18 2015 - Suspected)
• Erot.co (New- added June 26 2015 - Suspected)
• 3g2upl4pq6kufc4m.onion (New- added July 04 2015 - Suspected)
• Traffic2money.com (New- added July 28 2015 - Suspected)

Note: Suspected items- do appear to follow the same pattern of ownership, and may not be tied to the same offender.

A rather exhaustive list of spam referrers maintained by Piwik can be found here: https://github.com/piwik/referrer-spam-blacklist/blob/master/spammers.txt (Thank You - user2428118)

To Quote:

BlackHatWorth.com is a relatively new domain created only on January 7th, 2015 which is now being used for referrer spam. As a matter of fact, this referral spam website is being hidden behind the name of shopping search engine and beautiful scenery images.

...the IP address of BlackHatWorth.com which is 78.110.60.230 is the same one associated with other referral spam websites...

In fact, the domain BlackHatWorth.com is owned by the same Russian who owns the other referral spam domains such as ILoveVitaly.com, Econom.co, and Darodar.com. The domain owner’s name is supposedly Vitaly A Popov of Samara (city), Samaraskaya Oblast (state), Russia.

You cannot block this!

From: http://www.blackmoreops.com/2014/12/19/darodar-com-referrer-spam/

To Quote:

Here’s a quick primer on how Google Analytics works.

So, you get setup on GA and get a code from them. The code looks like UA-number-1 or some such thing. That number is your “account number” on GA. Now, this code and a bit of javascript go onto your webpage. Now, somebody visits your page, and their browser runs that javascript code.

That javascript code is what “records” their visit. It makes their browser talk to Google Analytics. Specifically, it makes certain types of HTTP requests that Google records information about, and then GA displays summaries of that information to you.

Pretty basic, right? Still with me? Okay, now, if all it is is this Javascript sending the “visit” to them, then anybody can fake that. Anybody at all. All I have to do to make your GA show false information is to send my fake information directly to GA.

I don’t need to visit your site at all. I don’t need to run javascript at all. I just need to reproduce those HTTP requests, which are public and so anybody can see them and how they work. They’re even fairly well documented, publicly, by Google themselves.

So, now, let’s say I’m a spammer jerk. I want to get people to see my spammy site. So, what do I do? I write a small bit of code to send thousands upon thousands of these fake requests to GA, and I simply cycle through all the UA numbers, in order, at random, whatever. I send a fake visit, with a fake referrer, and my spammy domain name. And guess what? It shows up in your Google Analytics screens.

You see this spam like any other normal visit. Because as far as GA is concerned, it was a normal visit. All they’re recording are those HTTP requests, which normally come from the GA javascript code. But a request is a request, and making a fake one is very, very easy.

That is what is going on. All I need is your UA number and with only a minor bit of effort I can fake a visit to your site without ever actually connecting to your site at all. That fake visit can have any domain name and any referrer in it that I choose.

This is an attack on Google Analytics, to promote whatever site is showing up. You cannot block it on your server, because your server is not involved at all.

You can do two things: one, set-up a filter as John Conde suggests; and two, see if there is a way to inform Google. For that I do not have an answer, but I have an idea.

[Update]

This is beginning to reach outrageous proportions from hundreds of spam hits a day to full out advertising such as this one:

Solution 3

You can exclude them by creating a filter. You need to find something specific enough so you don't accidentally block good visitors and it is tedious as you have to manually add each spammer but this will do the trick.

Solution 4

To answer your title question directly "How to fight off referrer spammers" the simplest answer is to drop Google Analytics and switch to Piwik, which automatically blocks all referrer spam by default.

I realise you are probably used to Google Analytics and wish to keep using it, but if you look at the bigger picture you do have another option which works very well. Piwik can also be configured to log visitor IP addresses if you like, and it does not leak visitor data to third-party advertising companies which may appeal to some users.

Solution 5

In case you are still searching for a solution, and can't understand the references made, here's the Definitive Guide to Removing Referral Spam, now on it's 133rd revision since Dec 28th, 2014: http://www.analyticsedge.com/2014/12/removing-referral-spam-google-analytics/

TL;DR:

• a valid hostname filter will remove all of the ghost referrals (Include hostname mydomain.com)

• a specific exclude filter (or website blocking) will remove the much shorter list of spam crawlers (semalt.com|kambasoft.com|7makemoneyonline.com|best-seo-offer.com|best-seo-solution.com|buttons-for-website.com|buttons-for-your-website.com|-musicas*-gratis|anticrawler.org|savetubevideo.com|ranksonic)

• an Advanced Segment can be used to remove them from your historical reports.

The Advanced Segment can be imported from the Google Analytics Solutions Gallery: https://www.google.com/analytics/gallery/#posts/search/%3F_.sort%3DDATE%26_.start%3D0%26_.type%3DADVANCED_SEGMENT%26_.viewId%3DGjpPQhFgS9aVzniXH4MTIg/

View more solutions

Comments

[closetnoc]

Please check you log files for hits with referrer spam and let us know if you see any. If you can, please edit the question and include the referrer spam in question. I suspect I know the answer already. I am doing some research and there is a fair amount of referrer activity lately some of which I have experienced myself. Also, if you have WordPress installed, please make sure it is up to date. One of the sites seems to be also going after WP vulnerabilities. Also, do not visit any of these sites unless you use a text browser like Curl.

[deepfritz]

@closetnoc, these spammers aren't logged on my server. As I wrote, they are spamming directly to GA. I don't use WP. "include the referrer spam in question" - what do you mean?

[closetnoc]

I know who they are and I posted an answer.

[Rody]

Definitely the best and easiest way to prevent this is to use: referrerspamblocker.com It's an automated setup of filters in Google Analytics. Been using it for months now.

[sam]

Does setting 'bot filtering' to true in the google analytics settings help resolve the issue ? - lunametrics.com/blog/2014/08/07/…

[deepfritz]

@sam, no. Only the valid hostname filter does.

[closetnoc]

Do you have any IIS advice that can help? I can help with Apache, but the OP seems to have IIS.

[John Conde]

If they're not hitting their website, which is what I believe they said, then this wouldn't be an effective method of blocking them. Or did I read the question wrong?

[closetnoc]

I read it as the OP is getting hits with referrer spam.

[John Conde]

Indeed, but apparently they are hitting GA directly instead of hitting their site. It seems these spammers are using scripts to spam directly to GA (i.e. they are not logged in my IIS).

[closetnoc]

Okay. You are right. My bad!! I am not sure why I did not pick this up because I was just reading about some of this just prior to this question. I am getting hit by referrer spam too and one article mentioned (briefly) that there were no entries in their log files. Interesting new method.

[closetnoc]

@JohnMueller This my not be within your realm, but can you send some e-mails around to see if G has knowledge on this? Thanks in Advance!!

[closetnoc]

I got it John. I posted an answer.

[John Conde]

Good call on pinging JM. If anyone can find out for us, he's the one.

[closetnoc]

@JohnConde Thanks! John Mueller may not know about this, but someone in G should. Even if there is no answer, at least G should have a heads up if they do not already. It is the right thing to do for us all. Cheers!

[closetnoc]

@JohnConde BTW- I saw question(s) on this on other SE (and possibly related) sites. I did not take notes, but is there a good way of broadcasting a note to the other Mods? Or should I just Google these domain names and look for answers and post?

[John Conde]

Questions relating to what exactly?

[closetnoc]

@JohnConde related to these referrer spam domain names and the activity. I read one at least.

[closetnoc]

@JohnMueller [Update] I noticed that the referrers that showed up in GA have now stopped. I want to notify you of a new one JIC it is useful. The domain name is hulfingtonpost.com. I updated the answer. Thanks in Advance!!

[closetnoc]

@user626528 [Update] I noticed that the referrers that showed up GA have now stopped. I updated the answer with a new one JIC you are filtering these out.

[joelmdev]

Link provided is heavy on LAMP, light on IIS. This article explains what to do on Windows machines in a bit more depth: tusksoft.com/blog/posts/6/…

[closetnoc]

@joelmdev You missed the point that these cannot be blocked. This is a direct attack on Google. No-one has any control except Google. They have been notified and these have stopped for the most part.

[joelmdev]

In the cases where it is logged directly through GA JS, that is correct- nothing you can do about it. But that method is small potatoes for referer spammers. The main goal of referer spam is to get the urls they are trying to promote into publicly accessible logs so that they are subsequently indexed by search engines. While the OP's issue might be direct manipulation of the GA JS, the majority of referer spam hits are not- too much work for too little return. Implying that referer spam is direct manipulation of GA JS and there's nothing you can do about it tells an incomplete story.

[closetnoc]

@joelmdev I agree. However, this question is specifically about referrer spam being directly aimed at Google. We have other questions regarding referral spam and how to block on the server. I have answered a few. BTW- there is no payoff from Google on referrer spam, but there are enough other search engines out there especially in Russia that can be spammed that way. Hence the reason why it still exists and comes primarily from Russia and Poland last I looked.

[Alfred Myers]

This answer has nothing to do with OP's question as spammers are spamming directly Google Analytics bypassing OP's web server.

[Adam]

@AlfredMyers Your comment is partially true since some spammers DO use the webserver method, others use the direct method as you mention. In the latter case the only solution is adding filters in Google Analytics to get clean reporting. To prevent this I'd prefer Google used a longer hashed unique identifier for GA accounts so that spammers can not simply loop through available numbers and fire queries.

[Alfred Myers]

The OP specifically states "...these spammers are using scripts to spam directly to GA (i.e. they are not logged in my IIS).". So for this specific question, they aren't accessing his web server.

[Steve]

This is a really interesting answer, but what is the point of spammers doing this? So what if their link shows up in GA? What might they be trying to achieve?

[closetnoc]

@Steve I cannot see that there is a big payoff, however, there has to be one somewhere. The one individual mentioned in the answer hates Google and likely wants to embarrass Google at the very least. But they are also clearly using this spam to drive traffic to various websites that keep being registered. I have even seen something like an ad using this method. It is annoying as hell!! It is hurting innocent people more than Google really. I assume that it works... very sad.

[user2428118]

Piwik has published a list of referrer spammers that they're actively updating: github.com/piwik/referrer-spam-blacklist/blob/master/…

[Matthieu Napoli]

By the way the referrer spam blacklist that Piwik uses is open source: github.com/piwik/referrer-spam-blacklist

[JamesRyan]

How about add a custom variable to your site then filter out all visitors that don't have that variable?

[closetnoc]

@JamesRyan I guess that can work! It is at least a clever and original thought!! I admit to not filtering out all the bad referrals- I should- it p1$$3$ me off to see them.

[Venkat Nori]

I am glad that this is the accepted answer, it really should be more appreciated. The vast majority of spam referrers in Analytics are ghost referrers. There are some you need to block with .htaccess, but that's an ongoing battle to stop them skewing the figures (Semalt being the prime culprit of this - legitimate bot my Aunt Fanny).

[Ooker]

Oh Carlos I'm about to recommend the guide to fight spam in your site. I know this comment is going to be perceived as a way to SEO/sell but it's not. I really admire your input on that guide.

[Don Dilanga]

their spam database is quite small. it doesn't even the domains noted in my analytics report.