How can I find out what other websites are hosted from my website IP address and why is that important from SEO point of view?

Question

I heard that is important from a SEO perspective what other websites are hosted on the IP address on which my website is stored. Do other websites hosted on my server/VPS affect my search results in the search engines?

Is there a command I can run in a telnet client to find out how many/ what websites are stored on the IP address of my website? (like I send GET requests from telnet client for example)

Solution 1

To complete the other answers, there is no authoritative database of complete IP -> name mappings.

DNS provides for two kinds of mappings:

• name -> IP. Multiple names can map to a single IP
• IP -> name (aka "reverse"). A given IP can map only to a single name.

Whether there is actually a reverse (IP -> name) mapping and what it points to is subject to the policies of the "owner" of the IP address (the web hosting company in this case). But it will not provide an exhaustive list of all names that map to that IP.

The services referred to in the previous answers try to assemble such a database, by making a not of all name -> IP mappings they find which point to said IP. But the number of domain names is huge and constantly changing, so they can hardly have a complete list, just whatever they found.

The only ones who may have the whole list (and that is actually not necessarily the case) is your web host, however I doubt they would share this information.

Of course, this would only be an issue in the case of shared hosting where many different sites are hosted on the same server with the same IP address. If you have a dedicated server (or equivalent) with you own IP address, then this would not be an issue.

However, note that all sorts of classification methods (e.g. for spam) will consider whole blocks rather than individual IP addresses, so even in that case, if you're in "a bad neighbourhood", you may get penalised by the behaviour of your neighbours.

Solution 2

Don't know if it's possible with simple command, but there are special "Reverse-IP" services just for that, check that one for example: http://viewdns.info/reverseip/

Apparently if you share IP with other domain, that is considered... malicious, like it's sending spam or spreading some viruses or having some "illegal" content, then your good domain also will be affected or even completely blocked by search engines. However if all domains on this IP are good, it should not have much effect (theoretically).

Solution 3

To answer your question directly: No, in most cases there is no negative impact for sharing an IP with a bad domain. How do I know this? I am the SEO oracle. I come from the future to save humanity.

Kidding, but only sort-of. If there was such a thing as IP penalization, we would be seeing far far different search results than what we do today because many domains wouldn't have gained authority. Here is what I mean:

• Shared Hosting Plans. It's not uncommon for poor quality hosts such as hostgator, 1&1, or others to pack over 400 tenants onto a single server. Allowing one of those to wreck the momentum of 399 others would be devastating. Riots would ensue, flaming torches, pitchforks, and hordes of angry OP's would be at Google's doorstep overnight. Google is not that stupid.
• Rented Ecommerce & SaaS. Let's take Bigcommerce for example. They use multi-tenant IP's so you will find many stores along side yours when looking up rdns. Looking at the domains you may notice that they are extremely low quality, repetitious, etc. In many cases they are straight up spam. Now let's go a bit further. BC allows you to run a demo store, which uses some of the same routing assets as the live stores. Your demo utilizes things such as util subdomain routes. Demos are often abused, used for spam, etc. Those routes should be toasted from SERP's, but they aren't. Again, Google is not that stupid.
• High Churn Cloud Servers. The biggest brands in the world use Google/AWS/Digitalocean alongside the biggest slimeball botnets in the world. They are always in close proximity to each other. Maybe Netflix can afford AWS enterprise which guarantees a cleaner IP, but 99% of the rest of folks using the service can't. More slimeballs churning throwaway instances to get away from banned/wrecked IP's via AWS means more IP's with shady historicals. The ratio of bad IP's to good IP's may someday get significant, which can make AWS look bad. Google is not that stupid to make AWS look bad. It's in their best interest to keep on indexing that service in a fair manner.
• IPv6 Saturation. There are just too many IP's to keep track of. 340,282,366,920,938,000,000,000,000,000,000,000,000 of them to be precise. IPv6 means each person alive on earth can have 5x1028 IP's allocated to them...enough to make them "disposable". Thats too many to track and far too many to worry about until you make a big big ripple. So in light of these numbers, is 1 IP hosting 1 crap spam domain out of 400 on a server big enough to cause a ripple for the rest? I don't think so. Is 1 botnet C&C running its own VPS with a couple of phishing sites beside it enough to cause a ripple? You bet. Worrying about Mr Spam's IP in an infinite sea of disposable malevolency is not sustainable. Worrying about heavy hitters doing the same thing on their own instances IS sustainable. Google isn't that stupid to spend all resources on small fries, at the IP level. They use the domain and other levels instead, which makes sense.

As you can see, IP isn't as important as people think for SEO authority. All the rank factors start to hit at domain level, and they are semi-conservative + transparent (as in, it's obvious if you are doing something wrong). Here is a link exploring that thought: https://moz.com/search-ranking-factors/correlations#6

So when does IP authority actually matter? Server side email delivery and client IP bans. That's about it. Does that matter for SEO? Nope.

PS: there are a few tools to rdns, you should try them all. I don't know of one that runs easily since it requires an "index" of the net to find neighbors. Since they are "crawlers" the website tool results may vary. One rdns site may show a domain on the IP that the other missed. Another thing you can try is SSH into your shared server if they let you do those sorts of things. Check out the user list and start hitting the server IP followed by username like so to see what kind of neighbors you have: 123.123.123.123/~username

PSS: We tried Bigcommerce once. Our IP was pre-junked, mail server abused into the ground by demo store spammers, mail server was in RBL's, and neighbors were crap halfbaked stores, but our brand new domain gained PR3 and first page rankings in less than 2 months, without any backlinks whatsoever. If that IP mattered, we would have stayed buried.

Solution 4

You could use a service that does "reverse IP lookup" to check for websites hosted on the same server / the same IP address. This service http://viewdns.info for example lets you simply enter a domain name into "reverse IP lookup" and spits out a list of other websites / domains hosted on the same address. The same here: http://www.yougetsignal.com/tools/web-sites-on-web-server/. Results may differ – so I always check with two services.

In my honest opinion it will hurt a little, if there are for example dozens of websites about the same subject – especially if they link to each other heavily. But I suppose the bug G is clever enough to distinguish between black hat SEO practice and a good neighborhood. Image a record label hosting all their bands on their own server – I suppose that would (should!) for example not get punished.

If or if not this actually hurts SEO has been discussed for probably as long as there are search engines and web servers – for example here on quora or here on warriorforum.

Solution 5

I am getting very high server loads that my host explains away as it is because I am on a shared hosting platform. The way I look at it is that if Google is truly using speed as an indicator and it is crawling my site when the server load is off the Richter scale, this would mean that it would perceive my site as being slow. Hence, this could not be good for SEO.

View more solutions

Comments

[Admin]

www.bing.com search engine will provide what you're looking for if the website has been indexed. For example, the following search term returns GoDaddy shared hosting websites: ip:50.63.202.1

[closetnoc]

To put it to rest, yes Google does penalize sites that exist on an bad IP address. It is not even debatable when you think about it. We have seen this effect here. However, it does have to be an extreme case before anything recognizable as a penalty occurs. Otherwise, any effect may be small enough if at all to be totally missed. This effect is a SERP filter and effects placement in the SERPs. This means that any effect can easily be reversed.

[Naomi Rison]

Thank you for the comprehensive answer. What does authoritative database means in this context? And what does bad neighborhood means? I tested some services and some found more than 80 other websites hosted at my ip address... i guess i have some cheap hosting... some other services only found 4 or 5 other websites.

[Naomi Rison]

tillinberlin, what does a "friendly neighborhood" mean in this context ? @closetnoc what does SERP filtering do? I couldn't find too much about this on the internet?

[closetnoc]

Friendly is a misleading term used by SEOs. It is actually whether an IP address, IP address block, host, or registrar is found to be abusive in some way. The correct terms are good neighborhood or bad neighborhood. Search engines work in a handful of phases, indexing, metrics gathering and calculation, query look-up, filtering the query results, presenting the results in the search engine result page (SERP). Much of what people refer to rank can either be applied at the metrics gathering and calculation or within a SERP filter (filtering the query results).

[jcaron]

The DNS is authoritative for name -> IP mappings. Either there is such a mapping for a given name, or there is not. But it does not provide a way to query all these mappings by IP, only by name. So databases you can find just "try to" find all these mappings (and index them by IP so you can lookup those mappings by IP), but they can never be guaranteed to hold all the mappings, or that the mappings they have are up-to-date.

[jcaron]

A "bad neighbourhood" in this context means having spammers, hackers, hacked machines, etc. in the same block. It's usually a sign of lax enforcement by the ISP/web host, and it has a strong tendency to draw more people doing the same kind of stuff. There may be lots of legitimate sites in the middle, but they will often get thrown with the bathwater.

[jcaron]

A shared web hosting server can frequently host hundreds, possibly thousands of sites.

[closetnoc]

We have seen (here) sites drop in the SERPs radically the day after starting to use a CDN where the IP address is also used by a spamdexer. Just complaining and getting onto another IP address solves the problem nearly as fast. You are right, generally, the effect is small and imperceptible or none at all. Just because you cannot see something does not mean it does not exist. For the reasons you detail extremely well, there is a lot of forgiveness, however, there are times where Googles wrath is self-evident. Boy is it! Cheers!!

[dhaupin]

@closetnoc Yeah for real, good point, automation running like that def causes enough of a ripple to be noticed by the islanders. And the free services like CF are more or less a pirate bazaar, especially since the origin site IP's can hide behind the DNS, essentially anonymized. Listen to closetnoc and choose your provider for CDN's, load balancers, etc wisely :)

[closetnoc]

I was also happy to see you mention the sleazy cloud services including Amazon (within the super sleazy - why bother with them category) and Google who bitches about site quality and does a 5h1tty business damned near everywhere else. Apparently it is okay for Google to offer services that are chocked full of sleazy customers doing really bad things and then bludgeon you over the head for being slightly mistaken in your sites content, structure, etc. Give me a break! "G" stand for Going for the Gold Good or bad Gal-dangit.

[Simon Hayter]

400 accounts on a single server?... you serious under estimated this number. Most shared hosting companies nowadays will use between 4-8 Quad Core Xeons, with 96gb+ ram and a fast raid setup, and will host websites in the 10,000s... Why? because its cheaper to host within one rack than multiples. Also, the amount of users is irrelevant, its about the quality of hardware and software, and not over-selling.

[jcaron]

@Lightness Races in Orbit, name to IP and IP to name mappings are completely independent in the DNS. Many names may map to a single IP (as stated above), but the independent IP to name mapping (via PTR records in the in-addr.arpa domain) only points to a single name (which may not even map back to the same IP!).

[Naomi Rison]

@jcaron what does block mean? On the internet i found that a block are ip addresses assinged to a country but I guess here it has abother meaning.

[Harper - Reinstate Ukraine]

Tracking by IP is infeasible? Oh. Because many people do it and it's not that hard. You're just looking at it forwards. Look at it backwards and it becomes easy. Well, by big-data standards anyway. By "neighborhood" we usually mean server or ISP, and that is what the SE's are concerned with. Spammers resist this with IP games, it's an arms race. That's why some of them hack innocent web sites to avoid clustering

[Harper - Reinstate Ukraine]

"Friendly neighborhood" means the ISP is not a spamhaus.

[Harper - Reinstate Ukraine]

Nothing wrong with sharing an IP with thousands of domains. That is how the web Actually Is. If that was penalized, nothing would rank.

[Harper - Reinstate Ukraine]

Reverse IP is not trustworthy and is irrelevant when thousands of domains typically host on a single IP. It's not that hard to get a complete list of domains from the registrar. You don't need every registrar, just do a lightweight web crawl of known domains and you'll discover most that are linked. You could do a modestly effective job of this at home, it's no problem for a commercial entity that does crawling for a living. For them, discovering every domain is not a problem.

[daboross]

@LightnessRacesinOrbit RDNS is what he's referring to, I think. Only place I've seen it actually used is in what your IP is displayed as in IRC servers. RDNS only provides one domain name per IP. This isn't related to multiple domains pointing to the IP, as far as I know - all it means is there's one single domain which points to the IP which is marked by the network host as the rdns "name".

[jcaron]

@LightnessRacesinOrbit, I think we must not be talking about the same thing. Again, having many names pointing to the same IP address is definitely not an issue, and is indeed very, very common (it's actually the whole reason for the question). The problem is that there is no way via DNS to query all those mappings just by providing an IP, but only by looking up names and finding that they point to said IP. The DNS system does however provide an IP->name mapping, but that one is completely independent form the name->IP mapping, and returns only one name, or none at all.

[Harper - Reinstate Ukraine]

You'd think, right? Actually what I see in the wild is: a guy with $50,000 in domains is pretty paranoid about detection. Whereas ISPs with tens of thousands of small site customers will obliviously stack them all onto a single IP. They are all penny ante sites, like bloggers, barbers and food trucks, and you never know when one will explode and get a million hits an hour. That's why there's deep infrastructure behind that.

[Harper - Reinstate Ukraine]

And those small sites are the bread and butter of search. On the web, the long tail is quite long.

[dhaupin]

@Harper It's about logistics, volume, and lethality; not the simplistic action of flagging/banning an IP that comment spammed. From a perspective of Google, how are you going to maintain a massive IPv6 list without piggybacking on top of 3rd party lists? And once you start piggybacking on top of 3rd party community/automation/plugin built lists how can you ensure that the IP's are not innocent or false positive, as in, a hacked site or immature mod? How can they ensure that they broke the rules when the list doesn't know their rules? How can their bot scan faster than new IP's are assigned?

[dhaupin]

@Harper And, I'm speaking about the future too when i talk about the IPv6. Right now it's rather easy. Hosts are very stringent about assigning their IP's. IPS's may only have a certain amount of IPv6 in their pool. Checking the allocation sizes for a decent sized organization, on the lowest tier of ISP allocation, (let's think OVH spambay) they could get the equivalent to the total of all IPv4 in one shot. Example, 2001:db8::/32 is equal to 4,294,967,296 addresses....and that's just 1 of thousands of large hosts. There are 20 tiers above them that go up to 4,503,599,627,370,496 addresses.