Why does baidu crawl my site like crazy

Question

I have Microsoft SQL Server 2008 installed. It appears in the add/remove programs window.

How can I create/delete/alter tables and find the address of my server if I don't have Management Studio installed?

I'd like to create the tables and such directly in Visual Studio 2010 premium.

Solution

It doesn't sound like something you should worry about too much, google crawls some of my sites in a similar way, but it could be a fake bot, so check out the links below...

The Baidu Spider FAQ's (Chrome's built in translation) have this to say:-

4. Why Baiduspider crawls my site continuously? In order to ensure the latest information is presented, Baiduspider crawls new pages or pages frequently renewed at your site. Please check the log to see whether the crawling from Baiduspider is reasonable.

To avoid the excess crawling by spammers or other trouble makers who pretend to be Baiduspider, you can check the log. When you find any abnormal crawling, please inform us at [email protected] and provide the log of Baiduspider.

Baidu does offer it's own version of webmaster tools (chrome's translation function will be needed when accessing the tools themselves) perhaps signing up for them might yield some answers. Google WMT offers limited ability to manage crawl rate, perhaps their tool offers some similar functionality.

Comments

[bekay]

Do you receive any organic traffic from Baidu.com? If no I would block the spider if it's eating up your bandwidth like that.

[user1394704]

Not sure about what you mean by organic traffic, but if you mean from users using baidu search engine, the answer is no i don't. It's not taking much bandwith or ressources. It's just that i'm curious why baidu is crawling it so much.

[bekay]

Yes that's what I mean. If it's not taking up much bandwidth I wouldn't worry unless it continued. You could always email them. " If you find any unreasonable access from Baiduspider, please inform us at [email protected]"

[Lèse majesté]

I'd only block a search engine spider if you never want to receive any search traffic from them, not just because you currently don't receive any--especially considering that Baidu is the 3rd largest search engine in the world in terms of market share.

[Fiasco Labs]

Baidu is very aggressive till it gets enough of your site indexed. There are scrapers out there that claim to be Baidu, I use botsvsbrowsers.com to identify them and ban them when things get out of hand. Yandex can generate the same kind of traffic as well.

[toomanyairmiles]

@FiascoLabs It would be great if you could share some of your experience with the Chinese search engines with us via the community wiki, we don't have much content about them.

[Admin]

I only see Connect to Database/Connect to Server. Neither show me the name of the Server. They expect me to type in the name manually. :(

[SLaks]

Try the down arrow. Alternatively, type localhost (or, for Express, .\SQLExpress)

[Admin]

It appears I only have SQL Express (judging from services.msc window) - what can I do?