How to prevent Google from finding my admin index page?

Question

I am running a website but for some days i stopped it and put the under-construction page because the Index of admin page is visible to the outside world through the Google search. One of my friend told me that your websites index is visible and its one step away to access the password file and he shows me that very simply using the Google search.

How can i prevent this and i am hosting my site with a hosting company and i report about this to them but they simply replied to me still its secure so you no need to worry...

am i really don need to worry and continue my site with the visible index of admin page?

Solution 1

Google webmaster tools allows you to remove pages from their search index for sites that you own.

You can then use robots.txt to ask most major search engines not to index parts of your site Google cover how to create a robots file in some detail and webmaster tools has a useful generator. Example code here

Block a page:-

Disallow: /private_file.html

Block a directory

Disallow: /directory/ 

Solution 2

There are several ways to block your admin. The easiest way would be to use htaccess to send an HTTP header to tell the search engines not to index any of those pages. Put the following code in an htaccess file in your admin directory:

Header: "X-robots-tag: noindex"

This will apply this header to every page in that directory and subdirectory below it and prevent those pages from being indexed.

You can read more about this in Google's Webmaster Blog.

Solution 3

robots.txt doesn't block Google or others from indexing URLs, it simply asks please do not crawl these URL's while you're here http://www.mattcutts.com/blog/robots-txt-remove-url/ if these blocked pages are linked to from other pages on the web Google will still index the page, noindex should be used as well as blocking with robots.txt

Solution 4

Google and other search engines can easily crawl and index your WordPress theme files and scripts. It might reduce your page rank and SEO value. To fix this, create or open a robots.txt file in the root of WordPress and add the following code:

User-agent: *
Disallow:  /wp-admin/
Disallow:  /wp-includes/
Disallow:  /wp-content/themes/
Diasllow:  /wp-content/plugins/

Solution 5

robots.txt is very dangerous. When hackers find your robots.txt, they will see all the forbidden files and directories and will find a way to access them. Use .htaccess on each folder and state the Options -Indexes.

View more solutions

Comments

[thitami]

The fact that your page is 'visible' is only because google cache the information. My friend has a 'under construction' on his page before it was setup. And he said that for 2 months later his page appeared in google as 'under construction', which really annoyed him.

[Krish]

i am on configure i will tell the result about it thanks for your answer

[Andrew Lott]

You're assuming the site is WordPress, which it doesn't sound to be.

[Anthony Hatzopoulos]

This is the best answer. Besides tediously manually removing pages using Google Webmaster Tools. Once it's indexed this will make it removed. For a protected resource I would do X-Robots-Tag: noindex, nofollow. See the dev docs

[Kishan Oza]

what if when i don not have that kind of set up..instead of file i handle admin panel through url. Can i block the url ??

[Abdul Sadik Yalcin]

@AndrewLott this answer can be easily adopted to any other site. Just change the links and place the robots.txt in the root directory.