Is there some kind of API for submitting a site to Google for indexing?

Question

I have a client that is insisting there is a method to send a URL automatically to Google to get it indexed quickly.

Here are the methods I know of to tell Google about your site for indexing: 1. Link to it from another site already in the Google index 2. Submit it here https://www.google.com/webmasters/tools/submit-url?pli=1 3. Use Webmaster Tools to fetch as Googlebot

Does anyone know of such an API or method that would allow you programmatically submit a URL to Google for indexing?

Solution 1

If you sign up an verify the site with Google Webmaster Tools you can submit a sitemap. It's a simple XML format and pretty easy to generate each time your site is updated, this allows you to quickly submit large numbers of URLs to Google for indexing (though bear in mind that adding a URL to a sitemap does not mean Google will index it, rank it or speed up it's submission - there are no guarantees!).

Example:-

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
      <loc>http://www.example.com/</loc>
      <lastmod>2005-01-01</lastmod>
      <changefreq>monthly</changefreq>
      <priority>0.8</priority>
   </url>
</urlset> 

This is about as close as they come to such an API, my guess is your client has heard of this and is giving you the Chinese whispers version of the explanation they received.

That said blogging software like Wordpress does 'ping' Google Blog Search and other sites with new posts - see http://blogsearch.google.com/ping / http://blogsearch.google.com / 2012 Ping List, this service does appear to list blog postings faster than the standard Googlebot.

Blog Search did have an API for displaying results on websites but this was depreciated in 2011 - your client could also have heard about this as well and perhaps is conflating the faster listing of blog postings with the listing API thus the mangled explanation you've received.

Solution 2

If Google is your only concern then simply use Google Analytics. One click on a new URL/page and Google will add the page to the crawl list. This is the fastest way to let Google know about new content.

My pages are crawled quickly (within hours) and besides the internal links it is about the only method I use. There is basically no other way Google knows about such a new page.

Update
Actually, I just checked. For a new page today it took GoogleBot only 16 minutes to show up in my logs.

Solution 3

I love when customers know better then the people they hired to do the job.

Anyway, the ways you have cited are the nornal ways to get indexed. As a registrar Google may potentially find sites through registrations but that's about it. Additionally, there is no direct way to make Google crawl or index your content faster. You can control the speed relative to its current speed in Webmaster Tools, but that is, once again, relative to the current crawl rate.

And, most importantly, there is no API for submitting to Google that will get a page indexed faster. It's a myth. So have fun telling your customer, again, that they're incorrect.

Comments

[m_pGladiator]

Care to try that method again with NO internal links to your new page? I'm just trying to eliminate cause/correlation confusion.

[mogoh]

@Mike Google Analytics simply picks up the "current URL" not any internal links. If you have hidden pages and want them to be hidden you'll have to remove Analytics on those pages. Took me a while to realize how in the world some hidden pages showed up in SERP. The cause is Analytics.

[Guy Thomas]

There are numerous free tools to generate xml sitemaps automatically.