How long for Google to index pages after submit to index? How to check the progress?

Question

After a revamp of our website, I have submitted an updated sitemap then fetched the homepage URL / as Google and submitted it and its direct links for indexing in Google Search Console more than 2 weeks ago. Since then I haven't seen a change at all in total number of indexed pages. The pages crawled per day has been consistent throughout this time but nothing to indicate the whole site has been re-crawled for indexing (I would've expected to see a spike here).

If I go to Crawl -> Sitemaps, I can see some pages have been marked as indexed but it's only a very small percentage of total number of submitted URLs and the figures haven't changed in over two weeks. There are no sitemap errors or index errors.

Does anyone know how long it takes before I should see some movement? And am I looking in the right place to see it, i.e. the crawl stats and total number of indexed pages in Google Search Console? How do I know if Google has crawled the pages and decided not to index them versus not crawled them at all?

Solution

There is no specified timeframe in which Google, or any search engine, will start to show more of your URL's in the SERPs. (That said, Bing is usually easier to convince.) The initial crawl alone will take up to several days. The problem is that the algorithms may not always deem all of your sitemap pages worthy of being listed.

You can try resubmitting your XML sitemap a few times. With one client, I got the site fully indexed only after three sitemap submissions. That said, in asking Google to reevaluate your site, it can potentially work against you too, if it decides that your content isn't as good as it initially thought.

Google won't index pages it sees as thin or unoriginal. If you have several pages where the header and footer are the main content areas, and the body of the content is very thin, this sort of page will be viewed as low quality or duplicate content.

You can take the following steps to troubleshoot:

• Is your content good enough to index? If not, flesh out those pages.
• Are your title tags up to SEO standards? Do you have any meta tags, such as noindex or nofollow, on any of those pages that would prevent indexation?
• Are you blocking any sitemap URL's in your robots.txt?
• Is most of your content generated by JavaScript?
• Are any of your pages really slow, or not mobile friendly?

After making some changes, resubmit the sitemap and see what happens.

You can look for specific URL's using combined search operators, such as: site:example.com inurl:sample-page. Here's a great resource for that type of fun: https://moz.com/blog/mastering-google-search-operators-in-67-steps

In your GSC, you can see, in your sitemap area, how many pages were submitted vs how many were indexed. That means, whatever wasn't indexed, was ruled out by the search engine. When you run your search queries using the operators, you may see way more pages than what Google said was indexed. That means you should go deep into the SERPs and check for duplicate content, indexed taxonomy nodes, anything that you can take action on.

If you see no improvement, wait a few weeks or months. Try to build more backlinks from reputable sources. Try to get your content shared on social media. Publish more content, if possible. Convince Google that the sitemap pages you're submitting are worthy of living in the index. How long that takes depends on your marketing strategy.

Comments

[closetnoc]

Just because you changed a bunch of stuff on your site does not mean Google will suddenly say to itself, Huh. Perhaps I should see what this is all about. Google will explore your site about as fast as it has before. So how long will it take? How the hell should I know? ;-) Check your metrics and see how fast Google visited your site in the past. Google may see enough changes and speed up a little. But do not expect too much. Cheers!!

[Mel]

thanks for your comment @closetnoc .. i assumed that by submitting it for indexing that it is like requesting google to revisit the site and acknowledge the changes..google themselves say this should happen almost instantly but i haven't seen any evidence of this unless i'm looking in the wrong place?

[closetnoc]

And my tongue in cheek answer is... And you believed them? Sometimes they do speed up a bit. It is all a matter of scale, popularity, trends, and so forth. However, most of the time Google will continue on the same scale as before with bursts of What the hell else I got to do? Cheers Mate!!

[Mel]

Thanks for the detailed answer @Henry! Looks like I'll need to go back to the start and check each point off individually and hopefully things will turn around. Appreciate the advice.

[Trebor]

Thanks Henry. I've wondered how long it took for Google's indexing to trickle down into the SERPs. I don't treat indexing and SERP positioning as simultaneous. While some content seems to place high up in the SERPs immediately due to current events, other content may take weeks to months before it moves up in the SERPs.

[Henry Visotski]

That's the correct approach, @Trebor. Indexed content can also be very low ranking content. (Like many of those Contact Us pages.)