An SEO professional asked John Mueller about what they should be looking for in the Google Search Console Crawl Stats report.
They are trying to understand if there is an issue on the technical side with Google crawling their site.
So their main question is: What are some of the signals, or things, to identify in Google Search Console’s crawl stats report, that may indicate that Google is struggling to crawl something?
John explained that the crawl stats report will not be useful for the SEO pro, simply because with the crawl stats report, you are looking at an aggregate view of data representing the crawling of your site.
Usually, for most cases, this makes more sense. If you have something like a few 100,000 pages, then you can examine that and say “Oh, well on average the crawling is slow.”
Whereas if you have a website that has, I don’t know, maybe around 100 pages or so, then essentially, even if crawling is really, really slow, Google can still get them once a day, or in the worst case scenario maybe once a week.
It’s not going to be a technical issue regarding crawling. It’s more a matter of understanding that the website actually offers something unique and valuable that Google needs to have indexed.
So it’s much less of an issue on the crawling side and more about the indexing side of things.
The exception here would be if there’s really a big technical issue with your site. But this is something that you would see right away, because you would probably check these URLs individually and find that Google can’t crawl them at all.
Perhaps there’s an error that’s returned, or there’s a noindex that’s returned. This will be very obvious.
John’s assumption here, especially for a smaller website, is you have to convince Google at first that it should try to crawl and index your site.
This happens at approximately the 25:40 mark in the video.