Sometimes, errors are introduced when Google processes site elements on their side. One SEO professional was concerned about a lot of errors in Google Search Console that began in mid-July.
They found that there was a noindexed error in Google Search Console even though none of the URLs were marked as noindexed.
When they were crawled and showing up in GSC, they were presenting with this error. On subsequent crawls, they would get indexed.
John Mueller’s hunch, without having looked at the pages, was to double-check if there was really a problem with indexing.
Maybe it’s more of a problem on the rendering side.
He explained that Google uses the Chromium flavor of Google Chrome in order to process site elements on their side. If it has to load many page elements such as CSS files and JS files in order to render the page, it’s possible this is producing the intermittent indexing issues that are being described here.
This discourse occurs at approximately the 5:59 mark in the video.
John Mueller Hangout Transcript
SEO Professional 3 5:59
There! Can you hear me? Yes, good, disconnecting from work and connecting from home is different. Anyway, back in mid-July, we started noticing a lot of errors in Search Console submitted but noindexed. The URLs themselves do not have a noindex on them. But on the subsequent crawl, they get indexed.
The problem is, is that, you know, we get 300 errors, no index, and then on subsequent crawls, only five get crawled before they recrawl, you know, so many more. So given that they are noindexed, and granted, things can’t render or they can’t find a page, they’re directed to our page not found, which does have a noindex.
And so I know somehow, they’re getting directed there. Is this just a memory issue? Or? Like, since they’re a sub get subsequently called fine? Is it just a…?
John 7:00
It’s hard to say, without looking at the pages. So I would really try to double-check if this was a problem then and is it not a problem anymore. Or if it’s still something that kind of intermittently happens. Because if it doesn’t matter, if it doesn’t kind of take place now any more than like…
SEO Professional 3 7:21
No, it does still take place now like it just crawled on 10/4 and then on 10/5 it recrawled again, and the ones that got recrawled were indexed fine. So I don’t know if it’s caching it somehow or if it was just a memory issue? I’m not sure.
John 7:37
Yeah, my hunch, without knowing your site, is that something with the rendering is sometimes going wrong, and it’s reaching that error page that you mentioned. And that’s, if that’s something that still takes place, I would try to figure out what might be causing that.
And it might be that, like, when you test the page in Search Console, nine times out of ten, it works well. But kind of that one time out of ten, when it doesn’t work well and redirects to the error page, or we think it redirects to the error page, that’s kind of the case I will try to drill down into and try to figure out is, is it that, there are too many requests to render this page, or there’s something complicated with the JavaScript that sometimes takes too long and sometimes works well.
And then try to try to narrow things down from that point of view.
SEO Professional 3 8:35
Okay, okay. So could it be that if there’s too much volume in the pages… so many resources were blocked, and so then it looks like page not found, and then when there’s less traffic on the website, the resources are able to be rendered?
John 8:50
Yeah, could be and I mean, what happens on our side is we crawl the HTML page. And then we try to process the HTML page in the Chromium kind of Chrome-type browser. And for that, we try to pull in all of the resources that are mentioned there.
So if you go to the developer console in Chrome, and you look at the network section, it shows you a waterfall diagram of everything that it loads to render the page. And if, if there’s lots of things that need to be loaded, then it can happen that things time out, and then we might run into the error situation. So that’s kind of the direction I would go.
And if you can’t isolate exactly what is going wrong, then I would try to take the number of requests there and just see if there are ways that you can minimize that. Maybe the developer team can combine the different JavaScript files or combine the CSS files, minimize the images, or things like that.
