An SEO professional was concerned about their content showing up crawled but not indexed in Google Search Console.
Recently, they have been facing issues for some of their client websites, where they are writing and posting content to their client’s blog for around the past four months.
But, most of the content is not getting indexed by Google.
When they inspect the URL, they get two messages. The first message is crawled but not indexed yet.
The next message is that the URL is discovered, but not indexed.
So his question is: if a URL is crawled, but not indexed, does this mean that Google thinks the content is not perfect for indexing?
John answered that he doesn’t think it means anything in particular. He believes this is always an easy early assumption to make- that Google crawled it, but decided not to index it.
Most of the time when Google crawls something, it doesn’t necessarily mean that they will automatically index it.
John said he would treat the two categories of ‘crawled but not indexed’ as a similar thing.
It’s tricky, however, because Google doesn’t index everything. This can happen as well.
But, if they believe that their site is a good site, and Google should be indexing this more, John would be happy to take any example URLs to the search team.
This happens at approximately the 21:06 mark in the video.
John Mueller Hangout Transcript
SEO Professional 5 21:06
I have a question about Google crawling and indexing. So recently, we are facing some issues for some of our client websites. So we are writing content and posting content on their blog. What happened in the last four months, we are posting content on the website.
But most of the content is not getting indexed by Google. And when we inspect the URL, we usually get two messages. One is the URL is crawled, but not indexed yet, or the URL is discovered, but not indexed yet.
So my first question is about if a URL is crawled, but not indexed, does it mean Google thinks this content is not perfect for indexing? Or what does it mean?
John 21:58
I don’t think it means anything in particular. So I think that’s always kind of an easy early assumption to say, oh, Google looked at it, but decided not to index it.
Most of the time, like when we still crawl something, it doesn’t necessarily mean that we will automatically index it. So I would almost treat those two categories of ‘not indexed’ as a similar thing. And it’s tricky, because we don’t index everything.
And so that can happen. If you think this is a good website, where we should be indexing more, I would love to have some example URLs so that we can pass that on to the team. If you want to drop them in the chat, that would be super useful. Okay.
SEO Professional 5 22:53
And the next one is, in this scenario, it’s not only for one client, we are facing this for multiple clients, with five or six clients at the moment.
So is there any other way, like we are trying to inspect the URL, then requesting for indexing, we are trying to use Google Search Console.
Is there any other way to make it faster, or is there any other thing maybe that will help for getting indexed?
John 23:20
I mean, there are, there are different things, which perhaps you’re already doing. On the one hand, kind of making sure that it’s easy for us to recognize the important content on a website is really good. Which sometimes means making less content and making better content.
So having fewer pages that you want to have indexed. The other thing is internal linking is very important for us to understand what you would consider to be important on a website. So things for example, that are linked from the homepage are usually a sign that, like you care about these pages, so maybe we should care about them more.
Things with external links, they also kind of go into that category of where we see other people think that these pages are important, then maybe we will see them as important too.
And then sitemaps and RSS feeds, from a technical point of view, also help us a little bit better to understand, like, these pages are new, or they have changed recently, we should check them out again and see what is there.
But all of these things kind of come together. And it’s something where it’s rarely that there’s like one trick that you’re missing to get these pages indexed.
SEO Professional 5 24:43
And the last question, so what happens when we see that those articles are not getting indexed?
So we told our client to share this article on their social media accounts like Facebook or Twitter, Instagram, and those actually help us to draw some links from some other website to those blog posts. Now, we are using those blog posts for interlinking as well to other pages.
So getting linked to those blog posts, which are not indexed by Google, will have any impact on the website ranking at any value from SEO point of view?
John 25:17
I mean, if we find External links to those pages, then chances are we might crawl and index that page a little bit higher, I guess. It depends a little bit on what kind of external links of course. There are links from social media directly, usually have nofollow attached so we don’t really forward any signals there.
And if it’s something where we can recognize, well, these are maybe problematic links or not that useful links, maybe we will ignore those too. But obviously, if we can tell that something is seen as being important, we’ll probably go off and crawl and index that page, more likely.
What you generally won’t see is that we will kind of forward value to the rest of the website, if we don’t actually index that page.
Because if we decide not to index your page, then it’s still that situation that “Well, we don’t have a destination for those links.” So we can’t do anything with those links for the rest of the website.
