One SEO professional was concerned about a handshake domains setup they have been working with, which was part of their new DNS systems.
They implemented a test on several servers where they would use an HTML file (the same HTML file) on several different servers.
One server took just a few days to crawl. This was very fast compared to the other two. Another server took nearly 20 days before any crawling took place. And yet another server took approximately two to three weeks before it was crawled.
They were curious as to how this could affect how their site is indexed and if there is any way to make this indexing process faster.
John explained that the aspect of fluctuating indexing speeds across different domains is normal and to be expected.
Google tries to understand the overall context of a domain or site, and they try to determine exactly where it would make sense for them to spend their resources in indexing.
If they don’t already know it’s a great website, it may take longer for Google to begin indexing the site.
However, if they already know it’s a great website, then they will crawl and index the site much faster.
He also warned against using alternate DNS systems to set up websites, saying that using alternate DNS systems can interfere with crawling and indexing overall.
This happens at approximately the 46:00 mark in the video.
John Mueller Hangout Transcript
John 46:00
All right. And then I guess a complicated question about handshake domains. So I had to look this up. I think, Damien, you’re here as well. Can you explain a bit more what handshake domains are? And what has happened?
SEO Professional 7 46:25
Sure. Um, I guess the question and what I’m trying to look for is–I guess the best way is, I’ll give you the example of how I went about it. I had three, I guess you’d say of the handshake domain – basically, handshake is set across for it being a decentralized platform, the way the files rest on the platform is scattered, scattered across, let’s just say, for time’s sake, a few FTP servers.
With that known, I put a HTML file for test on three different of these, let’s just say, servers, that being the three different companies. And I noticed that the indexing of that file, the HTML file, varied drastically between, you know, one, one actually, actually indexed probably within two to three to three weeks, another one that was maybe about a few days, actually- fairly quick.
And then the third final test that was probably about, geez, about almost 20 days before it even got any light of search, you know, showing. I know it’s a little tricky question here. And it’s a little out of the realms of normal search. So that’s why I was kind of curious.
John 48:05
Now, so I think the the aspect of indexing across different domains that that’s kind of normal and to be expected, in the sense that we, we try to understand kind of the overall context of a domain or of a website, and try to figure out, like, where does it make sense for us to invest our resources.
And that can mean, if we don’t know much about a domain, maybe it’ll take a little bit longer for us to start crawling and indexing stuff there. Whereas if we already know that this is like a really great website, then we’ll probably crawl and index a lot faster. So that’s, I think, kind of the aspect that you’re seeing there. I think the other part with the handshake domains, I, I took a quick like, like, I don’t know, five minute look at what it was. So from what I can tell, it’s basically kind of like an alternate DNS system, which is kind of based on blockchain technology. And usually, what happens there is, on our side, our systems are essentially tuned to using kind of, I don’t know, the traditional DNS system, which, for the most part works fairly well. So that’s kind of what we would focus on there.
However, we do have, let’s say, an open mind in general, at Google and lots of people are playing around with all of the new technologies around the blockchain as well. So I wouldn’t say that, that will never shift but it’s probably not something that’s going to happen very quickly. And it’s I think, similar to the onion domains, I don’t know, if you remember those, it’s kind of like also an alternate DNS system.
And that’s something where I think in the end, we never index the onion domains directly. But if over time, we recognize that this is something that is very popular that people are using this actually to host their content and make it accessible then, right, I could work and I could understand that our systems would be like, well, we have to figure out a way to deal with this and to index this content as well. Because if people are searching for information, and that information is only findable through this way, then maybe we have to find a way to make that accessible.
So from that point of view, if you need something short term to work well in search, I would use the traditional DNS setup. If you kind of want to kind of grow this, I don’t know alternate setup, then I think that’s worthwhile trying out and kind of, like keeping as more like I’d say, like a side project or something. To kind of, I know, try things out. And it could also be that there’s some kind of a middle ground there in that, since these are also still IP addresses that are hosted on the internet, normally, maybe there’s a way for kind of a gateway setup to help out there. So that we can still kind of crawl one set of URLs that lead to the final content.
