An SEO professional wondered about problems they were having with Google AdsBots.
They want to block the ads URL from Googlebot in the robots.txt file. They were curious whether this would impact their Google ads, and if this would cause ads not to work normally.
John explained that the AdsBot does not follow the usual robots.txt directives. You must use the user-agent directly for Google AdsBot.
However, he does not know what the exact impact would be on the ads side if you block ads. His understanding is that Google uses this primarily as a quality check for the landing page.
The SEO professional then asked about the fact that they found that Googlebot crawled their ads URLs more so than their normal URLs. The thing they want Googlebot to do is promote their product pages more, instead of the ads.
In order to do this, they want to block ads URLs from Googlebot using robots.txt.
John explained that that is perfectly fine. If you block the ads landing pages from Googlebot, that is just fine. However, he does not know the impact if they were to block the ads pages from Google AdsBot.
The SEO professional explained that this is not something they do. Their next question was whether Google AdsBot shared the same crawl budget with Googlebot.
John replied that yes, Google AdsBot does share the same crawl budget with Googlebot.
It’s possible that crawl requests from Google AdsBot will increase and Googlebot can crawl less. John also said that it settles down fast.
This caused the SEO professional to express their concern that even if they’re blocking the ads URLs in Googlebot, Googlebot may still crawl too many ads URLs.
John then said that if they’re blocking ads landing pages for Googlebot, then Google should never ever be crawling the ads landing pages with Googlebot. It would only be the AdsBot that is actually crawling it.
This happens at approximately the 7:08 mark in the video.
Looking for a new way to improve your SEO audits? Our Ultimate SEO Audit Template could be right up your alley!