The Latest In Google Search for Site Owners

John 0:05
Hi, I’m John Mueller, a search advocate from Google Switzerland. Today, we’re going to share the latest on Google Search focusing on new APIs to better understand your site in Google search and ways that you can diagnose indexing of embedded videos for search. This session is recommended for developers of websites wanting to make the most out of the modern opportunities in Google Search.

Before we start, let’s take a step back. Google search is the reason we’re here together in this session. Did you know that over 15% of all queries submitted every day on Google search are completely new? We’ve been tracking this metric for many years now. And it’s surprising to even those involved that it’s been so stable.

My takeaway is that every day people are looking for something new and unique. Every day, there are chances for new websites to be found, it’s not too late for you to get started. However, if you’re already working on websites, that also means that even if your website is doing well now, it’s still worth monitoring how users find your site and looking at ways to make your website even better. And the opportunity is not just limited to new searches.

Year over year, Google has sent more and more traffic to websites on the open web. You can be a part of this too, helping to make websites that are easily findable, and that get traffic through Google search. So let’s dive right in. Google Search Console is the primary tool from us for understanding how a site is doing in search. It’s available to all owners of websites. It includes reports on the site’s status, its performance, and provides settings and diagnostics tools. There’s a lot already available on Search Console, so we’ll focus on just a small part, there are links to more information in the description. The most important part is verifying ownership of a website. Without this, it’s not possible to use Search Console. We recommend verifying on the domain name level, as it gives you the most options.

If you’re making a website for others to use, make it as easy as possible for them to verify their websites. There’s an API for that. If you’re using an existing content management system, or building on a hosted platform, this may already be implemented. It can take a couple of days for data to start appearing after verification. This is why we recommend it as an early first step. Are you verified? All right, let’s check out the new APIs. The first API on our list is the Search Analytics API.

This maps to the performance report in Search Console and gives you information on how your pages performed in search. New in this API is support for Google Discover, and Google News. This is interesting for developers, since you can find creative ways to display and add value on top of the data. For example, while Search Console interface shows only 1000 rows, you could potentially show more or add a layer of advanced analysis and data visualization.

As an aside, note that Google Analytics and Search Analytics in Search Console are different things. Roughly speaking Search Console and its APIs cover everything in Google Search leading up to a user going to a website. Google Analytics, on the other hand, looks at everything that happens afterwards once a user is on the website. The measurements are slightly different too. So while trends are often the same, the exact numbers won’t match.

With the Search Analytics API, you can request information about clicks, impressions, ranking position and search queries for the whole website or for parts of it. The data is anonymized according to Google’s high internal standards and can be safely stored on your end. This allows you to create useful dashboards within your websites for the content creators. These dashboards help them to understand the current situation and to work out ways to improve. If you track the data over time, you can also collect insights over stable and periodic trends. How might that look in practice?

Google Site Kit is an open source plugin for WordPress that integrates the Search Analytics API among others. At a glance in the owners dashboard, it can show how traffic from Search has developed over time and use that to give you tips on how to improve. Site Kit is now being used on over 2 million websites. Proof that people find the integration of data in their CMS useful. If you’re using WordPress, I’d recommend giving it a shot. If you’re working on a CMS for others, I’d recommend trying it out for inspiration. It’s open source, so you can also check out the code.

We’re already working with a few platforms to build similar tools for their users with the same APIs as Site Kit, and we’re looking for even more CMS partners to join. Please get in touch with us if you’re interested. For example, EC|Cube, an ecommerce platform from Japan, has made use of the Search Analytics API to create a custom plugin for their users. The report shows clicks and impressions, plus average ranking for top keywords and pages. The plugin now comes pre-installed for new EC|Cube based sites.

If you’re looking for further inspiration of how to make use of the search APIs, we recommend taking a look at Search Console Insights. This is an easy to use content analytics tool tailored for smaller creators. It provides an intuitive UI with insights such as what are the most common search queries leading to the site.

This helps creators to track their content’s performance and inspire them with ideas for how they can improve. And, something users seem to appreciate a lot, it motivates by celebrating achievements and progress. The metrics are easy to fetch from the API, so you too can add motivational milestones to your site. Shifting gears, let’s move on to the other new API, the URL Inspection API. With this new API, developers can check the indexing data for a specific URL on their website.

This API maps to the URL Inspection Tool in Search Console, which is shown here. The API response includes, among other things, the current indexing status, any structured data detected and its mobile usability status. The index status includes whether the URL is known to Google systems, if it’s been crawled, indexed, and what the canonical URL is. For example, a page can have a declared preferred canonical URL, but sometimes Google won’t choose it.

Through the API, you can test both URLs and quickly check the difference. This makes it possible to regularly check a critical sample of your site’s pages for their availability in search. Keep in mind that Google doesn’t guarantee indexing of all pages.

That said, it’s easier to check for potential technical issues when you know that some pages have dropped from indexing. We’ve seen amazing integrations of this API in the brief time it has been live so far. Here’s one example: The Screaming Frog SEO crawler is a tool that runs locally and can crawl your website locally to analyze it, giving you insight into how pages are currently linked. This tool has integrated the URL inspection API to make it easy to check a sample of your pages for their indexing status, allowing you to export it to a spreadsheet for further analysis.

You could use the API in a similar way for manual checks or implement it for automatic periodic checks to catch issues early on. Another neat implementation is the lost Google impressions index check from Dave Smart. This tool combines the search analytics tool with the URL inspection API. In short, it looks for pages that have seen a drop in impressions in Search Console, and then checks those with the URL inspection API to display their status.

This is a neat way to combine the two APIs. If you’re building a website, you could combine the APIs in a similar way to automatically inform the operator of potential trouble before it becomes a lasting problem. And now over to Dikla for a video segment on videos in Search Console.

Dikla Cohen 9:28
Thanks, John. Hi, I’m Dikla Cohen, a web ecosystem consultant from Google London. There is no arguing that video is an important content format on the web. It is therefore not surprising that more and more users appreciate finding links to videos as answers in search. With that, many websites want to highlight not only their textual content, but also their videos in the Search Results. People often wonder how videos are indexed and appear on search. Google discovers videos on the web just like any other web page.

The first step is indexing the landing page where the video appears, then, Google needs to recognize that there’s a video prominently located on the page.

Finally, Google needs to be able to extract key information about the video, like thumbnail, title, and other metadata like duration. Sites can provide this easily through structured data or other methods. Worth noting that video indexing does not rely on where and how you host your videos. You can host them yourself, or use most of the popular video hosting platforms and services.

To make video indexing easier to track and diagnose, we will soon be launching a new set of reports and tools in Search Console. Let’s take a quick look. The video page indexing report will show a summary of all the indexed pages our systems found with a video on them while crawling and indexing your website.

At a glance, you’ll be able to see in how many video landing pages a video was indexed or not indexed. Recognized reasons are grouped with a trendline and counts of the affected videos. For example, which videos are missing a thumbnail URL.

Reasons with the largest impact are ranked on top. Click on one of the rows to get more details such as affected video pages URLs. You will be able to download the list of examples and double check them. With this, you could narrow down the details and more easily understand what is happening and how to resolve issues.

After updating the website, when you’re fairly confident that the issue is resolved, you can go back to the video indexing report. Go to that specific issue type and use the Validate fix button to inform Google. This will initiate the recrawling of known URLs affected by the issue. And as the pages are reprocessed, you’ll be notified if everything is resolved, or if there are remaining issues. Note that it can take some time to go through all the URLs. You can also check the video indexing status, which John mentioned earlier.

To get there, just enter the URL of the video landing page into the top of Search Console. This will first show you the current status of that URL, as well as the video indexing status of the page. It will inform you whether we detect a video on the page, and if so, did we manage to index it. If a detected video was not indexed, it will list the reasons preventing video from being indexed. Please note that Google indexes only one video per page, even if there are multiple videos on the page.

To summarize, with the new tools, you will be able to see how many video landing pages Google discovered and in how many of them a video was indexed. Examine reasons for unindexed videos in video landing pages. Use the list of affected video pages URLs to debug and fix issues. Validate fix to initiate recrawling of known URLs affected by the issue. And check the video indexing status of a specific video using the URL inspection tool. I hope this helps streamline the publishing of videos on your site bringing you more users through search.

Dikla Cohen 13:47
All right, one last update. Last year, we mentioned the page experience ranking change. The update included combining a set of signals that measure how users experience a page. We call them core web vitals, and they measure speed, interactivity, and stability. There are other talks this I/O that discuss core vitals in more detail. And we recommend checking them out.

For the page experience update, we started off with mobile. And just recently, we rolled out the ranking change on desktop search as well. We’ve been following along how sites have improved with these metrics. And you can head to the page experience report on Search Console to better understand how your site is doing.

We’re excited to share that in Q4 2021 alone, we saved our users over 1000 years of time waiting for pages to load due to the page experience program. That’s an amazing result. And a reminder that the web is both extremely popular, and that there’s still a lot to achieve in making it truly fantastic.

We’re looking forward to working together with you all to continue to move the web forward. And that’s it for the updates from Google search. For more information about any of these topics, check out the search central developer documentation. If there’s anything on your mind that you’d like to find out more about, check out the Google Search Central help community. In the forums there, you can meet like-minded folks who can often help you by answering questions around websites and Google search.

And if you’d like to stay in the loop with us, make sure to subscribe to the Google Search central YouTube channel. Thanks for joining us here. Bye.