If you want to make sure that your website is being crawled and indexed by search engines as effectively as possible, then you need to create and audit XML sitemaps regularly. Adding an XML Sitemap not only makes crawling easier, it could potentially make indexing easier as well.
An XML sitemap is a list of the pages on your website that you want search engines to crawl and index. It’s a simple text file that contains information about each page on your site, such as when it was last updated and how often it changes.
- Easier crawling and indexing
- Helps identify broken links
- Shows which pages need the most attention
- Can help improve site architecture
You can create an XML sitemap manually, but there are also many automatic XML sitemap generators available. Once you’ve generated your sitemap, you need to upload it to your website and submit it to search engines.
If you are not keen on doing all of these steps, you may want to check out our Ultimate SEO Audit Template. This SEO audit template is your free guide to performing your own SEO audit.
Which Websites Should Include an XML Sitemap?
Google’s XML Sitemap developer’s guide explains that sitemaps are necessary for the following sites:
You might need a sitemap if:
- Your site is really large. As a result, it’s more likely Google web crawlers might overlook crawling some of your new or recently updated pages.
- Your site has a large archive of content pages that are isolated or not well linked to each other. If your site pages don’t naturally reference each other, you can list them in a sitemap to ensure that Google doesn’t overlook some of your pages.
- Your site is new and has few external links to it. Googlebot and other web crawlers crawl the web by following links from one page to another. As a result, Google might not discover your pages if no other sites link to them.
- Your site has a lot of rich media content (video, images) or is shown in Google News. If provided, Google can take additional information from sitemaps into account for search, where appropriate.
You might not need a sitemap if:
- Your site is “small.” By small, we mean about 500 pages or fewer on your site. (Only pages that you think need to be in search results count toward this total.)
- Your site is comprehensively linked internally. This means that Google can find all the important pages on your site by following links starting from the homepage.
- You don’t have many media files (video, image) or news pages that you want to show in search results. Sitemaps can help Google find and understand video and image files, or news articles, on your site. If you don’t need these results to appear in image, video, or news results, you might not need a sitemap.
A Step-by-step Guide to Auditing Your XML Sitemap
Here’s a step-by-step guide on how to audit your XML sitemaps:
1. Check for errors. The first thing you need to do is check your sitemap for any errors. This can be done using a tool like Xenu’s Link Sleuth. Simply enter your sitemap URL and hit “Start.” Xenu will then crawl your sitemap and report any errors it finds.
2. Check for broken links. Once you’ve fixed any errors, you need to check for broken links. This can be done using a tool like Screaming Frog SEO Spider. Just enter your sitemap URL and hit “Start.” Screaming Frog will then crawl your sitemap and report any broken links it finds.
3. Check the pages on your sitemap. Once you’ve fixed all the broken links, you need to check the actual pages on your sitemap. This can be done using a tool like Google PageSpeed Insights. Simply enter the URL of each page on your sitemap and hit “Analyze.” Google will then give you a report on how fast each page loads and what you can do to improve its speed.
4. Check for duplicate pages. The next thing you need to do is check for duplicate pages on your sitemap. This can be done using a tool like Copyscape. Just enter the URL of each page on your sitemap and hit “Search.” Copyscape will then search the web for any duplicates of that page.
5. Check the traffic to your sitemap pages. The last thing you need to do is check the traffic to your sitemap pages. This can be done using a tool like Google Analytics. Simply create a new report and add your sitemap pages as the primary dimension. Then check the report to see how much traffic each page is getting.
Types of Errors You May Encounter in Your Sitemaps
When you audit your sitemaps, you may come across a few different types of errors. Here are some of the most common errors you may encounter:
- Broken links: These are links that lead to pages that don’t exist.
- Mixed content: This is when a page contains both secure (https) and non-secure (http) content.
- Redirects: These occur when one page URL redirects to another page URL.
- 404 errors: These occur when a page can’t be found.
How to Fix Sitemap Errors
Once you’ve found an error in your sitemap, you need to fix it as soon as possible. Here are some tips on how to fix common sitemap errors:
Broken links: If you find a broken link, you need to either remove it from your sitemap or update it to point to the correct page.
Mixed content: If you find mixed content on a page, you need to either remove the non-secure content or update the page to use https.
Redirects: If you find a redirect, you need to either update the redirect or remove it from your sitemap.
404 errors: If you find a 404 error, you need to either update the page or remove it from your sitemap.
Additionally, there are some less common sitemap errors, and these should also be looked into:
Syntax errors with the sitemap code: If you find issues with syntax errors in the sitemap coding, this needs to be fixed and re-validated. Things like:
Initial opening sitemap code doesn’t reference the correct version, along with the correct character encoding.
This is an example line:
If another type of character encoding is required, then this will need to be repaired as well. But, for most situations you shouldn’t need this.
How Often Should You Audit Your Sitemaps?
You should audit your sitemaps at least once a month. However, if you make a lot of changes to your website, you may need to audit your sitemaps more often.
Here are some signs that you may need to audit your sitemaps more often:
- You’re not getting the traffic you want
- You’re losing rankings in search engines
- Your pages are loading slowly
- You’re getting a lot of 404 errors
If you notice any of these signs, then you should audit your sitemaps as soon as possible.
Attributes of URLs That Should Be in Your Sitemap
Any healthy and accurate sitemap file should have the following as part of it. Here are a few XML sitemap do’s and don’ts:
Any URLs should have a final 200 OK status code when Google arrives on it.
Any URLs should have canonical versions of that URL that self-canonical.
Make sure that your sitemap code is properly formatted and validated.
Make sure that all URLs are properly crawlable and indexable.
You also want to remove any duplicate URLs from your sitemap.
Don’t exceed the 50,000 URL sitemap limit.
The XML sitemap should also not exceed a 50 MB file size.
There shouldn’t be any 301 redirected URLs in the sitemap file.
There shouldn’t be any 404 errors in the sitemap file.
No social media URLs should be in the sitemap file.
The Correct XML Sitemap Can Make or Break Your Website
If you want to make sure your website is running smoothly, you need to audit your XML sitemaps regularly.
We have outlined the steps you need to take to audit your sitemaps and fix any errors. We’ve also provided a guide on how often you should audit your sitemaps and what attributes of URLs should be included in them. Finally, we reviewed the more common – and not-so-common – XML Sitemap errors that you may encounter.
By following these tips, you can ensure that your website is running efficiently and error-free.