Your Guide to Creating and Editing XML Sitemaps

Your Guide to Creating and Editing XML Sitemaps For SEO

In this article we’ll cover the various types of sitemaps that exist, and when to use those.

We’ll also dive into exactly where you can find sitemaps, and how they can be created or edited using various tools.

We explain best practices for your sitemap and exactly how you can submit that for crawling to the likes of Google and Bing.

Below there is a thorough explanation of each of these points, so keep reading to learn more.

What is a Sitemap and How Do Search Engines Use Sitemaps?

So what exactly is a sitemap? In its simplest form, it is a list of pages on a website.

Essentially a sitemap can tell search engines which URLs are in place on a site and how they can be reached. They can also contain additional information such as when pages were last modified.

Think of a sitemap as your website’s blueprint if you like. Its purpose is to help search engines discover and crawl your pages efficiently, often leading to these URLs being indexed in search results.

If you have pages on your site which are important to you – but aren’t linked to internally – a sitemap can help to ensure these pages are discovered by search engines.

This is why a sitemap can be crucial to your SEO efforts.

This can be especially important for large websites, or sites that add new pages regularly. However, we’ll come onto the full benefits of having a sitemap in place very shortly.

It’s also worth noting that a sitemap is a file, rather than a traditional page on a website.

Google’s John Mueller confirmed this on X when asked if there was an issue with an XML sitemap featuring a noindex instruction. He also explained that there’s no real “nofollow” for sitemaps:

John also confirmed that sitemap files are just data files, which means they aren’t processed in the same way that normal pages would be.

The Various Types of Website Sitemaps

There are primarily two main types of sitemaps that you can have in place on a site.

The first is an XML sitemap which is created for search engine crawlers such as Google and Bing. These are the main types of sitemap used for SEO. Each individual sitemap can contain up to 50,000 URLs. with a size limit of 50MB. However, a site can have multiple XML sitemaps, which we’ll come onto shortly.

The second is an HTML sitemap which is designed to help users navigate through your website, with the appearance of a regular page.

Though, this isn’t a substitute for strong navigation elements on the site. You should still feature solid menu links within the header and footer of the site.

Again, John Mueller expressed those thoughts clearly on Mastodon:

This basically means users shouldn’t have to rely on an HTML sitemap to navigate the key pages on your site.

One top tip if you do have an HTML sitemap in place on your site, is to review in Analytics how many clicks this page receives. If the number is high then it could be an indication that your site navigation is somewhat lacking for real users.

Going back to the SEO-focused XML sitemap, there are variations which can be utilised by websites.

There is a News Sitemap which can be used by Google News-approved websites. Helping the likes of Google find breaking news content quickly.

This is something The New York Times has taken to another level with their “Today” sitemap which features their latest content. This was discussed on Linkedin:

As Chris goes on to point out:

This implementation from the New York Times is incredibly smart. It ensure (sic) that their most recent content is never more than “3 clicks” away from any page on the site.

This goes to show just how important sitemaps can be for news sites.

There is also an Image Sitemap which is used to help search engines discover and index images on a website. This can be especially useful for stock photo or eCommerce-type sites but is often not necessary on a standard content website. This is because those images are usually discovered as part of the page URL within the sitemap.

Additionally, there is a Video Sitemap which is used by search engines to find video content on a website. Once again this is mainly for sites that feature videos as a critical component of their content. Videos embedded within a page would normally be discovered as part of the standard XML sitemap.

There is also a Mobile Sitemap, but this is just for mobile phone-only pages.

So what exactly does an XML sitemap look like?

Here’s an example from our site Detailed.com which was generated by Yoast SEO:

As you can see there are three sitemaps included in the file. One for posts, one for pages and another for categories.

This is a fairly basic sitemap which allows search engines to discover content in various areas, and the last modified date of those locations.

Though, in honesty, a sitemap doesn’t often need to be more complex than this.

Do You Need an XML Sitemap for SEO?

Perhaps the question could be ‘should you have a sitemap’, rather than do you need one?

If the pages on your site are properly linked then search engine crawlers can most likely discover them. However, sitemaps are perfect for helping search engines easily find all of those key pages on your website. Especially if any new content doesn’t yet have links pointing to that page.

In Google’s own words:

A sitemap helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed. However, in most cases, your site will benefit from having a sitemap.

There is also the issue of helping search engines understand when pages were last updated, in which case XML sitemaps can be incredibly helpful for SEO.

We’d strongly suggest that most websites should have a sitemap in place. Especially if the site is particularly large or shares news.

How Can You Find a Sitemap on a Website?

There are a number of ways in which you can locate the existing sitemap of a website.

Visit the Sitemap URL

The easiest way to search for a sitemap on a website is to search for https://domain.com/sitemap.xml.

This will often bring up the sitemap of that website in a browser.

On occasions, especially if WordPress sites are using the likes of Yoast SEO plugin, the sitemap page will then redirect to https://domain.com/sitemap_index.xml.

As an example, if you were to search for https://detailed.com/sitemap.xml you would be redirected to https://detailed.com/sitemap_index.xml.

You can then view the individual sitemap files by clicking on the links within the sitemap:

In our case clicking on the post sitemap would show the posts contained within that sitemap.

Robots.txt File

Another way is to check the robots.txt file of a site, as the sitemap URL is often displayed here.

If we take the detailed.com robots.txt file as an example, you can see the sitemap URL is displayed here:

This is the case with many other robots.txt files also.

If you find that your sitemap exists but is not included in the robots.txt file, then we strongly suggest that it does appear here.

Google also advises the following:

Insert the following line anywhere in your robots.txt file, specifying the path to your sitemap. We will find it the next time we crawl your robots.txt file:

Sitemap: https://example.com/my_sitemap.xml

This is incredibly easy to do, so something that’s definitely worth checking is in place.

Using Google Search Console

Sitemaps are often found within Google Search Console, so if you have access to the page you can check there.

Simply click on the Sitemaps tab in the left sidebar:

You can then see a list of the submitted sitemaps for that website.

Using Sitemap Search Terms

Sitemaps can also be found using search operators within a web browser.

Two variations that may work to locate the sitemap page of a site are:

site:domain.com inurl:sitemap
and
site:domain.com filetype:xml

Replacing domain.com with the URL of the actual website you’re searching for.

If after exhausting all of the above methods, you’re still unable to locate the sitemap of a site, it’s likely that the website doesn’t actually have an XML sitemap in place. In which case, you may wish to create one.

Using the Detailed SEO Extension

A shameless plug perhaps, but the free Detailed SEO Extension makes finding the XML sitemap of a website incredibly easy.

When viewing a site, click on the browser extension and simply scroll down to “Sitemap.xml”:

Clicking on this link will open the sitemap of the site in your browser. It really is that easy.

XML Sitemap Best Practices

How To Create A Sitemap

When deciding what pages to include within your sitemap, a great rule of thumb is would you want people to land on these pages and would you like them indexed in search results? If so, then those URLs should absolutely appear in your sitemap.

One way of creating a sitemap for your site is by using the default sitemap generator in your CMS such as WordPress or Wix. This is a very viable option for many sites and these can be created with ease.

Each CMS tends to have its own guide on how to do so which is linked to from their dashboard.

Another popular option is the Yoast SEO plugin. The XML sitemap generator is available in both the free and premium versions.

You simply select Yoast SEO within the WordPress dashboard, click on Settings and then Enable the XML sitemaps feature.

Using the Yoast SEO plugin is an incredibly easy and straightforward way of creating a sitemap. Through the WordPress dashboard, you’re able to determine exactly what pages you would like to include within the sitemap.

Yoast SEO can also support splitting large XML sitemaps into smaller sections of up to 1,000 URLs. You can even then connect your sitemap to Google Search Console once completed.

If you have already created a sitemap within your CMS, then Yoast SEO will override this with any new sitemap you create.

An alternative way to create a sitemap for your site is via XML-Sitemaps.com, which is a free tool you can use to generate the sitemap and then submit it to Search Console (explained below).

Additionally, there is also another WordPress plugin named XML Sitemap Generator for Google which has over 1 million installations and does exactly the same thing.

If you have a custom-made website, then you may need support from your site developers to help you create this. They will be able to provide you with a lot more clear and concise information on how to write this code for your site.

Whichever option you choose to create your sitemap, the same method can be used going forward to update your sitemap if required.

Tip, check your sitemap to ensure you’re not including noindexed pages within your file.

Only pages you wish to be crawled should feature here, and as search engines shouldn’t be showing these pages in search results, there’s no need for them to appear within your sitemap.

Alternatively, if you do wish to index these pages going forward, you can remove the noindex directive and add those to your sitemap in future.

In relation to this, let’s touch on other pages you can choose to exclude.

Which Pages to Exclude From Your XML Sitemaps

You can optimise the way search engines crawl your site and discover key pages by excluding non-SEO relevant pages from your sitemap.

As well as pages that are noindexed, there are a number of pages you should therefore consider removing from your sitemap.

Here’s a list to make that easier to digest:

• Thin content pages
• Duplicate content pages
• Internal site search results (unless you believe they will generate relevant search traffic)
• Pagination pages
• Filtering URLs that you deem not relevant to your SEO aims
• 404 (page not found) pages
• 5XX (server error) pages
• Login pages
• Redirected pages
• Non-canonical pages
• Comment URLs
• Share via email URLs
• Tracking ID pages

The requirements for every site will be unique, but those are some of the key page types that are often excluded, especially by large websites.

Excluding the above, where relevant for you, can help the likes of Google to focus on indexing the pages that really matter to your site.

Submitting Your Sitemap to Google and Bing

To help Google find your XML sitemap, you can add this to your Search Console account.

As shown in the previous image, simply click on the Sitemaps tab in the left sidebar and add your sitemap URL to the Add a new sitemap section and then click submit.

You can then view the status of your sitemap and Google will highlight if there are any issues or errors. The number of discovered URLs is also displayed as well as the last read date of the sitemap.

Clicking on the sitemap will show whether or not that has been processed successfully:

You can also click on the See Page Indexing option to show the number of indexed URLs. This can be incredibly useful for finding issues, as if the number of submitted pages is significantly more than the number of indexed pages, this could suggest a problem with indexation.

This report can then show any not-indexed pages, as well as other issues such as 404 pages and redirects appearing in the sitemap:

Clicking on any of these options will expand upon the information shown.

If you do have any issues when submitting your sitemap, and get messages such as “has errors” or “couldn’t fetch”, then clicking on these will bring up a clear explanation of what the issue was as well as how to fix it.

The process is very similar within Bing Webmaster tools. Simply click on Sitemaps from the left-hand menu, and Submit sitemap.

Bing will then show you information such as the success status of your sitemap as well as when it was last crawled.

If there are any problems with the sitemap submission these will be highlighted with explanations of the issue.

In Summary, Are Sitemaps Important for SEO?

An effective XML sitemap is absolutely crucial to support your site’s SEO efforts.

In order for search engines such as Google and Bing to find the content that you wish to rank, you should make it easy for them to find these pages. Having an XML sitemap in place helps search engines discover and crawl your pages efficiently.

Creating (or editing) a sitemap and excluding non-relevant pages from that file – then submitting to search engines – is certainly worth taking the time to do.