We’ve all experienced it: a site with hundreds of pages, navigation that reveals only a fraction of the content, and no quick way to find out what’s hidden in the corners. The sitemap, an XML file located at the root of the domain, provides a list of the URLs the site owner wants to make visible.
But a sitemap is not a complete map of the site. It doesn’t guarantee that every listed page will be crawled or indexed. To get something useful out of it, you need to read it methodically, compare it with other data, and avoid reading more into it than it actually says.
XML sitemap: what it shows and what it hides
When you open a sitemap file, you’ll find a raw list of URLs enclosed in XML tags. Each entry may include a last-modified date (lastmod), a change frequency (changefreq), and a priority (priority). On paper, these three fields help you sort pages. In practice, two of them are useless for organizing a crawl.
Google says it ignores the priority and changefreq values. These declarative fields don’t reflect how the site actually behaves. A webmaster can assign a priority of 1.0 to every page without making any difference to crawling. To classify URLs, we rely on observable criteria: page type (category, product page, article), modification date, or indexability status.
The lastmod tag can be useful, provided it’s reliable. It should reflect a significant content update, not simply the regeneration of the file or a change to the footer. When the date is consistent and verifiable, it makes it easy to identify recently updated pages on a large site.
To see what a sitemap looks like in practice, the Série Live sitemap is a good example of this structure: it organizes URLs by content type and includes their modification dates.

Compare a sitemap with a crawl to find orphan pages
The real value of a sitemap becomes clear when you compare it with a site crawl. A crawl, performed with a tool such as Screaming Frog, Sitebulb, or an online crawler, navigates the site by following internal links. It reconstructs the structure as a search engine would discover it. The sitemap, on the other hand, lists the URLs the site owner wants to expose.
Comparing these two datasets reveals three types of discrepancies that would otherwise go unnoticed.
- URLs included in the sitemap but missing from the crawl: these may be orphan pages, accessible only through the sitemap, with no internal links pointing to them. A search engine will have trouble finding them through navigation alone.
- Sitemap URLs that return a 404 error, a 301 redirect, or a 5xx status code: these are outdated URLs that clutter the file and waste crawl budget.
- URLs found in the crawl but missing from the sitemap: pages exposed through navigation that the webmaster didn’t consider worth listing, or simply forgot. Depending on their nature, they may need to be added or deindexed.
You can make this comparison in a spreadsheet. Export the sitemap URL list, export the crawl report, then join the two datasets by URL. Rows with no match in either column flag discrepancies that need investigating.
Sitemap and navigation: finding buried sections of a website
Beyond orphan pages, a sitemap helps identify sections of a site that are difficult to discover through navigation. On an e-commerce site with thousands of product pages, some categories may be four or five clicks away from the homepage. A crawl measures this depth. The sitemap, by contrast, puts every URL on the same level.
Sort the sitemap URLs by directory (the path in the URL: /blog/, /products/, /help/) to get a section-by-section view of the site. You can then check, for each section, how many URLs appear in the first levels of the crawl and how many are buried deeper.
A practical example of a site diagnosis
Consider a site whose sitemap contains 800 URLs across /articles/, /guides/, and /tools/. The crawl shows that the 200 pages under /tools/ are accessible only through a single link in the footer. As a result, these pages have a high crawl depth and receive little internal link equity. The sitemap alone wouldn’t have been enough to make this diagnosis, but it helped reveal that the section existed and warranted further investigation.
The sitemap is the map; the crawl is the terrain. Either one on its own gives you an incomplete picture.

Finding a site’s sitemap: methods that work
When exploring a third-party site, the first step is to locate its sitemap. The most direct method is to try the /sitemap.xml URL at the root of the domain. Most CMS platforms (WordPress, Shopify, Webflow) generate a sitemap at this location by default.
If this URL returns nothing, check the site’s robots.txt file, available at /robots.txt. This file often contains a “Sitemap:” directive followed by the full URL of the file. This is the most reliable method when the sitemap isn’t at the standard location.
When the sitemap is an index
On large sites, the sitemap.xml file is often a sitemap index: it doesn’t list the site’s URLs directly, but points to several topic-specific sitemaps (sitemap-posts.xml, sitemap-pages.xml, sitemap-products.xml). Each subfile contains a batch of URLs. For a complete crawl, download and merge all the sitemaps before comparing them with the crawl.
Some sites don’t provide a sitemap at all. In that case, crawling is the only way to reconstruct the site’s structure. A sitemap makes the job easier, but it isn’t a prerequisite for exploring a site.
A sitemap doesn’t replace a technical audit, a crawl, or an analysis of internal linking. Its real value becomes clear when you move beyond its passive role as a file submitted to Google and use it as a comparison tool. Orphan pages, broken URLs, poorly linked sections: all of these can be detected by comparing the sitemap with what the site actually shows its visitors and search engines.



