Imagine you built a library. Thousands of books. Carefully organised. Lovingly written. And then when the librarian showed up, you handed them zero catalogue, zero floor plan, and just said: “It’s all in there somewhere. Good luck.”
That is a website without an XML sitemap.
Google’s crawlers are good at finding pages — but they’re not psychic. They follow links. If a page has no links pointing to it, if it’s buried five clicks deep in your architecture, or if it was published yesterday and hasn’t been linked from anywhere yet — Google may not find it for weeks.
A sitemap solves this. You’re handing Google the catalogue.
Table of Contents
What Is an XML Sitemap?
An XML sitemap is a structured file — written in XML format — that lists the URLs you want Google to know about on your website.
It looks like this in its simplest form:
xml<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://yourwebsite.com/</loc>
<lastmod>2026-04-01</lastmod>
</url>
<url>
<loc>https://yourwebsite.com/technical-seo/</loc>
<lastmod>2026-03-28</lastmod>
</url>
</urlset>
Each <url> entry tells Google: this page exists, this is its address, and this is when it was last updated.
You don’t write this by hand (please don’t write this by hand). Your CMS generates it automatically. WordPress does it. Shopify does it. Webflow does it. Most modern platforms do it. Your job is to make sure it exists, it’s accurate, and Google knows where to find it.
Why It Matters for SEO
A common misconception: “Google finds everything anyway, so sitemaps don’t matter.”
Google does find a lot. But “a lot” is not “everything,” and “eventually” is not “quickly.” Here’s where sitemaps make a measurable difference:
New websites — fresh domains have very few external links pointing to them. Googlebot has no natural path in. Without a sitemap, it may find your homepage and little else for weeks.
Deep pages — a page five or six clicks away from your homepage may not be discovered by following internal links alone. A sitemap puts it directly on Google’s radar.
Frequently updated content — news sites, blogs, and e-commerce stores with constantly changing inventory benefit enormously from sitemaps. The lastmod field tells Google when a page was updated, prompting a recrawl. Without it, Google decides recrawl frequency on its own — which is slower.
Large sites — on a 50,000-page e-commerce site, sitemaps aren’t optional. They’re essential. Google needs a structured map to know what exists.
The sitemap doesn’t guarantee indexing. Google still decides what it wants to index based on quality and authority. But it removes the “Google simply didn’t know this existed” problem from the equation.
What Should (and Shouldn’t) Be In Your Sitemap
This is where most sites get it wrong. People assume a sitemap should contain every URL on the site. It should not.
Include:
- All pages you want indexed and ranked — blog posts, service pages, product pages, landing pages
- Your most important category and archive pages
- Any new content you want Google to discover quickly
Do not include:
- Pages with
noindextags — if you’ve told Google not to index a page, don’t also tell Google it’s important via your sitemap. That’s contradictory and confusing. - 301 redirect URLs — include only the final destination URL
- 404 pages — listing broken pages in your sitemap tells Google “this is important” and then immediately disappoints it
- Duplicate content URLs — if you have canonical tags managing duplicates, only include the canonical URL
- Admin pages, login pages, cart and checkout pages, search results pages — nothing that has zero standalone search value
A clean sitemap that lists 200 high-quality URLs is more effective than a bloated sitemap listing 2,000 URLs that include your login page, your privacy policy in four language variants, your checkout confirmation page, and every filtered product URL you’ve ever generated.
Sitemap Types You Should Know About
Standard XML sitemap — the default. Lists all your indexable URLs. Lives at /sitemap.xml or wherever your CMS puts it.
Sitemap index — for large sites with more than 50,000 URLs (the per-sitemap limit). A sitemap index is a parent file that lists multiple smaller sitemaps, each covering a section of the site. You submit the index to Google Search Console and it handles the rest.
Image sitemap — specifically for image URLs. Tells Google about images that may not be discoverable through regular page crawls. Useful if your images are a significant traffic source.
Video sitemap — same concept, for video content. Includes metadata like title, description, duration, and thumbnail.
News sitemap — for Google News-approved publishers. Contains only articles published within the last two days and is subject to strict formatting requirements.
For most websites, a single standard XML sitemap — or a sitemap index if you’re large — is all you need.
How to Create and Submit Your Sitemap
Step 1: Generate the sitemap
The Fast Way: Subu’s XML Sitemap Generator
I’m building a free XML Sitemap Generator tool — because “ask your developer to check if the sitemap exists” is advice that leads to a lot of awkward email chains.
The tool will let you generate a clean, correctly formatted XML sitemap for your website without touching a single line of code. No CMS dependency. No plugin required. Just your URLs in, a valid sitemap out, ready to submit directly to Google Search Console.
(Tool coming soon — I’ll drop the link here the moment it’s live. If you’re reading this and the link still isn’t here, check the Tools section or leave a comment below and I’ll send it to you directly.)
In the meantime, here’s how to get your sitemap sorted right now:
- WordPress: Yoast SEO, Rank Math, and All in One SEO all generate sitemaps automatically. It’s usually at
yoursite.com/sitemap.xmloryoursite.com/sitemap_index.xml - Shopify: built-in at
yourshopify.com/sitemap.xml - Webflow: auto-generated, found in site settings
- Custom or static sites: use a tool like Screaming Frog (can export a sitemap after crawling) or an online generator
Step 2: Add it to your robots.txt
At the bottom of your robots.txt file, add:
textSitemap: https://yourwebsite.com/sitemap.xml
This ensures any crawler that visits your site immediately knows where the sitemap lives, even if you never submitted it manually.
Step 3: Submit to Google Search Console
- Open Search Console → Sitemaps (left sidebar)
- Enter your sitemap URL
- Click Submit
- Done
Google will show you how many URLs were submitted versus how many were actually indexed. That gap — submitted vs indexed — is useful diagnostic information.
Sitemap Mistakes That Make Subu Sigh Heavily
The Never-Updated Sitemap — auto-generated in 2022, never touched again. Missing 300 pages published since then. Google’s map of your site is two years out of date.
The Noindex-Included Sitemap — pages explicitly tagged with noindex appearing in the sitemap. You’re sending Google a conflicting instruction set. It generates Sitemap warnings in Search Console and forces Google to choose which directive to follow.
The Redirect Graveyard — 400 URLs in the sitemap that all 301 redirect to other URLs. You’re making Google do extra work for every single entry before it reaches the real destination.
The Never-Submitted Sitemap — the sitemap was generated by the CMS and has been sitting quietly at /sitemap.xml for three years. Nobody ever opened Search Console to submit it. Google may have found it. Or may not.
The Missing Sitemap on a 10,000-Page Site — no sitemap at all on a large e-commerce store, relying entirely on internal links for Google to discover product pages. Pages added three months ago that Google still hasn’t crawled. Revenue invisibly leaking.
The Everything-Included Sitemap — 8,000 URLs including every parameter variation, every filtered search result, every admin URL that somehow got crawled once. Completely diluted signal. Google spends crawl budget visiting junk URLs from the sitemap instead of your actual content.
How to Check If Your Sitemap Is Working
Google Search Console → Sitemaps tab
This is your primary dashboard. It shows:
- Whether your sitemap was successfully fetched
- How many URLs were submitted
- How many of those were indexed
- Any errors Google encountered when processing it
A submitted count of 500 with an indexed count of 80 is a conversation worth having. Either you have 420 pages that aren’t worthy of indexing, or you have 420 pages with quality issues Google is quietly declining to rank.
Screaming Frog sitemap analysis
Screaming Frog can crawl your sitemap and flag issues: pages returning 404s, pages returning 301 redirects, pages tagged noindex that are still listed. Run this quarterly or after any significant site changes.
The TL;DR
- An XML sitemap is a structured list of URLs you want Google to know about — it’s your website’s catalogue for crawlers
- Sitemaps matter most for new sites, large sites, frequently updated sites, and deep pages with limited internal links
- Only include indexable, canonical, live URLs — no noindex pages, no redirects, no 404s, no checkout pages
- Generate via your CMS (Yoast, Rank Math, Shopify), add the URL to robots.txt, and submit in Google Search Console
- Monitor the submitted vs indexed gap in Search Console — it tells you a lot about your content quality and crawl health
- Regenerate and resubmit after major site restructures, migrations, or content additions
Your sitemap is not a set-it-and-forget-it file. It’s a live document that should reflect exactly what you want Google to prioritise at any given moment.
Questions about why your indexed page count is much lower than your submitted count? Drop it in the comments. That gap usually tells a story.
— Subu, SEO by Subu
