How to validate sitemap xml

Save the file, then run a rules check over it: python sitemap-lint.py sitemap.xml https://host/sitemap.xml. An XML parser settles well-formedness and nothing else. The failures that cost you are a relative URL, a URL on another host, more than 50,000 entries and more than 50 MB uncompressed, and all four parse cleanly.

Why check this

Run this on staging sign-off and after any release that changes routing, the host name, or the generator that writes the file. The defect it catches is a sitemap that every tool accepts and no crawler acts on: a build that emits /products/widget/ instead of the absolute URL, or one that moves assets to a CDN host and keeps listing them in the site's own sitemap.

The two failure classes look identical in an editor. One is a well-formedness error, which any parser reports with a line and a column. The other is a protocol rule that lives in prose on sitemaps.org and only partly in the published schema, so a file can be perfect XML and be ignored entry by entry.

Prerequisites

• Python 3.8 or later. xml.etree.ElementTree and gzip are in the standard library, so there is nothing to install.
• The sitemaps.org protocol for the rules, and sitemap.xsd for the content model the prose does not fully cover.
• The URL a crawler reaches the file through: the Sitemap: line in robots.txt, or the index that names it. Read it with How to check robots.txt.
• Save this as sitemap-lint.py. It is a rules check against the protocol, not an XSD validation: it parses with a standard parser, then applies the limits the schema cannot express.

# python sitemap-lint.py <sitemap.xml|.xml.gz> <URL the crawler was pointed at>
import sys, gzip, xml.etree.ElementTree as ET
from urllib.parse import urlsplit

NS = "{http://www.sitemaps.org/schemas/sitemap/0.9}"
path, pointed_at = sys.argv[1], sys.argv[2]
raw = gzip.open(path, "rb").read() if path.endswith(".gz") else open(path, "rb").read()
print("uncompressed bytes:", len(raw))
try:
    root = ET.fromstring(raw)
except ET.ParseError as e:
    sys.exit("FATAL not well-formed XML: %s" % e)
print("root element:", root.tag)
if root.tag != NS + "urlset":
    sys.exit("FATAL root is not %surlset" % NS)
base = urlsplit(pointed_at)
scope = base.path.rsplit("/", 1)[0] + "/"
locs = [(e.findtext(NS + "loc") or "").strip() for e in root.findall(NS + "url")]
print("<url> entries: %d, scope: %s%s" % (len(locs), base.netloc, scope))
err, out_of_scope = [], 0
for i, loc in enumerate(locs, 1):
    p = urlsplit(loc)
    if p.scheme not in ("http", "https") or not p.netloc:
        err.append("url %d: loc is not an absolute URL: %r" % (i, loc))
    elif p.netloc != base.netloc:
        err.append("url %d: host %s is not %s" % (i, p.netloc, base.netloc))
    elif not p.path.startswith(scope):
        out_of_scope += 1
    if len(loc) > 2048:
        err.append("url %d: loc is %d characters, the limit is 2048" % (i, len(loc)))
if len(locs) > 50000:
    err.append("file: %d URLs, the limit is 50000" % len(locs))
if len(raw) > 52428800:
    err.append("file: %d bytes uncompressed, the limit is 52428800" % len(raw))
for e in err[:20]:
    print(e)
print("errors: %d, outside the scope of %s: %d" % (len(err), scope, out_of_scope))

Steps

1. 1

   Step 1.

   Fetch the file once, keep the headers, and write the body to disk.

   curl -sS -D mdn-headers.txt -o sitemap.xml https://developer.mozilla.org/sitemap.xml
   grep -iE '^HTTP|^content-type|^content-length|^last-modified' mdn-headers.txt
   

   HTTP/2 200 
   content-type: application/xml
   last-modified: Fri, 11 Sep 2026 01:34:47 GMT
   content-length: 1282

   application/xml and text/xml are both accepted. A text/html answer here is a catch-all route, not a sitemap.

2. 2

   Step 2.

   Run the rules check against a real sitemap, naming the URL the crawler was pointed at.

   python sitemap-lint.py mdn-en-us.xml.gz https://developer.mozilla.org/sitemap.xml
   

   uncompressed bytes: 1928405
   root element: {http://www.sitemaps.org/schemas/sitemap/0.9}urlset
   <url> entries: 14759, scope: developer.mozilla.org/
   errors: 0, outside the scope of /: 0

   Read it line by line. The size is measured after decompression, because the 50 MB limit is on uncompressed bytes: this file is 125,812 bytes on the wire and 1,928,405 after gzip. The root element carries the namespace, so a file declaring no xmlns fails on that line instead. Zero errors and zero out of scope is the pass.

3. 3

   Step 3.

   Build a file that is valid XML and breaks three protocol rules, then check it.

   python sitemap-lint.py bad-sitemap.xml https://shop.example.com/sitemap.xml
   

   uncompressed bytes: 2349
   root element: {http://www.sitemaps.org/schemas/sitemap/0.9}urlset
   <url> entries: 4, scope: shop.example.com/
   url 2: loc is not an absolute URL: '/products/widget/'
   url 3: host cdn.example.net is not shop.example.com
   url 4: loc is 2054 characters, the limit is 2048
   errors: 3, outside the scope of /: 0

   The four entries were the site root, a relative path, an image on a CDN host, and a search URL padded past the length limit.

4. 4

   Step 4.

   Hand the same file to a general-purpose XML parser and watch it pass.

   [xml]$a = Get-Content bad-sitemap.xml -Raw; "parsed OK, url count " + $a.urlset.url.Count
   

   parsed OK, url count 4

   This is how a broken sitemap reaches production. Three rule violations, and the parser reports a clean document with four URLs in it.

5. 5

   Step 5.

   Put one unescaped ampersand in a <loc> and run both tools again.

   python sitemap-lint.py amp-sitemap.xml https://shop.example.com/sitemap.xml
   

   uncompressed bytes: 187
   FATAL not well-formed XML: not well-formed (invalid token): line 3, column 56

   The value was https://shop.example.com/list?sort=price&dir=asc. System.Xml refuses the same file: '=' is an unexpected token. The expected token is ';'. Line 3, position 57. Both point at the same character, one counting from zero and one from one. The fix is &, and it applies to every query string a generator writes out.

6. 6

   Step 6.

   Generate 50,001 entries and confirm the count rule fires.

   python sitemap-lint.py big-sitemap.xml https://shop.example.com/sitemap.xml
   

   uncompressed bytes: 2789059
   root element: {http://www.sitemaps.org/schemas/sitemap/0.9}urlset
   <url> entries: 50001, scope: shop.example.com/
   file: 50001 URLs, the limit is 50000
   errors: 1, outside the scope of /: 0

   One entry over the limit, and the whole file is out of specification. Split it and put an index in front, which is the next page.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | errors: 0 and outside the scope of /: 0 | The file satisfies the protocol rules | Move on to the lastmod values | | FATAL not well-formed XML | A parse error, usually a bare & or < in a URL | Escape the character in the generator, not in the output file | | loc is not an absolute URL | A relative path where a full URL is required | Prefix scheme and host in the generator | | host X is not Y | Entries for a host the file does not cover | Move them to a sitemap on that host | | FATAL root is not ...urlset | Missing or wrong xmlns, or an index file | Check the namespace string, then read the index page | | 50001 URLs or over 52428800 bytes | Past a hard limit of the protocol | Split the file and add a sitemap index |

Common mistakes

Thresholds

What to check next

• How to check sitemap index file: what to do once one file passes 50,000 URLs, and the two ways an index is built wrong.
• How to check sitemap lastmod: the one element in the file with a date format of its own, and the values that parse but do not validate.
• How to check sitemap and robots txt: how a crawler finds the file, which decides the scope the check above uses.
• How to check if a page is indexable: a listed URL still needs to be crawlable and indexable on its own.
• How to check robots.txt: the Sitemap: line that names the discovery URL.

FAQ

How to find the sitemap.xml path on a website?

Read the Sitemap: line in /robots.txt first, because that is the declared location and there can be several. /sitemap.xml and /sitemap_index.xml are conventions, not requirements. A file reachable at neither is still valid when it is submitted directly to a search engine.

How to validate a compliant XML sitemap?

Two passes. A parser for well-formedness, which reports a line and a column, then a rules pass for the absolute URL, host, count and size limits, which no parser checks. The script above does both and prints the two classes separately.

Should a sitemap have every URL on the site?

No. It should hold the URLs you want crawled and nothing else. Listing a page that is noindex, canonicalised elsewhere, or blocked in robots.txt sends contradictory instructions. A sitemap is a list of candidates, not a site inventory.

How to test a sitemap without a public URL?

Point the script at the generated file in the build output and pass the URL it will be served from. That runs in CI, before deploy, and catches the relative and cross-host cases early.

Verified