How to test a 404 page

Request a URL that cannot exist, then read three things: the status line, which must be 404 and not 200; the robots signals on the response; and whether the links on the error page itself resolve. The version under test returned 200, carried no robots directive, and its navigation led back to itself.

Why check this

An error page is tested after a framework upgrade, after a router or CDN rule changes, and after any move to client-side routing, because those are the changes that quietly turn a 404 into a 200. It also belongs on the release checklist of any site that ships a custom error page, since a redesign can drop the status code without touching a pixel.

The failure it catches has three parts and they arrive together. Unknown paths answer 200 with the application shell, so nothing in monitoring fires. The shell has no robots directive, so the URLs are eligible for indexing. The navigation on that shell is built with relative links, so a visitor who lands deep in the tree cannot get out.

Prerequisites

curl 8 or later, and Node 22 for the link resolution in step 4. The curl manual covers -I and -D.
A site you own. One request to a live third-party site to see its error page is fine. Sweeping a site you do not own for 404s is a crawl, and it belongs on your own build.
Save the test site as e404-server.mjs and run node e404-server.mjs. It serves the same error page two ways: under /legacy/ as most sites ship it, and everywhere else as it should be. Stop it when you are done.

import { createServer } from 'node:http';

const html = { 'content-type': 'text/html; charset=utf-8' };
const nav = (rel) => `<nav><a href="${rel ? 'index.html' : '/'}">Home</a>
<a href="${rel ? 'catalogue.html' : '/catalogue'}">Catalogue</a></nav>`;
// rel=true is the version most sites ship: relative links, and no robots directive.
const errorPage = (rel) => `<!doctype html><html lang="en"><meta charset="utf-8">
<title>Page not found</title>${rel ? '' : '<meta name="robots" content="noindex">'}
<link rel="stylesheet" href="${rel ? 'style.css' : '/style.css'}">
${nav(rel)}<h1>We could not find that page</h1>
<form action="${rel ? 'search' : '/search'}"><input name="q"><button>Search</button></form>`;

createServer((req, res) => {
  const path = req.url.split('?')[0];
  if (path === '/' || path === '/index.html') {
    return res.writeHead(200, html).end(`<!doctype html><title>Shop</title>${nav(false)}<h1>Shop</h1>`);
  }
  if (path === '/catalogue') return res.writeHead(200, html).end('<!doctype html><title>Catalogue</title><h1>Catalogue</h1>');
  if (path === '/search') return res.writeHead(200, html).end('<!doctype html><title>Search</title><h1>Search</h1>');
  if (path === '/style.css') return res.writeHead(200, { 'content-type': 'text/css' }).end('body{font-family:sans-serif}');
  // The version under test before the fix: soft 404.
  if (path.startsWith('/legacy/')) return res.writeHead(200, html).end(errorPage(true));
  // The version after the fix.
  return res.writeHead(404, { ...html, 'x-robots-tag': 'noindex' }).end(errorPage(false));
}).listen(8733, () => console.log('404 test site on http://localhost:8733/'));

Steps

1
Step 1.
Start the test site. Stop it with Ctrl-C at the end.
```
node e404-server.mjs
```
```
404 test site on http://localhost:8733/
```
Both http://localhost:8733/legacy/spring-sale and http://localhost:8733/shop/spring-sale render the same error page in a browser. Nothing on screen separates them.

Read the status line for a URL that cannot exist. This is the part a browser hides.

for p in /legacy/spring-sale /shop/spring-sale; do echo "--- $p"; curl -sI "http://localhost:8733$p" | tr -d '\r' | grep -E '^(HTTP|x-robots-tag|content-type)'; done

--- /legacy/spring-sale
HTTP/1.1 200 OK
content-type: text/html; charset=utf-8
--- /shop/spring-sale
HTTP/1.1 404 Not Found
content-type: text/html; charset=utf-8
x-robots-tag: noindex

The first response claims the page exists, and everything downstream believes it: the uptime monitor, the link report and the crawler.

3
Step 3.
Check the indexing directive in the markup, which is the only defence left when the status is wrong.
```
for p in /legacy/spring-sale /shop/spring-sale; do printf '%-22s ' "$p"; curl -s "http://localhost:8733$p" | grep -o -E '<meta name="robots"[^>]*>' || echo '(no robots meta)'; done
```
```
/legacy/spring-sale    (no robots meta)
/shop/spring-sale      <meta name="robots" content="noindex">
```
On the fixed route the directive is redundant, because a 404 response is already a removal signal. On the broken route it is the only signal there could have been, and it is absent. Check both the meta tag and the header, since either one is enough.

Check that the error page is usable from where it was served. Resolve every link and form action against the request URL, fetch each one, and print the heading that comes back.

node -e "
const pages = ['http://localhost:8733/legacy/spring-sale', 'http://localhost:8733/shop/spring-sale'];
const h1 = (b) => (b.match(/<h1>([^<]*)<\/h1>/) || [, '(no h1)'])[1];
(async () => {
  for (const url of pages) {
    const r = await fetch(url);
    console.log(url, '->', r.status);
    for (const m of (await r.text()).matchAll(/(?:href|action)=\"([^\"]+)\"/g)) {
      const ref = new URL(m[1], url).href;
      const t = await fetch(ref, { redirect: 'manual' });
      console.log('   ', String(t.status).padEnd(4), ref.padEnd(46), h1(await t.text()));
    }
  }
})();
"

http://localhost:8733/legacy/spring-sale -> 200
  200  http://localhost:8733/legacy/style.css         We could not find that page
  200  http://localhost:8733/legacy/index.html        We could not find that page
  200  http://localhost:8733/legacy/catalogue.html    We could not find that page
  200  http://localhost:8733/legacy/search            We could not find that page
http://localhost:8733/shop/spring-sale -> 404
  200  http://localhost:8733/style.css                (no h1)
  200  http://localhost:8733/                         Shop
  200  http://localhost:8733/catalogue                Catalogue
  200  http://localhost:8733/search                   Search

Every link on the broken version answers 200, and every one of them serves the error page again. Its relative hrefs resolved into /legacy/, where nothing exists. The stylesheet resolved the same way, so the page is also unstyled. On the fixed version the same four requests reach the shop, the catalogue and the search page.

5
Step 5.
Look at what a live site returns, so the local result has a reference point. One request is enough.
```
curl -s -o /dev/null -D - https://developer.mozilla.org/en-US/docs/Web/HTTP/h2check-does-not-exist | tr -d '\r' | grep -iE '^(HTTP|x-robots-tag|content-type|cache-control)'
```
```
HTTP/2 404
cache-control: no-store
content-type: text/html
```
MDN answers 404 and sends no X-Robots-Tag at all, read on 2026-09-11. That is the normal shape: the status carries the meaning, and the robots header is what you add when the status cannot be fixed.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | HTTP/1.1 404 Not Found | The response says what the page says | Nothing. Keep a test on it. | | 200 OK with a not-found body | A soft 404 | Return 404 from the route. Details in How to check soft 404. | | 404 and no robots directive | Normal, and sufficient | Leave it. Add noindex only while a 200 is still being fixed. | | 200 and no robots directive | The URL is a candidate for indexing | Fix the status first, add the directive as a stopgap. | | Links on the error page returning the error page | Relative hrefs resolved into a path that does not exist | Make every link on the error page root-relative or absolute. | | 301 to the home page instead of 404 | The site hides errors behind a redirect | Serve the error at the requested URL, with 404. |

Common mistakes

Sign: The 404 page looks right in a browser and every automated check passes.Cause: The browser renders the body and says nothing about the status line. A page that answers 200 with an error design looks identical to a real 404 in the viewport. Read the status with curl, or the Status column in the DevTools Network panel, before judging the page.

Sign: The error page works when tested at the site root and breaks in production.Cause: Its links are relative. Served at /legacy/spring-sale, the href index.html resolved to /legacy/index.html, which does not exist, and the stylesheet went the same way. Test the error page from a deep path, never only from the top level.

Sign: A link checker reports no 404s on a site that has plenty.Cause: If unknown routes answer 200, there is no 404 for a checker to find. The count of zero is the symptom. Compare the body of each 200 against a page you know does not exist, as in [How to check for broken links on a website](/check/check-broken-links/).

Sign: Only one made-up URL was tested.Cause: Frameworks route by pattern. An unknown path under an existing prefix, an unknown file extension, and a path with a query string can each be handled by different rules. The two routes above differ only by prefix and behave differently.

What to check next

How to check soft 404: the crawler side of a 200 error page and what it does to indexing.
How to check 410 vs 404 response: which code to return for content removed on purpose.
How to check for broken links on a website: finding the links that lead to the error page in the first place.
How to check x robots tag: reading the header used in step 2, and where else it applies.
How to check if a page is indexable: the full set of signals, of which the status code is one.

FAQ

What is a 404 page?

The page a server returns when the requested URL has no content, together with a 404 status on the response. The status is the part that other software reads. The page is the part a person reads, and it needs working navigation and a search entry point.

How do I check a 404 page returns the right status code?

Request a URL that cannot exist and read the status line: curl -sI https://example.com/no-such-page. It must start with HTTP/1.1 404 or HTTP/2 404. The DevTools Network panel shows the same value in its Status column.

My web server does not return a 404 code. How do I find where it happens?

Request several shapes of missing URL: an unknown path at the root, an unknown path under a real prefix, one with a file extension, and one with a query string. Frameworks match these with different rules, and the 200 usually comes from a catch-all route.

How do I find 404 pages across a whole site?

Collect the links from your own build and request each one, then group the responses by status. Run it against staging, not against a site you do not control.

Verified

Verified 2026-09-11by Maks Vernycurl 8.21.0node 22.23.2

Each output block is what the command above it printed on that date, on the host named in the step. Figures read from a live site move between runs. Compare the shape of the answer rather than the digits, and see the methodology for how a page is re-verified.

All frontend and browser checks checkscluster

basic7 minpublished 2026-09-11updated 2026-09-11Maks Verny