How to check soft 404
A soft 404 is a page whose body says the thing is missing while the status line says 200. No status check can see it. Ask for an identifier that cannot exist, then compare the answer with a page that does exist: node soft404.mjs http://host/docs widget-42 prints the status, the final URL and the byte count of each.
Why check this
Run this after any change to routing, to a data source, or to an error template, and whenever a monitor calls a section healthy while support calls it broken.
The failure it prevents is a missing page that reports success. A catalogue drops a product, the route still matches, the template renders with no data, and the server answers 200 with an empty panel. Every status-based test passes, the link checker passes, and the page is indexed as real content.
Three shapes turn up in practice: a 200 carrying error text, a redirect to the home page, and a 200 carrying the site chrome around an empty content area. This procedure builds all three, then reads them the only way that works: comparing a known-missing URL against a known-good one.
Prerequisites
- Node 18 or later. The detector uses
fetch, shipped since Node 18. - curl 7.0 or later for step 5, which repeats the verdict with a different toolchain.
- A free port.
netstat -ano | grep 9137prints nothing when 9137 is free.
Save this as soft404-server.js. Route a answers a miss correctly. Routes b to e are the four ways of getting it wrong.
// node soft404-server.js listens on http://127.0.0.1:9137
// Five routes that answer a missing page in five different ways.
const http = require('http');
const shell = (title, main) => `<!doctype html>
<html lang="en"><head><meta charset="utf-8"><title>${title}</title></head>
<body><header><a href="/">Docs</a></header>
<main>${main}</main>
<footer><p>Example docs, build 2026-09-11</p></footer></body></html>
`;
const found = (id) => shell(`Widget ${id}`, `<h1>Widget ${id}</h1><p>The ${id} widget takes one argument.</p>`);
const missing = shell('Not found', '<h1>Not found</h1><p>No page with this address.</p>');
http.createServer((req, res) => {
const html = (status, body) => {
res.writeHead(status, { 'content-type': 'text/html; charset=utf-8' });
res.end(body);
};
if (req.url === '/') return html(200, shell('Docs home', '<h1>Docs home</h1><p>Start here.</p>'));
const m = /^\/([a-e])\/([A-Za-z0-9-]+)$/.exec(req.url);
if (!m) return html(404, missing);
const [, route, id] = m;
if (id === 'widget-42') return html(200, found(id));
if (route === 'a') return html(404, missing); // correct
if (route === 'b') return html(200, missing); // soft: error text, 200
if (route === 'c') { res.writeHead(302, { location: '/' }); return res.end(); } // soft: home
if (route === 'd') return html(200, shell('Docs', '')); // soft: empty content area
return html(200, shell('Not found', `<h1>Not found</h1><p>We could not find ${id}.</p>`)); // soft: echoes the id
}).listen(9137, '127.0.0.1', () => console.log('listening on 127.0.0.1:9137'));
Save this as soft404.mjs. It asks for one id that exists and two that cannot.
// node soft404.mjs <base-url> <id-that-exists>
// Compares one page that exists against two ids that cannot exist.
const [base, good] = process.argv.slice(2);
const nonce = () => 'zz' + Math.random().toString(16).slice(2, 10);
const strip = (body, id) => body.split(id).join('');
async function get(id) {
const url = `${base}/${id}`;
const res = await fetch(url); // follows redirects, like a crawler
const body = await res.text();
return { id, url, final: res.url, status: res.status, bytes: body.length,
echoed: body.includes(id), body };
}
const rows = [await get(good), await get(nonce()), await get(nonce())];
for (const [i, r] of rows.entries()) {
const label = i === 0 ? 'exists ' : 'impossible';
console.log(`${label.padEnd(10)} ${r.status} ${String(r.bytes).padStart(4)} bytes ` +
`id-echoed=${r.echoed ? 'yes' : 'no '} ${r.final === r.url ? 'no redirect' : '-> ' + r.final}`);
}
const [g, a, b] = rows;
const verdict =
a.status >= 400 ? `HARD ${a.status}. The status states the page is missing.` :
a.final !== a.url ? `SOFT 404. A missing id answered 200 at ${a.final}, not at the URL asked for.` :
strip(a.body, a.id) === strip(b.body, b.id) && a.bytes !== g.bytes
? 'SOFT 404. Two ids that cannot exist returned the same page once their id is removed.' :
'INCONCLUSIVE. Widen the comparison.';
console.log(`VERDICT ${base} ${verdict}`);
Start it and keep the Windows PID, which step 6 needs.
node soft404-server.js & netstat -ano | grep 9137
Steps
- Step 1.
Ask every route for an id that does not exist and read the status alone, the way a monitor does.
for r in a b c d e; do curl -sS -o /dev/null -w "%{http_code} %{size_download} bytes %{url_effective}\n" "http://127.0.0.1:9137/$r/widget-999" done404 270 bytes http://127.0.0.1:9137/a/widget-999 200 270 bytes http://127.0.0.1:9137/b/widget-999 302 0 bytes http://127.0.0.1:9137/c/widget-999 200 214 bytes http://127.0.0.1:9137/d/widget-999 200 273 bytes http://127.0.0.1:9137/e/widget-999Five requests for a page that cannot exist, and only the first line says so.
- Step 2.
Repeat it with redirects followed, which is what a crawler and a browser do.
for r in a b c d e; do curl -sSL -o /dev/null -w "%{http_code} %{size_download} bytes %{url_effective}\n" "http://127.0.0.1:9137/$r/widget-999" done404 270 bytes http://127.0.0.1:9137/a/widget-999 200 270 bytes http://127.0.0.1:9137/b/widget-999 200 255 bytes http://127.0.0.1:9137/ 200 214 bytes http://127.0.0.1:9137/d/widget-999 200 273 bytes http://127.0.0.1:9137/e/widget-999Line 3 became a 200 and the URL changed. A missing page redirected to the home page is the third shape, and
%{url_effective}is the field that exposes it. - Step 3.
Read a page that does exist, on the same route, for the number to compare against.
curl -sSL -o /dev/null -w "%{http_code} %{size_download} bytes %{url_effective}\n" "http://127.0.0.1:9137/b/widget-42"200 298 bytes http://127.0.0.1:9137/b/widget-42298 bytes against 270. A 28-byte gap, which is why a threshold on body length alone is not a detector. Route
areturns those same 270 bytes under a 404. - Step 4.
Run the detector over each route. It asks for the id that exists, then for two ids that cannot.
for r in a b c d e; do node soft404.mjs "http://127.0.0.1:9137/$r" widget-42; echo; doneexists 200 298 bytes id-echoed=yes no redirect impossible 404 270 bytes id-echoed=no no redirect impossible 404 270 bytes id-echoed=no no redirect VERDICT http://127.0.0.1:9137/a HARD 404. The status states the page is missing. exists 200 298 bytes id-echoed=yes no redirect impossible 200 270 bytes id-echoed=no no redirect impossible 200 270 bytes id-echoed=no no redirect VERDICT http://127.0.0.1:9137/b SOFT 404. Two ids that cannot exist returned the same page once their id is removed. exists 200 298 bytes id-echoed=yes no redirect impossible 200 255 bytes id-echoed=no -> http://127.0.0.1:9137/ impossible 200 255 bytes id-echoed=no -> http://127.0.0.1:9137/ VERDICT http://127.0.0.1:9137/c SOFT 404. A missing id answered 200 at http://127.0.0.1:9137/, not at the URL asked for. exists 200 298 bytes id-echoed=yes no redirect impossible 200 214 bytes id-echoed=no no redirect impossible 200 214 bytes id-echoed=no no redirect VERDICT http://127.0.0.1:9137/d SOFT 404. Two ids that cannot exist returned the same page once their id is removed. exists 200 298 bytes id-echoed=yes no redirect impossible 200 273 bytes id-echoed=yes no redirect impossible 200 273 bytes id-echoed=yes no redirect VERDICT http://127.0.0.1:9137/e SOFT 404. Two ids that cannot exist returned the same page once their id is removed.Read the last block first. Route
eanswersid-echoed=yesfor two ids invented during the run, so the rule "the body mentions what I asked for, therefore the page is real" passes a miss. The rule that holds: strip each id from its own response, and if two impossible ids leave identical bytes, nothing on the page came from data. - Step 5.
Repeat the surprising verdict with different tools, so it is not a property of one script.
curl -sS "http://127.0.0.1:9137/e/zz11111111" | sed 's/zz11111111//g' > miss1.html curl -sS "http://127.0.0.1:9137/e/zz22222222" | sed 's/zz22222222//g' > miss2.html curl -sS "http://127.0.0.1:9137/e/widget-42" | sed 's/widget-42//g' > real.html diff miss1.html miss2.html && echo "miss1 vs miss2: identical" diff real.html miss1.htmlmiss1 vs miss2: identical 2c2 < <html lang="en"><head><meta charset="utf-8"><title>Widget </title></head> --- > <html lang="en"><head><meta charset="utf-8"><title>Not found</title></head> 4c4 < <main><h1>Widget </h1><p>The widget takes one argument.</p></main> --- > <main><h1>Not found</h1><p>We could not find .</p></main>Two clients, one answer. The page that exists differs from a miss in the title and in the content area. The two misses do not differ at all, which is the signature.
- Step 6.
Stop the server by the PID
netstatreported.taskkill //PID 42636 //F; netstat -ano | grep LISTENING | grep 9137SUCCESS: The process with PID 42636 has been terminated.No line after it means nothing holds the port. The doubled slashes are for Git Bash. Kill the PID, never the image name.
How to read the result
| What you see | What it means | What to do |
| --- | --- | --- |
| 404 or 410 for an impossible id | The server states the absence | Nothing. Confirm the page is usable with How to test a 404 page |
| 200, and two impossible ids return identical bytes | A template with a hole in it. The classic soft 404 | Answer 404 from the route that fails to load its record |
| 200 at a URL other than the one asked for | The miss was redirected, usually to the home page | Return 404 at the requested URL, or redirect to a real replacement |
| 200 that is shorter than a real page and has an empty content area | The chrome rendered and the data did not | Make the missing record set the status, not only the view |
| 200 whose body repeats the id you invented | An error template that echoes the request | Ignore the echo. Compare two impossible ids with the id removed |
| Verdicts differ between two impossible ids | One of the ids collided with real data, or the response varies | Rerun with longer random ids |
Common mistakes
What to check next
- How to check 410 vs 404 response: once the route answers a real status, decide which one it should be.
- How to test a 404 page: the page a visitor reaches once the status is right.
- How to check rel next and rel prev pagination tags: pages past the last one are a common source of these.
- How to check if a page is indexable: a soft 404 is indexable, which is what makes it costly.
FAQ
What is a soft 404 error?
A URL that returns a success status while its content reports the resource is missing. The common forms are a 200 carrying error text and a redirect to the home page. The name comes from the mismatch: the absence is stated in the body only, where no status check reads it.
How do I find soft 404s on a site I did not build?
Take one URL shape that carries an identifier, such as a product or article path. Ask for it twice with random identifiers and once with an identifier that exists, then compare the three. Keep it to a few requests on a host you do not own.
Is a 200 with "nothing found" ever correct?
Yes, when the URL is a query rather than a record. A search page with no results is a working page with an empty result set, and 200 is right. A record URL naming something that does not exist answers 404.
Does a redirect to the home page count as a soft 404?
Yes. Step 2 shows the shape: a request for a missing id ends at 200 with url_effective pointing at /. The visitor is told nothing, and every URL that was ever wrong now reports a working page.
Verified
Verified by Maks Vernynode 22.23.2curl 8.21.0
Each output block is what the command above it printed on that date, on the host named in the step. Figures read from a live site move between runs. Compare the shape of the answer rather than the digits, and see the methodology for how a page is re-verified.
Related on this site
intermediate8 minpublished updated Maks Verny