How to check for broken links on a website
Extract every href from the page, request each URL with GET rather than HEAD, and compare the body of each 200 response against the body of a URL that cannot exist. On a local test site, a HEAD-only run called two working links broken and passed one broken link as healthy.
Why check this
Links break on a schedule you can predict: after a content migration, after a route rename, after a CMS export, and after any redirect map is rewritten. That makes this a sign-off check on staging and a scheduled check on production, not something to run once a year.
The failure it catches is the link that still answers. A help article is renamed, the old path now falls through to the application shell, and the shell answers 200 with a "page not found" panel. Uptime monitoring sees 200. Analytics sees a visit. The reader sees nothing, and no alert fires.
Prerequisites
- Node 22 and curl 8 or later. The curl manual documents
-I,-Land the-wvariables used below. - A site you own. Requesting every URL on a site you do not control is a crawl, it costs the operator money, and it is what rate limiting exists to stop. Point the run at your own build, your staging host, or the test site below.
- Save the test site as
links-server.mjsand start it withnode links-server.mjs. It carries one of every case this procedure separates. Stop it when you are done.
import { createServer } from 'node:http';
const html = { 'content-type': 'text/html; charset=utf-8' };
const home = `<!doctype html><html lang="en"><meta charset="utf-8"><title>Link test</title>
<h1>Link test page</h1><ul>
<li><a href="/ok">Working page</a></li>
<li><a href="/missing">Deleted page</a></li>
<li><a href="/sale">Summer sale</a></li>
<li><a href="/report.pdf">Quarterly report</a></li>
<li><a href="/members">Members area</a></li>
<li><a href="/old">Old address</a></li></ul>`;
const notFound = `<!doctype html><html lang="en"><meta charset="utf-8"><title>Page not found</title>
<h1>Sorry, we could not find that page</h1><p><a href="/">Back to the home page</a></p>`;
createServer((req, res) => {
switch (req.url) {
case '/': return res.writeHead(200, html).end(home);
case '/ok': return res.writeHead(200, html).end('<!doctype html><title>Working</title><h1>Working page</h1>');
case '/missing': return res.writeHead(404, html).end(notFound);
// Soft 404: the not-found page, served with 200.
case '/sale': return res.writeHead(200, html).end(notFound);
// Rejects HEAD, serves GET. Common on static file handlers.
case '/report.pdf':
if (req.method === 'HEAD') return res.writeHead(405, { allow: 'GET' }).end();
return res.writeHead(200, { 'content-type': 'application/pdf' }).end('%PDF-1.4 stub');
// A rule that answers any method other than GET with 403.
case '/members':
if (req.method === 'HEAD') return res.writeHead(403, html).end();
return res.writeHead(200, html).end('<!doctype html><title>Members</title><h1>Members area</h1>');
case '/old': return res.writeHead(301, { location: '/ok' }).end();
default: return res.writeHead(404, html).end(notFound);
}
}).listen(8731, () => console.log('link test site on http://localhost:8731/'));
Steps
- Step 1.
List the links the markup declares, before any script runs.
curl -s http://localhost:8731/ | grep -o -E 'href="[^"]+"' | sort -uhref="/members" href="/missing" href="/ok" href="/old" href="/report.pdf" href="/sale"Six candidates. A real extraction also resolves relative paths against the page URL and drops
mailto:,tel:and bare fragments. - Step 2.
Request each one with HEAD, which is what most link checkers send by default.
for p in /ok /missing /sale /report.pdf /members /old; do printf '%-12s ' "$p"; curl -s -o /dev/null -w '%{http_code}\n' -I "http://localhost:8731$p"; done/ok 200 /missing 404 /sale 200 /report.pdf 405 /members 403 /old 301Read as a verdict, this says one link is broken and two more are suspect.
- Step 3.
Request the same six with GET and put the two columns next to each other.
for p in /ok /missing /sale /report.pdf /members /old; do printf '%-12s HEAD %s GET %s\n' "$p" "$(curl -s -o /dev/null -w '%{http_code}' -I http://localhost:8731$p)" "$(curl -s -o /dev/null -w '%{http_code}' http://localhost:8731$p)"; done/ok HEAD 200 GET 200 /missing HEAD 404 GET 404 /sale HEAD 200 GET 200 /report.pdf HEAD 405 GET 200 /members HEAD 403 GET 200 /old HEAD 301 GET 301Two rows disagree.
/report.pdfand/membersanswer 405 and 403 to HEAD and 200 to GET. A reader clicking either link gets the file. Neither answer is a statement about the URL: one comes from a handler that implements GET only, the other from a rule that filters by method. - Step 4.
Catch the links that answer 200 with an error page. Request a URL that cannot exist, keep its body, and compare every other body against it.
node -e " const base = 'http://localhost:8731'; const paths = ['/ok', '/missing', '/sale', '/report.pdf', '/members', '/old']; (async () => { const probe = await fetch(base + '/h2check-probe-' + Date.now()); const baseline = await probe.text(); console.log('probe:', probe.status, baseline.length, 'bytes'); for (const p of paths) { const r = await fetch(base + p, { redirect: 'manual' }); const body = await r.text(); console.log(String(r.status).padEnd(4), p.padEnd(12), String(body.length).padStart(4), 'bytes', body === baseline ? 'identical to probe' : ''); } })(); "probe: 404 170 bytes 200 /ok 58 bytes 404 /missing 170 bytes identical to probe 200 /sale 170 bytes identical to probe 200 /report.pdf 13 bytes 200 /members 58 bytes 301 /old 0 bytes/salereturns 200 and the byte-for-byte not-found page. That is the link no status check finds. On a site that injects a timestamp or a request id into every page, compare the<title>or a container element instead of the whole body. - Step 5.
Follow the redirects, because a 301 is a verdict about the wrong URL.
for p in /old /missing /sale; do printf '%-12s ' "$p"; curl -sL -o /dev/null -w 'final %{http_code} hops %{num_redirects} %{url_effective}\n' "http://localhost:8731$p"; done/old final 200 hops 1 http://localhost:8731/ok /missing final 404 hops 0 http://localhost:8731/missing /sale final 200 hops 0 http://localhost:8731/sale/oldis healthy: one hop to a page that exists. Record the hop count as well as the final code, since a link that works through four redirects is still a defect worth a ticket.
How to read the result
| What you see | What it means | What to do | | --- | --- | --- | | 404 or 410 on GET | The link is broken | Fix the href, or redirect the old path to its replacement. | | 403 or 405 on HEAD, 200 on GET | The server rejects the method, not the URL | Re-request with GET before reporting. Switch the checker to GET. | | 200 with the not-found body | A soft 404 | The worst case: no status says so. Read How to check soft 404. | | 301 to a URL that then 404s | A redirect map points at a deleted page | Follow the chain to its end and judge the final response. | | 429 during a run | You are sending faster than the host allows | Lower concurrency, add a delay, and never point the run at a third party. |
Common mistakes
What to check next
- How to find broken images on a website: the same defect in
<img>, where the status code is the wrong signal again. - How to test a 404 page: what the destination of a broken link should return and render.
- How to check soft 404: the crawler side of a 200 error page, and how search engines treat it.
- How to check redirect chain: the full hop list behind the single number step 5 prints.
- How to check HTTP status code: reading and comparing status codes with curl.
FAQ
What are broken links on a website?
Links whose target no longer serves the content they promise. That includes 404 and 410 responses, hosts that no longer resolve, and pages that answer 200 with an error body. The last group is the largest on sites that render routes in JavaScript.
How do I find broken links on a website?
Crawl your own build, collect every href, and request each one with GET. Compare the response body of every 200 against a known not-found body. Do the run on staging, where a crawl costs nothing and the results are not confused by a CDN.
How do I check if a single link is broken?
Run curl -sIL -o /dev/null -w '%{http_code} %{url_effective}\n' URL. If the code is not 2xx, repeat without -I before deciding, because the server may be rejecting HEAD rather than the URL.
Do broken links show up in the browser console?
A broken <a href> produces nothing until someone clicks it. Failed subresources do log there, which is a separate check: How to check console errors on a website.
Verified
Verified by Maks Vernycurl 8.21.0node 22.23.2
Each output block is what the command above it printed on that date, on the host named in the step. Figures read from a live site move between runs. Compare the shape of the answer rather than the digits, and see the methodology for how a page is re-verified.
Related on this site
basic8 minpublished updated Maks Verny