Webhook timeout

Post to the endpoint with a generous deadline to learn what it really costs, then post again with the sender's real budget. Here curl measured total=12.018436s while the sender aborted at 5.018 s and retried. The receiver committed the handler twice, and the sender recorded no delivery at all.

Why check this

Run this whenever the webhook handler gains work: a new database write, a call to a third party, an image resize. Those additions move the endpoint's response time, and the sender's deadline does not move with it. The failure it prevents is a handler that grows past the sender's budget and starts producing a duplicate for every event, while every dashboard on the receiving side reports success.

That is the part worth holding on to. A timeout is not a lost event. It is one event, processed as many times as the sender retries, recorded as zero deliveries on one side and several on the other.

Prerequisites

• Node 22 or later. The sender's deadline is AbortSignal.timeout, which covers the whole exchange, not the connect phase alone.
• curl 7.32 or later for %{time_total}, documented in the curl manual.
• A free TCP port. Everything here uses 127.0.0.1:8931. Confirm with netstat -ano | grep 8931 first.
• No provider account, and none is used. The 5 s budget below is the one set in sender.mjs. Providers publish their own deadline and change it.
• Save the receiver as receiver.mjs. Posting to /slow/12000 makes it sleep 12 s before replying.

// receiver.mjs - a webhook endpoint you can make fail, hang or lose its reply.
import { createServer } from 'node:http';

const port = Number(process.argv[2] ?? 8931);
const t0 = Date.now();
const perEvent = new Map();
let order = 0;

createServer((req, res) => {
  let raw = '';
  req.on('data', (c) => (raw += c));
  req.on('end', async () => {
    const ev = JSON.parse(raw || '{}');
    const n = (perEvent.get(ev.id) ?? 0) + 1;
    perEvent.set(ev.id, n);
    const arrival = ++order;
    const [, mode, arg] = req.url.split('/');
    const say = (status, note = '') =>
      console.log(
        `#${arrival} t=${((Date.now() - t0) / 1000).toFixed(3)}s id=${ev.id} seq=${ev.seq} attempt=${n} -> ${status} ${note}`
      );
    if (mode === 'down' || (mode === 'fail' && n <= Number(arg)) || (mode === 'flaky' && arrival <= Number(arg))) {
      say(500);
      return res.writeHead(500).end('not today');
    }
    if (mode === 'slow') await new Promise((r) => setTimeout(r, Number(arg)));
    if (mode === 'drop') {
      say('200-then-lost', '(handler committed, reply discarded)');
      return res.socket.destroy();
    }
    say(200, '(handler committed)');
    res.writeHead(200).end('ok');
  });
}).listen(port, () => console.log(`receiver listening on ${port}`));

• Save the sender as sender.mjs. TIMEOUT_MS is its deadline per attempt, and it appends to dlq.jsonl when the attempts run out.

// sender.mjs - delivers one event with exponential backoff, then dead-letters it.
import { appendFileSync } from 'node:fs';

const [url, id, seq] = process.argv.slice(2);
const MAX = Number(process.env.ATTEMPTS ?? 4);
const BASE = Number(process.env.BASE_MS ?? 1000);
const TIMEOUT = Number(process.env.TIMEOUT_MS ?? 5000);

const event = { id, seq: Number(seq ?? 1), type: 'payment.succeeded', createdAt: new Date().toISOString() };
const t0 = Date.now();
const history = [];
let prev = null;

for (let attempt = 1; attempt <= MAX; attempt += 1) {
  const started = Date.now();
  let outcome;
  try {
    const r = await fetch(url, {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body: JSON.stringify(event),
      signal: AbortSignal.timeout(TIMEOUT),
    });
    outcome = `HTTP ${r.status}`;
    await r.text();
    if (r.ok) { report(attempt, started, outcome); history.push(rec(attempt, started, outcome)); done('delivered'); }
  } catch (e) {
    outcome = e.name === 'TimeoutError' ? `timeout at ${TIMEOUT} ms` : (e.cause?.code ?? e.name);
  }
  report(attempt, started, outcome);
  history.push(rec(attempt, started, outcome));
  if (attempt === MAX) break;
  const ideal = BASE * 2 ** (attempt - 1);
  const delay = Math.round(ideal / 2 + Math.random() * (ideal / 2));
  console.log(`         sleeping ${delay} ms before attempt ${attempt + 1}`);
  await new Promise((r) => setTimeout(r, delay));
}
appendFileSync('dlq.jsonl', JSON.stringify({ event, reason: `gave up after ${MAX} attempts`, lastOutcome: history.at(-1).outcome, failedAt: new Date().toISOString(), attempts: history }) + '\n');
done(`dead-lettered after ${MAX} attempts`);

function rec(attempt, started, outcome) {
  return { attempt, startedAt: new Date(started).toISOString(), tookMs: Date.now() - started, outcome };
}
function report(attempt, started, outcome) {
  const gap = prev === null ? '   -  ' : `${((started - prev) / 1000).toFixed(3)}s`;
  prev = started;
  console.log(`attempt ${attempt}  t=${((started - t0) / 1000).toFixed(3)}s  gap=${gap}  took=${String(Date.now() - started).padStart(5)} ms  ${outcome}`);
}
function done(msg) { console.log(`result: ${msg}`); process.exit(0); }

Steps

1. 1

   Step 1.

   Start the receiver with its console going to a file.

   node receiver.mjs 8931 > receiver.log 2>&1 &
   

   receiver listening on 8931

   Every t= the receiver prints from now on is measured from this line.

2. 2

   Step 2.

   Measure what the endpoint costs, with a deadline far above anything you expect.

   curl -s -o /dev/null -X POST -H 'content-type: application/json' -d '{"id":"evt_probe","seq":0}' -w 'http=%{http_code} total=%{time_total}s\n' --max-time 20 http://127.0.0.1:8931/slow/12000
   

   http=200 total=12.018436s

   12.02 s, and a 200 at the end of it. This number is the only honest input to the timeout decision, and it has to be measured on the slowest path the handler has, not the empty one.

3. 3

   Step 3.

   Send the same event through a sender whose budget is 5 s.

   ATTEMPTS=2 BASE_MS=1000 TIMEOUT_MS=5000 node sender.mjs http://127.0.0.1:8931/slow/12000 evt_2002 7
   

   attempt 1  t=0.000s  gap=   -    took= 5018 ms  timeout at 5000 ms
          sleeping 787 ms before attempt 2
   attempt 2  t=5.819s  gap=5.819s  took= 5015 ms  timeout at 5000 ms
   result: dead-lettered after 2 attempts

   Both attempts aborted 18 ms and 15 ms past the deadline, and the event went to dlq.jsonl. From the sender's records this event was never delivered.

4. 4

   Step 4.

   Read the receiver's version of the same 11 seconds.

   cat receiver.log
   

   receiver listening on 8931
   #1 t=14.005s id=evt_probe seq=0 attempt=1 -> 200 (handler committed)
   #2 t=26.179s id=evt_2002 seq=7 attempt=1 -> 200 (handler committed)
   #3 t=31.926s id=evt_2002 seq=7 attempt=2 -> 200 (handler committed)

   Two commits for evt_2002, each about 12 s after its request arrived, each answering 200 into a socket the sender had already closed. One side counts zero deliveries, the other counts two.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | total=12.018436s against a 5 s budget | The handler is slower than the deadline | Reply 202 and do the work in a worker, or raise the budget knowingly. | | timeout at 5000 ms on every attempt | The sender gave up before any reply | Read the receiver log before calling the event lost. | | Receiver logs a 200 the sender never got | The work committed, the reply had nowhere to go | Deduplicate on the event id. See Webhook idempotency. | | took a few ms past the deadline | Abort fires on a timer, not at the instant | Assert on a band, for example 5000 ms to 5200 ms. | | The event is in dlq.jsonl and the handler ran twice | Both sides are right and they disagree | Reconcile by event id, not by delivery count. See Webhook dead letter queue. |

Common mistakes

What to check next

• How to test webhook retries: the schedule the sender follows after each of these timeouts.
• Webhook idempotency: the receiver-side key that makes a double commit harmless.
• Webhook dead letter queue: where the event above landed once the attempts ran out.
• How to test API timeout handling: the same deadline question for an ordinary client call, with curl exit codes.

FAQ

How many seconds should a webhook timeout be?

Set it above the endpoint's measured slow path and below the point where holding the connection costs more than a retry. Measure first, as in step 2, which returned 12.02 s here. A budget picked before that measurement is a guess with a number attached.

What happens to the event when the receiver times out?

Nothing, on the receiver. The handler keeps running and commits. The sender sees an abort, retries, and the handler commits again. The event is not lost, it is duplicated, which is why the receiver needs a deduplication key.

Why does the receiver log a success the sender never saw?

The reply was written to a socket the sender had already closed. Node's HTTP server does not cancel a handler when the client leaves, and neither do most frameworks. The log line records the commit, not the delivery.

Should the handler do the work before or after replying?

After. Store the raw payload, reply 202, and process it from a queue. The endpoint then answers in milliseconds regardless of what the work costs, and the sender's deadline stops being a variable in your design.

Verified