How to stress test an api

Raise the connection count in steps and record throughput, p99 and failures at each. On the Node target below, throughput stopped improving at 4 connections, p99 crossed 200 ms at 128, and the first request failed at 512. Those are three different breaking points from one ramp.

Why check this

"The breaking point" names three different loads, and a report that does not say which one it used cannot be compared with the next. Run the ramp before a capacity commitment, and after a change to a pool size, a worker count or a queue depth.

The failure it prevents is a capacity number taken from the load at which errors began. Here that load was 128 times the load at which the service stopped getting faster, and everything in between was served slowly.

Generator and target share this machine's eight cores. A saturated generator looks exactly like a server that stopped scaling, which is what step 5 separates.

Prerequisites

• Node 22 or later, and npx autocannon@8, which fetches autocannon on first use.
• A free port. 9743 is used below.
• The target, ramp-server.mjs. /work spends fixed work on the libuv thread pool through pbkdf2, and /noop does none.

// ramp-server.mjs
import { createServer } from 'node:http';
import { pbkdf2 } from 'node:crypto';
import { availableParallelism } from 'node:os';
const ROUNDS = Number(process.env.ROUNDS ?? 12_000);
createServer((req, res) => {
  res.writeHead(200, { 'content-type': 'application/json' });
  if (req.url === '/noop') return res.end('{"ok":true}');
  pbkdf2('password', 'salt', ROUNDS, 32, 'sha512', (err, key) =>
    res.end(`{"k":"${key.toString('hex').slice(0, 8)}"}`));
}).listen(9743, '127.0.0.1', () => console.log(
  `listening 9743 rounds=${ROUNDS} cores=${availableParallelism()} UV_THREADPOOL_SIZE=${process.env.UV_THREADPOOL_SIZE ?? 'unset'}`));

• The ramp, ramp.mjs. It reads the JSON output, not the printed tables, which are formatted in the machine locale and do not parse.

// ramp.mjs <path> <comma separated connection counts>
import { execFileSync } from 'node:child_process';
const path = process.argv[2];
const levels = process.argv[3].split(',').map(Number);
console.log(['conns', 'rps', 'mean_ms', 'p99_ms', 'non2xx', 'errors', 'timeouts'].join('\t'));
for (const c of levels) {
  const out = execFileSync('npx', ['autocannon@8', '-c', String(c), '-d', '5', '-t', '1', '-j', `http://127.0.0.1:9743${path}`],
    { encoding: 'utf8', shell: true, stdio: ['ignore', 'pipe', 'ignore'] });
  const r = JSON.parse(out);
  console.log([c, r.requests.average, r.latency.average, r.latency.p99, r.non2xx, r.errors, r.timeouts].join('\t'));
}

• The reader, rules.mjs, which applies three stopping rules to one ramp.

// rules.mjs <ramp output file>
import { readFileSync } from 'node:fs';
const rows = readFileSync(process.argv[2], 'utf8').trim().split(/\r?\n/).slice(1).map((l) => {
  const [c, rps, mean, p99, non2xx, errors, timeouts] = l.split('\t').map(Number);
  return { c, rps, mean, p99, non2xx, errors, timeouts };
});
const peak = Math.max(...rows.map((r) => r.rps));
const knee = rows.find((r) => r.rps >= 0.95 * peak);
const budget = rows.find((r) => r.p99 > 200);
const failed = rows.find((r) => r.errors + r.non2xx + r.timeouts > 0);
const show = (label, r) => console.log(`${label.padEnd(24)} ${String(r.c).padStart(4)} conns   rps ${r.rps}   p99 ${r.p99} ms   failed ${r.errors + r.non2xx + r.timeouts}`);
console.log(`peak throughput          ${peak} req/s`);
show('throughput knee', knee);
show('p99 over 200 ms', budget);
show('first failed request', failed);
for (const r of rows) console.log(`c=${String(r.c).padStart(4)}  in flight by Little's law ${(r.rps * r.mean / 1000).toFixed(1)}`);

• -t 1 sets a one second client timeout. That budget is a choice and it decides where the third rule fires.
• Load goes to a host you own.

Steps

1. 1

   Step 1.

   Confirm the port has no listener.

   netstat -ano | grep ":9743 " | grep -c LISTENING
   

   0

   Any other number means another process owns it.

2. 2

   Step 2.

   Start the target and time one request.

   node ramp-server.mjs & sleep 2 && curl -s -w "  %{time_total}s\n" http://127.0.0.1:9743/work
   

   listening 9743 rounds=12000 cores=8 UV_THREADPOOL_SIZE=unset
   {"k":"156d4f74"}  0.011083s

   11 ms with nothing else running. Every later figure is read against it.

3. 3

   Step 3.

   Ramp the connection count over eleven levels.

   MSYS_NO_PATHCONV=1 node ramp.mjs /work 1,2,4,8,16,32,64,128,256,512,1024 | tee ramp-work.txt
   

   conns	rps	mean_ms	p99_ms	non2xx	errors	timeouts
   1	167	5.31	7	0	0	0
   2	311	5.92	11	0	0	0
   4	510.2	7.37	12	0	0	0
   8	491.6	15.76	23	0	0	0
   16	478	32.91	45	0	0	0
   32	473.6	66.72	84	0	0	0
   64	428.6	147.02	200	0	0	0
   128	475.2	262.26	306	0	0	0
   256	477	509.35	559	0	0	0
   512	87.6	541.45	994	0	2030	2030
   1024	60.2	729.66	1002	0	4137	4137

   Two quantities move in opposite directions. Throughput climbs to 510,2 at 4 connections, then stays between 428,6 and 491,6 for six more doublings. Mean latency doubles at every doubling from 8 on, because the extra connections wait rather than work. Nothing fails until 512.

4. 4

   Step 4.

   Apply three stopping rules to the same table.

   node rules.mjs ramp-work.txt
   

   peak throughput          510.2 req/s
   throughput knee             4 conns   rps 510.2   p99 12 ms   failed 0
   p99 over 200 ms           128 conns   rps 475.2   p99 306 ms   failed 0
   first failed request      512 conns   rps 87.6   p99 994 ms   failed 4060
   c=   1  in flight by Little's law 0.9
   c=   2  in flight by Little's law 1.8
   c=   4  in flight by Little's law 3.8
   c=   8  in flight by Little's law 7.7
   c=  16  in flight by Little's law 15.7
   c=  32  in flight by Little's law 31.6
   c=  64  in flight by Little's law 63.0
   c= 128  in flight by Little's law 124.6
   c= 256  in flight by Little's law 243.0
   c= 512  in flight by Little's law 47.4
   c=1024  in flight by Little's law 43.9

   Three rules, three answers: 4, 128 and 512 connections. Each is defensible and none is "the breaking point" alone. The lower block checks that the ramp still measured what it claimed: throughput times mean latency gives the requests in flight, and it tracks the connection count to 243 at 256. At 512 it reads 47,4, a tenth of what the command asked for.

5. 5

   Step 5.

   Run the control. The same ramp against the route that does no work.

   MSYS_NO_PATHCONV=1 node ramp.mjs /noop 1,4,64,256,512,1024
   

   conns	rps	mean_ms	p99_ms	non2xx	errors	timeouts
   1	19406.41	0.01	0	0	0	0
   4	31736	0.01	0	0	0	0
   64	29784	1.62	4	0	0	0
   256	23423.2	10.47	36	0	0	0
   512	25867.2	19.35	30	0	0	0
   1024	25342.4	40	52	0	0	0

   At 512 connections the generator sustained 25 867,2 a second with no timeouts, against 87,6 and 2 030 timeouts on /work. The client was not the limit, so the collapse belongs to the handler. The control still drifts from 31 736 down to 25 342,4: the generator slows as its own connection count rises.

6. 6

   Step 6.

   Name the knee by moving it. Restart with a thread pool of eight and ramp the low levels again.

   P=$(netstat -ano | grep "127.0.0.1:9743 " | grep LISTENING | awk '{print $NF}'); powershell -Command "Stop-Process -Id $P -Force"; sleep 1; UV_THREADPOOL_SIZE=8 node ramp-server.mjs & sleep 2; MSYS_NO_PATHCONV=1 node ramp.mjs /work 1,2,4,8,16,32
   

   listening 9743 rounds=12000 cores=8 UV_THREADPOOL_SIZE=8
   conns	rps	mean_ms	p99_ms	non2xx	errors	timeouts
   1	164.6	5.44	8	0	0	0
   2	317.8	5.78	11	0	0	0
   4	486.6	7.74	13	0	0	0
   8	543	14.19	23	0	0	0
   16	541.8	28.96	45	0	0	0
   32	534.6	59.07	97	0	0	0

   The knee moved from 4 connections to 8, the size of the pool, and the plateau rose from about 480 to about 540. A knee that moves when one named resource is resized has a cause. Reported without that test it is a shape on a chart.

7. 7

   Step 7.

   Stop the target and confirm the port is clear.

   netstat -ano | grep "127.0.0.1:9743 " | grep LISTENING; powershell -Command "Stop-Process -Id 38560 -Force"; netstat -ano | grep ":9743 " | grep -c LISTENING
   

     TCP    127.0.0.1:9743         0.0.0.0:0              LISTENING       38560
   0

   Stop that id alone. Stopping every node.exe takes other servers with it.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | Throughput flat, latency rising | Past the knee: work is queued, not done | Report the knee as the useful load, not the failure load. | | Throughput falling and latency falling with it | The run stopped holding the load it asked for | Check requests in flight against the connection count before using the row. | | First failure far above the knee | The service degrades long before it breaks | State which rule the reported number came from. | | errors equal to timeouts | Every failure was the client timeout, not a server response | Change -t and rerun. The number moves with the budget. | | Control route slows at the same level | The generator is the limit | Move the generator to another machine before quoting anything. |

Common mistakes

What to check next

• How to test concurrent users: the same sweep read for latency, not limits.
• Error rate in performance testing: dividing failures by total once failures appear.
• How to check p95 latency: the percentile autocannon never prints, and how to compute it.
• How to check connection limit of a server: the other break, where the socket is refused rather than queued.
• How to read load test results: the rows in the summary that invalidate a run.

FAQ

Load testing vs stress testing

A load test runs at a load you expect and asks whether the targets hold. A stress test raises the load until something gives. The ramp above is the second: the first is one of its rows, run longer.

How to test autoscaling

No cloud, container runtime or orchestrator exists on this machine, so no page here shows a scaling event. The ramp gives the signal a scaling rule reads: the load where the knee appears, and the metric that crosses first. Test the rule itself where it runs.

How to measure server capacity

Capacity is the load at which your own target still holds, so pick the target first. With a p99 budget of 200 ms this service holds 64 connections and 428,6 a second. With a budget of one failed request it holds 256.

Does a breaking point on localhost transfer to production?

The shape does, the number does not. Loopback has no network latency, the generator competes for the same cores, and the handler is one pbkdf2 call. Find which resource bends first, then measure the real deployment.

Verified