Load testing checklist
Fifteen checks for a load test, in the order that keeps the result honest. The first group is about the measurement, because a number produced before you know what the generator was doing is not evidence: throughput moves with the duration of the run, and a p95 computed the usual way is not a p95 at all. The second group reads a result you have been handed, where the common failure is a report that looks better because the service is failing faster. The third group is where the service stops coping, and the fourth is what it does with a shape of traffic rather than a volume. Every page here was measured with the generator and the target on the same machine, and each one says so, because a client that is itself the bottleneck produces a curve identical to a server that stopped scaling.
15 checksabout 233 min end to end
Before the number means anything
- How to check requests per second an api can handle
The figure changes with the duration of the run, and a one-second run reports an average over a single sample, so a short run is not a fast measurement but an absent one. - How to check p95 latency
A percentile taken from per-second averages is a different statistic from one taken from raw durations, and on the same requests the two differ by a factor of four. - How to test concurrent users
Throughput staying flat while latency climbs is queueing, and the arithmetic relating the two to the connection count is what tells you the run was measuring the server. - Load test ramp up time
The first seconds of a run move the reported average by more than most changes you would make to the code, and discarding them can also discard the service's real behaviour after a deploy.
Reading a result
- How to read load test results
A fast error is counted as a fast response, so throughput and latency can both improve while the service gets worse, and only the response-code split gives it away. - Error rate in performance testing
A connection refused, a timeout, a 5xx and a 429 are four server states that one error counter collapses into one number, and the client's count and the server's log answer different questions. - How to measure event loop lag
Loop delay and request latency measure different things, and a service can have a quiet loop and a terrible tail, or the reverse.
Where it stops coping
- How to stress test an api
The load where throughput stops improving, where the tail crosses a budget and where the first request fails are three different numbers, and the phrase has to name one. - How to check connection limit of a server
Several unrelated limits all present as a refused connection, and one of them belongs to the machine generating the load rather than to the server. - How to check memory growth under load
Memory that rises during load and returns afterwards is not a leak, and without a forced collection the two curves are the same shape. - How to run a soak test
Sockets, timers and listeners run out long before memory does, and the count that matters returns to its baseline between cycles or it does not.
Shapes of traffic, not volumes
- How to test rate limiter under burst
A fixed window and a token bucket agree mid-window and disagree at the reset, where the fixed window passes about twice its nominal limit. - Spike testing in software testing
A step and a ramp fail differently, and the half most tests omit is the recovery, where a draining queue or a retry storm keeps the service down after the spike has ended. - How to test graceful degradation
An unbounded queue accepts everything and reports as a healthy server while every answer arrives after the caller gave up. - How to test load balancing
A perfectly even split is the symptom when one instance is slow, and how long a dead backend keeps receiving traffic is a property of the health check rather than of the backend.
Take it with you
The file is the same list as Markdown checkboxes, ready to paste into a release ticket or a pull request description.
Ticks are kept in this browser only. They are not sent anywhere and other people do not see them.