How to test concurrent users

Run one route at several connection counts and read throughput and latency together: npx autocannon@8 -c 25 -d 10 http://127.0.0.1:9663/work. Here throughput held near 930 rps from 2 connections to 100, while mean latency went from 1.99 ms to 107.3 ms. Flat throughput with rising latency is a queue.

Why check this

One connection count tells you nothing about a second one. Run the sweep before a release that changes how work is scheduled: a thread pool resized, a synchronous call added to a handler, a connection pool shared with a new consumer.

The failure it catches is a service that meets its latency target in a single-user test and misses it by 100 times at the concurrency the release will actually see. Nothing in the single-user run predicts that, because at one connection there is no queue to stand in.

Generator and target share this machine, and both compete for the same eight cores. Step 4 is the control that says whether a flat line came from the server or from the generator.

Prerequisites

• Node 22 or later, and npx autocannon@8, which fetches autocannon on first use.
• A free port. Port 9663 is used below. netstat -ano | grep ":9663 " | grep -c LISTENING prints 0 when it is free.
• The target, saved as conc-server.mjs and started with node conc-server.mjs &. /work burns 1 ms of CPU per request on the single Node thread. /free answers at once and exists only as the control in step 4.

// conc-server.mjs
import { createServer } from 'node:http';
createServer((req, res) => {
  if (req.url === '/work') { const end = performance.now() + 1; while (performance.now() < end); }
  res.writeHead(200, { 'content-type': 'application/json' });
  res.end('{"ok":true}');
}).listen(9663, '127.0.0.1', () => console.log('listening on 9663'));

Steps

1. 1

   Step 1.

   Measure one connection first. Everything later is read against this line.

   npx autocannon@8 -c 1 -d 10 http://127.0.0.1:9663/work
   

   Running 10s test @ http://127.0.0.1:9663/work
   1 connections
   
   
   ┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────┐
   │ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max  │
   ├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────┤
   │ Latency │ 1 ms │ 1 ms │ 1 ms  │ 1 ms │ 1.01 ms │ 0.09 ms │ 8 ms │
   └─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────┘
   ┌───────────┬────────┬────────┬────────┬────────┬────────┬─────────┬────────┐
   │ Stat      │ 1%     │ 2.5%   │ 50%    │ 97.5%  │ Avg    │ Stdev   │ Min    │
   ├───────────┼────────┼────────┼────────┼────────┼────────┼─────────┼────────┤
   │ Req/Sec   │ 797    │ 797    │ 831    │ 856    │ 825,37 │ 18,61   │ 797    │
   ├───────────┼────────┼────────┼────────┼────────┼────────┼─────────┼────────┤
   │ Bytes/Sec │ 147 kB │ 147 kB │ 153 kB │ 158 kB │ 152 kB │ 3.42 kB │ 147 kB │
   └───────────┴────────┴────────┴────────┴────────┴────────┴─────────┴────────┘
   
   Req/Bytes counts sampled once per second.
   # of samples: 11
   
   9k requests in 11.01s, 1.67 MB read

   Latency Avg 1.01 ms is the handler's service time, and it matches the 1 ms the route burns. Req/Sec Avg 825,37 is one connection sending one request at a time, so it is roughly one second divided by the service time. No queue exists yet.

2. 2

   Step 2.

   Sweep the same route across seven connection counts.

   for c in 1 2 5 10 25 50 100; do npx autocannon@8 -c $c -d 10 -n -j http://127.0.0.1:9663/work > s-$c.json; node -e "const r=require('./s-$c.json');console.log('-c $c  rps',r.requests.average,' p50',r.latency.p50,'ms  p97.5',r.latency.p97_5,'ms  mean',r.latency.average,'ms')"; done
   

   -c 1  rps 817.55  p50 1 ms  p97.5 1 ms  mean 1.01 ms
   -c 2  rps 934.3  p50 2 ms  p97.5 2 ms  mean 1.99 ms
   -c 5  rps 952.55  p50 5 ms  p97.5 5 ms  mean 5.01 ms
   -c 10  rps 925.2  p50 10 ms  p97.5 12 ms  mean 10.3 ms
   -c 25  rps 937.4  p50 26 ms  p97.5 28 ms  mean 26.18 ms
   -c 50  rps 934.19  p50 52 ms  p97.5 58 ms  mean 52.87 ms
   -c 100  rps 923.3  p50 106 ms  p97.5 138 ms  mean 107.3 ms

   Read it as a curve, not as seven results. Throughput rises once, from 817.55 to 934.3 between one connection and two, then stops: 952.55, 925.2, 937.4, 934.19, 923.3. Latency does not stop. It doubles with each doubling of connections, 1.99 to 5.01 to 10.3 to 26.18 to 52.87 to 107.3 ms. The knee is between 1 and 2 connections, and every connection added after it buys queue time instead of work.

3. 3

   Step 3.

   Multiply throughput by mean latency and compare the product with the connection count.

   node -e "for (const c of [1,2,5,10,25,50,100]) { const r=require('./s-'+c+'.json'); const L=r.requests.average*r.latency.average/1000; console.log(('c='+c).padEnd(7),'rps',String(r.requests.average).padStart(7),' mean ms',String(r.latency.average).padStart(6),' rps x mean',L.toFixed(2).padStart(6),' vs c',((L-c)/c*100).toFixed(1)+'%'); }"
   

   c=1     rps  817.55  mean ms   1.01  rps x mean   0.83  vs c -17.4%
   c=2     rps   934.3  mean ms   1.99  rps x mean   1.86  vs c -7.0%
   c=5     rps  952.55  mean ms   5.01  rps x mean   4.77  vs c -4.6%
   c=10    rps   925.2  mean ms   10.3  rps x mean   9.53  vs c -4.7%
   c=25    rps   937.4  mean ms  26.18  rps x mean  24.54  vs c -1.8%
   c=50    rps  934.19  mean ms  52.87  rps x mean  49.39  vs c -1.2%
   c=100   rps   923.3  mean ms  107.3  rps x mean  99.07  vs c -0.9%

   Requests in flight equal arrival rate times time spent, so throughput times latency returns the connection count. At 100 connections it lands at 99.07, an error of 0.9%. This is why the three numbers are not independent: fix throughput at 930 rps and latency is decided by the connection count alone. The error at one connection, 17.4%, has its own reason and the pitfalls below give it.

4. 4

   Step 4.

   Point the same generator at a handler that does no work, at three of the same levels.

   for c in 1 10 100; do npx autocannon@8 -c $c -d 10 -n -j http://127.0.0.1:9663/free > f-$c.json; node -e "const w=require('./s-$c.json'),f=require('./f-$c.json');console.log('-c $c   work',w.requests.average,'rps  mean',w.latency.average,'ms    free',f.requests.average,'rps  mean',f.latency.average,'ms')"; done
   

   -c 1   work 817.55 rps  mean 1.01 ms    free 19792.73 rps  mean 0.01 ms
   -c 10   work 925.2 rps  mean 10.3 ms    free 44356.37 rps  mean 0.01 ms
   -c 100   work 923.3 rps  mean 107.3 ms    free 37501.82 rps  mean 2.26 ms

   At 100 connections the same client drove 37 501.82 rps into /free while /work held 923.3. The generator was 40 times away from being the limit, so the flat line in step 2 belongs to the server. Without this control, a client that ran out of sockets would have produced the same shape.

5. 5

   Step 5.

   Stop the target and confirm the port is clear.

   netstat -ano | grep "127.0.0.1:9663 " | grep LISTENING; powershell -Command "Stop-Process -Id 44008 -Force"; netstat -ano | grep ":9663 " | grep -c LISTENING
   

     TCP    127.0.0.1:9663         0.0.0.0:0              LISTENING       44008
   0

   Read the process id from the last column and stop that id alone.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | Throughput rises with each level | Still below the knee | Keep raising -c until it stops rising. That level is the capacity. | | Throughput flat, latency proportional to -c | Saturated, and queueing cleanly | Report the level at the knee. Adding load past it only adds waiting. | | Latency rises faster than -c | Contention on top of the queue | Look for locks, garbage collection or a pool smaller than the load. | | Throughput falls as -c rises | Past capacity into collapse | Find the last level that held, and cap concurrency below it. | | The control route flattens too | The generator is the limit, not the server | Add -w workers, or move the generator to another machine. |

Common mistakes

What to check next

• How to check requests per second an api can handle: the throughput axis of this sweep, and how much it moves between runs.
• How to check p95 latency: the tail behind each mean above, which grows faster than the mean does.
• How to stress test an api: what happens past the last level that held.
• How to check connection limit of a server: the ceiling on -c itself, before throughput is involved.
• How to test API concurrency: correctness under parallel writes, which this procedure does not measure.

FAQ

How do I test concurrent users using JMeter?

A thread group with the thread count set to the level, one HTTP sampler, and the Aggregate Report listener for throughput and percentiles. The sweep is the same: one level per run, results compared as a curve. JMeter is not installed here, so no JMeter output appears on this page.

Throughput or latency: which one do I report?

Both, at a named connection count. Either alone is unreadable: 930 rps was true at 2 connections and at 100, where latency differed by 50 times. Report the level, the throughput and a latency percentile as one row.

Which concurrent user testing tool should I use?

autocannon for a quick HTTP sweep from a terminal, k6 or JMeter when the load needs scripted steps, think time or a data set per user. All three produce the same curve. The figures on this page came from autocannon 8.0.0.

Does one connection equal one user?

No. A connection sends its next request as soon as the response lands. A user pauses. Model people by arrival rate, measure with connections, and write down which of the two a number refers to.

Verified