How to test load balancing

Put two or three instances that name themselves behind a balancer, send load through it, then count per instance. Round robin gave 33.3% to each of three. With one instance delayed 50 ms it still gave 33.4%, and throughput fell from 7 533,6 to 580,4 requests a second.

Why check this

A balancer is judged by a share, and a share that looks correct can still be wrong. Run this after a change to the pool, the strategy, the sessions or the health check.

The failure it prevents is an even split onto an uneven pool. One slow instance took its full third and dropped throughput to a thirteenth, while the counters reported a perfect balance.

Backends and generator share this machine's eight cores, so the shares are exact and the throughput figures describe the whole set.

Prerequisites

Node 22 or later, and npx autocannon@8, which fetches autocannon on first use.
Four free ports: 9745 for the balancer, 9746 to 9748 for instances.
The instance, backend.mjs. It names itself in x-instance and counts its own requests.

// backend.mjs   PORT=9746 ID=a DELAY=0 node backend.mjs
import { createServer } from 'node:http';
const PORT = Number(process.env.PORT), ID = process.env.ID, DELAY = Number(process.env.DELAY ?? 0);
let served = 0;
createServer((req, res) => {
  if (req.url === '/count' || req.url === '/reset') {
    if (req.url === '/reset') served = 0;
    res.writeHead(200, { 'content-type': 'application/json' });
    return res.end(JSON.stringify({ id: ID, served, delay: DELAY }));
  }
  served += 1;
  const send = () => {
    res.writeHead(200, { 'content-type': 'application/json', 'x-instance': ID });
    res.end(`{"id":"${ID}"}`);
  };
  if (DELAY > 0) setTimeout(send, DELAY); else send();
}).listen(PORT, '127.0.0.1', () => console.log(`backend ${ID} on ${PORT} delay=${DELAY}ms`));

The balancer, balancer.mjs. No nginx or HAProxy is installed here, so the three strategies and the health check are written out.

// balancer.mjs   STRATEGY=rr|least|sticky  HEALTH=<ms, 0 = off>  node balancer.mjs
import { createServer, request, Agent } from 'node:http';
const STRATEGY = process.env.STRATEGY ?? 'rr';
const HEALTH = Number(process.env.HEALTH ?? 0);
const agent = new Agent({ keepAlive: true, maxSockets: 64 });
const pool = [
  { id: 'a', port: 9746, inflight: 0, up: true },
  { id: 'b', port: 9747, inflight: 0, up: true },
  { id: 'c', port: 9748, inflight: 0, up: true },
];
let rr = 0, failed = 0;
const hash = (s) => { let h = 7; for (const ch of s) h = (Math.imul(h, 31) + ch.charCodeAt(0)) | 0; return Math.abs(h); };
const pick = (req) => {
  const live = pool.filter((b) => b.up);
  if (live.length === 0) return null;
  if (STRATEGY === 'least') return live.reduce((m, b) => (b.inflight < m.inflight ? b : m));
  if (STRATEGY === 'sticky') return live[hash(String(req.headers['x-session'] ?? '')) % live.length];
  rr += 1;
  return live[rr % live.length];
};
createServer((req, res) => {
  if (req.url === '/lb-stats') {
    res.writeHead(200, { 'content-type': 'application/json' });
    return res.end(JSON.stringify({ strategy: STRATEGY, health: HEALTH, failed, pool: pool.map((b) => ({ id: b.id, up: b.up })) }));
  }
  const b = pick(req);
  if (!b) { res.writeHead(503, { 'content-type': 'application/json' }); return res.end('{"error":"no backend"}'); }
  b.inflight += 1;
  const up = request({ host: '127.0.0.1', port: b.port, path: req.url, method: req.method, agent }, (r) => {
    b.inflight -= 1;
    res.writeHead(r.statusCode, { ...r.headers, 'x-backend': b.id });
    r.pipe(res);
  });
  up.on('error', () => { b.inflight -= 1; failed += 1; res.writeHead(502, { 'x-backend': b.id }); res.end('{"error":"backend unreachable"}'); });
  req.pipe(up);
}).listen(9745, '127.0.0.1', () => console.log(`balancer 9745 strategy=${STRATEGY} health=${HEALTH}ms`));
if (HEALTH > 0) setInterval(() => {
  for (const b of pool) {
    const r = request({ host: '127.0.0.1', port: b.port, path: '/count', timeout: 200 }, (x) => { b.up = x.statusCode === 200; x.resume(); });
    r.on('error', () => { b.up = false; });
    r.on('timeout', () => r.destroy());
    r.end();
  }
}, HEALTH).unref();

Two readers, counts.mjs and reset.mjs.

// counts.mjs
const ports = [[9746, 'a'], [9747, 'b'], [9748, 'c']];
const rows = [];
for (const [p, id] of ports) {
  try {
    const r = await fetch(`http://127.0.0.1:${p}/count`);
    rows.push({ id, served: (await r.json()).served });
  } catch (e) { rows.push({ id, served: null, error: e.cause?.code ?? e.code }); }
}
const total = rows.reduce((s, r) => s + (r.served ?? 0), 0);
for (const r of rows) console.log(`${r.id}  ${r.served ?? r.error}\t${r.served === null ? '' : ((100 * r.served) / total).toFixed(1) + '%'}`);
console.log(`total ${total}`);

// reset.mjs
for (const p of [9746, 9747, 9748]) {
  try { await fetch(`http://127.0.0.1:${p}/reset`); } catch { /* down */ }
}

sessions.mjs, because autocannon sends one fixed header set per run and sticky routing needs several.

// sessions.mjs <sessions> <requests per session>
import http from 'node:http';
const S = Number(process.argv[2]), N = Number(process.argv[3]);
const agent = new http.Agent({ keepAlive: true, maxSockets: 16 });
const byBackend = {}, bySession = {};
const one = (sess) => new Promise((done) => {
  http.get({ host: '127.0.0.1', port: 9745, path: '/', agent, headers: { 'x-session': sess } }, (r) => {
    const b = r.headers['x-backend'] ?? `status-${r.statusCode}`;
    byBackend[b] = (byBackend[b] ?? 0) + 1;
    (bySession[sess] ??= new Set()).add(b);
    r.resume(); r.on('end', done);
  }).on('error', (e) => { byBackend[e.code] = (byBackend[e.code] ?? 0) + 1; done(); });
});
const run = async (sess) => { for (let i = 0; i < N; i += 1) await one(sess); };
await Promise.all([...Array(S)].map((_, i) => run(`s${i + 1}`)));
console.log('per backend :', JSON.stringify(byBackend));
console.log('per session :', Object.entries(bySession).map(([k, v]) => `${k}=${[...v].join('+')}`).join(' '));

health-case.sh, which restarts instance c, starts the balancer at one health interval, then stops c two seconds into a six second run.

#!/bin/bash
# health-case.sh <health interval ms>
H=$1
P=$(netstat -ano | grep "127.0.0.1:9745 " | grep LISTENING | awk '{print $NF}')
[ -n "$P" ] && powershell -Command "Stop-Process -Id $P -Force"
P=$(netstat -ano | grep "127.0.0.1:9748 " | grep LISTENING | awk '{print $NF}')
[ -n "$P" ] && powershell -Command "Stop-Process -Id $P -Force"
sleep 1
PORT=9748 ID=c node backend.mjs > bc.log 2>&1 &
sleep 1
STRATEGY=rr HEALTH=$H node balancer.mjs > lb.log 2>&1 &
sleep 1
npx autocannon@8 -c 12 -d 6 -j http://127.0.0.1:9745/ > h-$H.json 2>/dev/null &
sleep 2
CPID=$(netstat -ano | grep "127.0.0.1:9748 " | grep LISTENING | awk '{print $NF}')
powershell -Command "Stop-Process -Id $CPID -Force"
sleep 6
node -e "const r=require('./h-$H.json');console.log('health=$H ms  total',r.requests.total,' non2xx',r.non2xx,' 2xx',r['2xx'],' share',(100*r.non2xx/r.requests.total).toFixed(1)+'%')"

Steps

1
Step 1.
Confirm the four ports are free.
```
netstat -ano | grep -E ":974[5-8] " | grep -c LISTENING
```
```
0
```
Any other number means one belongs to somebody else.

Start three instances and the balancer, then send six requests by hand.

PORT=9746 ID=a node backend.mjs & PORT=9747 ID=b node backend.mjs & PORT=9748 ID=c node backend.mjs & sleep 2; STRATEGY=rr node balancer.mjs & sleep 2; for i in 1 2 3 4 5 6; do curl -s -D - -o /dev/null http://127.0.0.1:9745/ | grep -i "x-instance\|x-backend" | tr -d '\r' | paste -sd' '; done

backend a on 9746 delay=0ms
backend b on 9747 delay=0ms
backend c on 9748 delay=0ms
balancer 9745 strategy=rr health=0ms
x-instance: b x-backend: b
x-instance: c x-backend: c
x-instance: a x-backend: a
x-instance: b x-backend: b
x-instance: c x-backend: c
x-instance: a x-backend: a

Two independent names on every response: x-instance from the instance, x-backend from the balancer's record of where it sent the request. When they disagree, every share below is untrustworthy.

Send load through the balancer and count per instance.

node reset.mjs && npx autocannon@8 -c 12 -d 5 http://127.0.0.1:9745/ && node counts.mjs

│ Latency │ 1 ms │ 1 ms │ 3 ms  │ 3 ms │ 1.15 ms │ 0.65 ms │ 18 ms │
│ Req/Sec   │ 5 199  │ 5 199  │ 8 163   │ 8 319   │ 7 533,6 │ 1 179,49 │ 5 196  │

38k requests in 5.01s, 7.99 MB read
a  12559	33.3%
b  12560	33.3%
c  12559	33.3%
total 37678

One request apart across 37 678. An even share is the right answer only here, because the three instances are identical.

Make one instance slow and repeat, changing nothing else.

P=$(netstat -ano | grep "127.0.0.1:9748 " | grep LISTENING | awk '{print $NF}'); powershell -Command "Stop-Process -Id $P -Force"; sleep 1; PORT=9748 ID=c DELAY=50 node backend.mjs & sleep 2; node reset.mjs && npx autocannon@8 -c 12 -d 5 http://127.0.0.1:9745/ && node counts.mjs

backend c on 9748 delay=50ms
│ Latency │ 0 ms │ 1 ms │ 63 ms │ 71 ms │ 20.22 ms │ 27.8 ms │ 75 ms │
│ Req/Sec   │ 562    │ 562    │ 576    │ 603    │ 580,4  │ 13,49   │ 562    │

3k requests in 5.04s, 615 kB read
a  971	33.3%
b  971	33.3%
c  972	33.4%
total 2914

The share is unchanged and the service is thirteen times slower: 580,4 against 7 533,6. The latency row holds both populations, 1 ms at the median and 63 ms at 97.5%. Round robin counts turns, and a turn on the slow instance costs 50 ms.

Switch the strategy to least connections against the same three instances.

P=$(netstat -ano | grep "127.0.0.1:9745 " | grep LISTENING | awk '{print $NF}'); powershell -Command "Stop-Process -Id $P -Force"; sleep 1; STRATEGY=least node balancer.mjs & sleep 2; node reset.mjs && npx autocannon@8 -c 12 -d 5 http://127.0.0.1:9745/ && node counts.mjs

balancer 9745 strategy=least health=0ms
│ Latency │ 0 ms │ 0 ms │ 1 ms  │ 2 ms │ 0.48 ms │ 4.25 ms │ 79 ms │
│ Req/Sec   │ 7 323   │ 7 323   │ 10 871 │ 11 647  │ 10 411,6 │ 1 574,65 │ 7 323   │

52k requests in 5.01s, 11 MB read
a  27894	53.6%
b  23919	45.9%
c  247	0.5%
total 52060

Same instances, same load, 18 times the throughput. The slow instance took 0.5% rather than 33.4%, because a request on it is still in flight 50 ms later. Here the uneven share is correct, and a dashboard that flags it is flagging the fix.

Switch to sticky routing and drive four sessions through it.

P=$(netstat -ano | grep "127.0.0.1:9745 " | grep LISTENING | awk '{print $NF}'); powershell -Command "Stop-Process -Id $P -Force"; sleep 1; STRATEGY=sticky node balancer.mjs & sleep 2; node reset.mjs && node sessions.mjs 4 300 && node counts.mjs

balancer 9745 strategy=sticky health=0ms
per backend : {"a":600,"b":300,"c":300}
per session : s1=a s2=b s4=a s3=c
a  600	50.0%
b  300	25.0%
c  300	25.0%
total 1200

Four identical sessions over three instances cannot be even, and 50 / 25 / 25 is the best split available. Every session stayed on one instance, which is the property under test. The cost is on the same line: one session in four spent its life on the 50 ms instance.

Stop an instance during a run, at three health check intervals.

./health-case.sh 0; ./health-case.sh 1000; ./health-case.sh 200; curl -s http://127.0.0.1:9745/lb-stats

health=0 ms  total 34349  non2xx 10204  2xx 24145  share 29.7%
health=1000 ms  total 55999  non2xx 1411  2xx 54588  share 2.5%
health=200 ms  total 60808  non2xx 30  2xx 60778  share 0.0%
{"strategy":"rr","health":200,"failed":30,"pool":[{"id":"a","up":true},{"id":"b","up":true},{"id":"c","up":false}]}

One instance of three died at the same moment in all three runs. The failure rate was 29.7%, 2.5%, then 30 requests of 60 808, and only the health interval changed. Note that 29.7% is above the third the dead instance was owed: a refused connection returns faster than a served one, so it took more turns.

Stop the four processes and confirm the ports are clear.

netstat -ano | grep -E ":974[5-8] " | grep LISTENING; for p in 9745 9746 9747 9748; do P=$(netstat -ano | grep "127.0.0.1:$p " | grep LISTENING | awk '{print $NF}'); [ -n "$P" ] && powershell -Command "Stop-Process -Id $P -Force"; done; netstat -ano | grep -E ":974[0-9] " | grep -c LISTENING

  TCP    127.0.0.1:9745         0.0.0.0:0              LISTENING       24832
TCP    127.0.0.1:9746         0.0.0.0:0              LISTENING       44436
TCP    127.0.0.1:9747         0.0.0.0:0              LISTENING       27568
0

Instance c was already stopped by the last outage case, so three listeners remain. Stop the ids you started, never every node.exe.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | Even share, low throughput | Turns are even, capacity is not | Compare per instance latency, then try least connections. | | Uneven share, high throughput | The strategy is routing around a slow member | Confirm which member is slow before calling it a fault. | | x-instance and x-backend disagree | The balancer's record does not match where the request went | Stop reading its counters and count at the instances. | | One instance at zero with the others even | It is out of the pool, or its health check fails | Read up in /lb-stats before restarting anything. | | Error share above one instance's share | The dead member is answering faster than the live ones | Expect the error rate to exceed the traffic share during an outage. |

Common mistakes

Sign: An even split is treated as proof the balancer is healthy.Cause: Round robin held 33.3 / 33.3 / 33.4 while one instance ran 50 times slower, and throughput fell from 7 533,6 to 580,4 a second. The share was perfect and the service was broken. Least connections on the same pool gave 53.6 / 45.9 / 0.5 and 10 411,6 a second.

Sign: Error rate during an outage is quoted as a property of the failed instance.Cause: One instance of three died in three identical runs and produced 10 204, 1 411 and 30 failed requests. The only difference was the health check interval: off, 1 000 ms and 200 ms. The number describes the detector, not the fault.

Sign: Distribution is measured from the balancer's own counters.Cause: Those counters record the routing decision, not the delivery. Every response here carries both x-instance from the instance and x-backend from the balancer, and the instances keep their own totals at /count. Three sources that agree are what makes the share evidence.

Sign: Sticky routing is tested with one session, or with autocannon alone.Cause: autocannon sends one fixed header set for a whole run, so a sticky pool under it looks like a pool of one. Four sessions here produced 50 / 25 / 25, which is the best split three instances can give four sessions. Skew under stickiness is arithmetic, not a bug.

What to check next

How to test concurrent users: what connection count does to latency behind a balancer.
Error rate in performance testing: counting the failures the outage produced.
How to check connection limit of a server: when an instance stops accepting rather than slowing.
Health check timeout: the probe whose interval set the error rate above.
How to check requests per second an api can handle: the single instance figure to compare the pool against.

FAQ

How to check load balance

Send load through the balancer, then ask each instance what it served. Counting at the instances, as counts.mjs does, survives a balancer whose records are wrong. For one request, read a response header naming the instance.

Round robin or least connections?

Round robin when the instances are identical and requests cost the same. Least connections when either varies: it cut a slow instance from 33.4% to 0.5% and raised throughput 18 times. Round robin holds no state, which is why it is the common default.

Do sticky sessions break load balancing?

They bound it. Distribution follows sessions rather than requests, so four sessions over three instances is 50 / 25 / 25 at best, and one long session pins load to one instance. Measure the skew rather than assume it is small.

Verified

Verified 2026-09-12by Maks Vernynode 22.23.2autocannon 8.0.0curl 8.21.0Windows 11 build 22631

Each output block is what the command above it printed on that date, on the host named in the step. Figures read from a live site move between runs. Compare the shape of the answer rather than the digits, and see the methodology for how a page is re-verified.

Load testing checklist15 checks
All load and stress checks checkscluster

intermediate20 minpublished 2026-09-12updated 2026-09-12Maks Verny