How to check memory growth under load

Sample process.memoryUsage() before load, after load, and after a forced collection. On the Node target below, three cycles left heapUsed at 7.4 MB every time on a route that allocates and discards, and at 16.3, 24.8 and 33.2 MB on one that keeps a reference. Growth that survives a collection is the leak.

Why check this

Run this before a release that changed a cache, a queue, a metrics counter or anything registered once per request. The failure it catches is a process that serves traffic for two hours and is restarted by the platform on the third, with no error in the log beyond the restart.

A single high reading cannot say which of two things happened. Memory that rose under load and was collected afterwards is a working set. Memory that stayed after a collection is retained. The forced collection between the readings is what separates them.

Prerequisites

• Node 22 or later, started with --expose-gc so that global.gc() is callable (command-line options).
• npx autocannon@8, which fetches autocannon on first use.
• A free port. Port 9750 is used below. Confirm nothing is listening on it first.
• The generator shares eight cores with the target, and loopback has no network in it. MB per second belongs to this machine; MB per request is the figure that travels.
• The target, mem-server.mjs, started with node --expose-gc mem-server.mjs &. /churn allocates and drops, /leak keeps every request, /buffers and /buffers-filled retain a 64 KB buffer each.

// mem-server.mjs   start with: node --expose-gc mem-server.mjs
import { createServer } from 'node:http';
import { writeHeapSnapshot } from 'node:v8';

const kept = [];
const buffers = [];

const mb = (b) => +(b / 1048576).toFixed(1);
const mem = () => {
  const m = process.memoryUsage();
  return { rss: mb(m.rss), heapTotal: mb(m.heapTotal), heapUsed: mb(m.heapUsed),
    external: mb(m.external), arrayBuffers: mb(m.arrayBuffers), kept: kept.length };
};
const json = (res, o) => {
  res.writeHead(200, { 'content-type': 'application/json' });
  res.end(JSON.stringify(o));
};

createServer((req, res) => {
  const path = req.url.split('?')[0];
  if (path === '/churn') {
    const rows = Array.from({ length: 500 }, (_, i) => ({ i, s: `row-${i}` }));
    return json(res, { ok: rows.length });
  }
  if (path === '/leak') {
    kept.push({ at: Date.now(), url: req.url, headers: { ...req.headers }, body: 'x'.repeat(512) });
    return json(res, { ok: true });
  }
  if (path === '/buffers') {
    buffers.push(Buffer.allocUnsafe(65536));
    return json(res, { ok: true });
  }
  if (path === '/buffers-filled') {
    buffers.push(Buffer.alloc(65536, 7));
    return json(res, { ok: true });
  }
  if (path === '/mem') return json(res, mem());
  if (path === '/gc') { global.gc(); return json(res, mem()); }
  if (path === '/snapshot') {
    const name = new URL(req.url, 'http://x').searchParams.get('name') ?? 'snap';
    return json(res, { file: writeHeapSnapshot(`${name}.heapsnapshot`) });
  }
  if (path === '/reset') { kept.length = 0; buffers.length = 0; global.gc(); return json(res, mem()); }
  res.writeHead(404).end();
}).listen(9750, '127.0.0.1', () => console.log('listening on 9750'));

• cycles.sh, which runs one fixed load, reads memory, forces a collection and reads again.

#!/bin/sh
# cycles.sh <route> <cycles> <requests per cycle>
ROUTE=$1; N=${2:-3}; A=${3:-20000}
B=$(curl -s http://127.0.0.1:9750/gc)
echo "cycle 0  after gc   $B"
i=1
while [ $i -le $N ]; do
  npx autocannon@8 -c 10 -a $A -j http://127.0.0.1:9750$ROUTE > /dev/null 2>&1
  D=$(curl -s http://127.0.0.1:9750/mem)
  G=$(curl -s http://127.0.0.1:9750/gc)
  echo "cycle $i  after load $D"
  echo "cycle $i  after gc   $G"
  i=$((i+1))
done

• snapdiff.mjs, which counts nodes in two snapshots by constructor and walks the retainers of the largest array.

// snapdiff.mjs <before.heapsnapshot> <after.heapsnapshot>
import { readFileSync } from 'node:fs';

function load(file) {
  const s = JSON.parse(readFileSync(file, 'utf8'));
  const NF = s.snapshot.meta.node_fields, EF = s.snapshot.meta.edge_fields;
  const nt = s.snapshot.meta.node_types[0], et = s.snapshot.meta.edge_types[0];
  const n = NF.length, e = EF.length;
  const node = (i) => ({
    type: nt[s.nodes[i * n]], name: s.strings[s.nodes[i * n + 1]],
    id: s.nodes[i * n + 2], self: s.nodes[i * n + 3], edges: s.nodes[i * n + 4],
  });
  return { s, n, e, et, count: s.nodes.length / n, node };
}

function tally(h) {
  const m = new Map();
  for (let i = 0; i < h.count; i += 1) {
    const d = h.node(i);
    const k = `${d.type}:${d.name}`;
    const v = m.get(k) ?? { count: 0, self: 0 };
    v.count += 1; v.self += d.self;
    m.set(k, v);
  }
  return m;
}

const A = load(process.argv[2]), B = load(process.argv[3]);
const ta = tally(A), tb = tally(B);
const rows = [];
for (const [k, v] of tb) {
  const p = ta.get(k) ?? { count: 0, self: 0 };
  rows.push([k, v.count - p.count, v.self - p.self]);
}
rows.sort((x, y) => y[2] - x[2]);
console.log(`nodes  ${A.count} -> ${B.count}`);
console.log('grew by self size:');
for (const [k, dc, ds] of rows.slice(0, 6)) {
  console.log(`  ${String(k).padEnd(28)} ${String(dc).padStart(8)} objects  ${(ds / 1048576).toFixed(2)} MB`);
}

let big = 0;
for (let i = 1; i < B.count; i += 1) if (B.node(i).type === 'array' && B.node(i).self > B.node(big).self) big = i;
const first = new Int32Array(B.count + 1);
for (let i = 0; i < B.count; i += 1) first[i + 1] = first[i] + B.node(i).edges;
const back = new Map();
for (let i = 0; i < B.count; i += 1) {
  for (let j = first[i]; j < first[i + 1]; j += 1) {
    const to = B.s.edges[j * B.e + 2] / B.n;
    if (!back.has(to)) back.set(to, []);
    back.get(to).push([i, j]);
  }
}
console.log(`\nlargest array: ${(B.node(big).self / 1048576).toFixed(2)} MB, retained by:`);
let cur = big; const seen = new Set([big]);
for (let hop = 0; hop < 8; hop += 1) {
  const r = (back.get(cur) ?? []).find(([f]) => !seen.has(f));
  if (!r) break;
  const [from, edge] = r;
  const type = B.et[B.s.edges[edge * B.e]];
  const raw = B.s.edges[edge * B.e + 1];
  const label = type === 'element' || type === 'hidden' ? `[${raw}]` : B.s.strings[raw];
  const d = B.node(from);
  console.log(`  <- ${String(type + ' ' + label).padEnd(22)} of ${d.type} "${d.name}"`);
  seen.add(from); cur = from;
}

Steps

1. 1

   Step 1.

   Run three cycles of 20 000 requests against the route that allocates and drops.

   sh cycles.sh /churn 3 20000
   

   cycle 0  after gc   {"rss":53.6,"heapTotal":8.3,"heapUsed":6.6,"external":3.6,"arrayBuffers":0.1,"kept":0}
   cycle 1  after load {"rss":62.1,"heapTotal":16.3,"heapUsed":9.6,"external":3.6,"arrayBuffers":0.1,"kept":0}
   cycle 1  after gc   {"rss":62.5,"heapTotal":16.1,"heapUsed":7.3,"external":3.6,"arrayBuffers":0.1,"kept":0}
   cycle 2  after load {"rss":62.6,"heapTotal":16.3,"heapUsed":10.5,"external":3.6,"arrayBuffers":0.1,"kept":0}
   cycle 2  after gc   {"rss":62.7,"heapTotal":16.3,"heapUsed":7.4,"external":3.6,"arrayBuffers":0.1,"kept":0}
   cycle 3  after load {"rss":62.6,"heapTotal":16.3,"heapUsed":9,"external":3.6,"arrayBuffers":0.1,"kept":0}
   cycle 3  after gc   {"rss":62.7,"heapTotal":16.3,"heapUsed":7.4,"external":3.6,"arrayBuffers":0.1,"kept":0}

   Two series, read separately. heapUsed after load is 9.6, 10.5 and 9.0 MB, which depends on when the last collection ran. After a forced collection it is 7.3, 7.4 and 7.4 MB every cycle. rss went from 53.6 to 62.7 MB and stayed: 60 000 requests left the process 9.1 MB larger with nothing retained.

2. 2

   Step 2.

   Force a collection and write the first heap snapshot.

   curl -s http://127.0.0.1:9750/gc > /dev/null; curl -s "http://127.0.0.1:9750/snapshot?name=before"
   

   {"file":"before.heapsnapshot"}

   v8.writeHeapSnapshot() stops the process while it writes. Take it after a collection, so the file holds live objects.

3. 3

   Step 3.

   Run the same three cycles against the route that keeps every request.

   sh cycles.sh /leak 3 20000
   

   cycle 0  after gc   {"rss":61.3,"heapTotal":12.8,"heapUsed":7.6,"external":3.6,"arrayBuffers":0.1,"kept":0}
   cycle 1  after load {"rss":89.6,"heapTotal":49.8,"heapUsed":18.8,"external":3.6,"arrayBuffers":0.1,"kept":20000}
   cycle 1  after gc   {"rss":89.7,"heapTotal":50.5,"heapUsed":16.3,"external":3.6,"arrayBuffers":0.1,"kept":20000}
   cycle 2  after load {"rss":105.5,"heapTotal":59.2,"heapUsed":27.2,"external":3.6,"arrayBuffers":0.1,"kept":40000}
   cycle 2  after gc   {"rss":105.5,"heapTotal":59.2,"heapUsed":24.8,"external":3.6,"arrayBuffers":0.1,"kept":40000}
   cycle 3  after load {"rss":114.6,"heapTotal":69.2,"heapUsed":35.4,"external":3.6,"arrayBuffers":0.1,"kept":60000}
   cycle 3  after gc   {"rss":114.6,"heapTotal":69.2,"heapUsed":33.2,"external":3.6,"arrayBuffers":0.1,"kept":60000}

   The after-collection series is now 7.6, 16.3, 24.8 and 33.2 MB. Each cycle adds 8.7, 8.5 then 8.4 MB for the same 20 000 requests, or 0.43 KB retained per request. A leak is that straight line, not the height of one reading.

4. 4

   Step 4.

   Write the second snapshot.

   curl -s "http://127.0.0.1:9750/snapshot?name=after"
   

   {"file":"after.heapsnapshot"}

   The two files here were 8 947 222 and 59 418 490 bytes. Take the pair around one cycle rather than hours apart.

5. 5

   Step 5.

   Compare the snapshots and walk the retainers.

   node snapdiff.mjs before.heapsnapshot after.heapsnapshot
   

   nodes  92455 -> 872593
   grew by self size:
   concatenated string:(concatenated string)   360000 objects  10.99 MB
   object:Object                  120005 objects  6.41 MB
   string:127.0.0.1:9750           60000 objects  1.83 MB
   string:keep-alive               60000 objects  1.83 MB
   string:/leak                    60000 objects  1.37 MB
   string:xxxxxxxx                 60000 objects  1.37 MB
   
   largest array: 0.58 MB, retained by:
   <- internal elements      of object "Array"
   <- context kept           of object "system / Context"
   <- internal 40            of synthetic "(Stack roots)"
   <- element [19]           of synthetic "(GC roots)"
   <- element [1]            of synthetic ""

   The counts name the shape: 120 005 new Object nodes for 60 000 requests, one record and one header copy each.

   The retainer walk names the variable. context kept is the module-scope const kept, reached from (GC roots), so nothing will release it. That line is the answer a growth chart cannot give.

6. 6

   Step 6.

   Clear the target, then retain 1 000 uninitialised 64 KB buffers.

   curl -s http://127.0.0.1:9750/reset > /dev/null; curl -s http://127.0.0.1:9750/reset; npx autocannon@8 -c 10 -a 1000 -j http://127.0.0.1:9750/buffers > /dev/null 2>&1; curl -s http://127.0.0.1:9750/gc
   

   {"rss":126,"heapTotal":9.3,"heapUsed":6.7,"external":3.6,"arrayBuffers":0.1,"kept":0}
   {"rss":139.9,"heapTotal":10.8,"heapUsed":7.1,"external":66.1,"arrayBuffers":62.6,"kept":0}

   heapUsed moved from 6.7 to 7.1 MB while arrayBuffers moved from 0.1 to 62.6 MB. A monitor plotting heapUsed alone reports this process as flat. The first /reset drops the references and the second returns the backing stores, so the baseline takes two calls.

7. 7

   Step 7.

   Repeat with buffers that are written rather than left uninitialised.

   curl -s http://127.0.0.1:9750/reset > /dev/null; curl -s http://127.0.0.1:9750/reset; npx autocannon@8 -c 10 -a 1000 -j http://127.0.0.1:9750/buffers-filled > /dev/null 2>&1; curl -s http://127.0.0.1:9750/gc
   

   {"rss":127.3,"heapTotal":10.6,"heapUsed":6.8,"external":3.6,"arrayBuffers":0.1,"kept":0}
   {"rss":192.7,"heapTotal":10.8,"heapUsed":7.1,"external":66.1,"arrayBuffers":62.6,"kept":0}

   arrayBuffers reports 62.6 MB in both runs, and rss rose 13.9 MB in step 6 against 65.4 MB here. Buffer.alloc writes every page, Buffer.allocUnsafe does not, and resident memory counts touched pages.

8. 8

   Step 8.

   Stop the target and confirm the port is clear.

   netstat -ano | grep "127.0.0.1:9750 " | grep LISTENING; powershell -Command "Stop-Process -Id 39092 -Force"; netstat -ano | grep -c ":9750 .*LISTENING"
   

     TCP    127.0.0.1:9750         0.0.0.0:0              LISTENING       39092
   0

   Read the process id from the last column and stop that id alone.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | heapUsed after collection returns to the same value | Allocation and release, nothing retained | Record that value as the baseline the next release is compared against. | | heapUsed after collection rises by a similar amount each cycle | Retention proportional to load | Take two snapshots around one cycle and read the retainer walk. | | rss rises once and then holds | The heap grew to its working size and pages are not returned | Not a leak. Read the after-collection series instead. | | heapUsed flat while rss climbs | Growth is off the V8 heap | Read external and arrayBuffers in the same sample. | | Only the during-load reading is high | Garbage that no collection has reached yet | Force a collection before reading, as every cycle above does. | | kept rising with no memory change | The retained objects are small | Still a leak. It reaches the limit later, not never. |

Common mistakes

What to check next

• How to run a soak test: sockets, timers and listeners leak sooner.
• How to measure event loop lag: collection pauses appear there as delay.
• How to check memory leak in a web page: the browser half of the same question.
• How to read load test results: what the generator reported meanwhile.
• How to stress test an api: the load level these cycles need.

FAQ

How do I check for a memory leak in Node.js?

Force a collection, run a fixed load, force another, read heapUsed, and repeat three times. Here that gave 7.4, 7.4, 7.4 MB on one route and 16.3, 24.8, 33.2 MB on another.

Which number shows a leak, rss or heapUsed?

heapUsed sampled after a forced collection, across cycles. rss includes the heap V8 reserved and pages the allocator never returned, so it rises on a healthy process too.

What is memory leakage in performance testing?

Memory kept after the work that needed it finished. Load finds it where a functional test cannot: one request retaining half a kilobyte is invisible, 60 000 of them are 25 MB.

Do I need --expose-gc to do this?

For the forced collection, yes. Without it global.gc is undefined, so every reading mixes live objects with garbage. Use the flag in a test environment only.

Verified