Websocket compression

Open the handshake with Sec-WebSocket-Extensions: permessage-deflate and read the 101. A server that agrees echoes the extension with its parameters. That only settles negotiation, so read the first byte of each frame: 0xc1 means the payload is compressed, 0x81 means the same message went out whole.

Why check this

Run this when a socket carries repetitive JSON, and again after any change to the server or the proxy in front of it. The failure it prevents: a dashboard that streams a 930 byte row set forty times a second, with permessage-deflate in the handshake and every frame going out at full size because the negotiated parameters turned the size rule on. The handshake looks correct in DevTools and the egress bill does not.

The browser cannot settle it either. Chrome reports message sizes after inflation, so its number is the one your code sees rather than the one the socket carried. Step 6 shows the gap.

Prerequisites

// ws-deflate.js
const { WebSocketServer } = require('ws');

function serve(port, perMessageDeflate) {
  const wss = new WebSocketServer({ port, perMessageDeflate });
  wss.on('connection', (ws, req) => {
    console.log(port, 'offered :', req.headers['sec-websocket-extensions'] || '(none)');
    ws.on('message', (data) => {
      const text = data.toString();
      const sock = ws._socket;
      const before = sock.bytesWritten;
      ws.send(text, () => {
        console.log(`${port} payload ${Buffer.byteLength(text)} bytes, frame ${sock.bytesWritten - before} bytes`);
      });
    });
  });
}

serve(8872, { threshold: 0 }); // compress every message
serve(8873, true);             // ws default, threshold 1024
console.log('listening on 8872 and 8873');
// ws-raw.js  <port> <deflate|nct|plain> <rows>
const net = require('net');

const [port, mode, rows] = [Number(process.argv[2]), process.argv[3], Number(process.argv[4])];
const payload = JSON.stringify({ rows: Array(rows).fill({ id: 1, status: 'ok' }) });
const offers = {
  deflate: 'Sec-WebSocket-Extensions: permessage-deflate\r\n',
  nct: 'Sec-WebSocket-Extensions: permessage-deflate; server_no_context_takeover\r\n',
  plain: '',
};
const ext = offers[mode];

const sock = net.connect(port, '127.0.0.1', () => {
  sock.write(
    `GET / HTTP/1.1\r\nHost: 127.0.0.1:${port}\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n` +
      `Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\nSec-WebSocket-Version: 13\r\n${ext}\r\n`
  );
});

let buf = Buffer.alloc(0);
let upgraded = false;
sock.on('data', (chunk) => {
  buf = Buffer.concat([buf, chunk]);
  if (!upgraded) {
    const end = buf.indexOf('\r\n\r\n');
    if (end === -1) return;
    console.log(buf.subarray(0, end).toString().trim());
    buf = buf.subarray(end + 4);
    upgraded = true;
    sock.write(maskedTextFrame(payload));
    console.log(`\nsent payload: ${Buffer.byteLength(payload)} bytes`);
    if (buf.length === 0) return;
  }
  console.log(`reply frame : ${buf.length} bytes`);
  console.log(`byte 0      : 0x${buf[0].toString(16)} = ${buf[0].toString(2).padStart(8, '0')}`);
  sock.end();
});

function maskedTextFrame(text) {
  const body = Buffer.from(text);
  const mask = Buffer.from([1, 2, 3, 4]);
  const head = body.length < 126
    ? Buffer.from([0x81, 0x80 | body.length])
    : Buffer.from([0x81, 0xfe, body.length >> 8, body.length & 0xff]);
  const masked = Buffer.from(body.map((b, i) => b ^ mask[i % 4]));
  return Buffer.concat([head, mask, masked]);
}

Steps

  1. Step 1.

    Offer the extension with parameters and read what the server confirms.

    curl -s --max-time 2 -o /dev/null -D - --http1.1 -H 'Connection: Upgrade' -H 'Upgrade: websocket' -H 'Sec-WebSocket-Version: 13' -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' -H 'Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits; server_max_window_bits=10' http://127.0.0.1:8872/
    
    HTTP/1.1 101 Switching Protocols
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
    Sec-WebSocket-Extensions: permessage-deflate; server_max_window_bits=10

    The answer is a subset of the offer: server_max_window_bits=10 accepted, client_max_window_bits dropped.

  2. Step 2.

    Send a 930 byte message and read the reply as bytes, not as a message.

    node ws-raw.js 8872 deflate 40; tail -1 ws-deflate.log
    
    HTTP/1.1 101 Switching Protocols
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
    Sec-WebSocket-Extensions: permessage-deflate
    
    sent payload: 930 bytes
    reply frame : 47 bytes
    byte 0      : 0xc1 = 11000001
    8872 payload 930 bytes, frame 47 bytes

    Bit 1 of byte 0 is RSV1, and RSV1 set is the per-frame flag for "this payload is deflated". The reader and the server counted 47 bytes each.

  3. Step 3.

    Repeat with the same payload and no extension offered, to get the uncompressed baseline.

    node ws-raw.js 8872 plain 40
    
    HTTP/1.1 101 Switching Protocols
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
    
    sent payload: 930 bytes
    reply frame : 934 bytes
    byte 0      : 0x81 = 10000001

    No extension line in the 101, RSV1 clear, 930 payload bytes plus a 4 byte header. The saving is 934 bytes against 47.

  4. Step 4.

    Send one small message through the same compressed socket.

    node ws-raw.js 8872 deflate 1
    
    sent payload: 33 bytes
    reply frame : 37 bytes
    byte 0      : 0xc1 = 11000001

    33 bytes of JSON left as a 37 byte frame. Deflate emits a block header per message, and on a payload this short there is no redundancy to pay for it.

  5. Step 5.

    Offer server_no_context_takeover to the server on 8873, which runs the ws defaults, and send two sizes.

    for n in 40 50; do node ws-raw.js 8873 nct $n | tail -3; done
    
    sent payload: 930 bytes
    reply frame : 934 bytes
    byte 0      : 0x81 = 10000001
    sent payload: 1160 bytes
    reply frame : 48 bytes
    byte 0      : 0xc1 = 11000001

    Same server, same negotiated extension, opposite results. The 930 byte message sat under the size rule and went out whole; the 1160 byte one was deflated to 48.

  6. Step 6.

    Connect from Chrome at http://127.0.0.1:8872/ and compare what DevTools reports with what the socket carried.

    const ws = new WebSocket('ws://127.0.0.1:8872/');
    ws.onopen = () => ws.send(JSON.stringify({ rows: Array(40).fill({ id: 1, status: 'ok' }) }));
    
    sent     opcode=1 payloadData length=930
    received opcode=1 payloadData length=930

    Those are the Network.webSocketFrameSent and webSocketFrameReceived events behind the Messages tab. The server logged 8872 payload 930 bytes, frame 47 bytes for that same connection, so the browser number is 20 times the traffic.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | Sec-WebSocket-Extensions: permessage-deflate in the 101 | The extension is negotiated for this connection | Continue. Negotiated is not the same as applied. | | No extension line in the 101 | The server declined or never offered it | Turn it on at the server, and check the proxy is not stripping the header. | | Byte 0 is 0xc1 | RSV1 set, this frame is deflated | Nothing. Compare the frame size with the payload size to size the win. | | Byte 0 is 0x81 on a negotiated socket | This frame went out uncompressed | Check the size rule and the negotiated context-takeover parameters. | | Frame larger than the payload | Deflate cost more than it saved | Raise the size rule so short messages skip compression. | | DevTools length matches the payload exactly | You are reading the inflated message | Measure at the server or from a raw socket, not in the browser. |

Thresholds

1024 bytes Source: ws 8.21.3, permessage-deflate option threshold: size in bytes below which messages should not be compressed if context takeover is disabled (https://github.com/websockets/ws/blob/master/doc/ws.md)

The number applies only when server_no_context_takeover or client_no_context_takeover is part of the agreement. Step 2 and step 5 differ in that parameter alone, and a 930 byte message is compressed on one socket and not on the other.

Common mistakes

Sign: The handshake shows permessage-deflate, and traffic is unchanged.Cause: Negotiation is per connection and compression is per frame. Only RSV1 in byte 0 says whether a given message was deflated, and no header read and no browser panel exposes that bit.
Sign: The Messages tab in Chrome DevTools shows the full JSON length, so compression looks broken.Cause: Chrome reports the payload after inflation. A 930 byte message that crossed the wire as a 47 byte frame is listed as 930. Measure bytes written at the server or read the socket directly.
Sign: Turning compression on made a chatty socket use more bandwidth.Cause: Every deflated frame carries a block header, and a payload under a few dozen bytes grows. Here 33 bytes became a 37 byte frame. A socket that sends many tiny events needs a size rule, not a switch.
Sign: The server confirms fewer parameters than the client offered.Cause: The response is an agreement, not an echo. The server here accepted server_max_window_bits=10 and dropped client_max_window_bits. Read the response line rather than the request line when you record what a connection negotiated.

What to check next

FAQ

Is WebSocket compression on by default?

No. The client has to offer permessage-deflate and the server has to confirm it. Browsers offer it on every connection. The ws library ships with it off on the server, so the confirmation is the line to check.

How do I know whether a single message was compressed?

Read bit 1 of the frame's first byte. 0xc1 is a compressed text frame, 0x81 an uncompressed one. That is the only per-message signal, and it is visible from a raw socket or from a proxy, not from a client API.

Why is my compressed frame bigger than the message?

Deflate emits a block header for every message. Below roughly forty bytes of payload that header is larger than the redundancy it removes. Step 4 shows 33 bytes leaving as a 37 byte frame.

Does context takeover matter for testing?

Yes, in two ways. It changes compression ratios, because the deflate window carries over between messages. It also switches on the size rule in ws, which decides whether short messages are compressed at all.

Verified

Verified by Maks Vernynode 22.23.2ws 8.21.3curl 8.21.0Chrome 152.0.7977.76

Each output block is what the command above it printed on that date, on the host named in the step. Figures read from a live site move between runs. Compare the shape of the answer rather than the digits, and see the methodology for how a page is re-verified.

All byte counts came from a local server on ports 8872 and 8873 on one Windows machine. Ratios depend on the payload: forty repeated rows compress far better than mixed text. The Chrome figures are one capture from one headless run of Chrome 152.

intermediate8 minpublished updated Maks Verny