How to check vary header

Read it next to the encoding it protects: curl -s -H 'accept-encoding: gzip' -D - -o /dev/null https://example.com/ | grep -i -E 'vary|content-encoding'. A response that carries content-encoding and no Vary: Accept-Encoding lets a shared cache store one body under the URL alone and hand compressed bytes to a client that cannot decode them.

Why check this

This defect never appears on your machine. A single client always gets the body that matches its own request, so the page renders, the tests pass, and the bug only exists once a cache sits between two clients with different Accept-Encoding values. Check it after any change that adds compression, adds a caching layer, or moves a route onto a new reverse proxy. The failure it prevents is a corporate proxy serving gzip bytes to an internal tool that sends no Accept-Encoding, which reads as random binary garbage in one office and nowhere else.

Prerequisites

// naive-cache.js. A shared cache keyed on the URL alone, in front of an
// origin that compresses and sends no Vary header.
const http = require('node:http');
const zlib = require('node:zlib');
const BODY = Buffer.from('<html><body>' + 'compressible text. '.repeat(200) + '</body></html>');
const cache = new Map();
http.createServer((req, res) => {
  const hit = cache.get(req.url);
  if (hit) {
    res.writeHead(200, { ...hit.headers, 'x-cache': 'HIT' });
    return res.end(hit.body);
  }
  const gzip = (req.headers['accept-encoding'] || '').includes('gzip');
  const headers = { 'content-type': 'text/html' };
  if (gzip) headers['content-encoding'] = 'gzip';
  const body = gzip ? zlib.gzipSync(BODY) : BODY;
  cache.set(req.url, { headers, body });
  res.writeHead(200, { ...headers, 'x-cache': 'MISS' });
  res.end(body);
}).listen(8147, () => console.log('naive cache on 8147'));

Steps

  1. Step 1.

    Read Vary and Content-Encoding together on a host that gets it right.

    curl -s -H 'accept-encoding: gzip' -D - -o /dev/null https://www.cloudflare.com/ | grep -i -E 'vary|content-encoding'
    
    vary: accept-encoding
    content-encoding: gzip

    Two lines, both present. This is the pass condition, and reading them in one grep stops you approving a Vary that guards nothing.

  2. Step 2.

    Run the same grep on a host that compresses without the header.

    curl -s -H 'accept-encoding: gzip' -D - -o /dev/null https://example.com/ | grep -i -E '^HTTP|content-encoding|vary|cf-cache-status'
    
    HTTP/2 200
    cf-cache-status: HIT
    content-encoding: gzip

    A compressed body, a cache hit, and no Vary line. That combination is the finding to open a ticket on.

  3. Step 3.

    With the cache running, send the first request as a client that accepts gzip.

    curl -s -H 'accept-encoding: gzip' -D - -o /dev/null http://127.0.0.1:8147/page | grep -i -E '^HTTP|content-encoding|x-cache'
    
    HTTP/1.1 200 OK
    content-encoding: gzip
    x-cache: MISS

    A miss, so the gzipped body is now stored under the key /page.

  4. Step 4.

    Send the second request as a client that cannot decode anything.

    curl -s -H 'accept-encoding: identity' -D - -o /dev/null http://127.0.0.1:8147/page | grep -i -E '^HTTP|content-encoding|x-cache'
    
    HTTP/1.1 200 OK
    content-encoding: gzip
    x-cache: HIT

    The client asked for identity and the cache answered gzip from store. No error, status 200.

  5. Step 5.

    Look at the bytes that client received.

    curl -s -H 'accept-encoding: identity' http://127.0.0.1:8147/page | head -c 8 | xxd
    
    00000000: 1f8b 0800 0000 0000                      ........

    1f 8b is the gzip magic number. The body starts with a gzip member where the client expected <html.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | vary: accept-encoding with content-encoding | Correct. Caches key on the encoding | Nothing. | | content-encoding and no vary | Any shared cache in the path can mix bodies | Open a ticket on the server that compresses, not on the cache. | | vary: accept-encoding and no content-encoding | Harmless, slightly wasteful | The cache splits its keyspace for nothing. Low priority. | | vary: * | Nothing may be cached for this URL | Almost always a mistake. It disables shared caching entirely. | | vary naming User-Agent or Cookie | Cache hit rate collapses | High-cardinality fields multiply the stored copies. Ask why it is there. |

Common mistakes

Sign: No Vary header, yet every client still receives the right body through the CDN.Cause: Some CDNs normalise Accept-Encoding internally and key on it whether or not the origin says so. example.com returned cf-cache-status: HIT with no Vary and still served the correct encoding. That covers one cache in the path and none of the others, so the missing header is still a defect.
Sign: Vary lists the same field name twice.Cause: Two layers each append their own value instead of merging. api.github.com returned vary: Accept,Accept-Encoding, Accept, X-Requested-With on a compressed response. It is still correct, because caches treat the list as a set, but it means two components are writing the header and only one of them is under your control.
Sign: The bug reproduces through the proxy and never against the origin.Cause: A single client sends one Accept-Encoding value, so the stored body always matches what it asked for. The defect needs two clients with different values against one cache key, which is what steps 3 and 4 arrange on purpose.

What to check next

FAQ

How to check Vary: Accept-Encoding?

Request the URL with an Accept-Encoding header and grep the response headers for both vary and content-encoding, as in step 1. Checking Vary on a request that asked for no encoding can hide the problem, because some servers add the header only when they compress.

Is a missing Vary header a real bug if the site works?

Yes. It works because nothing in your path cached the two variants under one key yet. RFC 9111 section 4.1 makes Vary the only signal a shared cache has, so the correctness depends on every intermediary, not on your code.

Should Vary list anything besides Accept-Encoding?

Only fields the response body actually depends on. Accept-Language on a localised route is right. User-Agent multiplies the stored copies by every browser build and usually points at server-side device detection that belongs somewhere else.

Does Vary matter on a response marked no-store?

No. A response a cache may not store has nothing to key. Check Cache-Control first: if the route is cacheable for anyone, the Vary line matters.

Verified

Verified by Maks Vernycurl 8.21.0node 22.23.2

Each output block is what the command above it printed on that date, on the host named in the step. Figures read from a live site move between runs. Compare the shape of the answer rather than the digits, and see the methodology for how a page is re-verified.

intermediate8 minpublished updated Maks Verny