Access log format

Drive a fast request, a slow one and a failing one, then ask each log format the questions an incident asks: which route, which user, how long, which request. A line of timestamp and status answers none of them. A line with request id, user, path, status and duration answers all four.

Why check this

Run this when a service is added to an environment, when a proxy is put in front of one, and before a load test whose numbers you intend to attribute to routes. The format is cheap to change on the day the service is written and expensive to change once dashboards and alerts read it.

The failure it prevents is an incident review that stops at "there were four hundred 500s at 07:35". Without a route, a user reference and a duration in the same line, the next question needs a code change and a redeploy.

Prerequisites

• Node 22. No dependencies: the target writes its own log lines.
• The field names of the classic combined format are in the Apache mod_log_config documentation. The combined line below follows that shape without the referer field.
• A local target that writes the same requests three ways, so the formats can be compared on one run rather than across three deployments. Save this as access-fields.mjs.

// access-fields.mjs  Node 22, no dependencies. Writes the same requests in three formats.
import { createServer } from 'node:http';
import { appendFileSync } from 'node:fs';
import { randomUUID } from 'node:crypto';

const w = (file, line) => appendFileSync(file, line + '\n');
const anonIp = (ip) => ip.replace(/^::ffff:/, '').replace(/\.\d+$/, '.0');

createServer((req, res) => {
  const t0 = process.hrtime.bigint();
  const id = req.headers['x-request-id'] ?? randomUUID();
  const url = new URL(req.url, 'http://127.0.0.1');
  const user = req.headers['x-user-id'] ?? '-';
  const ip = req.socket.remoteAddress ?? '-';
  const status = url.pathname === '/orders/9f31' ? 500 : 200;
  setTimeout(() => {
    const out = status === 500 ? '{"error":"upstream"}' : '{"ok":true}';
    res.writeHead(status, { 'content-type': 'application/json', 'x-request-id': id });
    res.end(out);
    const ms = Number(process.hrtime.bigint() - t0) / 1e6;
    const ts = new Date().toISOString();
    w('./access-min.log', `${ts} ${status}`);
    w('./access-combined.log',
      `${ip} - - [${ts}] "${req.method} ${req.url} HTTP/1.1" ${status} ${out.length} "${req.headers['user-agent'] ?? '-'}"`);
    w('./access-fields.log', JSON.stringify({
      ts, id, ip: anonIp(ip), user, method: req.method, path: url.pathname,
      status, ms: Number(ms.toFixed(1)), bytes: out.length,
    }));
  }, url.pathname === '/checkout' ? 120 : 0);
}).listen(8319, '127.0.0.1', () => console.log('access-fields on 127.0.0.1:8319'));

• Durations measured against a local port contain no network time. Treat them as a check that the field works, not as latency figures.

Steps

1. 1

   Step 1.

   Start the target.

   node access-fields.mjs
   

   access-fields on 127.0.0.1:8319

2. 2

   Step 2.

   Send three requests that differ in the ways an incident cares about: a query string, a slow route, and a failure from another user.

   curl -s -H 'x-user-id: u_4471' 'http://127.0.0.1:8319/profile?email=dana.roswell@ordermail.test' \
     --next -s -H 'x-user-id: u_4471' http://127.0.0.1:8319/checkout \
     --next -s -H 'x-user-id: u_8820' http://127.0.0.1:8319/orders/9f31
   

   {"ok":true}{"ok":true}{"error":"upstream"}

3. 3

   Step 3.

   Read the minimal format: a timestamp and a status, which is what a log written by hand usually carries.

   cat access-min.log
   

   2026-09-12T07:35:08.763Z 200
   2026-09-12T07:35:08.892Z 200
   2026-09-12T07:35:08.900Z 500

   The file can answer "how many 500s" and nothing else. Line 2 took 129 ms longer to appear than line 1 and the file does not say so.

4. 4

   Step 4.

   Ask it the first incident question.

   grep " 500" access-min.log
   

   2026-09-12T07:35:08.900Z 500

   The answer to "which route failed" is a timestamp. Correlating it with anything means guessing from other systems by clock, which is how an hour disappears.

5. 5

   Step 5.

   Read the combined format. It adds the fields a web server writes by default.

   cat access-combined.log
   

   127.0.0.1 - - [2026-09-12T07:35:08.763Z] "GET /profile?email=dana.roswell@ordermail.test HTTP/1.1" 200 11 "curl/8.21.0"
   127.0.0.1 - - [2026-09-12T07:35:08.892Z] "GET /checkout HTTP/1.1" 200 11 "curl/8.21.0"
   127.0.0.1 - - [2026-09-12T07:35:08.900Z] "GET /orders/9f31 HTTP/1.1" 500 20 "curl/8.21.0"

   Now the route is there, and so is an address a user typed into a form, because the request line holds the whole URL. This file is a store of personal data that no logger redaction covers.

6. 6

   Step 6.

   Read the field format: path without the query, a request id, a user reference, a duration, and a truncated client address.

   cat access-fields.log
   

   {"ts":"2026-09-12T07:35:08.763Z","id":"cee8287c-ed3b-472e-906f-4a1af1cb3f4e","ip":"127.0.0.0","user":"u_4471","method":"GET","path":"/profile","status":200,"ms":12.2,"bytes":11}
   {"ts":"2026-09-12T07:35:08.892Z","id":"7544bd8e-d8bb-4b93-8aca-3481017a26c1","ip":"127.0.0.0","user":"u_4471","method":"GET","path":"/checkout","status":200,"ms":126.6,"bytes":11}
   {"ts":"2026-09-12T07:35:08.900Z","id":"61be7c71-a150-4630-984b-1f351650efee","ip":"127.0.0.0","user":"u_8820","method":"GET","path":"/orders/9f31","status":500,"ms":6.1,"bytes":20}

   The query string is gone, the last octet of the address is zeroed, and the user is a pseudonymous id rather than a name.

7. 7

   Step 7.

   Answer both questions from that file with one pass.

   node -e "const r=require('fs').readFileSync('access-fields.log','utf8').trim().split('\n').map(JSON.parse);const s=[...r].sort((a,b)=>b.ms-a.ms)[0];console.log('slowest', s.path, s.ms+'ms', s.user, s.id);for(const e of r.filter(x=>x.status>=500))console.log('5xx    ', e.path, e.ms+'ms', e.user, e.id)"
   

   slowest /checkout 126.6ms u_4471 7544bd8e-d8bb-4b93-8aca-3481017a26c1
   5xx     /orders/9f31 6.1ms u_8820 61be7c71-a150-4630-984b-1f351650efee

   Route, user and duration for both answers, and an id to search for in the application log.

8. 8

   Step 8.

   Check that the id in the file is the one the client was given. Send a request with your own id and read the response header.

   curl -s -D - -o /dev/null -H 'x-request-id: 9d41-batch-042' -H 'x-user-id: u_4471' http://127.0.0.1:8319/checkout | grep -i x-request-id && tail -1 access-fields.log
   

   x-request-id: 9d41-batch-042
   {"ts":"2026-09-12T07:35:36.508Z","id":"9d41-batch-042","ip":"127.0.0.0","user":"u_4471","method":"GET","path":"/checkout","status":200,"ms":121,"bytes":11}

   An id that the response returns and the log records is what lets a support ticket reach a single line. An id generated per line and never returned does not.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | Timestamp and status only | The file counts errors and explains none of them | Add path, duration, request id and a user reference before the next release. | | A full URL in the request line | Query parameters are stored, including anything a form put there | Log the path, and keep the parameter names you need in a separate field. | | A full client address | A field that several jurisdictions treat as personal data | Truncate on write, or set a retention that matches your policy. | | A request id that no response carries | The log cannot be reached from a support ticket | Return the id in a response header and record the same value. | | A duration field that is always near zero | The timer stops before the response is written | Measure to the end of the response, as step 6 does, not to the start of the handler. |

Common mistakes

What to check next

• How to check logs for pii: scanning both files for the values this format is meant to keep out.
• X request id header: the response side of the id that step 8 matched.
• How to check if analytics ip is anonymized: the same truncation question where the address leaves your systems.
• Json logging format: why the field format above is one JSON object per line rather than a delimited string.

FAQ

What is the standard HTTP access log format?

The combined format, as step 5 shows: client address, identity fields, time, request line, status, response size, referer and user agent. It predates request ids and durations, so a service that needs those adds fields rather than inheriting them.

Which fields does an access log need?

The ones an incident asks for: time, method, path without the query, status, duration, request id, and a pseudonymous user or session reference. Response size and upstream name earn their place when a proxy or a CDN sits in front.

Should the query string go in the access log?

Not whole. It carries whatever a form or a link put there, and no logger redaction reaches the access log. Record the path, and add the specific parameters you need as named fields, after checking what they hold.

Is an IP address in an access log personal data?

Treat it as personal data. In Breyer, C-582/14, decided 19 October 2016, the Court of Justice held that a dynamic address is personal data for a site operator that has the legal means to identify the visitor from data the internet provider holds. Truncation on write, as in step 6, keeps the field useful.

Why is the duration on a local run so small?

There is no network in it. The 126.6 ms on /checkout is a delay the service creates itself, and the other two lines are single-digit milliseconds. A local target proves the field is populated and measured correctly, never what a client experiences.

Verified