How to test an interrupted file upload

Start an upload, kill the client part way, then read the server. Its log line declared=177151 received=61440 names both numbers. Look in the storage directory for a fragment under the final name, and in the temp directory for a .part file nothing will ever remove. A retry must leave one object, not two.

Why check this

An upload is the request a tester rarely finishes on purpose. A tab closes, a phone changes cell, a proxy gives up at 60 seconds. The last byte never arrives and the server decides alone what to do with the part that did.

Run this after a change to the upload route, after a storage backend swap, and after a proxy or a size limit goes in front of the endpoint. The failure it prevents: a complete 177151 byte object replaced by a 24576 byte fragment under the same name, by a request that never finished and returned no status.

Hashing a transfer that did finish is How to check sha256 of a file. This page takes the one that stopped.

Prerequisites

• Node 22 for the two scripts below. See the aborted event.
• curl 8.0 or later for -T and --limit-rate. See the curl manual.
• The target. /upload/ streams to a temp file and renames on end, /direct/ streams into the store, /cap/ closes the socket at 1 KB, /list reports both directories. Save it as upload-server.mjs.

import { createServer } from 'node:http';
import { createWriteStream, mkdirSync, readdirSync, statSync, renameSync, rmSync } from 'node:fs';

const STORE = 'store', TMP = 'tmp';
for (const d of [STORE, TMP]) { rmSync(d, { recursive: true, force: true }); mkdirSync(d); }
const ls = (d) => readdirSync(d).map((f) => `${f}:${statSync(`${d}/${f}`).size}`);

createServer((req, res) => {
  const name = req.url.split('/').pop();
  const declared = req.headers['content-length'] ?? '(absent)';
  if (req.url === '/list') {
    res.writeHead(200, { 'content-type': 'application/json' });
    res.end(JSON.stringify({ store: ls(STORE), tmp: ls(TMP) }) + '\n');
    return;
  }
  // /direct/ streams into the store directory. /upload/ streams into tmp and renames on end.
  const direct = req.url.startsWith('/direct/');
  const cap = req.url.startsWith('/cap/') ? 1024 : 0;
  const path = direct ? `${STORE}/${name}` : `${TMP}/${name}.part`;
  const out = createWriteStream(path);
  let got = 0;
  req.on('data', (c) => {
    got += c.length;
    out.write(c);
    if (cap && got >= cap) {
      console.log(`ABORT  ${req.method} ${req.url} declared=${declared} received=${got} outcome=server-closed-at-cap`);
      out.end();
      req.socket.destroy();           // no status line is sent
    }
  });
  req.on('aborted', () => {
    console.log(`ABORT  ${req.method} ${req.url} declared=${declared} received=${got} outcome=client-gone`);
    out.end();                        // the partial bytes stay where they were written
  });
  req.on('end', () => {
    out.end(() => {
      if (!direct) renameSync(path, `${STORE}/${name}`);
      console.log(`DONE   ${req.method} ${req.url} declared=${declared} received=${got} outcome=complete`);
      res.writeHead(201, { 'content-type': 'application/json' });
      res.end(JSON.stringify({ name, declared, received: got }) + '\n');
    });
  });
}).listen(8932, '127.0.0.1', () => console.log('upload target on 8932'));

• The cutting client. It declares the whole file, sends N bytes, then leaves. rst destroys the socket, fin half closes it. Save it as cut-upload.mjs.

// Declares the whole file, sends `sent` bytes, then leaves. mode: rst (destroy) | fin (half-close)
import { request } from 'node:http';
import { readFileSync } from 'node:fs';

const [path, sent, mode] = [process.argv[2], Number(process.argv[3]), process.argv[4]];
const file = readFileSync('report.csv');

const req = request({
  host: '127.0.0.1', port: 8932, method: 'PUT', path: `/${path}`,
  headers: { 'content-length': file.length },
});
req.on('error', (e) => console.log('client error:', e.code));
req.on('response', (res) => {
  console.log('status', res.statusCode, JSON.stringify(res.headers));
  let b = '';
  res.on('data', (d) => (b += d)).on('end', () => console.log('body:', JSON.stringify(b)));
});
req.write(file.subarray(0, sent), () => {
  console.log(`declared ${file.length}, sent ${sent}, closing with ${mode.toUpperCase()}`);
  if (mode === 'rst') req.socket.destroy();
  else req.socket.end();
  setTimeout(() => process.exit(0), 1500);
});

• A payload big enough to cut. The run used a 177151 byte CSV.

node -e "let s='id,email,amount\n';for(let i=1;i<=5000;i++)s+=i+',user'+i+'@example.test,'+(i*17)+'.00\n';require('node:fs').writeFileSync('report.csv',s)"

Start it with node upload-server.mjs and keep its output in view.

Steps

1. 1

   Step 1.

   Upload the whole file, so the complete case is on record.

   curl -s -T report.csv http://127.0.0.1:8932/upload/report.csv
   

   {"name":"report.csv","declared":"177151","received":177151}

   The server echoes the declared header and the bytes it counted. Two equal numbers and a 201.

2. 2

   Step 2.

   Kill the client process part way through a slow upload, as a closed tab does.

   ( curl -s --limit-rate 20k -T report.csv http://127.0.0.1:8932/upload/killed.csv; echo "curl exit $?" ) &
   sleep 2
   PID=$(netstat -ano | awk '$3=="127.0.0.1:8932" && $4=="ESTABLISHED" {print $5}')
   echo "client pid $PID"
   powershell -Command "Stop-Process -Id $PID -Force"
   

   client pid 22868
   curl exit 127
   [2]+  Done                    ( curl -s --limit-rate 20k -T report.csv http://127.0.0.1:8932/upload/killed.csv; echo "curl exit $?" )

   Take the pid from netstat: Get-Process curl would take every curl on the machine. The 127 is the shell's report for a terminated job, not a curl exit code, and no status or body reaches anyone.

3. 3

   Step 3.

   Cut the connection with a reset after 40960 bytes, declared length unchanged.

   node cut-upload.mjs upload/cut-rst.csv 40960 rst
   

   declared 177151, sent 40960, closing with RST
   client error: ECONNRESET

   ECONNRESET and nothing else. A client that retries on a status code has none here.

4. 4

   Step 4.

   Cut the same upload with a half close instead of a reset.

   node cut-upload.mjs upload/cut-fin.csv 40960 fin
   

   declared 177151, sent 40960, closing with FIN
   status 400 {"connection":"close"}
   body: ""

   Same bytes, same server, a different client experience. Node's HTTP parser answers 400 Bad Request with Connection: close and an empty body. The route never wrote it.

5. 5

   Step 5.

   Let the server do the cutting, as a size limit does.

   curl -sS -T report.csv http://127.0.0.1:8932/cap/capped.csv; echo "curl exit $?"
   

   curl: (56) Recv failure: Connection was reset
   curl exit 56

   Exit 56 is a transport failure, not a rejection. No status line was sent at all.

6. 6

   Step 6.

   Interrupt an upload on the route that writes straight into the store, over a stored object.

   node cut-upload.mjs direct/report.csv 24576 rst
   

   declared 177151, sent 24576, closing with RST
   client error: ECONNRESET

   The same interruption as step 3 against a different storage pattern. Step 9 shows the cost.

7. 7

   Step 7.

   Retry the upload killed in step 2, same file, same name.

   curl -s -T report.csv http://127.0.0.1:8932/upload/killed.csv
   

   {"name":"killed.csv","declared":"177151","received":177151}

   The retry is a fresh request that resends every byte. HTTP PUT has no resume of its own.

8. 8

   Step 8.

   Read the server's account of all seven attempts.

   cat server.log
   

   upload target on 8932
   DONE   PUT /upload/report.csv declared=177151 received=177151 outcome=complete
   ABORT  PUT /upload/killed.csv declared=177151 received=61440 outcome=client-gone
   ABORT  PUT /upload/cut-rst.csv declared=177151 received=40960 outcome=client-gone
   ABORT  PUT /upload/cut-fin.csv declared=177151 received=40960 outcome=client-gone
   ABORT  PUT /cap/capped.csv declared=177151 received=65420 outcome=server-closed-at-cap
   ABORT  PUT /cap/capped.csv declared=177151 received=65420 outcome=client-gone
   ABORT  PUT /direct/report.csv declared=177151 received=24576 outcome=client-gone
   DONE   PUT /upload/killed.csv declared=177151 received=177151 outcome=complete

   Five aborted requests, six ABORT lines: the capped request logged twice, once for the close the server chose and once for the aborted event behind it. declared against received measures the damage.

9. 9

   Step 9.

   Read both directories.

   curl -s http://127.0.0.1:8932/list
   

   {"store":["killed.csv:177151","report.csv:24576"],"tmp":["capped.csv.part:65420","cut-fin.csv.part:40960","cut-rst.csv.part:40960"]}

   killed.csv is one object at full size, so the retry did not double. report.csv is 24576 bytes: step 6 destroyed what step 1 stored. Three .part files hold 147340 bytes no request will claim.

How to read the result

| What you see | What it means | What to do | | --- | --- | --- | | received equals declared on every accepted request | The body arrived whole before the route acted on it | Keep the pair in the access log. It is the cheapest truncation alarm there is. | | A fragment under the final name in the store | The route streams into its destination and has no commit step | Write to a temp path and rename after end. The rename is atomic on one filesystem. | | .part files older than the longest legal upload | Interrupted transfers leave temp files and nothing sweeps them | Add a reaper on age and alert on the count. Three interrupted requests left 147340 bytes here. | | A 400 on a short body, and nothing in the route's own log | The HTTP parser answered, not the application | Do not assert on 400 as proof the route rejected the file. Assert on the route's own log line. | | curl exit 56 or 55 with no status code | The socket died before a response line | Test the client's retry path against a dead socket, not only against a 4xx. | | The server accepted more bytes than its own limit | The limit is checked after the first chunk arrives | Compare against How to test file upload size limit. |

Common mistakes

What to check next

• How to check sha256 of a file: confirms the bytes that arrived are the right ones.
• How to test multipart form data: a cut multipart body loses its closing boundary.
• How to test file upload size limit: the deliberate version of step 5.
• How to test API timeout handling: the dead socket from the retry side.
• How to test API idempotency: whether a retry can produce a second object.

FAQ

What should a server do with a partially uploaded file?

Keep the bytes outside the storage namespace until the request ends, then move them in one atomic rename. The /upload/ route does that: the four interruptions aimed at it left bytes in the temp directory and the store kept only complete objects.

How do I simulate a dropped connection during an upload?

Three ways, all on this page: kill the client process by pid, destroy the socket from a Node client, or have the server destroy it. The server records one outcome; the client sees three.

Can an interrupted HTTP upload be resumed?

Not with a plain PUT or POST. The retry in step 7 resent all 177151 bytes. Resuming needs an offset in the protocol, a chunked upload API or an agreed Content-Range, and a server that keeps the partial object.

How long should temp upload files be kept?

Longer than the slowest legal upload and shorter than a day. Sweep on age and alert on the file count: a count rising while the upload rate holds steady means transfers are failing.

Verified