How to test a unicode file name in content disposition
Upload eight files whose names carry Cyrillic, CJK, an emoji, a combining accent and a right-to-left override, then compare the filename bytes in the part header against the name your parser produced and the name on disk. All three should agree. Here busboy read every non-ASCII name as Latin-1.
Why check this
Run this when an upload route is added, when its parser changes version, and at staging sign-off for a product that takes files in more than one language. The name passes four layers that each guess an encoding: client, parser, storage code, filesystem.
The failure it prevents is a file that uploads with a 200 and cannot be found afterwards. A user sends звіт.csv, the response says success, and the admin list shows eight Latin-1 characters where four letters were. The report arrives weeks later, as a search problem.
A quieter one: two uploads whose names look identical land as two rows, because one accent is precomposed and the other a combining mark.
Prerequisites
- Node 22 and
npm i busboy. busboy 1.6.0 is the parser under multer and fastify-multipart. - A local target on
127.0.0.1:8935, because the check needs a parser and a store you can read. Save it asupload-server.mjsand stop it when you finish./rawprints the part header as bytes,/uploadparses and stores,/listreads the disk.
import { createServer } from 'node:http';
import { mkdirSync, writeFileSync, readdirSync, statSync } from 'node:fs';
import { join } from 'node:path';
import busboy from 'busboy';
const STORE = new URL('./store/', import.meta.url).pathname.slice(1);
mkdirSync(STORE, { recursive: true });
const hex = (c) => c.codePointAt(0).toString(16).toUpperCase().padStart(4, '0');
const pts = (s) => [...s].map((c) => 'U+' + hex(c)).join(' ');
// Format characters (Unicode Cf) reorder or vanish in a terminal and in HTML, so
// print them escaped. U+202E in a name flips everything after it on screen.
const show = (s) => JSON.stringify(s).replace(/\p{Cf}/gu, (c) => '\\u' + hex(c));
createServer((req, res) => {
const send = (t) => { res.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' }); res.end(t); };
if (req.url === '/list') {
send(readdirSync(STORE).map((n) => `${statSync(join(STORE, n)).size} B ${pts(n)}`).join('\n') + '\n');
return;
}
if (req.url === '/raw') { // the part header exactly as it arrived
const c = [];
req.on('data', (d) => c.push(d)).on('end', () => send(
Buffer.concat(c).toString('latin1').split('\r\n')
.filter((l) => /^content-disposition/i.test(l))
.map((l) => l.replace(/[^\x20-\x7e]/g, (ch) => '\\x' + ch.charCodeAt(0).toString(16)))
.join('\n') + '\n'));
return;
}
if (req.url === '/upload') {
const bb = busboy({ headers: req.headers });
const out = [];
bb.on('file', (field, stream, info) => {
const raw = info.filename; // what busboy 1.6.0 hands you
const utf8 = Buffer.from(raw, 'latin1').toString('utf8');
const body = [];
stream.on('data', (d) => body.push(d)).on('end', () => {
out.push(`raw ${pts(raw)}`);
out.push(`utf8 ${show(utf8)} ${pts(utf8)}`);
out.push(`nfc ${utf8 === utf8.normalize('NFC')}`);
try { writeFileSync(join(STORE, utf8), Buffer.concat(body)); out.push('disk ok'); }
catch (e) { out.push(`disk ${e.code}`); }
});
});
bb.on('close', () => send(out.join('\n') + '\n'));
req.pipe(bb);
return;
}
res.writeHead(404).end();
}).listen(8935, '127.0.0.1', () => console.log('listening on 127.0.0.1:8935'));
upload.mjs, the client. Every name is built from code points, so no shell or editor alters it. Node 22FormDatawrites the part header a browser writes.
const url = process.argv[2] ?? 'http://127.0.0.1:8935/upload';
const cp = (...n) => String.fromCodePoint(...n);
const names = {
ascii: 'report.csv',
cyrillic: cp(0x437, 0x432, 0x456, 0x442) + '.csv', // zvit, Ukrainian
cjk: cp(0x5831, 0x544a, 0x66f8) + '.csv', // houkokusho, Japanese
emoji: 'report' + cp(0x1f4c4) + '.csv',
nfc: 'caf' + cp(0xe9) + '.csv', // e with acute, one code point
nfd: 'cafe' + cp(0x301) + '.csv', // e plus a combining acute
rtl: 'annex' + cp(0x202e) + 'fdp.exe', // right-to-left override
colon: 'q1' + cp(0x3a) + 'q2.csv', // the colon that NTFS reserves
};
const body = new TextEncoder().encode('id,amount\n1,42\n');
for (const [label, name] of Object.entries(names)) {
const fd = new FormData();
fd.set('file', new File([body], name, { type: 'text/csv' }));
const res = await fetch(url, { method: 'POST', body: fd });
console.log(`=== ${label}`);
process.stdout.write(await res.text());
}
- A two-line
payload.csvfor step 1:printf 'id,amount\n1,42\n' > payload.csv. - RFC 7578 section 4.2 defines
filenamein a form part and forbids the RFC 5987filename*form there. - The disk figures come from one machine, Windows 11 on NTFS. Nothing here was run on Linux.
Steps
- Step 1.
Send one Cyrillic name with curl and read the bytes that left the client.
curl -s -F 'file=@payload.csv;filename=звіт.csv' http://127.0.0.1:8935/rawContent-Disposition: form-data; name="file"; filename="\xe7\xe2\xb3\xf2.csv"Four bytes for four letters, so this is not UTF-8.
e7 e2 b3 f2isзвітin Windows-1251, the ANSI code page of this machine. - Step 2.
Send all eight names from a browser-shaped client and read the part headers again.
node upload.mjs http://127.0.0.1:8935/raw=== ascii Content-Disposition: form-data; name="file"; filename="report.csv" === cyrillic Content-Disposition: form-data; name="file"; filename="\xd0\xb7\xd0\xb2\xd1\x96\xd1\x82.csv" === cjk Content-Disposition: form-data; name="file"; filename="\xe5\xa0\xb1\xe5\x91\x8a\xe6\x9b\xb8.csv" === emoji Content-Disposition: form-data; name="file"; filename="report\xf0\x9f\x93\x84.csv" === nfc Content-Disposition: form-data; name="file"; filename="caf\xc3\xa9.csv" === nfd Content-Disposition: form-data; name="file"; filename="cafe\xcc\x81.csv" === rtl Content-Disposition: form-data; name="file"; filename="annex\xe2\x80\xaefdp.exe" === colon Content-Disposition: form-data; name="file"; filename="q1:q2.csv"Eight bytes for four Cyrillic letters, three per CJK ideograph, four for the emoji: raw UTF-8 in a quoted string, with no charset label in the part.
nfcandnfddiffer on the wire,c3 a9against65 cc 81. - Step 3.
Parse the same eight uploads and read the name the application receives.
node upload.mjs http://127.0.0.1:8935/upload… === cyrillic raw U+00D0 U+00B7 U+00D0 U+00B2 U+00D1 U+0096 U+00D1 U+0082 U+002E U+0063 U+0073 U+0076 utf8 "звіт.csv" U+0437 U+0432 U+0456 U+0442 U+002E U+0063 U+0073 U+0076 nfc true disk ok … === emoji raw U+0072 U+0065 U+0070 U+006F U+0072 U+0074 U+00F0 U+009F U+0093 U+0084 U+002E U+0063 U+0073 U+0076 utf8 "report📄.csv" U+0072 U+0065 U+0070 U+006F U+0072 U+0074 U+1F4C4 U+002E U+0063 U+0073 U+0076 nfc true disk ok === nfc raw U+0063 U+0061 U+0066 U+00C3 U+00A9 U+002E U+0063 U+0073 U+0076 utf8 "café.csv" U+0063 U+0061 U+0066 U+00E9 U+002E U+0063 U+0073 U+0076 nfc true disk ok === nfd raw U+0063 U+0061 U+0066 U+0065 U+00CC U+0081 U+002E U+0063 U+0073 U+0076 utf8 "café.csv" U+0063 U+0061 U+0066 U+0065 U+0301 U+002E U+0063 U+0073 U+0076 nfc false disk ok === rtl raw U+0061 U+006E U+006E U+0065 U+0078 U+00E2 U+0080 U+00AE U+0066 U+0064 U+0070 U+002E U+0065 U+0078 U+0065 utf8 "annex\u202Efdp.exe" U+0061 U+006E U+006E U+0065 U+0078 U+202E U+0066 U+0064 U+0070 U+002E U+0065 U+0078 U+0065 nfc true disk ok …Every
rawcode point sits in U+0080 to U+00FF: the UTF-8 bytes read one at a time as Latin-1. Theutf8line re-decodes them into the real name.nfdprintscafé.csvexactly asnfcdoes, andnfc falseis the only signal that the two differ. - Step 4.
Read the names that reached the filesystem, as code points and sizes.
curl -s http://127.0.0.1:8935/list15 B U+0061 U+006E U+006E U+0065 U+0078 U+202E U+0066 U+0064 U+0070 U+002E U+0065 U+0078 U+0065 15 B U+0063 U+0061 U+0066 U+0065 U+0301 U+002E U+0063 U+0073 U+0076 15 B U+0063 U+0061 U+0066 U+00E9 U+002E U+0063 U+0073 U+0076 0 B U+0071 U+0031 15 B U+0072 U+0065 U+0070 U+006F U+0072 U+0074 U+002E U+0063 U+0073 U+0076 15 B U+0072 U+0065 U+0070 U+006F U+0072 U+0074 U+1F4C4 U+002E U+0063 U+0073 U+0076 15 B U+0437 U+0432 U+0456 U+0442 U+002E U+0063 U+0073 U+0076 15 B U+5831 U+544A U+66F8 U+002E U+0063 U+0073 U+0076Eight uploads, eight entries, two of them the accented word that NTFS kept apart. One entry is a name nobody sent:
q1, holding zero bytes. - Step 5.
Find the 15 bytes that the
q1:q2.csvupload reported as written.Get-Item -Path .\store\q1 -Stream * | Format-Table Stream,Length -AutoSizeStream Length ------ ------ :$DATA 0 q2.csv 15The colon read as a stream separator, so the body went into an NTFS alternate data stream on a zero-length file.
writeFileSyncraised nothing and the route logged success. - Step 6.
Confirm the finding on a server you do not control, against an ASCII control.
for n in report.csv звіт.csv; do printf '%-12s ' "$n"; curl -s -o resp.json -w "HTTP %{http_code} " -F "file=@payload.csv;filename=$n" https://httpbin.org/post; node -e "const j=require('./resp.json');console.log('files',JSON.stringify(Object.keys(j.files)),' form',JSON.stringify(Object.keys(j.form)))"; donereport.csv HTTP 200 files ["file"] form [] звіт.csv HTTP 200 files [] form []Same endpoint, same body, one character class apart. The Windows-1251 name gives
200 OKwith no file and no field, because httpbin's parser dropped the part in silence. That is the response a user gets when an upload vanishes.
How to read the result
| What you see | What it means | What to do |
| --- | --- | --- |
| Two bytes per Cyrillic letter in /raw | The client sent UTF-8, which is what browsers do | Nothing. Move to what the parser made of it. |
| One byte per letter, e7 e2 b3 f2 | The client sent a legacy code page, here Windows-1251 | Test with a browser or a scripted UTF-8 client. A desktop curl is not the reference. |
| Parser name is all U+0080 to U+00FF | The parser decoded UTF-8 bytes as Latin-1 | Re-decode with Buffer.from(name, 'latin1').toString('utf8') before any validation. |
| U+FFFD in the re-decoded name | The bytes were not UTF-8 to begin with | Reject the upload. Guessing the code page produces a name nobody can reproduce. |
| nfc false | The name carries a combining mark, not a precomposed character | Normalise to NFC before the duplicate check and before the write. |
| Two stored names that look the same | The store keeps both spellings as separate keys | Normalise on write, then deduplicate what is already there. |
| A stored file at 0 B after a successful write | A character in the name has meaning to the filesystem | Reject or replace the character. On NTFS the colon hides the body in a stream. |
| U+202E or another Cf code point in the name | The displayed name is not the stored name | Strip format characters, and escape them wherever the name is shown. |
Common mistakes
What to check next
- How to check content disposition header: the response half, where the same name needs the
filename*a request part must not use. - How to test multipart form data: the envelope these part headers live in.
- How to check path traversal in an uploaded file name: separators and
..in the same field. - How to test unicode input: the same characters typed into a text field.
- How to check for garbled characters from the wrong encoding: telling a decoding fault from a truncation.
FAQ
Can I put filename* in a multipart upload part?
RFC 7578 section 4.2 forbids the RFC 5987 encoding in a form part, and browsers do not send it. Send raw UTF-8 bytes in filename, as every client in step 2 did.
Which test names cover the most ground?
The eight in upload.mjs: a two-byte script, a three-byte script, an astral character, two spellings of one accented word, a right-to-left override, and a name whose colon NTFS reserves.
Why build the names from code points instead of typing them?
A shell, an editor or a paste can change them first. The step 1 command was typed as UTF-8 and left as Windows-1251.
Should the server rename uploads instead?
A generated id with the original name in metadata removes the filesystem questions. The decoding and normalisation ones stay, because the metadata still holds a name.
Does the emoji need its own case?
Yes. It is the only name above U+FFFF, so it separates a code-point count from a UTF-16 code-unit count. It arrived as 4 bytes and 1 code point, with length 2.
Verified
Verified by Maks Vernycurl 8.21.0node 22.23.2busboy 1.6.0Windows PowerShell 5.1
Each output block is what the command above it printed on that date, on the host named in the step. Figures read from a live site move between runs. Compare the shape of the answer rather than the digits, and see the methodology for how a page is re-verified.
Related on this site
intermediate8 minpublished updated Maks Verny