How Identify File Type is built
Identify File Type reads a file’s first bytes and reports what format they actually match, independent of the file’s name or extension. This post is about the detection pipeline behind that: the binary-signature library it runs on, the two of that library’s signatures that turned out not to be trustworthy as-is, and how it resolves — or declines to resolve — signatures that several formats share.
Tech used
Binary signatures (“magic bytes”) and magic-bytes.js
Most binary file formats begin with a fixed, recognizable byte sequence — a PNG always starts with the same 8 bytes (89 50 4E 47 0D 0A 1A 0A), a ZIP-based archive with 50 4B 03 04. This is usually called a magic number or magic signature: not a checksum or anything format-negotiated, just a convention format designers picked so a reader (or a Unix file command) can identify the format without trusting the file’s name.
The tool checks a file’s bytes against a table of these signatures using magic-bytes.js 1.13.1, a zero-dependency library whose filetypeinfo(bytes: Uint8Array) function returns every registered signature the input matches, each with a typename, mime, and common extension. An earlier candidate, file-type, was considered and rejected: by its own documentation it is binary-only and does not detect text-based formats (.txt, .csv, .svg and similar), which this tool also needs to cover — so a separate text-sniffing layer was always going to be necessary regardless of which binary library was chosen, and magic-bytes.js’s coverage was the closer match to begin with.
Reading a bounded prefix, not the file
readPrefix in src/utils/fileValidation.ts reads only the first 4096 bytes of the selected file, via file.slice(0, 4096).arrayBuffer(), and that Uint8Array is the only thing identifyBytes() in src/utils/identify.ts ever sees:
export const IDENTIFY_PREFIX_BYTES = 4096;
export async function readPrefix(file: File): Promise<Uint8Array> {
const buf = await file.slice(0, IDENTIFY_PREFIX_BYTES).arrayBuffer();
return new Uint8Array(buf);
}
Every registered magic-bytes.js signature lives well within the first few dozen bytes of a file, so 4 KB is generous headroom, not a tight budget — the bound exists so the tool’s cost is constant regardless of whether the dropped file is 4 KB or 4 GB, and so it never needs a Web Worker: classifying a 4 KB prefix is cheap enough for the main thread, unlike the Worker + WASM decode path used for actual format conversion in siblings like HEIC to JPG.
Two magic-bytes.js signatures that aren’t real magic numbers
Reading the library’s match output closely turned up two signatures that don’t behave like the others. filetypeinfo registers a generic Json match that fires on any content starting with { — verified empirically against input like "{ not json at all", which it matches with no structural validation at all — and a generic xml match that fires on any <?xml prefix without checking whether an <svg tag follows. Trusting either as an authoritative binary hit would misreport a plain-text config file that happens to start with { as “JSON (binary)”, and would make every SVG file that starts with an XML prolog trip the extension-mismatch warning (named .svg, “detected” as generic XML).
src/utils/identify.ts handles this with a small allowlist:
const TEXT_PSEUDO_SIGNATURES = new Set(['json', 'xml', 'svg']);
function isTextPseudoMatchOnly(matches: BinaryMatch[]): boolean {
return matches.length > 0 && matches.every((m) => TEXT_PSEUDO_SIGNATURES.has(m.typename.toLowerCase()));
}
When every returned match is one of these three, the binary result is discarded and the bytes fall through to the text-sniffing layer instead. The dedicated SVG signature (a bare <svg prefix, no XML prolog) is precise enough to trust on its own, but is routed through the same text path anyway, so an SVG file gets the same result shape — encoding, content kind, catalog link — as every other text format rather than a one-off binary card.
Text sniffing: TextDecoder, then a content-kind heuristic
When no binary signature applies, the prefix is decoded as text using the browser’s TextDecoder — first strict UTF-8, then strict Shift_JIS if that throws:
function decodeText(bytes: Uint8Array): { text: string; encoding: TextEncoding } | null {
try {
return { text: new TextDecoder('utf-8', { fatal: true }).decode(bytes), encoding: 'utf-8' };
} catch {
// fall through to Shift_JIS
}
try {
return { text: new TextDecoder('shift-jis', { fatal: true }).decode(bytes), encoding: 'shift-jis' };
} catch {
return null;
}
}
This is the same two-encoding idea as CSV Viewer’s detectEncoding/decodeBytes, but with one difference: CSV Viewer’s Shift_JIS attempt is lenient, because it always assumes the input is text and just needs a readable decoding. Here the input might genuinely be neither UTF-8 nor Shift_JIS text — it might be unrecognized binary data — so both attempts run with fatal: true, and a third outcome (null, reported as “unknown binary data”) is a real, distinct result rather than a fallback that never fires.
A successful decode is then sniffed for a rough content kind, deliberately conservative in what it claims: JSON requires the text to fully JSON.parse, not just start with { or [ — content that starts with a brace but fails to parse is flagged as possiblyTruncatedJson (the 4 KB read may have cut a real JSON file short) rather than being labeled JSON on a hunch. CSV and TSV require the delimiter count to be consistent across the first two non-empty lines, and a single-line comma sample needs at least two commas — so "Hello, this is a sentence." isn’t mistaken for a one-row CSV. HTML, XML, and SVG are resolved by prefix (<!doctype/<html, <?xml, <svg), with the same “does the prolog actually contain <svg” check used to route the SVG pseudo-signature applied here too. Markdown detection is intentionally not attempted — nothing in a byte prefix distinguishes Markdown from plain prose reliably enough to claim it, so unmatched text is reported simply as “plain text.”
Implementation & operational notes
Ambiguous signatures are listed, not resolved. ZIP, JAR, DOCX, XLSX, PPTX, ODT, ODS, ODP, VSDX, APK, and AAR all share the same 4-byte ZIP local-file-header signature (PK\x03\x04), because they’re all ZIP containers with different content inside. Telling them apart requires reading the archive’s central directory or looking for specific internal file names — information that usually lives past the 4 KB prefix this tool reads, and that this tool (being detection-only) doesn’t extract anyway, unlike ZIP Viewer, which reads exactly that central directory. So identifyBytes returns every matching signature as-is: result.matches[0] is shown as the primary detected format, and the rest are listed as “also could be” candidates rather than the tool guessing at one.
Extension-mismatch checking accounts for the ambiguity it can’t resolve. checkExtensionMismatch in src/utils/extensionMismatch.ts is the core feature: it compares the file’s own extension against the common extension(s) of every returned match, not just the primary one, so a file named report.docx with a ZIP-family signature is correctly treated as a non-mismatch (DOCX is a legitimate member of that family) rather than flagged because the primary match happened to be typed zip. A file with no extension at all (README, or a dotfile like .gitignore, where lastIndexOf('.') is 0) is never flagged as mismatched, since there’s no claim to contradict — though the detected format is still shown.
Confidence is presented as either a single answer or an explicit list, not a score. The result never shows a percentage or a “likely” qualifier. Either exactly one format is reported (and, when applicable, the extension-mismatch warning fires), or — for the ZIP family — every plausible candidate is listed under “also could be” so the ambiguity is visible instead of hidden behind a single guess, or the result is one of two distinct non-matches: empty for a genuine 0-byte file, and unknown (“unknown binary data”) when the bytes match no signature and fail both text decodes. Empty and unknown are kept as separate result kinds rather than collapsing an empty file into “unknown,” since an empty file isn’t really unidentifiable — there’s nothing there to identify.
Detection only funnels into other tools; it never acts on the file itself. src/utils/catalogLinks.ts maps a subset of detection results to a sibling tool in the same catalog — any ZIP-family match to ZIP Viewer, CSV/TSV to CSV Viewer, HEIF to HEIC to JPG, WebP to webp-to-jpg, EML to eml-viewer, and — the always-available fallback — anything reported as unknown binary data to Hex Viewer, so a file this tool can’t name can still be inspected byte-by-byte. Those five mappings are hardcoded to specific slugs rather than derived from the detected typename generically, since a link is only shown where a real corresponding tool exists in the catalog. The tool itself never extracts, converts, or previews content — that division of labor is deliberate: identification and hex inspection are separate, atomic tools, not one tool with two modes.
Try it / source
- Tool: Identify File Type
- Source: github.com/GeppettoAndRomero/identify-file