runlocally

runlocally engineering notes

Merge ZIP

How Merge ZIP is built

By Geppetto · · Open Merge ZIP →

These are the engineering notes for Merge ZIP: the technologies it is built on, what each one is, and how it is used in the tool.

Tech used

Many readers into one writer with @zip.js/zip.js

Merging is a fan-in: one @zip.js/zip.js ZipReader per input archive (from the Unzip notes) feeds a single shared ZipWriter(new BlobWriter('application/zip')). Each input’s getEntries() reads its central directory, then every entry’s data is streamed across — entry.getData(new BlobWriter()) out of the source, writer.add(name, new BlobReader(data)) into the one output — before the next input is opened. Folder structure survives because each entry keeps its full path (docs/readme.txt), and explicit directory entries are carried over once, so even empty folders come along. useUnicodeFileNames: true sets the UTF-8 filename flag (bit 11, from the Create ZIP notes) on the merged archive.

Explicit name-collision handling

The interesting problem in a merge is two inputs holding the same path. A ZIP would happily store both entries under one name, and an extractor would then overwrite one with the other — a silent data loss. To avoid that, a Set of names already written is tracked and every incoming file name is checked against it:

  • rename (default) keeps both. The later entry is disambiguated by inserting a numeric suffix before the extension, keeping any folder prefix: docs/readme.txt becomes docs/readme (1).txt.
  • skip keeps the first occurrence and drops later duplicates.

Either way, nothing is silently overwritten. The count of collisions resolved (and of duplicates skipped) is reported back so the result can state what happened. Shared folder entries are deduplicated separately and never counted as collisions.

Shell

Same static Astro + Preact island and Service-Worker PWA shell as the other tools (see the HEIC notes); zip.js runs deflate in the Web Workers described in the Create ZIP notes.

Implementation & operational notes

One writer, opened once, closed once. All inputs stream into the same ZipWriter; close() is called a single time at the end to resolve the merged Blob. If one input isn’t a readable ZIP, both the reader and the shared writer are closed and a clear error names the offending file, rather than leaving a half-built archive open.

At least two inputs. The merge requires two or more archives; a single file has nothing to merge into and is rejected before any work starts.

Re-packed, not copied through. Because the output is a new archive, entries are re-compressed as they are added. Contents are identical to the sources; the compressed bytes may differ.

Try it / source