single-file-core 1.5.119 → 1.5.120

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -3,7 +3,7 @@
3
3
  **Status: draft.** This document specifies the SingleFile archive, the polyglot file
4
4
  format produced by [SingleFile](https://github.com/gildas-lormeau/SingleFile) when it
5
5
  saves a page as a ZIP archive. It is written against the reference implementation,
6
- [single-file-core](https://github.com/gildas-lormeau/single-file-core) 1.5.108
6
+ [single-file-core](https://github.com/gildas-lormeau/single-file-core) 1.5.119
7
7
  (`processors/compression/`), and every byte-level statement has been verified on
8
8
  generated specimen files.
9
9
 
@@ -103,13 +103,15 @@ Three consequences shape everything below:
103
103
 
104
104
  ### 1.2 Non-goals
105
105
 
106
- - **Forward-only ZIP parsers.** SingleFile archives require central-directory-driven reading (the
107
- entries are preceded by non-ZIP bytes). Parsers that stream local headers from
108
- offset 0 are out of scope (§7).
106
+ - **Forward-only ZIP parsers.** Every archive with a face requires central-directory-driven
107
+ reading, since the entries are then preceded by non-ZIP bytes. The variant with no other
108
+ face is an ordinary ZIP file and streams from offset 0 like any other (§8.1, class A);
109
+ parsers that require that are out of scope for the rest (§7).
109
110
  - **In-place modification by generic ZIP tools.** The face invariants are global:
110
111
  the writer picks each hiding tag only after checking the exact bytes it must hide,
111
- the recovery payload of universal mode contains a checksum of the whole ZIP
112
- region, and the PDF and PNG structures wrap the archive (§5.4). A tool
112
+ the recovery payload of universal mode contains a checksum of the ZIP region
113
+ without its comment-length field, and the PDF and PNG structures wrap the
114
+ archive (§5.4). A tool
113
115
  that adds, removes or recompresses entries invalidates them, and most rewriters drop
114
116
  the prepended and appended regions outright. A generically rewritten file keeps at
115
117
  best its ZIP face. Editing an archive means producing a new one through the writer
@@ -133,12 +135,12 @@ Three consequences shape everything below:
133
135
  | **universal mode** | The variant whose HTML face can extract the archive from the *parsed page text*, the text and comment nodes the HTML parser produced, and therefore needs no access to its own raw bytes. Named "universal" because it works from any location, including the `file:` protocol. |
134
136
  | **wrapper tag** | The HTML construct that hides a binary region from the HTML parser, `<!--`…`-->` by default (§5.1). |
135
137
  | **appended data** | Bytes after the ZIP End Of Central Directory record. Readers tolerate them within the window their EOCD scan already covers: 65557 bytes from the end of the file (the 22-byte record plus the 65535-byte maximum comment length); "the 64 KB window" refers to this. It may be left undeclared or declared as the archive comment; both forms are valid ZIP and readers MUST accept both (§4.2). The recovery payload can be computed before that choice is made because it stops two bytes short of the record, excluding its comment-length field (see *recovered range* below). |
136
- | **ZIP region** | The contiguous byte range holding the archive proper: from the first local file header the ZIP writer emitted through the last byte of the End Of Central Directory record. It spans the `zip-entries`, `pdf-central-record` (when present) and `central-directory · eocd` blocks of §3, and in the HTML variants it is exactly the content of the last wrapper. It does **not** include `pdf-local-header` or the PDF document, which sit earlier in the file. |
138
+ | **ZIP region** | The contiguous byte range holding the archive proper: from the first local file header the ZIP writer emitted through the last byte of the End Of Central Directory record. It spans the `zip-entries`, `pdf-central-record` (when present) and `central-directory · eocd` blocks of §3, and in the HTML variants it is the content of the last wrapper, exactly so on the element rungs and preceded by the `sfz-data` identifier on the comment rung, which the extractor steps over. It does **not** include `pdf-local-header` or the PDF document, which sit earlier in the file. |
137
139
  | **archive** | The *logical* ZIP file: the set of entries the central directory describes, wherever their bytes lie. This is distinct from the ZIP region above, which is a contiguous byte range. Every entry but one has its bytes inside the region; `page.pdf` is the deliberate exception, an entry of the archive whose local header and data sit before the region (§4.2). "Archive" in this document always means the logical file, "ZIP region" always the byte range, and the two differ only in the PDF-with-HTML variants. |
138
140
  | **recovered range** | What the universal extractor reproduces (§4.5): the ZIP region minus its last two bytes, the comment-length field of the End Of Central Directory record. That field is the one part of the record whose value depends on what follows the region, so leaving it out is what lets a writer decide the appended-data form after the recovery payload is final (§4.2). The extractor supplies the two bytes itself, as zeroes — the recovered range carries no comment. |
139
141
  | **reference writer** | `createArchive()` in single-file-core `processors/compression/compression.js`. |
140
142
  | **bootstrap** | The inline script in the HTML face that locates, extracts and displays the archived page. |
141
- | **`sfz` identifiers** | Three byte-level identifiers carry the `sfz` prefix: `data-sfz`, `<sfz-extra-data>` and `sfz-data`. The prefix is inherited from SingleFileZ, the browser extension the format originated in (since merged into SingleFile), and is kept unchanged for compatibility with existing files. They are wire identifiers, not the format's name. Two of the three have a role in the format: `<sfz-extra-data>` is the element carrying the recovery payload, and `sfz-data` is the identifier the universal extractor addresses the ZIP region with — an `id` attribute on the wrapper element, or the first characters of the wrapper comment's data (§4.5). `data-sfz` is a marker the reference writer happens to put on the root element; this document mentions it only where it describes bytes those files contain. |
143
+ | **`sfz` identifiers** | Three byte-level identifiers carrying the `sfz` prefix matter to this document: `data-sfz`, `<sfz-extra-data>` and `sfz-data`. A conforming file may hold others the format says nothing about — the reference bootstrap gives its own status messages `sfz`-prefixed ids, which no reader has any reason to look for. The prefix is inherited from SingleFileZ, the browser extension the format originated in (since merged into SingleFile), and is kept unchanged for compatibility with existing files. They are wire identifiers, not the format's name. Two of the three have a role in the format: `<sfz-extra-data>` is the element carrying the recovery payload, and `sfz-data` is the identifier the universal extractor addresses the ZIP region with — an `id` attribute on the wrapper element, or the first characters of the wrapper comment's data (§4.5). `data-sfz` is a marker the reference writer happens to put on the root element; this document mentions it only where it describes bytes those files contain. |
142
144
 
143
145
  ## 2. Variants: composing faces
144
146
 
@@ -164,8 +166,9 @@ include it because the clients producing those variants enable it by default, bu
164
166
  `embeddedImage` and `embeddedPdf` compose with a non-universal self-extracting file
165
167
  just as well; the extension is then `.zip.html`. The one interaction: `page.pdf` lies
166
168
  outside the ZIP region (§1.3), so the page-text extraction path does not recover it
167
- with the rest and the extractor skips that entry (§4.5); every path that reads raw
168
- bytes sees it normally. In the last three rows there is no
169
+ with the rest. The extractor therefore filters that entry out unconditionally, on
170
+ every acquisition path including the ones that read raw bytes and could return it
171
+ (§4.5). In the last three rows there is no
169
172
  HTML face, so the option does not apply. The *Specimen* column names the measured
170
173
  reference files this document cites; §8 records how to regenerate them.
171
174
 
@@ -267,12 +270,39 @@ or overridden by a server still tends to be decoded the way the extractor expect
267
270
  also keeps the reverse table small, at 27 entries (§5.5).
268
271
 
269
272
  The second part is the extra-data payload, which does **not**
270
- contain the archive: it carries only what the round trip destroys, namely a checksum
271
- and the information needed to restore newline bytes, which the parser normalizes.
273
+ contain the archive: it carries only what the round trip destroys or leaves
274
+ undetermined, namely a checksum, the recovered range's length, and the information
275
+ needed to restore newline bytes, which the parser normalizes.
272
276
  The parser also replaces NUL bytes with U+FFFD; since no byte decodes to U+FFFD under
273
277
  a qualifying encoding, the extractor maps U+FFFD back to NUL unambiguously and the
274
278
  payload needs nothing for it (§5.5).
275
279
 
280
+ The declared charset governs the **whole document**, not only the regions the format
281
+ reasons about. Everything the parser reads is decoded with it, the bootstrap script
282
+ included, and the writer's own code is therefore subject to the same single-byte
283
+ decoding as the page it carries. In universal mode the bootstrap MUST contain no
284
+ character outside printable ASCII.
285
+
286
+ Unlike the `<title>` and the table of contents (§4.6), it cannot be rescued by
287
+ character references. A `<script>` element's content is script data, a tokenizer state
288
+ that does not resolve them: `&#9786;` written there stays seven literal characters and
289
+ reaches the program as seven characters. The escape has to happen one level down, in
290
+ the JavaScript source — `\u263A` rather than `☺`, an escape the language resolves when
291
+ the script is compiled, not one the HTML parser resolves when the file is read. A
292
+ minifier will undo this if allowed to, since printing the shortest form is its default
293
+ and the shortest form of `\u263A` is the literal character. A writer that assembles
294
+ the bootstrap through a minifier MUST configure it to emit ASCII only.
295
+
296
+ The consequence of getting this wrong is worse than the mojibake a raw title produces,
297
+ and that is the reason for the MUST. A garbled title is visible; a garbled string
298
+ inside the extractor is not. A lookup table is the sharpest case. Emitted as literal
299
+ characters, a CP437 table is re-decoded as windows-1252 and grows from 256 entries to
300
+ 508, shifting every lookup past the first 32 by 60 positions — and nothing about the
301
+ page looks wrong, because the damage is confined to names the table decodes. Under
302
+ §5.8 that can be a single entry, and one is enough when it is the entry the extractor
303
+ matches by name. The requirement is on the whole bootstrap rather than on any table
304
+ inside it, because a minifier does not know which strings are load-bearing.
305
+
276
306
  ### 2.2 File name conventions
277
307
 
278
308
  The reference implementation names files by variant: `.zip` (no HTML face),
@@ -309,21 +339,21 @@ face adds, then the regions the PNG face adds.
309
339
 
310
340
  | Region | Producer | Present | Contents |
311
341
  |---|---|---|---|
312
- | `html-prologue` | HTML | HTML face | Doctype, the root element start tag, an optional implementation-defined comment, `<meta charset>`, title, optional head elements (canonical link, `robots` meta, viewport, Content-Security-Policy), minimal CSS, `<body hidden>`, wait/error messages, optional table of contents, optional text body (§4.6). The leading comment, the title, the canonical link and the text body are withheld when a password is set (§5.6). In the plain variant an optional UTF-8 BOM MAY precede the doctype (`includeBOM`); universal and PNG variants never carry one. In the PNG variants the region is split: everything through `<body hidden>` is the data of the `tEXt "PNG"` chunk, while the messages, the optional table of contents and the optional text body follow the `tEXt "ZIP"` chunk header; the doctype and the leading comment are dropped. |
313
- | `bootstrap` | HTML | HTML face | One inline `<script>`: the embedded ZIP reader, the extractor, the display routine, and the content-acquisition logic (§4.1). The wrapper start tag that opens the ZIP region follows it, directly or after a relocated `extra-data`. |
314
- | `<!--` / `-->` | HTML | HTML face | The wrapper tag pair hiding a binary region from the HTML parser — comment tags by default, another pair when the hidden bytes contain `-->` (§5.1). Drawn at each opening and closing position. The close tag is absent when appended data is prevented (`preventAppendedData`, or the `<plaintext>` wrapper which cannot close): no markup follows the archive and the wrapper runs to end-of-file. That does not mean the file ends at the EOCD — the PNG face's tail still follows, inside the wrapper, where it parses as text (§5.1). |
342
+ | `html-prologue` | HTML | HTML face | Doctype, the root element start tag, `<meta charset>`, an optional implementation-defined comment, title, optional head elements (canonical link, `robots` meta, viewport, Content-Security-Policy), minimal CSS, `<body hidden>`, wait/error messages, optional table of contents, optional text body (§4.6). The leading comment, the title, the canonical link and the text body are withheld when a password is set (§5.6). In the plain variant an optional UTF-8 BOM MAY precede the doctype (`includeBOM`); universal and PNG variants never carry one. In the PNG variants the region is split: everything through `<body hidden>` is the data of the `tEXt "PNG"` chunk, while the messages, the optional table of contents and the optional text body follow the `tEXt "ZIP"` chunk header; the doctype and the leading comment are dropped. |
343
+ | `bootstrap` | HTML | HTML face | One inline `<script>`: the embedded ZIP reader, the extractor, the display routine, and the content-acquisition logic (§4.1). In universal mode its bytes MUST be pure ASCII, since the declared charset decodes this region like any other and character references do not apply inside script data (§2.1). The wrapper start tag that opens the ZIP region follows it, directly or after a relocated `extra-data`. |
344
+ | `<!--` / `-->` | HTML | HTML face | The wrapper tag pair hiding a binary region from the HTML parser — comment tags by default, another pair when the hidden bytes defeat them — which `-->` is only the commonest way to do, the full test being `<!--`, `--!>` or a trailing `<!-` (§5.1). Drawn at each opening and closing position. The close tag is absent when appended data is prevented (`preventAppendedData`, or the `<plaintext>` wrapper which cannot close): no markup follows the archive and the wrapper runs to end-of-file. That does not mean the file ends at the EOCD — the PNG face's tail still follows, inside the wrapper, where it parses as text (§5.1). |
315
345
  | `zip-entries` | ZIP | always | The archive's local file headers and entry data, written by the ZIP writer. The central directory of an archive written by the reference writer lists `index.html` (the page) first, then `manifest.json` (a JSON description of the archive: original URL, title, save time, resource-to-URL map — informative; the page displays without it), then the resources; the *physical* order of the local headers inside the region is not guaranteed to match, and readers MUST NOT rely on either order — entries are addressed by name (§7.1). |
316
346
  | `central-directory · eocd` | ZIP | always | The central-directory records followed by the End Of Central Directory record. All offsets are absolute file positions (§5.3). In the HTML+PDF variants the EOCD accounts for the injected `pdf-central-record` (how the writer achieves that is §6). |
317
347
  | `extra-data` | extractor | universal | `<sfz-extra-data>` element holding the base64, deflate-compressed recovery payload (§5.5). It always sits outside the wrapper, so it parses as a real element the extractor can address. Normal placement: after the EOCD, between the wrapper close tag and the end tags. Relocated placement, used when the payload exceeds the 64 KB appended-data window or `preventAppendedData` is set: immediately before the wrapper start tag. In the relocated form the element is followed by space padding: its room is reserved before the archive is written, because the region precedes the ZIP data and resizing it would shift every central-directory offset (§6). Neither placement carries positional meaning — the extractor finds the ZIP region by identifier, not relative to this element (§4.5). |
318
348
  | `</body></html>` | HTML | HTML face | The end tags closing the document after the wrapper close tag. Omitted when appended data is prevented, and in the PNG variants so the file can end with the PNG tail. |
319
- | `pdf-local-header` | ZIP | PDF face with HTML | The hand-built local file header for `page.pdf` (STORE, checksum precomputed), written immediately before the PDF document so ZIP readers see an ordinary entry whose data is the PDF (§6). |
320
- | `pdf-document` | PDF | PDF face | The raw PDF bytes. With the HTML face, wrapped together with `pdf-local-header` in a wrapper tag pair inside `html-prologue`, placed so `%PDF-` starts at offset 1024 or lower — the range PDF readers search for the header, which is what lets a PDF document start after other bytes at all (§4.3). Without the HTML face the file simply *starts* with the PDF document, as prepended data the ZIP face tolerates; `page.pdf` is then not an archive entry at all — no local header, no central record. |
349
+ | `pdf-local-header` | ZIP | PDF face with HTML | The hand-built local file header for `page.pdf` (STORE, checksum precomputed, language encoding flag set as on every other entry — §5.8), written immediately before the PDF document so ZIP readers see an ordinary entry whose data is the PDF (§6). |
350
+ | `pdf-document` | PDF | PDF face | The raw PDF bytes. With the HTML face, wrapped together with `pdf-local-header` in a wrapper tag pair inside `html-prologue`, placed so `%PDF-` starts at offset 1024 or lower — the range PDF readers search for the header, which is what lets a PDF document start after other bytes at all (§4.3). Without the HTML face and without the PNG face, the file simply *starts* with the PDF document, as prepended data the ZIP face tolerates; `page.pdf` is then not an archive entry at all — no local header, no central record. |
321
351
  | `pdf-central-record` | ZIP | PDF face with HTML | The central-directory record for `page.pdf`, injected *before* the writer's own central directory. The start of the central directory is the one place a record can be added without moving any offset the writer already committed, and it makes `page.pdf` the first entry ZIP tools list (§6). |
322
352
  | `png-signature · IHDR` | PNG | PNG face | The 8-byte PNG signature and the `IHDR` chunk declaring the source image's dimensions — the first 33 bytes of the file. |
323
353
  | `tEXt "PNG"` | PNG | PNG face with HTML | The length, type and keyword bytes of the first `tEXt` chunk. Its data is `html-prologue` (with the PDF face, the embedded PDF document rides inside it too), ending with the wrapper start tag. |
324
354
  | `tEXt "PDF"` | PNG | PNG + PDF faces without HTML | The length, type and keyword bytes of a `tEXt` chunk whose data is the raw PDF document. Written only when the PNG and PDF faces combine without HTML — with the HTML face the PDF rides inside `tEXt "PNG"` instead — and placed right after `IHDR` so `%PDF-` stays within the header scan window (§4.3). |
325
355
  | `pixel-data chunks` | PNG | PNG face | The source image's image-data chunks, copied unmodified. With the HTML face they sit inside the wrapper so the HTML parser skips them. |
326
- | `tEXt "ZIP"` | PNG | PNG face | The length, type and keyword bytes of the second `tEXt` chunk. Its declared length covers everything from there up to but not including the trailing chunk CRC, as a PNG chunk length always does, so the PNG decoder skips the archive — and, with the HTML face, the bootstrap and the appended data — as the data of one chunk. With the HTML face, the wrapper opened at the end of `tEXt "PNG"` closes immediately after these bytes: its content is the first chunk's CRC, the pixel-data chunks and this chunk's own header, and the prologue resumes as markup directly after the close tag. |
356
+ | `tEXt "ZIP"` | PNG | PNG face | The length, type and keyword bytes of the archive's own `tEXt` chunk — the second one when the HTML face or the PDF face put a chunk ahead of it, the only one otherwise. Its declared length covers everything from there up to but not including the trailing chunk CRC, as a PNG chunk length always does, so the PNG decoder skips the archive — and, with the HTML face, the bootstrap and the appended data — as the data of one chunk. With the HTML face, the wrapper opened at the end of `tEXt "PNG"` closes immediately after these bytes: its content is the first chunk's CRC, the pixel-data chunks and this chunk's own header, and the prologue resumes as markup directly after the close tag. |
327
357
  | `crc · IEND` | PNG | PNG face | The `tEXt "ZIP"` chunk's CRC, computed once the archive bytes are final (§6), followed by the empty `IEND` chunk — the last bytes of the file (PNG requires `IEND` to end the stream, which is why the PNG variants drop the end tags). |
328
358
 
329
359
  The reader-by-reader interpretation of these regions is §4; the mechanics that keep
@@ -381,8 +411,8 @@ The bootstrap script runs at parse time and proceeds in three stages:
381
411
  range reading, fetching only the central directory and the entries it needs (a
382
412
  large archive displays without downloading the ZIP region in full); otherwise it
383
413
  downloads the whole file. When the header probe fails it falls back to page-text
384
- extraction; a failure of the full download itself, past the probe, reveals the
385
- error message directly. Only when every applicable rung fails does the error
414
+ extraction; so does a failure of the full download itself, past the probe, which
415
+ is why the probe leaves the document in place. Only when every applicable rung fails does the error
386
416
  message appear, with recovery instructions that differ by variant (§2).
387
417
  2. **Extract.** The embedded ZIP reader reads the archive through the ZIP lens
388
418
  (§4.2) and rebuilds the page: text entries are decoded, binary entries become
@@ -461,7 +491,7 @@ both. **Raw** — the EOCD declares a zero-length comment and the trailing bytes
461
491
  simply outside the archive — is the default, because tools print a declared archive
462
492
  comment on ordinary operations (§8.1), and in universal mode that comment is the
463
493
  whole base64 recovery payload. **Declared** — the EOCD's comment length covers every
464
- byte after the record — is what older writers emitted, and it is the only form some
494
+ byte after the record — is the later addition, and it is the only form some
465
495
  readers accept at all: `java.util.zip`, and therefore Android and most JVM tooling,
466
496
  rejects an archive with undeclared trailing bytes outright (§8.1). A writer SHOULD
467
497
  offer both and default to raw.
@@ -495,9 +525,10 @@ compatibility appendix records real-world support (§8):
495
525
  the objects themselves.
496
526
 
497
527
  With the HTML face, the PDF document sits inside the head, wrapped in its own
498
- wrapper-tag pair chosen against the PDF's bytes (a PDF containing `-->` steps the
499
- ladder, §5.1), and preceded by the `page.pdf` local header so the same bytes are also
500
- a ZIP entry. That dual role is why the entry MUST be STOREd and MUST NOT be
528
+ wrapper-tag pair chosen against the local header and the PDF together (§5.1) — a PDF
529
+ containing `-->` steps the ladder, and so can the header, whose CRC-32 and size fields
530
+ hold arbitrary bytes — and preceded by the `page.pdf` local header so the same bytes
531
+ are also a ZIP entry. That dual role is why the entry MUST be STOREd and MUST NOT be
501
532
  encrypted: a viewer reads the entry's data region directly, and any transformation
502
533
  of it would break the face. Without the HTML face the document needs no wrapper, and
503
534
  where it sits depends on the PNG face: alone with the ZIP face it simply starts the
@@ -525,9 +556,12 @@ chunks (ancillary by construction, their type starting lowercase):
525
556
  ZIP region and the appended data. The decoder hops over all of it as the data of
526
557
  one chunk.
527
558
 
528
- Both chunks carry text the PNG standard does not strictly permit: a `tEXt` text
529
- string is Latin-1 text, and the payloads here contain NUL bytes — 78 in the
530
- `tEXt "ZIP"` chunk of the `png` specimen, 101 in the `png-pdf` one. Decoders skip
559
+ The `tEXt "ZIP"` chunk carries text the PNG standard does not strictly permit: a
560
+ `tEXt` text string is Latin-1 text, and its payload contains NUL bytes — 78 in the
561
+ `png` specimen, 101 in the `png-pdf` one. The first chunk is pure printable ASCII in
562
+ the plain `png` specimen, where the doctype and the provenance comment are suppressed
563
+ and the title is escaped to character references; only the `-pdf` variants put NULs
564
+ in it. Decoders skip
531
565
  ancillary chunks without inspecting their text, so this passes everywhere tested
532
566
  (§8.1). It exercises the PNG tolerance §1.1 lists at its limit: what decoders ignore
533
567
  in a `tEXt` chunk is text PNG does not permit.
@@ -635,7 +669,10 @@ entry out on every acquisition path, including the ones that read raw bytes and
635
669
  return it.
636
670
 
637
671
  The extractor MUST verify the three checkable payload fields — byte length, newline
638
- count and checksum — and fail to the error message on any mismatch.
672
+ count and checksum — and fail to the error message on any mismatch. It MUST also fail
673
+ on the unassigned newline code of step 2 rather than decode it, so that a payload
674
+ written against a later revision of the format is named as unsupported instead of
675
+ silently reconstructing the wrong bytes.
639
676
 
640
677
  The recovered region is a complete archive but **not an offset-self-contained one**.
641
678
  Its offsets are still absolute positions in the original file (§5.3), so every
@@ -649,7 +686,11 @@ zipfile`, adds `(attempting to process anyway)` and lists both entries, while re
649
686
  that compensate silently, such as Python's `zipfile`, show no diagnostic at all. The
650
687
  shift also puts `page.pdf` out of the
651
688
  offset-following path: its local header lies *before* the region, so its compensated
652
- offset is negative (−102092 in the `pdf` specimen) and no reader can seek to it. On
689
+ offset is negative and no reader can seek to it. That offset is the header's own
690
+ position — which §4.3 keeps inside the first 1024 bytes — minus the region's start,
691
+ which lies past the whole bootstrap, so it is negative for every archive and its
692
+ magnitude is essentially the region's start, so it moves with the size of the inlined
693
+ ZIP library and no particular value should be read into it. On
653
694
  success the shifted bytes enter the normal extraction path (§4.2).
654
695
 
655
696
  The shift is a file offset, and a universal-mode reader has no file. It does not need
@@ -665,10 +706,13 @@ where `eocdPosition` is the EOCD record's own offset within `region`, found by s
665
706
  backward for its signature the way any ZIP reader finds it. A reader arrives at the
666
707
  same number as a ZIP library's prepended-data compensation, which derives it from the
667
708
  record's position rather than from the buffer's end. Do not substitute
668
- `region.length - 22` for `eocdPosition`: the two are equal only when the EOCD is the
669
- last record in the region, which zip64 and a non-empty archive comment both break.
709
+ `region.length - 22` for `eocdPosition`. The two are in fact equal for every recovered
710
+ region, which always ends at the EOCD's last byte and declares a zero-length comment
711
+ (§1.3), but the habit fails the moment the same code is pointed at a file rather than a
712
+ recovered region: there a non-empty archive comment puts bytes after the record. Zip64
713
+ does not — its records precede the EOCD, which stays last.
670
714
 
671
- Under zip64 (§5.4) both of those EOCD fields are the `0xFFFFFFFF` sentinel, and the
715
+ Under zip64 (§5.7) both of those EOCD fields are the `0xFFFFFFFF` sentinel, and the
672
716
  zip64 end of central directory record carries the real values. Take them from there,
673
717
  using the same `eocdPosition` arithmetic against that record's own position — the
674
718
  zip64 locator states an absolute offset in the original file, so it needs the shift
@@ -689,18 +733,46 @@ universal mode this cuts its audience in two: a charset-oblivious tool that read
689
733
  raw bytes — `grep`, plain-text search — sees intact UTF-8, while any consumer that
690
734
  honors the declared `<meta charset>` decodes it as windows-1252 and garbles
691
735
  non-ASCII text. That includes the HTML parser itself — harmless there, because the
692
- region is hidden and replaced (§4.1) — but also HTML-aware indexers. The text body
736
+ region is hidden and replaced (§4.1) — but also HTML-aware indexers. In universal mode the text body
693
737
  opens with the page title, for the same raw-byte
694
738
  audience — it is the first *text* in the element, which is not necessarily the
695
739
  element's first line: the reference writer's serialization puts a newline before it.
696
-
697
- The `<title>` element takes the opposite route. Its content is RCDATA, where
698
- character references are resolved against Unicode independently of the declared
699
- encoding, so the writer emits every character outside printable ASCII — and `&`,
700
- `<`, `>` — as a numeric reference. The element's bytes are therefore pure ASCII and
701
- the title survives the single-byte declaration intact: a page titled 日本語 shows as
702
- 日本語 in the browser tab and to any conforming parser. Writers that emit the title
703
- raw MUST NOT do so in universal mode, where the same bytes decode as mojibake.
740
+ Outside universal mode the title is not repeated there, since the prologue's own
741
+ `<title>` is already readable as bytes.
742
+
743
+ The `<title>` element takes the opposite route, and so does every other piece of
744
+ prologue text the writer assembles itself. Character references are resolved against
745
+ Unicode independently of the declared encoding — in RCDATA, where the title's content
746
+ sits, in ordinary element text, and in attribute values alike — so the writer emits
747
+ every character outside printable ASCII, along with `&`, `<`, `>` and `"`, as a
748
+ numeric reference. Those bytes are therefore pure ASCII and the text survives the
749
+ single-byte declaration intact: a page titled 日本語 shows as 日本語 in the browser tab
750
+ and to any conforming parser. The reference writer passes the title and the multi-page
751
+ table of contents — its link text and its `href` values, which is what the `"` is for —
752
+ through one shared escaper. Writers that emit such text raw MUST NOT do so in
753
+ universal mode, where the same bytes decode as mojibake.
754
+
755
+ The text body above is the deliberate exception, not an oversight: it is left as raw
756
+ UTF-8 because its audience reads bytes rather than parsed text. The bootstrap script
757
+ is the other region outside this rule, and it is outside for a harder reason — script
758
+ data does not resolve character references at all, so the escape must happen in the
759
+ JavaScript source instead (§2.1).
760
+
761
+ An implementation-defined comment (§3.1) is the third region outside the rule, and the
762
+ only one with no escape available at all. Comment data does not resolve character
763
+ references either, and unlike script data it has no second language of its own to
764
+ escape in: `&#233;` written in a comment stays `&#233;` in every reader, so the escaper
765
+ does not restore the character, it replaces one unreadable form with another. The
766
+ reference writer therefore leaves the comment alone and serializes it as UTF-8 with
767
+ the rest of the prologue, deliberately. Its audience is whoever opens the raw file in an
768
+ editor or runs a text tool over it, and those decode the bytes as UTF-8 whatever the
769
+ declaration says; only a browser's raw view, which honors the declared charset, shows
770
+ the text as mojibake in universal mode. The bytes are harmless to extraction, since
771
+ the recovery payload covers the ZIP region alone, far past the prologue. A reader MUST
772
+ NOT rely on decoding this comment through the declared charset, and a writer that
773
+ wants it readable everywhere restricts it to printable ASCII. The page's own copy of
774
+ the comment, inside `index.html`, is UTF-8 in a UTF-8 document and is the one the
775
+ displayed page and the infobar carry.
704
776
 
705
777
  ## 5. Cross-cutting mechanics
706
778
 
@@ -736,12 +808,11 @@ case-insensitively: `</XMP>` and `</Script ` close their elements just as `</xmp
736
808
  as the end ones. A stored, uncompressed resource is the realistic source of an
737
809
  upper-case one.
738
810
 
739
- Every rung hides its content unconditionally. `<noscript>` has the right terminator
740
- and was a rung until core 1.5.108, but it is the one construct whose content is raw
741
- text only while scripting is enabled and markup when it is not, so on a page opened
742
- without scripting the archive bytes would reach the tree builder as tags. The rungs
743
- below it hide the same payloads at no extra cost, so it was removed rather than
744
- demoted.
811
+ Every rung hides its content unconditionally, which is why `<noscript>` is not one. It
812
+ has the right terminator, but it is the one construct whose content is raw text only
813
+ while scripting is enabled and markup when it is not, so on a page opened without
814
+ scripting the archive bytes would reach the tree builder as tags. The rungs below it
815
+ hide the same payloads at no extra cost, so there is nothing to weigh against that.
745
816
 
746
817
  The order under the comment is not arbitrary. Every rung hides its content from an HTML
747
818
  parser, but text extractors differ, and the ZIP region is large enough that the
@@ -786,7 +857,7 @@ matching, indistinguishably from the same archive on the comment rung. The same
786
857
  covered a payload holding every byte value, every rung's patterns, and the near-misses
787
858
  `]]x>`, `] ]>`, `]>` and `]]`, in both the prologue position and mid-document.
788
859
 
789
- Two things about the ladder *are* required. Whatever order a writer gives the seven
860
+ Two things about the ladder *are* required. Whatever order a writer gives the eight
790
861
  closable rungs, it MUST apply the selection test below to every rung it considers, and
791
862
  MUST keep `<plaintext>` available as the rung of last resort: §6.2's termination
792
863
  argument needs one rung no payload can defeat.
@@ -820,16 +891,13 @@ what differs is how far the ladder goes:
820
891
  data double escaped*, where `</script>` does **not** close the element. So a payload
821
892
  can hold `<!--` and then `<script`, contain no `</script` anywhere, pass the end test —
822
893
  and the wrapper then swallows its own end tag, the extra-data element and the rest of
823
- the document. On the other six the start test is genuine conservatism: a nested `<!--`
894
+ the document. On the other seven the start test is genuine conservatism: a nested `<!--`
824
895
  is a parse error inside a comment but does not close it, and the raw-text rungs hold a
825
896
  flat run of characters with no states at all, while CDATA sections do not nest. The
826
897
  rule is uniform deliberately: the
827
898
  exemption would save one pattern match per rung on bytes already in memory, at the
828
899
  cost of a special case an implementer has to remember correctly about the single rung
829
- where forgetting it destroys the document. An earlier draft offered exactly that
830
- latitude, in the broader form "a writer MAY skip the start test outside universal
831
- mode", and the reference writer's PDF and PNG faces took it and shipped the bug
832
- (§8.5).
900
+ where forgetting it destroys the document.
833
901
  - **The PDF and PNG payloads** apply the same two tests, for the same reason — a face
834
902
  that took the `<script>` rung on a payload holding `<!--` and `<script` would swallow
835
903
  the rest of the document, title, bootstrap and extra-data element included — but the
@@ -902,7 +970,8 @@ and the writer compares its total against 65535 before committing to it. The EOC
902
970
  record's own 22 bytes sit inside the window too, giving the 65557-byte figure of §1.3.
903
971
 
904
972
  Only the extra-data element can outgrow the budget: it carries one 2-bit code per
905
- newline sequence in the ZIP region — CR LF counts once, for two bytes (§5.5) — so it
973
+ newline sequence in the recovered range — the ZIP region without its comment-length
974
+ field (§4.5), CR LF counting once, for two bytes (§5.5) — so it
906
975
  grows with the archive. Newline bytes
907
976
  occur at their natural density in compressed and STOREd binary data — about two in
908
977
  every 256 bytes — and the codes are compressed and base64-encoded, which measures at
@@ -921,7 +990,12 @@ element moves in front of the wrapper start tag, ahead of the archive (§3.1). R
921
990
  for it MUST be reserved before the ZIP region is written, because inserting bytes
922
991
  ahead of the archive would shift every offset the ZIP writer has already committed;
923
992
  the reservation is padded with spaces and the real payload is written into it once
924
- its final size is known (§6).
993
+ its final size is known (§6). Relocation is final for the build, and a relocated
994
+ archive carries no appended run at all: the writer emits neither the wrapper's
995
+ terminator nor the end tags, so the file ends at the EOCD record like a plain ZIP file
996
+ and the readers that reject trailing bytes open it (§8.1). The parser closes the open
997
+ comment or element at end of file, and `</body></html>` are implied, so the page
998
+ renders the same.
925
999
 
926
1000
  ### 5.3 Offset bookkeeping
927
1001
 
@@ -938,9 +1012,12 @@ self-consistent:
938
1012
  interpreted from the `%PDF-` header, so embedding it needs no rewriting; the writer
939
1013
  only MUST keep the header inside the scan window (§4.3).
940
1014
  - **PNG has no offsets, only lengths.** Each chunk declares its data length. The
941
- second `tEXt` chunk's length covers the whole archive and the appended data, so it
942
- can only be written once the file's final size is known, and the writer patches it
943
- in place at the end (§6).
1015
+ `tEXt "ZIP"` chunk's length covers the whole archive, and the appended data too in
1016
+ the variants that have it, so it can only be written once the file's final size is
1017
+ known, and the writer patches it in place at the end (§6). A `tEXt` chunk precedes it
1018
+ whenever the HTML face or the PDF face is present — carrying the prologue or the PDF
1019
+ document respectively — and under the PNG face alone it is the only one. Appended
1020
+ data follows it only under the HTML face.
944
1021
 
945
1022
  The injected `page.pdf` central record exploits a fourth, deliberate discrepancy. It
946
1023
  is written directly to the output stream, bypassing the ZIP writer's own byte
@@ -1068,25 +1145,28 @@ ZIP tool can verify.
1068
1145
 
1069
1146
  ### 5.7 zip64
1070
1147
 
1071
- The archive uses the zip64 structures whenever the ordinary records cannot express
1072
- it: a central directory starting beyond 4 GiB (the prefix counts toward the offset,
1073
- §5.3), a directory 4 GiB or longer, or 65535 entries or more. The reference writer
1074
- never requests zip64 explicitly, so it appears only when reached, and given how large
1075
- that is, effectively never in a saved page.
1148
+ The archive uses the zip64 end of central directory structures whenever the ordinary
1149
+ records cannot express it: a central directory starting beyond 4 GiB (the prefix counts
1150
+ toward the offset, §5.3), a directory 4 GiB or longer, or 65535 entries or more. A
1151
+ single entry of 4 GiB or more also produces zip64 extra fields, in that entry's local
1152
+ and central headers, without any zip64 end of central directory record. The reference
1153
+ writer never requests zip64 explicitly, so it appears only when reached, and given how
1154
+ large that is, effectively never in a saved page.
1076
1155
 
1077
1156
  When it is reached, the EOCD record carries the sentinel values `0xFFFF` and
1078
- `0xFFFFFFFF` in the fields that overflowed, preceded by a zip64 end of central
1079
- directory record and its locator. The `page.pdf` record injection then applies its
1080
- accounting to the zip64 record instead — entry counts and directory size there, and
1157
+ `0xFFFFFFFF`, preceded by a zip64 end of central directory record and its locator.
1158
+ The sentinels are not selective: the writer saturates the entry counts, the directory
1159
+ size and the directory offset together once zip64 is emitted, whichever one of them
1160
+ overflowed. The `page.pdf` record injection then applies its accounting to the zip64
1161
+ record instead — entry counts and directory size there, and
1081
1162
  the locator's pointer moved by the record's length — while leaving each saturated
1082
1163
  field at its sentinel. A writer MUST NOT let the injection push a 16-bit or 32-bit
1083
1164
  field to its sentinel value without emitting the corresponding zip64 record: a count
1084
1165
  of `0xFFFF` sends readers looking for a zip64 record that does not exist.
1085
1166
 
1086
- This combination has been verified on a forced-zip64 build (§8): `page.pdf` is listed
1087
- first by both Info-ZIP and the reference reader, the central directory offset in the
1088
- zip64 record points at the injected record, and extraction produces the same page as
1089
- the non-zip64 build.
1167
+ `test/sfz-harness/zip64.js` covers this: the sentinels stay, the counts and the
1168
+ directory size land in the zip64 record, its directory offset points at the injected
1169
+ record, `page.pdf` is the first record in the directory, and a reader lists every entry.
1090
1170
 
1091
1171
  zip64 does not conflict with universal mode. Its commonest trigger, 65535 entries or
1092
1172
  more, is reached at any archive size, and §4.5 gives the offset arithmetic for a
@@ -1094,6 +1174,40 @@ recovered region whose EOCD fields are sentinels. What universal mode cannot car
1094
1174
  a ZIP region of 2^32 bytes or more, which the recovery payload's 32-bit length field
1095
1175
  cannot express (§5.5) — a size bound, not a zip64 one.
1096
1176
 
1177
+ ### 5.8 Entry name encoding
1178
+
1179
+ Entry names in this format are arbitrary Unicode, and how a name is decoded is an
1180
+ interoperability question rather than a detail.
1181
+
1182
+ The reference writer never exercises that range. Its names are a fixed prefix, an
1183
+ index and an extension — `index.html`, `manifest.json`, `stylesheet_0.css`,
1184
+ `images/1.png`, `fonts/2.woff2`, `scripts/3.js`, `frames/4/`, `page.pdf` — and the
1185
+ extension comes either from a table of content types or from a URL pathname, which is
1186
+ percent-encoded. Every name it writes is therefore ASCII, whatever the language of the
1187
+ captured page. That is a property of this writer, not a guarantee of the format: a
1188
+ conforming writer may name entries after the resources themselves, and §7.3's rule
1189
+ that entry names are untrusted assumes one does.
1190
+
1191
+ How a name is encoded is ZIP's own business, not this format's: bit 11 of the general
1192
+ purpose bit flag selects UTF-8, and its absence selects the legacy code page. This
1193
+ document adds two requirements to that and specifies nothing else about it.
1194
+
1195
+ **A writer MUST set bit 11 on every entry**, not only on the entries whose names need
1196
+ it. The two encodings agree over printable ASCII, so setting it unconditionally costs
1197
+ nothing, and it means no name in the archive is decoded through the legacy path at all.
1198
+
1199
+ **A reader MUST honor the flag** rather than assume one encoding, and MUST expect to
1200
+ meet a clear one: the hand-built `page.pdf` records (§3.1, §6) are the only ones the
1201
+ reference writer does not produce through its ZIP writer, and an archive may carry them
1202
+ with no flag set at all. That single entry is then decoded as legacy while every other
1203
+ name in the same file is UTF-8.
1204
+ Its name is ASCII, where the two encodings agree, so a correct reader sees `page.pdf`
1205
+ either way — but a reader that hardcodes UTF-8 on the strength of the other entries has
1206
+ not covered it.
1207
+
1208
+ A name is not a path. §7.3's rule that entry names are untrusted applies to the decoded
1209
+ name, and decoding is the step before that check, not a substitute for it.
1210
+
1097
1211
  ## 6. Writer algorithm
1098
1212
 
1099
1213
  This section specifies the reference writer's build order. It is normative in the
@@ -1111,7 +1225,8 @@ buys. The cost is not evenly spread:
1111
1225
  | Plus universal mode | The recovery payload, the character round trip of §5.5, the appended-data budget of §5.2, and the retry loops of §6.2. This is where the real complexity lives, and it buys opening the file from `file:` with no cooperation |
1112
1226
  | Plus the PDF or PNG face | The header window of §4.3 or the chunk patching of §5.3, plus a second wrapper choice for the embedded payload |
1113
1227
 
1114
- Only the third row needs the retry loops, and only the fourth needs a value that cannot
1228
+ The second row already needs a rebuild when the rung changes; the third adds the rest of
1229
+ the retry loops, and only the fourth needs a value that cannot
1115
1230
  be computed until the file is otherwise complete. A writer that only wants durable saved
1116
1231
  pages can stop at the first row; the files it produces are accepted by every reader in
1117
1232
  §8.1.
@@ -1123,15 +1238,46 @@ pages can stop at the first row; the files it produces are accepted by every rea
1123
1238
  `tEXt "PDF"` chunk holding the PDF document here, so its header falls inside the
1124
1239
  PDF scan window (§4.3).
1125
1240
  2. **HTML prologue.** With the HTML face, emit the doctype (omitted under the PNG
1126
- face, which owns the start of the file), the root element start tag, any comment the
1127
- implementation adds, the `<meta charset>` required by §2.1, the head elements (the
1241
+ face, which owns the start of the file), the root element start tag, the
1242
+ `<meta charset>` required by §2.1, any comment the implementation adds — after the
1243
+ charset declaration, since a comment carrying the page URL has no bound and would
1244
+ otherwise push that declaration out of the first 1024 bytes. The doctype is the
1245
+ other unbounded region ahead of the declaration, copied from the saved page with its
1246
+ identifiers verbatim, so a writer MUST emit a minimal doctype in its place when
1247
+ keeping it would push the declaration past 1024 bytes.
1248
+
1249
+ **Replace it; do not truncate it, and do not drop it.** Truncation is unsafe:
1250
+ a cut inside a quoted identifier leaves the tokenizer in the system-identifier
1251
+ state, where it consumes the markup that follows until the next `>` — swallowing
1252
+ the root element start tag and the `data-sfz` marker on it (§1.3), so the document
1253
+ loses both. Dropping the doctype parses
1254
+ cleanly but puts the document in quirks mode, which is the mode the blank-page
1255
+ backstop, the wait message and the error message are then rendered under (§4.1) —
1256
+ the error message most of all, since it is what a reader sees precisely when
1257
+ nothing else has worked. A minimal doctype is 15 bytes, keeps standards mode, and
1258
+ costs nothing else: the extracted page is written into the document with its own
1259
+ doctype (§4.1), so the outer one never governs the restored page.
1260
+
1261
+ The PNG face is the exception, and nothing is available to it either way. A PNG file
1262
+ MUST begin with its 8-byte signature, so no doctype can precede it, and one written
1263
+ after the PNG head is discarded: those bytes are character data, so the parser has
1264
+ left its initial insertion mode and ignores a DOCTYPE token. The variant renders in
1265
+ quirks mode until the extracted page replaces it, whatever the writer does, so this
1266
+ section requires nothing about the doctype there. A writer MAY drop it, as the
1267
+ reference writer does, or keep it — but a kept one is content like any other, and
1268
+ both windows are measured from the start of the *file*, which under this face begins
1269
+ 45 bytes before the HTML does: the signature, `IHDR`, and the chunk length, type and
1270
+ keyword. That is 45 bytes less room than the arithmetic above suggests.
1271
+
1272
+ Then the head elements (the
1128
1273
  `<title>` and the canonical link among them), the CSS and `<body hidden>`,
1129
1274
  the wait and error messages, the optional table of contents and text body, and the
1130
1275
  bootstrap script. With a password, five of those are left out: the comment, the
1131
1276
  title, the canonical link, the text body and the entry comments of step 6 (§5.6).
1132
1277
  With the PNG face the head of this region,
1133
1278
  through `<body hidden>`, is the data of the `tEXt "PNG"` chunk and the remainder is
1134
- emitted after the `tEXt "ZIP"` chunk header in step 12; with the PDF face the
1279
+ emitted after the `tEXt "ZIP"` chunk header, which step 1 has already written and
1280
+ step 12 only patches; with the PDF face the
1135
1281
  region is interrupted by step 3 as well.
1136
1282
 
1137
1283
  Whatever a writer puts in the prologue, closing every element it opens before the
@@ -1141,7 +1287,7 @@ pages can stop at the first row; the files it produces are accepted by every rea
1141
1287
  3. **Embedded PDF.** With the PDF face and the HTML face, the prologue is *split*
1142
1288
  around the PDF, which MUST come early enough for `%PDF-` to start at offset 1024
1143
1289
  or lower (§4.3). Only what a parser needs first precedes it — the doctype, the
1144
- root element, any leading comment and the charset declaration — and everything
1290
+ root element and the charset declaration — and everything
1145
1291
  else in the head (title, link and meta elements, the stylesheet, `<body hidden>`,
1146
1292
  the messages, the optional table of contents and text body) follows it. Emit the
1147
1293
  wrapper start tag chosen for the PDF payload (§5.1), the hand-built `page.pdf`
@@ -1152,26 +1298,29 @@ pages can stop at the first row; the files it produces are accepted by every rea
1152
1298
  where the format depends on the writer rather than on its own layout. The
1153
1299
  irreducible part of the prefix is small: the root element start tag, the charset
1154
1300
  declaration, the wrapper start tag and the 38-byte local file header for
1155
- `page.pdf`, plus a minimal doctype — 100 bytes in the reference layout, and a few
1156
- more with a longer charset label or a wrapper past the first rung (§5.1). But two
1301
+ `page.pdf`, plus a minimal doctype — 92 bytes in the reference layout with a
1302
+ `utf-8` label, 99 with `windows-1252`, and a few more with a wrapper past the first
1303
+ rung (§5.1). But two
1157
1304
  regions ahead of the header have no length the format controls: the doctype, which
1158
1305
  is copied from the saved page and carries its public and system identifiers
1159
1306
  verbatim, and any comment the implementation chooses to write there. Real doctypes
1160
1307
  are small; the longest in common use, XHTML 1.1 with MathML and SVG, is about 140
1161
- bytes. But nothing caps either region, so a writer MUST cap them itself, keeping
1162
- everything before the local file header inside the remaining budget of roughly 924
1163
- bytes (879 with the PNG face, whose signature, `IHDR` and first chunk header take
1164
- the first 45 bytes of the same window, and whose variant drops the doctype in
1165
- exchange), shortening, dropping or relocating that content instead of emitting a
1308
+ bytes, though a crafted one is bounded only by what the parser accepts. Step 2's
1309
+ MUST already caps the doctype, but only far enough to keep the charset declaration
1310
+ inside the window; this header sits further into the file, behind the wrapper tag
1311
+ and a 38-byte local header, so it needs the tighter bound below and the comment
1312
+ needs one too. A writer MUST cap them itself, keeping
1313
+ everything before the local file header inside the remaining budget of roughly 930
1314
+ bytes (about 900 with the PNG face, whose signature, `IHDR` and first chunk header
1315
+ take the first 45 bytes of the same window while its variant drops the 15-byte
1316
+ doctype in exchange), shortening, dropping or relocating that content instead of emitting a
1166
1317
  header outside the window. A writer that places nothing of unbounded length before
1167
1318
  the PDF block satisfies the rule by construction and needs no check at all.
1168
1319
 
1169
1320
  The reference writer does both. Its provenance comment is emitted after the PDF
1170
1321
  block, so the page URL it carries cannot reach the window at all, and the prefix is
1171
1322
  measured before the header is written: when the page's own doctype would push
1172
- `%PDF-` past 1024, `<!DOCTYPE html>` is emitted in its place. Substituting the
1173
- doctype changes the bootstrap document's rendering mode, which costs nothing here:
1174
- the extracted page is written into the document with its own doctype (§4.1). A
1323
+ `%PDF-` past 1024, `<!DOCTYPE html>` is emitted in its place, on step 2's rule. A
1175
1324
  writer that must keep the page doctype has to find the room elsewhere.
1176
1325
 
1177
1326
  Without the HTML face, the PDF is simply the first thing in the file and the
@@ -1193,16 +1342,20 @@ pages can stop at the first row; the files it produces are accepted by every rea
1193
1342
  8. **Close and patch.** Close the archive, then correct the end of central directory
1194
1343
  record for the injected record: entry counts, directory size, and the zip64
1195
1344
  record and locator when present (§5.7).
1196
- 9. **Universal payload.** In universal mode, read back the ZIP region and check it
1197
- against the current wrapper (§5.1); on a collision, restart (§6.2). Otherwise
1198
- compute the region's CRC-32 and its newline codes, build and compress the payload,
1345
+ 9. **Universal payload.** With the HTML face — not only in universal mode, since any
1346
+ self-extracting file needs it — read back the ZIP region and check it against the
1347
+ current wrapper (§5.1); on a collision, restart (§6.2). Then, in universal mode
1348
+ only, compute the region's CRC-32 and its newline codes, build and compress the payload,
1199
1349
  and decide its placement against the budget (§5.2), restarting if the decision
1200
1350
  differs from the current pass.
1201
- 10. **Appended run.** Unless appended data is prevented, emit the wrapper end tag,
1351
+ 10. **Appended run.** Unless appended data is prevented or the payload is relocated
1352
+ (§5.2), emit the wrapper end tag,
1202
1353
  the extra-data element when it is appended, and `</body></html>` — the end tags
1203
1354
  are omitted under the PNG face, which must end with `IEND`.
1204
1355
  11. **Fill the reservation.** In the relocated placement, write the payload into the
1205
- space reserved in step 4; if it no longer fits, restart (§6.2).
1356
+ space reserved in step 4; if it no longer fits, restart (§6.2). Under
1357
+ `declareAppendedData` (§4.2), the EOCD's comment-length field is patched here too,
1358
+ the appended run's length now being final.
1206
1359
  12. **PNG tail.** With the PNG face, patch the `tEXt "ZIP"` chunk's length field, now
1207
1360
  that the total size is known, compute that chunk's CRC over everything from its
1208
1361
  type to the last byte written, and append the CRC and the `IEND` chunk.
@@ -1210,8 +1363,10 @@ pages can stop at the first row; the files it produces are accepted by every rea
1210
1363
  ### 6.2 The retry loops
1211
1364
 
1212
1365
  Four conditions restart the build from step 1, and each restart carries forward what
1213
- the failed pass learned. The first three terminate because each of them advances a
1214
- monotone quantity:
1366
+ the failed pass learned. Nothing a restart changes reaches the entries' bytes: a writer
1367
+ may compress them once and copy them into every pass, rewriting only the central
1368
+ directory's offsets, which is what the reference writer does. The first three
1369
+ terminate because each of them advances a monotone quantity:
1215
1370
 
1216
1371
  - **Wrapper collision** (§5.1): the next pass starts at the next rung of the ladder.
1217
1372
  The ladder is finite and its last rung, `<plaintext>`, is exempt from both selection
@@ -1220,29 +1375,42 @@ monotone quantity:
1220
1375
  ahead of the archive, sized at the measured payload length plus a margin.
1221
1376
  - **Reservation too small**: relocating the payload changes the file's layout, hence
1222
1377
  its offsets, hence the payload, which can grow past the room reserved for it. The
1223
- next pass reserves the new length plus the same margin. For the loop to terminate,
1224
- each reservation MUST be strictly larger than the payload that sized it: a margin
1225
- that can round down to zero lets two passes measure the same length and reserve the
1226
- same room forever.
1227
-
1228
- The fourth is the converse of the second: a pass that reserved room but then found the
1229
- payload would fit in the appended window discards the reservation and rebuilds without
1230
- it, so the writer does not leave dead padding in the file. This step is not monotone,
1231
- and it is the only one that could keep the build alive forever: dropping the reservation
1232
- moves
1233
- the archive back, which changes the offsets, which changes the payload that made the
1234
- reservation necessary. A payload lying on the 65535-byte boundary can therefore be too
1235
- large appended and small enough relocated, and the build oscillates. A writer MUST
1236
- break that cycle: **the reservation is discarded at most once per build**, and a payload
1237
- that fits the appended window on a later pass stays in the reservation it already has.
1238
- The file then keeps at most the reservation's own margin of dead padding.
1378
+ next pass reserves the new length plus the same margin. This restart fires only
1379
+ when the payload outgrew its reservation and it reserves at least that payload, so
1380
+ every reservation is larger than the one before and the loop cannot revisit a size.
1381
+ What keeps it short is the margin. Shifting the offsets changes a few of the
1382
+ central directory's bytes, which changes the line-ending codes, the deflate output
1383
+ and the base64 rounding, so the payload moves by a few quanta of 4 characters
1384
+ between two layouts: measured between -16 and +20 characters over archives of 8 to
1385
+ 2000 entries. A margin smaller than that shift buys a third pass in about one build
1386
+ out of four. The writer reserves the measured length plus 1 % plus 32 characters,
1387
+ which absorbed every shift measured.
1388
+
1389
+ There is no converse of the second: a pass that reserved room never discards it,
1390
+ even when the relocated payload would have fit the appended window. Relocation moves
1391
+ the archive, which changes the offsets, which changes the payload that made the
1392
+ relocation necessary, so a payload lying on the 65535-byte boundary can be too large
1393
+ appended and small enough relocated, and a writer that dropped the reservation could
1394
+ rebuild the two placements forever. Relocation is therefore final (§5.2), and the file
1395
+ keeps at most the reservation's own margin of dead padding.
1396
+
1397
+ The fourth stands apart from the other three, and terminates trivially because it can
1398
+ fire only once: if the end of central directory record cannot be patched to account for
1399
+ the injected `page.pdf` record — its signature not where the accounting expects it —
1400
+ the writer rebuilds without that record rather than leave a central directory the EOCD
1401
+ does not count. That is the restart enforcing §5.7's requirement that the injection
1402
+ never leave the two disagreeing, and the rebuilt archive simply has no `page.pdf`
1403
+ entry.
1239
1404
 
1240
1405
  Given identical inputs, modification date and archive time, the process is
1241
- deterministic: the same page produces the same bytes, retries included. The archive
1242
- time is a separate input because `manifest.json` records when the archive was made
1243
- (§7.1), so two builds of one page at two moments differ in that entry and in the entry
1244
- sizes around it. A consumer MUST NOT treat the byte identity of two archives of the
1245
- same page as meaningful.
1406
+ deterministic: the same page produces the same bytes, retries included. `manifest.json`
1407
+ records when the archive was made (§7.1), so two builds of one page at two moments
1408
+ differ in that entry and in the entry sizes around it. A writer that retries MUST pin
1409
+ the archive time across the passes of one build rather than read the clock again on
1410
+ each: the reference writer takes it from the clock inside the callback that emits the
1411
+ entries, which runs once per pass, so a retried build is reproducible only when that
1412
+ clock is frozen — which is what its own determinism test does. A consumer MUST NOT
1413
+ treat the byte identity of two archives of the same page as meaningful.
1246
1414
 
1247
1415
  ## 7. Consuming SingleFile archives safely
1248
1416
 
@@ -1254,8 +1422,10 @@ handles every variant of §2 without knowing which one it has.
1254
1422
 
1255
1423
  - **Read through the central directory.** Locate the End Of Central Directory record
1256
1424
  by scanning backward from the end of the file, then follow its offset. A reader that
1257
- streams local headers from offset 0 will not find an archive: the file starts with
1258
- the HTML, PDF or PNG face (§1.2, and §8.1 measures what such readers actually do).
1425
+ streams local headers from offset 0 will not find an archive in any variant that has
1426
+ a face, since the file then starts with the HTML, PDF or PNG face. The variant with
1427
+ no face is an ordinary ZIP file and streams fine (§1.2, and §8.1 measures what such
1428
+ readers actually do).
1259
1429
  - **Tolerate bytes before and after the archive.** They are the other faces, not
1260
1430
  corruption. Offsets are absolute, so no compensation is needed (§5.3).
1261
1431
  - **Accept both forms of appended data.** The bytes after the EOCD record may be raw
@@ -1301,8 +1471,9 @@ handles every variant of §2 without knowing which one it has.
1301
1471
  MUST ignore what it does not recognize.
1302
1472
  - **Expect a `page.pdf` entry whose data lies outside the archive proper** (§4.2). It
1303
1473
  is an ordinary STORE entry at an ordinary offset, so nothing special is needed to
1304
- read it, but a reader that assumes every entry sits between the first local header
1305
- and the central directory will reject or mislocate it.
1474
+ read it, but a reader that assumes the entries are contiguous will reject or
1475
+ mislocate it: `page.pdf`'s local header is the first in the file, and the whole
1476
+ bootstrap lies between its data and the next one.
1306
1477
 
1307
1478
  ### 7.2 Modifying
1308
1479
 
@@ -1326,8 +1497,10 @@ alongside it.
1326
1497
 
1327
1498
  ### 7.3 Security considerations
1328
1499
 
1329
- - **Entry names are untrusted.** They derive from a captured page's resource URLs. A
1330
- reader MUST sanitize them before writing to a filesystem: reject absolute paths and
1500
+ - **Entry names are untrusted.** The reference writer's names are a fixed prefix, an
1501
+ index and an extension (§5.8), but nothing in the format requires that, and a writer
1502
+ may name entries after the resources themselves. A reader MUST sanitize them before
1503
+ writing to a filesystem: reject absolute paths and
1331
1504
  `..` segments, and be aware that names may be long, may collide after case folding,
1332
1505
  and may contain characters the local filesystem rejects.
1333
1506
  - **Declared sizes are untrusted.** Do not pre-allocate from the declared uncompressed
@@ -1338,8 +1511,9 @@ alongside it.
1338
1511
  the bootstrap in a privileged one. The format's own display path replaces the
1339
1512
  document with the extracted page, which is not an isolation boundary by itself.
1340
1513
  - **A password protects entry contents only** (§5.6). Entry names, sizes and dates
1341
- stay readable in the central directory, and a name commonly states the resource's
1342
- filename. The PNG and PDF faces render the page regardless. A conforming writer
1514
+ stay readable in the central directory, and while the reference writer's names carry
1515
+ no information about the resources (§5.8), another writer's may state their
1516
+ filenames. The PNG and PDF faces render the page regardless. A conforming writer
1343
1517
  withholds the five fields of §5.6, the source URLs among them, but a reader MUST NOT
1344
1518
  read their absence as protection: nothing in the format stops a writer from emitting
1345
1519
  any of them, so an archive of unknown provenance may state every URL in the clear.
@@ -1361,7 +1535,7 @@ only if it affects the bytes the page is built from:
1361
1535
  | A recovery payload field disagrees with the reconstruction — length, newline count or checksum | **MUST** fail (§4.5). The reconstruction is wrong and nothing built from it can be trusted |
1362
1536
  | An entry's CRC-32 or AES authentication code does not match | **SHOULD** fail for that entry, and MUST NOT present a page rebuilt from it as intact |
1363
1537
  | `page.pdf` was reconstructed from the parsed page and its CRC-32 does not match | **MUST** discard the reconstruction (§4.5). The bytes are a guess about newlines the recovery payload does not describe, and the checksum is the only thing that tests it — unlike the row above, there is no read to have gone wrong, only an inference |
1364
- | Bytes before the first local file header, or after the EOCD record | **MUST** tolerate: they are the other faces (§7.1) |
1538
+ | Bytes outside the archive proper — before the first local file header, after the EOCD record, or between an entry's data and the next header | **MUST** tolerate: they are the other faces (§7.1). The gap in the middle is not hypothetical: with the PDF face the bootstrap lies between `page.pdf`'s data and the ZIP region |
1365
1539
  | The appended run exceeds the 65535-byte budget (§5.2) | Not a reader's problem: if the EOCD record was found, the archive is readable. Readers MAY warn |
1366
1540
  | A `tEXt` chunk CRC does not match, or a chunk holds bytes PNG does not permit (§4.4) | Irrelevant to extraction; a reader of the archive MAY ignore both |
1367
1541
  | `page.pdf` is present but its data does not begin with `%PDF-` | Not an error. The entry is data like any other |
@@ -1370,9 +1544,12 @@ only if it affects the bytes the page is built from:
1370
1544
  | The recovered region (universal mode) disagrees with the same bytes read directly, in the EOCD's two comment-length bytes only | Expected, not an error. A recovered region always declares a zero-length comment (§4.5), so it differs here from any archive written in the declared form (§4.2). Compare the two only up to those bytes |
1371
1545
  | The recovered region (universal mode) disagrees with the same bytes read directly, anywhere else | The file is not well-formed, whichever side is at fault, and a reader that has both MUST NOT silently merge them or pick per entry. Prefer the direct read — it is the writer's own output, where the recovered region is a reconstruction of it — and surface the disagreement rather than displaying either as intact |
1372
1546
 
1373
- Anything the format does not constrain, a reader MUST NOT reject: entries may carry
1374
- any extra fields, timestamps, data descriptors or name-encoding flags a ZIP writer
1375
- would ordinarily emit, and none of it is specified here.
1547
+ Anything the format does not constrain, a reader MUST NOT reject: entries may carry any
1548
+ extra fields, timestamps or data descriptors a ZIP writer would ordinarily emit. The few
1549
+ this document does constrain — the `0x9901` field of an encrypted entry (§4.2), the
1550
+ zip64 records (§5.7), the name-encoding flag (§5.8) — say how an entry is read, not
1551
+ whether it is acceptable, so this row covers them too: each is something a reader meets
1552
+ and reads.
1376
1553
 
1377
1554
  ## 8. Appendices
1378
1555
 
@@ -1445,15 +1622,25 @@ measured except Apple's `ditto`.
1445
1622
  ### 8.2 Anatomy of a small archive
1446
1623
 
1447
1624
  Offsets in `universal.sfz.html` (123077 bytes, two entries, saved from `example.com`
1448
- with the §8.3 command, against core 1.5.108).
1625
+ with the §8.3 command, on the build named there).
1449
1626
  The layout is the *universal* row of the byte map (§3).
1450
1627
 
1628
+ These numbers are one capture, not a contract. Everything from the bootstrap onward
1629
+ moves whenever the inlined ZIP library changes size, so treat the table as an
1630
+ illustration of the shape and not as values to compare a file against. What *is* fixed
1631
+ is the set of relations between the rows — the doctype opening the file with the root
1632
+ element start tag immediately after it, the charset declaration immediately after that
1633
+ and the comment immediately after that, the identifier's twelve bytes ahead of the
1634
+ region, the EOCD's directory offset being an absolute file position, and the entry
1635
+ order. Those are checked by `test/sfz-harness/byte-map.js`, which builds an equivalent
1636
+ specimen without a network.
1637
+
1451
1638
  | Offset | Bytes | Region |
1452
1639
  |---|---|---|
1453
1640
  | 0 | `<!DOCTYPE html>` | `html-prologue` begins |
1454
- | 16 | `<html data-sfz>` | root element start tag; the attribute is the reference implementation's own marker (§1.3) |
1455
- | 31 | `<meta charset=windows-1252>` | the charset rule, inside the first 1024 bytes (§2.1) |
1456
- | 58 | `<!--` … `-->` (ends at 200) | comment written by the implementation, not part of the format; it follows the charset declaration so it cannot push it out of the prescan window |
1641
+ | 15 | `<html data-sfz>` | root element start tag; the attribute is the reference implementation's own marker (§1.3) |
1642
+ | 30 | `<meta charset=windows-1252>` | the charset rule, inside the first 1024 bytes (§2.1) |
1643
+ | 57 | `<!--` … `-->` (ends at 200) | comment written by the implementation; its content is implementation-defined, but where it may appear is not (§3.1, §4.6, §5.6). It follows the charset declaration so it cannot push it out of the prescan window |
1457
1644
  | 200 | `<title>` … `</title>` (ends at 229) | the page title, as numeric character references (§4.6) |
1458
1645
  | 677 | `<style>` | the stylesheet of the blank-page backstop (§4.1) |
1459
1646
  | 855 | `<body hidden>` | start of the blank-page backstop (§4.1) |
@@ -1479,7 +1666,8 @@ Generated with the command-line client running `single-file-core` against
1479
1666
  results of §8.1 were measured on the 1.5.107 build of the same specimen set, which
1480
1667
  differs only inside the prologue and so falls in the same classes: those are grouped
1481
1668
  by whether bytes precede the archive and follow the EOCD, which no prologue change
1482
- alters. `--compress-content` makes the output an archive; `extract-data-from-page`
1669
+ alters. The declared-form results are the exception, `declareAppendedData` being later
1670
+ than that build (§8.5); they were measured separately on a build that has it. `--compress-content` makes the output an archive; `extract-data-from-page`
1483
1671
  defaults to true there, so the plain variant has to switch it off:
1484
1672
 
1485
1673
  | Specimen | Command |
@@ -1502,15 +1690,13 @@ entry comment, the optional text body or table of contents (§4.6), a UTF-8 BOM,
1502
1690
  in it. Two omissions matter more than the rest, because they are the parts of §5.1 a
1503
1691
  writer is most likely to get wrong: no specimen defeats a rung by its **start**
1504
1692
  pattern, and none defeats one with an **upper-case** pattern. A writer that tested only
1505
- end patterns, or matched them case-sensitively, produces every specimen here unchanged
1506
- — and the first of those two mistakes is one the reference writer actually shipped
1507
- (§8.5).
1693
+ end patterns, or matched them case-sensitively, produces every specimen here unchanged.
1508
1694
 
1509
1695
  Two specimens cannot be produced from a URL alone. The **ladder** specimen, which
1510
1696
  forces the second rung of §5.1, needs a page referencing an image whose stored bytes
1511
1697
  contain `-->`; the archive then wraps in `<script type=sfz-data>`. The **zip64** specimen requires
1512
1698
  an archive past the thresholds of §5.7, so it is produced by calling the writer
1513
- directly with zip64 forced on the ZIP writer.
1699
+ directly with zip64 forced on the ZIP writer, as `test/sfz-harness/zip64.js` does.
1514
1700
 
1515
1701
  The measurements quoted elsewhere in this document come from the same harness: the
1516
1702
  payload growth rate of §5.2 (86 KB → 181 bytes, 283 KB → 465, 1.07 MB → 1645, 4.2 MB
@@ -1535,7 +1721,7 @@ multi-byte ones (`utf-8`, `utf-16le`, `utf-16be`, `gbk`, `gb18030`, `big5`, `euc
1535
1721
  `shift_jis`, `euc-kr`, `iso-2022-jp`) decode a lone byte sequence to U+FFFD or to fewer
1536
1722
  than 256 characters.
1537
1723
  The reverse table each one needs ranges from 8 entries (`iso-8859-15`) to 128
1538
- (`koi8-r`, `koi8-u` and `ibm866`); windows-1252 needs 27.
1724
+ (`koi8-r`, `koi8-u`, `ibm866` and `x-user-defined`); windows-1252 needs 27.
1539
1725
 
1540
1726
  **That the round trip is charset-independent.** The mechanism of §5.5 — parse, then
1541
1727
  re-encode with the reverse table, restoring newlines from the 2-bit codes and NUL from
@@ -1567,6 +1753,8 @@ predicts.
1567
1753
  | August 2026 | Core 1.5.110: a PNG face leaving the comment rung on its checksum resumes the rung search instead of taking the next rung untested (§5.1). Taking it put a payload holding `</script>` on the script rung, where its own bytes closed the wrapper 93 bytes in and left the image data, the chunk framing and the whole ZIP region to the parser |
1568
1754
  | August 2026 | Core 1.5.110: `<svg><![CDATA[` joins the ladder above `<plaintext>` (§5.1) — the one rung whose terminator, `]]>`, real payloads rarely carry. It gives a payload naming every element rung somewhere to go that does not cost the appended-data placement, and moves the self-nesting limit from the fifth level to the sixth |
1569
1755
  | August 2026 | Core 1.5.115: password-protected archives withhold the provenance comment and the canonical link as well (§5.6). Both wrote the page's own URL into the prologue, beside the title that was already withheld, so the address the archive was saved from stayed in the clear |
1756
+ | August 2026 | Core 1.5.119: the inlined ZIP library is built ASCII-only, and §2.1 now requires it of any bootstrap. Its CP437 table had been emitted as literal characters, which the page re-decoded as windows-1252, growing the table from 256 entries to 508 and shifting every lookup by 60 — so the one entry read without the UTF-8 flag, `page.pdf`, came back mangled and no archive with a PDF face extracted in any engine (§5.8) |
1757
+ | August 2026 | Core 1.5.120: the hand-built `page.pdf` records set the language encoding flag, like every entry the ZIP writer produces (§5.8). Its name is ASCII, so no decoded name changes; what changes is that no entry in an archive is read through CP437 any more, closing the path the 1.5.119 defect surfaced on |
1570
1758
 
1571
1759
  This document was itself revised in August 2026, against core 1.5.108, after several
1572
1760
  independent reviews. One of them was a reader built from this specification alone, with