single-file-core 1.5.107 → 1.5.109
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/test.yml +31 -0
- package/doc/assets/singlefile-archive-byte-map.svg +298 -0
- package/doc/assets/singlefile-archive-lenses.svg +245 -0
- package/doc/singlefile-archive.md +1455 -0
- package/package.json +2 -2
- package/processors/compression/compression-extract.js +4 -3
- package/processors/compression/compression-packager.js +4 -1
- package/processors/compression/compression.js +239 -131
- package/single-file.js +1 -0
- package/test/sfz-harness/README.md +58 -0
- package/test/sfz-harness/check-determinism.js +57 -0
- package/test/sfz-harness/common.js +154 -0
- package/test/sfz-harness/dom-stub.js +13 -0
- package/test/sfz-harness/format-rules.js +289 -0
- package/test/sfz-harness/gen-e2e-page.js +14 -0
- package/test/sfz-harness/search-triggers.js +35 -0
- package/test/sfz-harness/smoke.js +10 -0
- package/test/sfz-harness/stored-trigger.js +50 -0
- package/test/sfz-harness/trigger-seeds.json +120 -0
|
@@ -0,0 +1,1455 @@
|
|
|
1
|
+
# The SingleFile archive format
|
|
2
|
+
|
|
3
|
+
**Status: draft.** This document specifies the SingleFile archive, the polyglot file
|
|
4
|
+
format produced by [SingleFile](https://github.com/gildas-lormeau/SingleFile) when it
|
|
5
|
+
saves a page as a ZIP archive. It is written against the reference implementation,
|
|
6
|
+
[single-file-core](https://github.com/gildas-lormeau/single-file-core) 1.5.108
|
|
7
|
+
(`processors/compression/`), and every byte-level statement has been verified on
|
|
8
|
+
generated specimen files.
|
|
9
|
+
|
|
10
|
+
The key words MUST, MUST NOT, SHOULD and MAY are to be interpreted as
|
|
11
|
+
described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119) when, and only when,
|
|
12
|
+
they appear in all capitals.
|
|
13
|
+
|
|
14
|
+
Three kinds of statement appear throughout:
|
|
15
|
+
|
|
16
|
+
- **Format requirements**, in RFC 2119 capitals. Their subject is a writer producing an
|
|
17
|
+
archive, or a *conforming reader of this format* — third-party software that reads
|
|
18
|
+
SingleFile archives (§7). It is never the stock HTML, ZIP, PDF or PNG readers the
|
|
19
|
+
document describes: those cannot be placed under an obligation by this document, and
|
|
20
|
+
a requirement that appears to do so is describing what a writer must produce so that
|
|
21
|
+
their existing behavior lands correctly (§4).
|
|
22
|
+
- **Reference behavior** — what `createArchive()` does where the format allows other
|
|
23
|
+
choices. Always marked as the reference writer's, and never a requirement.
|
|
24
|
+
- **Measurements** — observed behavior of specific third-party software, with the
|
|
25
|
+
version and the specimen it was measured on. §8 collects them; where one appears
|
|
26
|
+
inline it is the evidence for a claim, not a guarantee about software in general.
|
|
27
|
+
|
|
28
|
+
## 1. Introduction
|
|
29
|
+
|
|
30
|
+
A SingleFile archive is **one byte string that is simultaneously a valid document in
|
|
31
|
+
several formats**. Every archive is a valid ZIP file containing the saved page and its
|
|
32
|
+
resources. Depending on the options used to produce it, the same byte string is also:
|
|
33
|
+
|
|
34
|
+
- a valid **HTML page** that extracts and displays the archived page when opened in a
|
|
35
|
+
browser, with no external dependency;
|
|
36
|
+
- a valid **PNG image**, typically a screenshot of the page, though the format does not
|
|
37
|
+
require the image to depict it;
|
|
38
|
+
- a valid **PDF document**, typically a rendering of the page, though the format does
|
|
39
|
+
not require the document to depict it either.
|
|
40
|
+
|
|
41
|
+
Each format's reader accepts the file as a complete document of its own format and
|
|
42
|
+
silently ignores the bytes that belong to the other formats. The word is *accepts*,
|
|
43
|
+
not *conforms*: several faces lean on reader tolerances the target standards do not
|
|
44
|
+
promise (§1.1). The large payloads are stored once and
|
|
45
|
+
shared: the archive entries, the PDF document and the PNG pixel data are single
|
|
46
|
+
regions that several readers reach, not per-face copies. The polyglot works by
|
|
47
|
+
*partitioning* the file into regions and arranging each region so that every reader
|
|
48
|
+
either interprets it or skips it. What the optional features add is never a copy of
|
|
49
|
+
another face's payload: the text body (an optional plain-text copy of the page stored
|
|
50
|
+
in the HTML face for text tools and indexers, §4.6) repeats the page text, and the
|
|
51
|
+
image and PDF a writer supplies are separate documents, whether or not they depict the
|
|
52
|
+
archived page.
|
|
53
|
+
|
|
54
|
+
### 1.1 Design goals
|
|
55
|
+
|
|
56
|
+
The format exists to keep saved pages readable for as long as possible, with as
|
|
57
|
+
little software as possible. Each way of opening the file has a simpler fallback:
|
|
58
|
+
|
|
59
|
+
1. In a JavaScript-capable browser, the file opens and displays the page.
|
|
60
|
+
2. When extraction fails, the file displays an error message with recovery
|
|
61
|
+
instructions. Without JavaScript, it renders as a blank page: the document body
|
|
62
|
+
is hidden by construction, so the browser displays neither the page nor raw
|
|
63
|
+
archive bytes (§4.1).
|
|
64
|
+
3. Renamed to `.zip`, the file opens in a ZIP tool; the page and each resource are
|
|
65
|
+
ordinary entries. The two measured readers that refuse a self-extracting variant are
|
|
66
|
+
named in the ranking below.
|
|
67
|
+
4. Renamed to `.pdf` or `.png` (when those faces are present), the file opens in a PDF
|
|
68
|
+
viewer or an image viewer.
|
|
69
|
+
|
|
70
|
+
Three consequences shape everything below:
|
|
71
|
+
|
|
72
|
+
- **The HTML face depends on the ZIP face.** The HTML bootstrap extracts the page
|
|
73
|
+
*from the ZIP structure of its own file*; the ZIP face is not an export feature, it
|
|
74
|
+
is the storage layer the HTML face reads from.
|
|
75
|
+
- **Readers must need no cooperation.** Every face works with stock, unmodified
|
|
76
|
+
readers. The format relies on two kinds of reader behavior: rules the target
|
|
77
|
+
format actually defines (such as HTML's parsing and error-recovery rules, §4.1),
|
|
78
|
+
and *customary tolerances* — behaviors that are near-universal in practice but
|
|
79
|
+
that no standard promises, such as the backward scan ZIP readers use to find the
|
|
80
|
+
End Of Central Directory record (§4.2, §5.2), the PDF header scan and
|
|
81
|
+
trailing-data tolerance (§4.3), and PNG decoders' indifference to what an ancillary
|
|
82
|
+
`tEXt` chunk contains (§4.4). The compatibility appendix records the customary
|
|
83
|
+
tolerances' real-world support (§8).
|
|
84
|
+
- **The faces are not equally durable.** They rest on different amounts of unpromised
|
|
85
|
+
behavior, and "the file is valid in four formats" is not four guarantees. A decision
|
|
86
|
+
about what to rely on years from now should follow this order:
|
|
87
|
+
1. **The ZIP face** is the one to trust, and the only one the format calls storage.
|
|
88
|
+
A variant with no other face is an ordinary ZIP file. The self-extracting variants
|
|
89
|
+
add prepended and appended bytes, which cost two measured readers: `ditto`, which
|
|
90
|
+
requires a local header at offset 0, and `java.util.zip`, which rejects undeclared
|
|
91
|
+
trailing bytes. A writer can fix the second case by declaring those bytes as the
|
|
92
|
+
archive comment (§4.2).
|
|
93
|
+
2. **The HTML face** rests mostly on *normative* behavior: HTML's tokenizer states
|
|
94
|
+
and error recovery are specified, and the blank-page backstop of §4.1 is ordinary
|
|
95
|
+
CSS. It is exposed on two other counts: it depends on the ZIP face beneath it, and
|
|
96
|
+
a browser must run its script for the page to appear.
|
|
97
|
+
3. **The PDF and PNG faces are conveniences.** Both rest entirely on tolerances no
|
|
98
|
+
standard promises: the PDF header scan, where PDFium already enforces its
|
|
99
|
+
1024-byte window exactly with no margin, and PNG decoders' indifference to a
|
|
100
|
+
`tEXt` chunk holding bytes the format does not permit. They are worth having
|
|
101
|
+
because they cost nothing the other faces need. For preservation purposes they
|
|
102
|
+
are exports that happen to share the file; they are not archival copies.
|
|
103
|
+
|
|
104
|
+
### 1.2 Non-goals
|
|
105
|
+
|
|
106
|
+
- **Forward-only ZIP parsers.** SingleFile archives require central-directory-driven reading (the
|
|
107
|
+
entries are preceded by non-ZIP bytes). Parsers that stream local headers from
|
|
108
|
+
offset 0 are out of scope (§7).
|
|
109
|
+
- **In-place modification by generic ZIP tools.** The face invariants are global:
|
|
110
|
+
the writer picks each hiding tag only after checking the exact bytes it must hide,
|
|
111
|
+
the recovery payload of universal mode contains a checksum of the whole ZIP
|
|
112
|
+
region, and the PDF and PNG structures wrap the archive (§5.4). A tool
|
|
113
|
+
that adds, removes or recompresses entries invalidates them, and most rewriters drop
|
|
114
|
+
the prepended and appended regions outright. A generically rewritten file keeps at
|
|
115
|
+
best its ZIP face. Editing an archive means producing a new one through the writer
|
|
116
|
+
rules (§6).
|
|
117
|
+
- **Multi-page archives.** The reference implementation can bundle several saved
|
|
118
|
+
pages into one archive behind a routing bootstrap (`multiPageArchive`). This
|
|
119
|
+
version of the document specifies single-page archives only; the multi-page
|
|
120
|
+
layout is out of scope.
|
|
121
|
+
- **Confidentiality outside the ZIP entries.** A password encrypts ZIP entry contents
|
|
122
|
+
only (AES). The PDF and PNG faces render the page content and are plaintext by
|
|
123
|
+
design; the writer withholds what it can without breaking a face, as described in
|
|
124
|
+
§5.6. The embedded PDF MAY itself be a PDF-encrypted document, since the format is
|
|
125
|
+
agnostic to the PDF's content, but the reference writer does not produce one.
|
|
126
|
+
|
|
127
|
+
### 1.3 Terminology
|
|
128
|
+
|
|
129
|
+
| Term | Meaning |
|
|
130
|
+
|---|---|
|
|
131
|
+
| **face** | One of the formats the file is valid in: HTML, ZIP, PNG, PDF. |
|
|
132
|
+
| **region** | A byte range with a single producer, named in §3. Regions are the units the rest of this document reasons about; a region can appear in several pieces — `html-prologue` resumes after the embedded PDF document in the PDF variants, and after the `tEXt "ZIP"` chunk header in the PNG ones, so with all four faces it comes in three. |
|
|
133
|
+
| **universal mode** | The variant whose HTML face can extract the archive from the *parsed page text*, the text and comment nodes the HTML parser produced, and therefore needs no access to its own raw bytes. Named "universal" because it works from any location, including the `file:` protocol. |
|
|
134
|
+
| **wrapper tag** | The HTML construct that hides a binary region from the HTML parser, `<!--`…`-->` by default (§5.1). |
|
|
135
|
+
| **appended data** | Bytes after the ZIP End Of Central Directory record. Readers tolerate them within the window their EOCD scan already covers: 65557 bytes from the end of the file (the 22-byte record plus the 65535-byte maximum comment length); "the 64 KB window" refers to this. It may be left undeclared or declared as the archive comment; both forms are valid ZIP and readers MUST accept both (§4.2). The recovery payload can be computed before that choice is made because it stops two bytes short of the record, excluding its comment-length field (see *recovered range* below). |
|
|
136
|
+
| **ZIP region** | The contiguous byte range holding the archive proper: from the first local file header the ZIP writer emitted through the last byte of the End Of Central Directory record. It spans the `zip-entries`, `pdf-central-record` (when present) and `central-directory · eocd` blocks of §3, and in the HTML variants it is exactly the content of the last wrapper. It does **not** include `pdf-local-header` or the PDF document, which sit earlier in the file. |
|
|
137
|
+
| **archive** | The *logical* ZIP file: the set of entries the central directory describes, wherever their bytes lie. This is distinct from the ZIP region above, which is a contiguous byte range. Every entry but one has its bytes inside the region; `page.pdf` is the deliberate exception, an entry of the archive whose local header and data sit before the region (§4.2). "Archive" in this document always means the logical file, "ZIP region" always the byte range, and the two differ only in the PDF-with-HTML variants. |
|
|
138
|
+
| **recovered range** | What the universal extractor reproduces (§4.5): the ZIP region minus its last two bytes, the comment-length field of the End Of Central Directory record. That field is the one part of the record whose value depends on what follows the region, so leaving it out is what lets a writer decide the appended-data form after the recovery payload is final (§4.2). The extractor supplies the two bytes itself, as zeroes — the recovered range carries no comment. |
|
|
139
|
+
| **reference writer** | `createArchive()` in single-file-core `processors/compression/compression.js`. |
|
|
140
|
+
| **bootstrap** | The inline script in the HTML face that locates, extracts and displays the archived page. |
|
|
141
|
+
| **`sfz` identifiers** | Three byte-level identifiers carry the `sfz` prefix: `data-sfz`, `<sfz-extra-data>` and `sfz-data`. The prefix is inherited from SingleFileZ, the browser extension the format originated in (since merged into SingleFile), and is kept unchanged for compatibility with existing files. They are wire identifiers, not the format's name. Two of the three have a role in the format: `<sfz-extra-data>` is the element carrying the recovery payload, and `sfz-data` is the identifier the universal extractor addresses the ZIP region with — an `id` attribute on the wrapper element, or the first characters of the wrapper comment's data (§4.5). `data-sfz` is a marker the reference writer happens to put on the root element; this document mentions it only where it describes bytes those files contain. |
|
|
142
|
+
|
|
143
|
+
## 2. Variants: composing faces
|
|
144
|
+
|
|
145
|
+
Every SingleFile archive has the ZIP face. The other faces are enabled independently by writer
|
|
146
|
+
options, and compose. Each row of the table lists its complete option set; every row
|
|
147
|
+
also implies `compressContent`, the option that stores the page and its resources as
|
|
148
|
+
ZIP entries and so makes the output an archive at all:
|
|
149
|
+
|
|
150
|
+
| Writer options (core names) | HTML | PNG | PDF | Extension | Specimen |
|
|
151
|
+
|---|---|---|---|---|---|
|
|
152
|
+
| *(none)* | — | — | — | `.zip` | pure zip |
|
|
153
|
+
| `selfExtractingArchive` | ✔ | — | — | `.zip.html` | plain |
|
|
154
|
+
| `selfExtractingArchive`, `extractDataFromPage` | ✔ universal | — | — | `.u.zip.html` | universal |
|
|
155
|
+
| `selfExtractingArchive`, `extractDataFromPage`, `embeddedImage` | ✔ universal | ✔ | — | `.u.zip.html` | png |
|
|
156
|
+
| `selfExtractingArchive`, `extractDataFromPage`, `embeddedPdf` | ✔ universal | — | ✔ | `.u.zip.html` | pdf |
|
|
157
|
+
| `selfExtractingArchive`, `extractDataFromPage`, `embeddedImage`, `embeddedPdf` | ✔ universal | ✔ | ✔ | `.u.zip.html` | png-pdf |
|
|
158
|
+
| `embeddedPdf` | — | — | ✔ | `.zip` | zip-pdf |
|
|
159
|
+
| `embeddedImage` | — | ✔ | — | `.zip` | zip-png |
|
|
160
|
+
| `embeddedImage`, `embeddedPdf` | — | ✔ | ✔ | `.zip` | zip-png-pdf |
|
|
161
|
+
|
|
162
|
+
`extractDataFromPage` is orthogonal to the PNG and PDF faces. The PNG and PDF rows
|
|
163
|
+
include it because the clients producing those variants enable it by default, but
|
|
164
|
+
`embeddedImage` and `embeddedPdf` compose with a non-universal self-extracting file
|
|
165
|
+
just as well; the extension is then `.zip.html`. The one interaction: `page.pdf` lies
|
|
166
|
+
outside the ZIP region (§1.3), so the page-text extraction path does not recover it
|
|
167
|
+
with the rest and the extractor skips that entry (§4.5); every path that reads raw
|
|
168
|
+
bytes sees it normally. In the last three rows there is no
|
|
169
|
+
HTML face, so the option does not apply. The *Specimen* column names the measured
|
|
170
|
+
reference files this document cites; §8 records how to regenerate them.
|
|
171
|
+
|
|
172
|
+
Notes on composition:
|
|
173
|
+
|
|
174
|
+
- **Universal mode requires the HTML face** (it is a property of the bootstrap) and is
|
|
175
|
+
independent of PNG/PDF. The command-line client enables it by default whenever it
|
|
176
|
+
produces a self-extracting file; the browser extension exposes it as the
|
|
177
|
+
"self-extracting ZIP (universal)" file format.
|
|
178
|
+
- **The extra-data payload is what separates the two self-extracting types in
|
|
179
|
+
practice.** A plain file has no way of its own to reach its bytes from a `file:`
|
|
180
|
+
URL: the bootstrap has no payload to rebuild the archive from and does not attempt
|
|
181
|
+
a self-read there (§4.1), so it goes to its error message. Opening one from disk
|
|
182
|
+
needs cooperating software that reads the file and hands it to the bootstrap — the
|
|
183
|
+
SingleFile extension does this once granted file access ("Allow access to file
|
|
184
|
+
URLs" in Chrome, Edge and Brave; "Disable Local File Restrictions" in Safari). Over
|
|
185
|
+
HTTP the plain file is self-sufficient, fetching its own URL. A
|
|
186
|
+
universal file instead carries the extra-data payload (§3.1, §5.5), so the bootstrap can
|
|
187
|
+
rebuild the archive from the parsed page text with no access to the raw bytes. Such a
|
|
188
|
+
file opens from disk in any browser, with no setting and no assistance.
|
|
189
|
+
- **All four faces at once is a supported combination**: the PDF document rides
|
|
190
|
+
inside the HTML head, which itself rides inside the PNG's first `tEXt` chunk
|
|
191
|
+
(the all-four-faces row of the byte map).
|
|
192
|
+
- **Faces without HTML** are plain polyglots with no self-extraction and no markup:
|
|
193
|
+
the ZIP data is appended after a raw PDF document (which is then plain prepended
|
|
194
|
+
data), or wrapped in PNG chunks — or both, the PNG chunk layout carrying the PDF
|
|
195
|
+
document as the data of its own `tEXt` chunk (keyword `PDF`) placed right after
|
|
196
|
+
`IHDR`, which keeps `%PDF-` within the header scan window (§4.3). In none of
|
|
197
|
+
these variants is `page.pdf` an archive entry.
|
|
198
|
+
|
|
199
|
+
### 2.1 The charset rule
|
|
200
|
+
|
|
201
|
+
The HTML face declares `<meta charset=utf-8>` when universal mode is off, and a
|
|
202
|
+
single-byte charset when it is on — `windows-1252` in the reference writer. The
|
|
203
|
+
declaration MUST appear within the first 1024 bytes of the file so the parser's
|
|
204
|
+
encoding prescan finds it. The prescan reads exactly that many bytes and does not
|
|
205
|
+
resume, so the whole `<meta>` tag has to fit: one that straddles the boundary is not
|
|
206
|
+
seen, and the parser falls back to its default encoding.
|
|
207
|
+
|
|
208
|
+
Universal mode works in two parts, and the charset carries the first. The archive
|
|
209
|
+
bytes themselves are recovered *from the parsed page text*: the browser decoded
|
|
210
|
+
them as characters when it parsed the file, and the bootstrap re-encodes those
|
|
211
|
+
characters back into bytes. The requirement this places on the charset is
|
|
212
|
+
**injectivity**: under the encoding's index in the
|
|
213
|
+
[WHATWG Encoding Standard](https://encoding.spec.whatwg.org/) — the mapping every
|
|
214
|
+
browser implements — each of the 256 byte values MUST decode to a distinct code
|
|
215
|
+
point, and no byte may decode to U+FFFD. Any encoding with that property carries
|
|
216
|
+
arbitrary bytes through the parse, and 20 of the standard's encodings qualify (§8.4).
|
|
217
|
+
Multi-byte encodings, `utf-8` included, do not: invalid sequences collapse to U+FFFD
|
|
218
|
+
and the bytes cannot be recovered.
|
|
219
|
+
|
|
220
|
+
The reference writer uses `windows-1252`, for reasons beyond injectivity. It is the
|
|
221
|
+
best-supported single-byte encoding there is: the standard resolves the `iso-8859-1`
|
|
222
|
+
and `ascii` label families to it, and it is the fallback the HTML standard prescribes
|
|
223
|
+
for unlabelled content in most locales — so a file whose `<meta charset>` is stripped
|
|
224
|
+
or overridden by a server still tends to be decoded the way the extractor expects. It
|
|
225
|
+
also keeps the reverse table small, at 27 entries (§5.5).
|
|
226
|
+
|
|
227
|
+
The second part is the extra-data payload, which does **not**
|
|
228
|
+
contain the archive: it carries only what the round trip destroys, namely a checksum
|
|
229
|
+
and the information needed to restore newline bytes, which the parser normalizes.
|
|
230
|
+
The parser also replaces NUL bytes with U+FFFD; since no byte decodes to U+FFFD under
|
|
231
|
+
a qualifying encoding, the extractor maps U+FFFD back to NUL unambiguously and the
|
|
232
|
+
payload needs nothing for it (§5.5).
|
|
233
|
+
|
|
234
|
+
### 2.2 File name conventions
|
|
235
|
+
|
|
236
|
+
The reference implementation names files by variant: `.zip` (no HTML face),
|
|
237
|
+
`.zip.html` (self-extracting), `.u.zip.html` (self-extracting, universal). These are
|
|
238
|
+
conventions for humans and pickers; **readers MUST NOT rely on the file name**. Every
|
|
239
|
+
face is discoverable from the bytes alone: PNG and PDF by their signatures, the ZIP
|
|
240
|
+
face by its End Of Central Directory record, and the HTML face by an `<html` start tag
|
|
241
|
+
occurring before the first local file header — inside the first `tEXt` chunk's data in
|
|
242
|
+
the PNG variants, where the markup begins after the chunk's keyword. Inside the
|
|
243
|
+
archive, the `index.html` and `manifest.json` entries mark it as a saved page; a
|
|
244
|
+
reader should identify it that way (§7.1). The self-extracting variants are told apart
|
|
245
|
+
the same way: only a universal file carries an `<sfz-extra-data>` element.
|
|
246
|
+
|
|
247
|
+
## 3. The byte map
|
|
248
|
+
|
|
249
|
+
Unless a row states otherwise, the layouts below are measured from specimen files
|
|
250
|
+
saved from `example.com` (the generation commands are in §8); the
|
|
251
|
+
oversized-payload layout is derived from the writer rules instead, because a payload
|
|
252
|
+
over 64 KB requires an archive too large for a readable specimen. The figure below shows
|
|
253
|
+
the regions and their order; the glossary of §3.1 is the normative list, and it states
|
|
254
|
+
in text everything the figure conveys.
|
|
255
|
+
|
|
256
|
+

|
|
257
|
+
|
|
258
|
+
### 3.1 Region glossary
|
|
259
|
+
|
|
260
|
+
The names below are the block labels of the figure (where space is tight the figure
|
|
261
|
+
merges adjacent blocks into one label, such as `--></body></html>`). In this table,
|
|
262
|
+
*Producer* names
|
|
263
|
+
the syntax the bytes belong to (*extractor* marks the recovery machinery of
|
|
264
|
+
universal mode), and *Present* names the faces, variants or modes in which the
|
|
265
|
+
region exists. Rows are grouped: the HTML/ZIP core first, then the regions the PDF
|
|
266
|
+
face adds, then the regions the PNG face adds.
|
|
267
|
+
|
|
268
|
+
| Region | Producer | Present | Contents |
|
|
269
|
+
|---|---|---|---|
|
|
270
|
+
| `html-prologue` | HTML | HTML face | Doctype, the root element start tag, an optional implementation-defined comment, `<meta charset>`, title, optional head elements (canonical link, `robots` meta, viewport, Content-Security-Policy), minimal CSS, `<body hidden>`, wait/error messages, optional table of contents, optional text body (§4.6). In the plain variant an optional UTF-8 BOM MAY precede the doctype (`includeBOM`); universal and PNG variants never carry one. In the PNG variants the region is split: everything through `<body hidden>` is the data of the `tEXt "PNG"` chunk, while the messages, the optional table of contents and the optional text body follow the `tEXt "ZIP"` chunk header; the doctype and the leading comment are dropped. |
|
|
271
|
+
| `bootstrap` | HTML | HTML face | One inline `<script>`: the embedded ZIP reader, the extractor, the display routine, and the content-acquisition logic (§4.1). The wrapper start tag that opens the ZIP region follows it, directly or after a relocated `extra-data`. |
|
|
272
|
+
| `<!--` / `-->` | HTML | HTML face | The wrapper tag pair hiding a binary region from the HTML parser — comment tags by default, another pair when the hidden bytes contain `-->` (§5.1). Drawn at each opening and closing position. The close tag is absent when appended data is prevented (`preventAppendedData`, or the `<plaintext>` wrapper which cannot close): no markup follows the archive and the wrapper runs to end-of-file. That does not mean the file ends at the EOCD — the PNG face's tail still follows, inside the wrapper, where it parses as text (§5.1). |
|
|
273
|
+
| `zip-entries` | ZIP | always | The archive's local file headers and entry data, written by the ZIP writer. The central directory of an archive written by the reference writer lists `index.html` (the page) first, then `manifest.json` (a JSON description of the archive: original URL, title, save time, resource-to-URL map — informative; the page displays without it), then the resources; the *physical* order of the local headers inside the region is not guaranteed to match, and readers MUST NOT rely on either order — entries are addressed by name (§7.1). |
|
|
274
|
+
| `central-directory · eocd` | ZIP | always | The central-directory records followed by the End Of Central Directory record. All offsets are absolute file positions (§5.3). In the HTML+PDF variants the EOCD accounts for the injected `pdf-central-record` (how the writer achieves that is §6). |
|
|
275
|
+
| `extra-data` | extractor | universal | `<sfz-extra-data>` element holding the base64, deflate-compressed recovery payload (§5.5). It always sits outside the wrapper, so it parses as a real element the extractor can address. Normal placement: after the EOCD, between the wrapper close tag and the end tags. Relocated placement, used when the payload exceeds the 64 KB appended-data window or `preventAppendedData` is set: immediately before the wrapper start tag. In the relocated form the element is followed by space padding: its room is reserved before the archive is written, because the region precedes the ZIP data and resizing it would shift every central-directory offset (§6). Neither placement carries positional meaning — the extractor finds the ZIP region by identifier, not relative to this element (§4.5). |
|
|
276
|
+
| `</body></html>` | HTML | HTML face | The end tags closing the document after the wrapper close tag. Omitted when appended data is prevented, and in the PNG variants so the file can end with the PNG tail. |
|
|
277
|
+
| `pdf-local-header` | ZIP | PDF face with HTML | The hand-built local file header for `page.pdf` (STORE, checksum precomputed), written immediately before the PDF document so ZIP readers see an ordinary entry whose data is the PDF (§6). |
|
|
278
|
+
| `pdf-document` | PDF | PDF face | The raw PDF bytes. With the HTML face, wrapped together with `pdf-local-header` in a wrapper tag pair inside `html-prologue`, placed so `%PDF-` starts at offset 1024 or lower — the range PDF readers search for the header, which is what lets a PDF document start after other bytes at all (§4.3). Without the HTML face the file simply *starts* with the PDF document, as prepended data the ZIP face tolerates; `page.pdf` is then not an archive entry at all — no local header, no central record. |
|
|
279
|
+
| `pdf-central-record` | ZIP | PDF face with HTML | The central-directory record for `page.pdf`, injected *before* the writer's own central directory. The start of the central directory is the one place a record can be added without moving any offset the writer already committed, and it makes `page.pdf` the first entry ZIP tools list (§6). |
|
|
280
|
+
| `png-signature · IHDR` | PNG | PNG face | The 8-byte PNG signature and the `IHDR` chunk declaring the source image's dimensions — the first 33 bytes of the file. |
|
|
281
|
+
| `tEXt "PNG"` | PNG | PNG face with HTML | The length, type and keyword bytes of the first `tEXt` chunk. Its data is `html-prologue` (with the PDF face, the embedded PDF document rides inside it too), ending with the wrapper start tag. |
|
|
282
|
+
| `tEXt "PDF"` | PNG | PNG + PDF faces without HTML | The length, type and keyword bytes of a `tEXt` chunk whose data is the raw PDF document. Written only when the PNG and PDF faces combine without HTML — with the HTML face the PDF rides inside `tEXt "PNG"` instead — and placed right after `IHDR` so `%PDF-` stays within the header scan window (§4.3). |
|
|
283
|
+
| `pixel-data chunks` | PNG | PNG face | The source image's image-data chunks, copied unmodified. With the HTML face they sit inside the wrapper so the HTML parser skips them. |
|
|
284
|
+
| `tEXt "ZIP"` | PNG | PNG face | The length, type and keyword bytes of the second `tEXt` chunk. Its declared length covers everything from there up to but not including the trailing chunk CRC, as a PNG chunk length always does, so the PNG decoder skips the archive — and, with the HTML face, the bootstrap and the appended data — as the data of one chunk. With the HTML face, the wrapper opened at the end of `tEXt "PNG"` closes immediately after these bytes: its content is the first chunk's CRC, the pixel-data chunks and this chunk's own header, and the prologue resumes as markup directly after the close tag. |
|
|
285
|
+
| `crc · IEND` | PNG | PNG face | The `tEXt "ZIP"` chunk's CRC, computed once the archive bytes are final (§6), followed by the empty `IEND` chunk — the last bytes of the file (PNG requires `IEND` to end the stream, which is why the PNG variants drop the end tags). |
|
|
286
|
+
|
|
287
|
+
The reader-by-reader interpretation of these regions is §4; the mechanics that keep
|
|
288
|
+
them from colliding (wrapper-tag selection, checksums, offsets, the 64 KB budget) are
|
|
289
|
+
§5.
|
|
290
|
+
|
|
291
|
+
## 4. Reader lenses
|
|
292
|
+
|
|
293
|
+
Each consumer of the file has a defined way of locating its own bytes and a defined
|
|
294
|
+
reason to ignore the rest. This section walks the same file through each reader. The
|
|
295
|
+
figure shows the all-four-faces variant of the byte map once per reader, fading the
|
|
296
|
+
regions that reader ignores; the subsections explain each row.
|
|
297
|
+
|
|
298
|
+

|
|
299
|
+
|
|
300
|
+
This section describes what stock readers do with the file. The convention stated at
|
|
301
|
+
the head of this document applies throughout it: a MUST about a face constrains the
|
|
302
|
+
bytes a writer produces, never the stock reader whose behavior the format cannot
|
|
303
|
+
change.
|
|
304
|
+
|
|
305
|
+
### 4.1 The browser
|
|
306
|
+
|
|
307
|
+
The HTML parser consumes the whole file as one document. Its encoding prescan finds
|
|
308
|
+
the `<meta charset>` declaration within the first 1024 bytes (§2.1) and the file is
|
|
309
|
+
decoded as a single text; every binary region therefore also exists as characters in
|
|
310
|
+
the parsed document, which is what universal mode exploits (§4.5).
|
|
311
|
+
|
|
312
|
+
The binary regions are kept out of the rendered page by the wrapper tags. The
|
|
313
|
+
default wrapper is an HTML comment, and the HTML standard defines exactly which
|
|
314
|
+
character sequences terminate one (`-->`, and the recovery form `--!>`); the writer
|
|
315
|
+
MUST select a wrapper only after checking the bytes it must hide against that
|
|
316
|
+
wrapper's patterns (the exact rules differ between the ZIP region and the PDF and
|
|
317
|
+
PNG payloads, §5.1), so hiding relies on normative parsing behavior. One
|
|
318
|
+
case weakens the wrapper itself: when no wrapper fits a PDF or PNG payload the
|
|
319
|
+
payload is emitted bare (§5.1). Some binary content always sits *outside* a
|
|
320
|
+
wrapper: in the PNG variants, the signature, IHDR and chunk framing bytes that
|
|
321
|
+
precede the root element start tag decode to a short run of text that HTML error recovery
|
|
322
|
+
places in the (hidden) body. The backstop for all these cases is the prologue: it
|
|
323
|
+
declares `<body hidden>` and a stylesheet that suppresses everything except the
|
|
324
|
+
wait and error messages, so the page comes up blank rather than showing raw bytes,
|
|
325
|
+
with or without scripting.
|
|
326
|
+
|
|
327
|
+
The bootstrap script runs at parse time and proceeds in three stages:
|
|
328
|
+
|
|
329
|
+
1. **Acquire the archive bytes.** On `file:` URLs it goes straight to page-text
|
|
330
|
+
extraction (§4.5): whether a `file:` page may read its own bytes varies by
|
|
331
|
+
browser and configuration (§2), so the bootstrap uses the rung that depends on
|
|
332
|
+
neither. On other protocols it requests its own URL, aborting at the response
|
|
333
|
+
headers: when the server advertises `Accept-Ranges: bytes` it switches to HTTP
|
|
334
|
+
range reading, fetching only the central directory and the entries it needs (a
|
|
335
|
+
large archive displays without downloading the ZIP region in full); otherwise it
|
|
336
|
+
downloads the whole file. When the header probe fails it falls back to page-text
|
|
337
|
+
extraction; a failure of the full download itself, past the probe, reveals the
|
|
338
|
+
error message directly. Only when every applicable rung fails does the error
|
|
339
|
+
message appear, with recovery instructions that differ by variant (§2).
|
|
340
|
+
2. **Extract.** The embedded ZIP reader reads the archive through the ZIP lens
|
|
341
|
+
(§4.2) and rebuilds the page: text entries are decoded, binary entries become
|
|
342
|
+
in-memory URLs, and references between entries are rewritten deepest-first.
|
|
343
|
+
3. **Display.** The rebuilt page replaces the bootstrap document. Around this stage
|
|
344
|
+
the saved page's `<noscript>` elements are neutralized — rewritten to inert
|
|
345
|
+
placeholders before parsing, restored afterwards — because the page was captured
|
|
346
|
+
with scripting available, so its noscript fallbacks must not activate in the
|
|
347
|
+
viewer.
|
|
348
|
+
|
|
349
|
+
The entry point is exposed as a run-once `globalThis.bootstrap(content)` function,
|
|
350
|
+
so cooperating software MAY hand the bootstrap bytes it acquired itself; `content` is
|
|
351
|
+
the file's bytes as a `Blob` or an array of byte values, or a reader object exposing
|
|
352
|
+
the embedded ZIP reader's `readUint8Array` interface, and the call returns a promise
|
|
353
|
+
that settles when the page has been displayed. This is how
|
|
354
|
+
the SingleFile extension assists a *plain* (non-universal) file on `file:`: granted
|
|
355
|
+
file-URL access, it reads the file and invokes the bootstrap. That is the recovery
|
|
356
|
+
path the plain variant's error message describes.
|
|
357
|
+
|
|
358
|
+
### 4.2 The ZIP reader
|
|
359
|
+
|
|
360
|
+
The ZIP face is read from the end. A reader locates the End Of Central Directory
|
|
361
|
+
record by scanning backward from end-of-file; the format guarantees it lies within
|
|
362
|
+
the window every reader must already scan to support archive comments (§1.3,
|
|
363
|
+
*appended data*), because everything after it — wrapper close tag, extra-data, end
|
|
364
|
+
tags, PNG tail — fits the appended-data budget (§5.2). Accepting *undeclared* bytes
|
|
365
|
+
in that window is itself a customary tolerance (§1.1): the ZIP specification
|
|
366
|
+
documents the comment, not trailing junk. From the EOCD
|
|
367
|
+
the reader jumps to the central directory and reads only what it references;
|
|
368
|
+
central-directory-driven reading is a requirement of the format (§1.2, §7).
|
|
369
|
+
|
|
370
|
+
Two properties keep ordinary ZIP tools comfortable:
|
|
371
|
+
|
|
372
|
+
- **Offsets MUST be absolute file positions** (§5.3). The stored central-directory offset
|
|
373
|
+
equals the record's actual position, and each entry's local-header offset points at
|
|
374
|
+
a real local header — tools that cross-check offset arithmetic (rather than
|
|
375
|
+
tolerating a uniform shift from prepended data) accept the file as-is.
|
|
376
|
+
- **The non-ZIP regions are invisible to the reader.** Bytes before the first local
|
|
377
|
+
header and after the EOCD are simply never referenced. The one deliberate
|
|
378
|
+
exception: in the PDF-with-HTML variants the first central-directory record points
|
|
379
|
+
*back into the HTML head*, where the hand-built `page.pdf` local header and the PDF
|
|
380
|
+
document sit (`pdf-local-header`, §3.1) — an ordinary STORE entry that happens to
|
|
381
|
+
live inside the prepended region.
|
|
382
|
+
|
|
383
|
+
A listing shows `page.pdf` first (when the PDF face is present with HTML), then
|
|
384
|
+
`index.html`, `manifest.json` and the page's resources. That order and the two
|
|
385
|
+
conventions below describe the reference writer rather than constraining the format —
|
|
386
|
+
readers address entries by name (§7.1):
|
|
387
|
+
|
|
388
|
+
- Entries for resources fetched from a URL carry that URL in their *comment* field; a
|
|
389
|
+
resource that came from a `data:` URL carries the literal marker `data:` instead,
|
|
390
|
+
and `manifest.json` and `page.pdf` have no comment. Comments are omitted entirely
|
|
391
|
+
from a password-protected archive, because the central directory is not encrypted
|
|
392
|
+
(§5.6).
|
|
393
|
+
- Entries whose content is already compressed (images, fonts, media, PDF) are STOREd
|
|
394
|
+
and the rest are deflated, a size optimization no reader depends on.
|
|
395
|
+
|
|
396
|
+
One rule here is a requirement. With a password, entry contents are AES-encrypted —
|
|
397
|
+
except `page.pdf`, which MUST stay unencrypted and MUST be STOREd, because its bytes
|
|
398
|
+
double as the PDF face (§5.6). The encryption is WinZip's AES scheme, the one ZIP
|
|
399
|
+
tools implement under compression method 99 with the `0x9901` extra field: AE-2,
|
|
400
|
+
AES-256, PBKDF2-HMAC-SHA1 key derivation and an HMAC-SHA1 authentication code. A
|
|
401
|
+
reader that already supports encrypted ZIP entries needs nothing specific to this
|
|
402
|
+
format, and the [WinZip AES specification](https://www.winzip.com/en/support/aes-encryption/)
|
|
403
|
+
is normative for it. A reader implementing the scheme from primitives rather than from
|
|
404
|
+
a ZIP library needs six parameters that specification supplies and this paragraph's
|
|
405
|
+
names do not: 1000 PBKDF2 iterations; a 16-byte salt at AES-256 strength, stored
|
|
406
|
+
before the data; a derived key of 32 + 32 + 2 bytes, read as encryption key,
|
|
407
|
+
authentication key, then a password verifier the reader MUST check before decrypting;
|
|
408
|
+
the HMAC-SHA1 code truncated to its first 10 bytes and stored after the data; and a
|
|
409
|
+
CTR counter that increments **little-endian**, starting at 1, which general-purpose
|
|
410
|
+
CTR interfaces do not do.
|
|
411
|
+
|
|
412
|
+
Appended data comes in two forms and both are valid ZIP, so readers MUST accept
|
|
413
|
+
both. **Raw** — the EOCD declares a zero-length comment and the trailing bytes are
|
|
414
|
+
simply outside the archive — is the default, because tools print a declared archive
|
|
415
|
+
comment on ordinary operations (§8.1), and in universal mode that comment is the
|
|
416
|
+
whole base64 recovery payload. **Declared** — the EOCD's comment length covers every
|
|
417
|
+
byte after the record — is what older writers emitted, and it is the only form some
|
|
418
|
+
readers accept at all: `java.util.zip`, and therefore Android and most JVM tooling,
|
|
419
|
+
rejects an archive with undeclared trailing bytes outright (§8.1). A writer SHOULD
|
|
420
|
+
offer both and default to raw.
|
|
421
|
+
|
|
422
|
+
Neither form constrains the other faces, and universal mode supports both, because
|
|
423
|
+
the recovery payload describes the recovered range rather than the whole region: the
|
|
424
|
+
comment-length field is excluded (§1.3), so its value can be decided after the
|
|
425
|
+
payload is final. A writer that instead covered the field would have to solve a
|
|
426
|
+
circular dependency — the field's value depends on the size of the appended run,
|
|
427
|
+
which contains the payload, whose content depends on the field.
|
|
428
|
+
|
|
429
|
+
### 4.3 The PDF viewer
|
|
430
|
+
|
|
431
|
+
The PDF face relies on two customary reader behaviors — conventions PDF
|
|
432
|
+
implementations follow rather than guarantees of the PDF specification, so the
|
|
433
|
+
compatibility appendix records real-world support (§8):
|
|
434
|
+
|
|
435
|
+
- **The header scan.** Viewers accept a file whose `%PDF-` header starts at offset
|
|
436
|
+
1024 or lower — the bound is on the header's first byte, and 1024 itself passes —
|
|
437
|
+
and treat the header's position as byte 0 of the document: every
|
|
438
|
+
offset in the file (cross-reference entries, `startxref`) is interpreted relative
|
|
439
|
+
to it. The writer MUST place the header inside that window; the document's own
|
|
440
|
+
offsets then need no rewriting. Engines differ in how strictly they hold to the
|
|
441
|
+
1024-byte figure, and the strict ones are the common ones: PDFium — Chrome, Edge and
|
|
442
|
+
everything else Chromium-based — accepts a header starting at offset 1024 and
|
|
443
|
+
rejects one at 1025 outright, while poppler and macOS PDFKit impose no limit at all
|
|
444
|
+
(§8.1).
|
|
445
|
+
- **Tolerance of trailing data.** The archive continues after `%%EOF`, so the
|
|
446
|
+
document's tail is not the file's tail. Viewers cope by searching backward for the
|
|
447
|
+
trailer over a larger window or by reconstructing the cross-reference table from
|
|
448
|
+
the objects themselves.
|
|
449
|
+
|
|
450
|
+
With the HTML face, the PDF document sits inside the head, wrapped in its own
|
|
451
|
+
wrapper-tag pair chosen against the PDF's bytes (a PDF containing `-->` steps the
|
|
452
|
+
ladder, §5.1), and preceded by the `page.pdf` local header so the same bytes are also
|
|
453
|
+
a ZIP entry. That dual role is why the entry MUST be STOREd and MUST NOT be
|
|
454
|
+
encrypted: a viewer reads the entry's data region directly, and any transformation
|
|
455
|
+
of it would break the face. Without the HTML face the document needs no wrapper, and
|
|
456
|
+
where it sits depends on the PNG face: alone with the ZIP face it simply starts the
|
|
457
|
+
file, at offset 0, exercising only the trailing-data tolerance; with the PNG face it is
|
|
458
|
+
the data of a `tEXt "PDF"` chunk placed right after `IHDR` (§3.1), which puts `%PDF-`
|
|
459
|
+
at roughly offset 45, inside the window but not at its start.
|
|
460
|
+
|
|
461
|
+
### 4.4 The PNG decoder
|
|
462
|
+
|
|
463
|
+
PNG offers no header scan: the standard requires the signature to be the first
|
|
464
|
+
8 bytes of the stream and `IEND` to be its last chunk. The PNG face therefore owns
|
|
465
|
+
both ends of the file — the HTML face gives up its doctype and leading comment
|
|
466
|
+
at the front, and the closing `</body></html>` at the back (§3.1).
|
|
467
|
+
|
|
468
|
+
After the signature, a decoder walks chunks — length, type, data, CRC — and skips
|
|
469
|
+
ancillary chunks it does not use. The face hides all foreign bytes inside two `tEXt`
|
|
470
|
+
chunks (ancillary by construction, their type starting lowercase):
|
|
471
|
+
|
|
472
|
+
- **`tEXt` with keyword `PNG`** carries the head of the HTML prologue, through
|
|
473
|
+
`<body hidden>` — and, in the all-faces variant, the embedded PDF document inside
|
|
474
|
+
it — ending with the wrapper start tag. The rest of the prologue, the messages and
|
|
475
|
+
the optional text body, comes later, inside the second chunk.
|
|
476
|
+
- **`tEXt` with keyword `ZIP`** declares a length that covers everything from its
|
|
477
|
+
keyword to the trailing chunk CRC: the rest of the HTML, the bootstrap, the whole
|
|
478
|
+
ZIP region and the appended data. The decoder hops over all of it as the data of
|
|
479
|
+
one chunk.
|
|
480
|
+
|
|
481
|
+
Both chunks carry text the PNG standard does not strictly permit: a `tEXt` text
|
|
482
|
+
string is Latin-1 text, and the payloads here contain NUL bytes — 78 in the
|
|
483
|
+
`tEXt "ZIP"` chunk of the `png` specimen, 101 in the `png-pdf` one. Decoders skip
|
|
484
|
+
ancillary chunks without inspecting their text, so this passes everywhere tested
|
|
485
|
+
(§8.1). It exercises the PNG tolerance §1.1 lists at its limit: what decoders ignore
|
|
486
|
+
in a `tEXt` chunk is text PNG does not permit.
|
|
487
|
+
|
|
488
|
+
Both chunks MUST carry correct CRCs — decoders are entitled to verify them, and the
|
|
489
|
+
second chunk's CRC can only be computed once the archive bytes are final (§6). The
|
|
490
|
+
`pixel-data chunks` region is copied bit-identically from the source image, so the
|
|
491
|
+
decoded image is exactly that image.
|
|
492
|
+
|
|
493
|
+
### 4.5 The universal-mode extractor
|
|
494
|
+
|
|
495
|
+
The last reader is the format's own: the extraction path of universal mode, used
|
|
496
|
+
when the raw bytes are unreachable (§4.1). Its input is not the file but the *parsed
|
|
497
|
+
document* — the characters the HTML parser produced — and its output is the exact
|
|
498
|
+
ZIP region, byte for byte.
|
|
499
|
+
|
|
500
|
+
It works in three steps:
|
|
501
|
+
|
|
502
|
+
1. **Locate.** The extractor finds the `<sfz-extra-data>` element for the payload, and
|
|
503
|
+
the ZIP region's node by its identifier `sfz-data` (§1.3): the element returned by
|
|
504
|
+
`getElementById`, or, when the wrapper is a comment, the first comment in the
|
|
505
|
+
document whose data starts with those characters. An element bearing the identifier
|
|
506
|
+
wins over a comment when both resolve; a reader that finds an id-bearing element
|
|
507
|
+
which is not one of §5.1's wrapper rungs SHOULD fall back to the comment, since the
|
|
508
|
+
`id` is then something else in the page. The two placements (§3.1) need no
|
|
509
|
+
telling apart, and neither the region's position in the tree nor its depth carries
|
|
510
|
+
meaning — a document that moved the node before extraction resolves the same way,
|
|
511
|
+
which matters because the reference extractor relocates `meta` and `style` elements
|
|
512
|
+
into the head before this step and one wrapper rung is a `style` element. With a
|
|
513
|
+
comment wrapper the identifier is part of the node's data, so the re-encoding in
|
|
514
|
+
step 3 starts after it.
|
|
515
|
+
2. **Decode the payload.** The element's text is base64 of a raw-deflate stream, and
|
|
516
|
+
the writer puts nothing else inside the element — the padding of the relocated
|
|
517
|
+
placement sits outside it (§6.1) — though a reader SHOULD ignore whitespace there
|
|
518
|
+
rather than reject the file. It
|
|
519
|
+
inflates to four fields: a checksum of the recovered range, its byte length, the
|
|
520
|
+
newline count, and the packed sequence of 2-bit codes recording each original
|
|
521
|
+
newline (LF, CR or CR LF). §5.5 gives their wire format — little-endian 32-bit
|
|
522
|
+
words, the codes packed 16 per word, least-significant pair first — which a reader
|
|
523
|
+
needs before it can read any of what follows.
|
|
524
|
+
Every one of the four describes the *recovered range*,
|
|
525
|
+
not the whole ZIP region: a newline formed by the two excluded bytes is neither
|
|
526
|
+
counted nor coded, and the checksum does not cover them. The declared length is the
|
|
527
|
+
**only** bound on the re-encoding: the extractor MUST stop there and append two zero
|
|
528
|
+
bytes to complete the EOCD record.
|
|
529
|
+
Those two bytes are its comment-length field, and only that field: every other byte
|
|
530
|
+
of the record is reconstructed from the parsed text like the rest of the region. The
|
|
531
|
+
field is excluded because its value depends on what follows the region, so a payload
|
|
532
|
+
covering it could not be computed until the appended data was final (§1.3); a
|
|
533
|
+
recovered archive therefore always declares a zero-length comment, whatever the
|
|
534
|
+
original declared. Never infer the bound from the node instead: the wrapper's close tag is
|
|
535
|
+
absent whenever appended data is prevented, and under `<plaintext>` the node always
|
|
536
|
+
runs to end of file. The final word is padded to 16 codes,
|
|
537
|
+
so codes beyond the declared newline count carry no meaning and a reader MUST ignore
|
|
538
|
+
them; within the count, the value 3 is unassigned and a reader MUST reject a payload
|
|
539
|
+
that uses it.
|
|
540
|
+
3. **Re-encode the characters.** Apply the inverse of the declared charset, exactly
|
|
541
|
+
as §5.5 defines it: a character whose code point is a byte value that decodes to
|
|
542
|
+
itself becomes that byte, every other code point goes through the reverse table,
|
|
543
|
+
and U+FFFD becomes NUL. Under windows-1252 this is nearly an identity — 229 of the
|
|
544
|
+
256 values map to themselves — but the shortcut "any code point ≤ 255 is that
|
|
545
|
+
byte" is **not** a valid substitute: under other qualifying charsets (§2.1) code
|
|
546
|
+
points below 256 can belong to a different byte, 75 of them under `macintosh`, and
|
|
547
|
+
the shortcut would silently corrupt the region. The two things parsing destroyed
|
|
548
|
+
are restored from the payload: each parsed newline consumes the next 2-bit code to
|
|
549
|
+
reproduce the original byte sequence.
|
|
550
|
+
|
|
551
|
+
In the PDF-with-HTML variants the recovered region is a complete archive except for
|
|
552
|
+
one entry: its central directory still lists `page.pdf`, but that entry's local
|
|
553
|
+
header and data sit in the HTML head, outside the ZIP region (§1.3). The extractor
|
|
554
|
+
skips the entry instead of failing — the displayed page never references it, and
|
|
555
|
+
the PDF stays reachable through every raw-bytes path (§4.2).
|
|
556
|
+
|
|
557
|
+
The entry's offset is unusable *within the recovered region* — it is a true file
|
|
558
|
+
position like every other (§5.3), and a reader of the whole file follows it normally;
|
|
559
|
+
what fails is only the shifted arithmetic of the paragraph below. Its bytes are not out
|
|
560
|
+
of reach either: the local header
|
|
561
|
+
and the document sit in a wrapper in the head, so they are in the parsed page like any
|
|
562
|
+
other region, and a reader MAY recover them by the same round trip — find a local file
|
|
563
|
+
header naming `page.pdf` among the other parsed nodes, take the declared number of
|
|
564
|
+
bytes after it, and check them against the CRC-32 the central directory holds. The CRC
|
|
565
|
+
check is mandatory. The recovery payload's newline codes cover the ZIP region only, so
|
|
566
|
+
a newline in the PDF block has no code and its original bytes must be guessed: assume
|
|
567
|
+
LF, the byte the parser normalized *to*. A PDF routinely contains CR, so the guess
|
|
568
|
+
often fails, and nothing may be written unless the CRC-32 agrees. Take the length from
|
|
569
|
+
the central-directory record, which is the entry's authority for it; the hand-built
|
|
570
|
+
local header of §6.1 carries the same value, but a reader cannot tell that from a
|
|
571
|
+
length deferred to a data descriptor. A matching CRC-32 is a 32-bit non-cryptographic
|
|
572
|
+
check over a reconstruction that differs from the original in at most a few newline
|
|
573
|
+
bytes, so an undetected error is improbable but cannot be ruled out. A reader MUST NOT
|
|
574
|
+
present a reconstructed `page.pdf` as verified, and MUST NOT let one displace bytes
|
|
575
|
+
obtained from a raw-bytes read.
|
|
576
|
+
|
|
577
|
+
Recovering the entry is therefore optional; the reference extractor skips it, which
|
|
578
|
+
conforms. A reader that *presents the archive's contents* — a listing, an
|
|
579
|
+
extract-to-disk, an entry enumeration offered to a caller (§7.1) — MUST report
|
|
580
|
+
`page.pdf` as present and unretrieved rather than omit it, whether it skipped the
|
|
581
|
+
recovery or tried and failed the CRC check. Its listing is then the same as a raw-bytes
|
|
582
|
+
reader's, with only the bytes missing. A reader that has no such surface is outside the
|
|
583
|
+
rule: the display path of §4.1 rebuilds a page from the entries it needs, `page.pdf` is
|
|
584
|
+
referenced by nothing in that page, and the reference extractor accordingly filters the
|
|
585
|
+
entry out on every acquisition path, including the ones that read raw bytes and could
|
|
586
|
+
return it.
|
|
587
|
+
|
|
588
|
+
The extractor MUST verify the three checkable payload fields — byte length, newline
|
|
589
|
+
count and checksum — and fail to the error message on any mismatch.
|
|
590
|
+
|
|
591
|
+
The recovered region is a complete archive but **not an offset-self-contained one**.
|
|
592
|
+
Its offsets are still absolute positions in the original file (§5.3), so every
|
|
593
|
+
local-header offset in its central directory, and the central-directory offset in its
|
|
594
|
+
EOCD record, overshoot by exactly the region's start position in the file. A reader of
|
|
595
|
+
the recovered region MUST therefore apply a uniform negative shift of that amount, the
|
|
596
|
+
prepended-data compensation ZIP readers already implement: from the region's point of
|
|
597
|
+
view the missing bytes look like a prefix that was stripped. In the
|
|
598
|
+
specimen of §8.2 the shift is 122005 bytes: Info-ZIP reports `missing 122005 bytes in
|
|
599
|
+
zipfile`, adds `(attempting to process anyway)` and lists both entries, while readers
|
|
600
|
+
that compensate silently, such as Python's `zipfile`, show no diagnostic at all. The
|
|
601
|
+
shift also puts `page.pdf` out of the
|
|
602
|
+
offset-following path: its local header lies *before* the region, so its compensated
|
|
603
|
+
offset is negative (−102092 in the `pdf` specimen) and no reader can seek to it. On
|
|
604
|
+
success the shifted bytes enter the normal extraction path (§4.2).
|
|
605
|
+
|
|
606
|
+
The shift is a file offset, and a universal-mode reader has no file. It does not need
|
|
607
|
+
one: the region carries the shift within itself, since its EOCD declares both the size
|
|
608
|
+
of the central directory and its absolute offset, while the directory's position inside
|
|
609
|
+
the region is known. With `region` the recovered bytes,
|
|
610
|
+
|
|
611
|
+
```
|
|
612
|
+
shift = eocd.centralDirectoryOffset - (eocdPosition - eocd.centralDirectorySize)
|
|
613
|
+
```
|
|
614
|
+
|
|
615
|
+
where `eocdPosition` is the EOCD record's own offset within `region`, found by scanning
|
|
616
|
+
backward for its signature the way any ZIP reader finds it. A reader arrives at the
|
|
617
|
+
same number as a ZIP library's prepended-data compensation, which derives it from the
|
|
618
|
+
record's position rather than from the buffer's end. Do not substitute
|
|
619
|
+
`region.length - 22` for `eocdPosition`: the two are equal only when the EOCD is the
|
|
620
|
+
last record in the region, which zip64 and a non-empty archive comment both break.
|
|
621
|
+
|
|
622
|
+
Under zip64 (§5.4) both of those EOCD fields are the `0xFFFFFFFF` sentinel, and the
|
|
623
|
+
zip64 end of central directory record carries the real values. Take them from there,
|
|
624
|
+
using the same `eocdPosition` arithmetic against that record's own position — the
|
|
625
|
+
zip64 locator states an absolute offset in the original file, so it needs the shift
|
|
626
|
+
this formula produces and cannot be used to find it. A reader that instead uses the
|
|
627
|
+
sentinels arithmetically gets a shift in the billions, with no diagnostic.
|
|
628
|
+
|
|
629
|
+
### 4.6 Text tools
|
|
630
|
+
|
|
631
|
+
The optional text body (`insertTextBody`) addresses one more consumer: software that
|
|
632
|
+
reads the file as plain text — `grep`, desktop search, indexers — and will never run
|
|
633
|
+
the bootstrap or unzip anything. It is a `<main hidden>` element at the end of the
|
|
634
|
+
visible prologue holding the page's text content, so the page stays searchable
|
|
635
|
+
without any extraction. It is searchable by everyone, so it is not written at all
|
|
636
|
+
when a password is set (§5.6): its text would be readable without the password.
|
|
637
|
+
|
|
638
|
+
The text body is always written in UTF-8, regardless of the declared charset. In
|
|
639
|
+
universal mode this cuts its audience in two: a charset-oblivious tool that reads
|
|
640
|
+
raw bytes — `grep`, plain-text search — sees intact UTF-8, while any consumer that
|
|
641
|
+
honors the declared `<meta charset>` decodes it as windows-1252 and garbles
|
|
642
|
+
non-ASCII text. That includes the HTML parser itself — harmless there, because the
|
|
643
|
+
region is hidden and replaced (§4.1) — but also HTML-aware indexers. The text body
|
|
644
|
+
opens with the page title, for the same raw-byte
|
|
645
|
+
audience — it is the first *text* in the element, which is not necessarily the
|
|
646
|
+
element's first line: the reference writer's serialization puts a newline before it.
|
|
647
|
+
|
|
648
|
+
The `<title>` element takes the opposite route. Its content is RCDATA, where
|
|
649
|
+
character references are resolved against Unicode independently of the declared
|
|
650
|
+
encoding, so the writer emits every character outside printable ASCII — and `&`,
|
|
651
|
+
`<`, `>` — as a numeric reference. The element's bytes are therefore pure ASCII and
|
|
652
|
+
the title survives the single-byte declaration intact: a page titled 日本語 shows as
|
|
653
|
+
日本語 in the browser tab and to any conforming parser. Writers that emit the title
|
|
654
|
+
raw MUST NOT do so in universal mode, where the same bytes decode as mojibake.
|
|
655
|
+
|
|
656
|
+
## 5. Cross-cutting mechanics
|
|
657
|
+
|
|
658
|
+
Section 3 named the regions and §4 read them one reader at a time. What remains are
|
|
659
|
+
the rules that span readers: how a payload is hidden, how much room is left at the
|
|
660
|
+
end of the file, which numbers are offsets into what, what each checksum covers, how
|
|
661
|
+
the character round trip is inverted, what a password protects, and what changes when
|
|
662
|
+
the archive is large enough to need zip64.
|
|
663
|
+
|
|
664
|
+
### 5.1 Wrapper-tag selection
|
|
665
|
+
|
|
666
|
+
Binary payloads inside the HTML face are hidden by a wrapper tag pair. What the format
|
|
667
|
+
requires of a wrapper is that the HTML parser not treat its content as markup, so the
|
|
668
|
+
payload survives parsing as text, and that the payload not contain the construct's
|
|
669
|
+
terminator. Any construct with those properties works; the reference writer picks from
|
|
670
|
+
this ladder, in order:
|
|
671
|
+
|
|
672
|
+
| Order | Wrapper | Parser treatment of the content | Terminated by |
|
|
673
|
+
|---|---|---|---|
|
|
674
|
+
| 1 | `<!--` … `-->` | comment | `-->`, or the recovery form `--!>` |
|
|
675
|
+
| 2 | `<script type=sfz-data>` | script data, not executed (the type is not a JavaScript MIME type) | `</script` followed by whitespace, `/` or `>` |
|
|
676
|
+
| 3 | `<style type=sfz-data>` | raw text, no style sheet built (the type is not a CSS MIME type) | `</style` + delimiter |
|
|
677
|
+
| 4 | `<noframes>` | raw text | `</noframes` + delimiter |
|
|
678
|
+
| 5 | `<noembed>` | raw text | `</noembed` + delimiter |
|
|
679
|
+
| 6 | `<iframe>` | raw text | `</iframe` + delimiter |
|
|
680
|
+
| 7 | `<xmp>` | raw text | `</xmp` + delimiter |
|
|
681
|
+
| 8 | `<plaintext>` | everything to end of file | nothing — the element cannot be closed |
|
|
682
|
+
|
|
683
|
+
The terminators are written lower case above, but HTML matches end tag names ASCII
|
|
684
|
+
case-insensitively: `</XMP>` and `</Script ` close their elements just as `</xmp>` and
|
|
685
|
+
`</script>` do. A writer's test MUST be case-insensitive, on the start patterns as well
|
|
686
|
+
as the end ones. A stored, uncompressed resource is the realistic source of an
|
|
687
|
+
upper-case one.
|
|
688
|
+
|
|
689
|
+
Every rung hides its content unconditionally. `<noscript>` has the right terminator
|
|
690
|
+
and was a rung until core 1.5.108, but it is the one construct whose content is raw
|
|
691
|
+
text only while scripting is enabled and markup when it is not, so on a page opened
|
|
692
|
+
without scripting the archive bytes would reach the tree builder as tags. The rungs
|
|
693
|
+
below it hide the same payloads at no extra cost, so it was removed rather than
|
|
694
|
+
demoted.
|
|
695
|
+
|
|
696
|
+
The order under the comment is not arbitrary. Every rung hides its content from an HTML
|
|
697
|
+
parser, but text extractors differ, and the ZIP region is large enough that the
|
|
698
|
+
difference is a user-visible one. Measured on macOS: Spotlight's HTML importer indexes
|
|
699
|
+
the content of `<noframes>`, `<noembed>`, `<iframe>`, `<xmp>` and `<plaintext>`, and
|
|
700
|
+
`textutil` reads the last two of those, `<xmp>` and `<plaintext>`, while the comment
|
|
701
|
+
and the `script` and `style` rungs are dropped by both. Those two therefore sit
|
|
702
|
+
directly under the comment, so that an escalating writer keeps the archive out of the
|
|
703
|
+
reader's local search index
|
|
704
|
+
for as long as the payload allows. Both are inert at those types: the script is not
|
|
705
|
+
executed and no style sheet is built.
|
|
706
|
+
|
|
707
|
+
That ordering is the one place the format optimizes against measured third-party
|
|
708
|
+
behavior rather than against a rule, and unlike the tolerances of §1.1 nothing depends
|
|
709
|
+
on the measurement holding. An extractor that starts reading `<script type=sfz-data>`,
|
|
710
|
+
or stops reading `<xmp>`, changes only which archives end up in a local search index;
|
|
711
|
+
every rung still hides its content from the HTML parser, and a writer whose ladder is
|
|
712
|
+
ordered differently produces files that are just as correct.
|
|
713
|
+
|
|
714
|
+
Two things about the ladder *are* required. Whatever order a writer gives the seven
|
|
715
|
+
closable rungs, it MUST apply the selection test below to every rung it considers, and
|
|
716
|
+
MUST keep `<plaintext>` available as the rung of last resort: §6.2's termination
|
|
717
|
+
argument needs one rung no payload can defeat.
|
|
718
|
+
|
|
719
|
+
The wrapper of the ZIP region also carries the identifier the extractor addresses it
|
|
720
|
+
with (§4.5): an element rung takes it as an `id` attribute — `<script type=sfz-data
|
|
721
|
+
id=sfz-data>`, `<noframes id=sfz-data>` — and the comment rung as the first characters of
|
|
722
|
+
its data, `<!--sfz-data`. The wrappers hiding the PDF and PNG faces MUST NOT carry it:
|
|
723
|
+
those payloads are found by byte structure, and a second node bearing the identifier
|
|
724
|
+
would shadow the archive.
|
|
725
|
+
|
|
726
|
+
The reference writer walks the ladder from the top and takes the first rung the payload
|
|
727
|
+
does not defeat. The test it applies is the format's, and is the same for every payload;
|
|
728
|
+
what differs is how far the ladder goes:
|
|
729
|
+
|
|
730
|
+
- **The ZIP region** rejects a rung whose *end* pattern the payload contains, and also
|
|
731
|
+
a rung whose *start* pattern it contains. A rung's start pattern is the tag's opening
|
|
732
|
+
delimiter and name, without attributes: `<!--` for the comment, then `<script`,
|
|
733
|
+
`<style`, `<noframes`, `<noembed`, `<iframe`, `<xmp` for the elements.
|
|
734
|
+
|
|
735
|
+
`<plaintext>` is exempt from **both** tests. It has no terminator to occur and no
|
|
736
|
+
tokenizer states to escape into — a `<plaintext` inside a `<plaintext>` is inert
|
|
737
|
+
text like everything else — so no payload can defeat it. §6.2's termination argument
|
|
738
|
+
rests on that exemption.
|
|
739
|
+
|
|
740
|
+
The other seven are all tested, and a writer MUST test all seven rather than the one
|
|
741
|
+
that needs it. The `<script>` rung needs it
|
|
742
|
+
to be correct at all, because script data has escape states no other rung has: `<!--`
|
|
743
|
+
in script data enters *script data escaped*, and a `<script` after that enters *script
|
|
744
|
+
data double escaped*, where `</script>` does **not** close the element. So a payload
|
|
745
|
+
can hold `<!--` and then `<script`, contain no `</script` anywhere, pass the end test —
|
|
746
|
+
and the wrapper then swallows its own end tag, the extra-data element and the rest of
|
|
747
|
+
the document. On the other six the start test is genuine conservatism: a nested `<!--`
|
|
748
|
+
is a parse error inside a comment but does not close it, and the raw-text rungs hold a
|
|
749
|
+
flat run of characters with no states at all. The rule is uniform deliberately: the
|
|
750
|
+
exemption would save one pattern match per rung on bytes already in memory, at the
|
|
751
|
+
cost of a special case an implementer has to remember correctly about the single rung
|
|
752
|
+
where forgetting it destroys the document. An earlier draft offered exactly that
|
|
753
|
+
latitude, in the broader form "a writer MAY skip the start test outside universal
|
|
754
|
+
mode", and the reference writer's PDF and PNG faces took it and shipped the bug
|
|
755
|
+
(§8.5).
|
|
756
|
+
- **The PDF and PNG payloads** apply the same two tests, for the same reason — a face
|
|
757
|
+
that took the `<script>` rung on a payload holding `<!--` and `<script` would swallow
|
|
758
|
+
the rest of the document, title, bootstrap and extra-data element included — but the
|
|
759
|
+
`<plaintext>` rung is excluded from their ladder: those payloads sit in the middle
|
|
760
|
+
of the file, so a wrapper that can never close is not an option. When no rung fits,
|
|
761
|
+
the payload is written bare, with no wrapper at all; it then reaches the parser as
|
|
762
|
+
text, and the blank-page backstop of §4.1 keeps it invisible.
|
|
763
|
+
|
|
764
|
+
The comment rung has one restriction more than a terminator. HTML forbids comment text
|
|
765
|
+
that *starts* with `>` or `->`, and the tokenizer enforces it: it closes the comment
|
|
766
|
+
right there, spilling the payload into the parser. What starts the comment differs by
|
|
767
|
+
payload — the ZIP region begins with the identifier, the PDF face with a local file
|
|
768
|
+
header or `%PDF-` — but the PNG face begins with the CRC of the chunk carrying the
|
|
769
|
+
start tag, four bytes that are only settled once the tag is chosen, and one in 256 of
|
|
770
|
+
them is `>`. A writer using a comment there MUST compute that checksum and take the
|
|
771
|
+
next rung when it opens with `>` or `->`; the next rung is an element, which has no
|
|
772
|
+
such restriction, so the test cannot cascade. HTML also forbids comment text ending
|
|
773
|
+
with `<!-`, which the terminator check covers by testing that pattern anchored at the
|
|
774
|
+
payload's end.
|
|
775
|
+
|
|
776
|
+
Choosing the last rung has consequences that reach the rest of the file: because
|
|
777
|
+
`<plaintext>` cannot be closed, selecting it sets `preventAppendedData` — no *markup*
|
|
778
|
+
may follow the ZIP region, which forces the relocated placement of the extra-data
|
|
779
|
+
element (§5.2) and drops the closing `</body></html>`.
|
|
780
|
+
|
|
781
|
+
That constraint is about markup, not about the last byte of the file, so the PNG face
|
|
782
|
+
composes with this rung: the `tEXt` chunk's checksum and the `IEND` chunk still follow
|
|
783
|
+
the region, as the PNG face requires, and `<plaintext>` reads them as the text they are.
|
|
784
|
+
Verified on a build forced onto this rung with a screenshot embedded: the file ends
|
|
785
|
+
`49 45 4e 44 ae 42 60 82`, decodes as a PNG, and its archive extracts from the parsed
|
|
786
|
+
page. The termination argument of §6.2 therefore holds for the faced variants too.
|
|
787
|
+
|
|
788
|
+
The check MUST be made against the payload's final bytes, for every payload the
|
|
789
|
+
writer hides and in every variant that hides one. Every rejection restarts the build
|
|
790
|
+
(§6): the wrapper choice changes the bytes preceding the archive, so the archive must
|
|
791
|
+
be rewritten at its new position.
|
|
792
|
+
|
|
793
|
+
### 5.2 The appended-data budget
|
|
794
|
+
|
|
795
|
+
Everything the writer emits after the EOCD record MUST fit in 65535 bytes — the
|
|
796
|
+
maximum length a ZIP archive comment may declare, and therefore the distance beyond
|
|
797
|
+
the record that every reader's backward scan already covers (§1.3). The appended run is:
|
|
798
|
+
|
|
799
|
+
```
|
|
800
|
+
wrapper close tag + extra-data element + end tags + (PNG face: 4-byte chunk CRC + 12-byte IEND)
|
|
801
|
+
```
|
|
802
|
+
|
|
803
|
+
and the writer compares its total against 65535 before committing to it. The EOCD
|
|
804
|
+
record's own 22 bytes sit inside the window too, giving the 65557-byte figure of §1.3.
|
|
805
|
+
|
|
806
|
+
Only the extra-data element can outgrow the budget: it carries one 2-bit code per
|
|
807
|
+
newline sequence in the ZIP region — CR LF counts once, for two bytes (§5.5) — so it
|
|
808
|
+
grows with the archive. Newline bytes
|
|
809
|
+
occur at their natural density in compressed and STOREd binary data — about two in
|
|
810
|
+
every 256 bytes — and the codes are compressed and base64-encoded, which measures at
|
|
811
|
+
one byte of element per 650 bytes of archive at scale (§8). The budget is therefore
|
|
812
|
+
exhausted at an archive of roughly 40 MB, so the relocated placement is rare in
|
|
813
|
+
practice. That ratio is the large-archive limit and must not be used to size a
|
|
814
|
+
particular file: deflate's overhead is a fixed cost spread over a growing payload, so
|
|
815
|
+
small archives are far less efficient. Measured on exact byte counts, a 6099-byte region
|
|
816
|
+
needs 69 bytes of element — a ratio of 88 — and a 74057-byte region needs 189, a ratio
|
|
817
|
+
of 392; §8.3's series then runs 475, 609, 650 and 646 as the archive grows from 86 KB to
|
|
818
|
+
4.2 MB, so the ratio approaches the headline figure from below and levels off rather
|
|
819
|
+
than climbing past it. A writer sizes its reservation from the payload it actually
|
|
820
|
+
produced (§6.2), never from this figure. When the payload does not fit, or when
|
|
821
|
+
`preventAppendedData` is set, the writer switches to the **relocated placement**: the
|
|
822
|
+
element moves in front of the wrapper start tag, ahead of the archive (§3.1). Room
|
|
823
|
+
for it MUST be reserved before the ZIP region is written, because inserting bytes
|
|
824
|
+
ahead of the archive would shift every offset the ZIP writer has already committed;
|
|
825
|
+
the reservation is padded with spaces and the real payload is written into it once
|
|
826
|
+
its final size is known (§6).
|
|
827
|
+
|
|
828
|
+
### 5.3 Offset bookkeeping
|
|
829
|
+
|
|
830
|
+
Three coordinate systems coexist in one file, and the format's job is to keep each
|
|
831
|
+
self-consistent:
|
|
832
|
+
|
|
833
|
+
- **ZIP offsets are absolute file positions.** The writer is told the size of
|
|
834
|
+
everything already emitted before the first local header, so the central directory
|
|
835
|
+
offset in the EOCD and every local-header offset in the central directory are true
|
|
836
|
+
file positions (§4.2), so a reader of the *whole file* never needs prepended-data
|
|
837
|
+
compensation — the repair by which a reader recomputes offsets that disagree with
|
|
838
|
+
the file size. A reader of the recovered ZIP region alone does need it (§4.5).
|
|
839
|
+
- **PDF offsets are header-relative.** The document's own cross-reference offsets are
|
|
840
|
+
interpreted from the `%PDF-` header, so embedding it needs no rewriting; the writer
|
|
841
|
+
only MUST keep the header inside the scan window (§4.3).
|
|
842
|
+
- **PNG has no offsets, only lengths.** Each chunk declares its data length. The
|
|
843
|
+
second `tEXt` chunk's length covers the whole archive and the appended data, so it
|
|
844
|
+
can only be written once the file's final size is known, and the writer patches it
|
|
845
|
+
in place at the end (§6).
|
|
846
|
+
|
|
847
|
+
The injected `page.pdf` central record exploits a fourth, deliberate discrepancy. It
|
|
848
|
+
is written directly to the output stream, bypassing the ZIP writer's own byte
|
|
849
|
+
counter, at exactly the position where the central directory is about to start.
|
|
850
|
+
The writer's counter is therefore left *behind* the true stream position by exactly
|
|
851
|
+
the record's length, so the central-directory offset it stores lands on the injected
|
|
852
|
+
record rather than after it: the stored offset needs no correction and the record
|
|
853
|
+
becomes the first entry of the directory. The accounting does need correcting: after
|
|
854
|
+
the archive is closed the writer increments the entry counts and adds the record's
|
|
855
|
+
length to the directory size (§6).
|
|
856
|
+
|
|
857
|
+
### 5.4 Checksum inventory
|
|
858
|
+
|
|
859
|
+
Four independent integrity mechanisms cover overlapping byte ranges. All three CRC-32
|
|
860
|
+
variants are the standard ZIP and PNG CRC-32: reflected polynomial `0xEDB88320`,
|
|
861
|
+
initial value `0xFFFFFFFF`, final complement, processing each byte
|
|
862
|
+
least-significant-bit first — the function `zlib.crc32` and its equivalents compute.
|
|
863
|
+
One table therefore serves all three, but they cover different ranges and live in
|
|
864
|
+
different structures:
|
|
865
|
+
|
|
866
|
+
| Checksum | Covers | Stored in |
|
|
867
|
+
|---|---|---|
|
|
868
|
+
| ZIP entry CRC-32 | one entry's *uncompressed* content | local file header and central-directory record of that entry, including the hand-built `page.pdf` records. Zero for AES-encrypted entries, whose integrity comes from their authentication code instead |
|
|
869
|
+
| PNG chunk CRC-32 | one chunk's type and data bytes | the 4 bytes following each chunk's data. For `tEXt "ZIP"` this spans the whole ZIP region and the appended data |
|
|
870
|
+
| Universal payload CRC-32 | the recovered range as the extractor re-encodes it — the ZIP region without its comment-length field (§1.3) | the recovery payload, with the range's length and newline count (§4.5) |
|
|
871
|
+
| AES authentication code | one encrypted entry's stored bytes | that entry's data, when a password is set |
|
|
872
|
+
|
|
873
|
+
The PDF face contributes none: PDF has no whole-file checksum, so the document can sit
|
|
874
|
+
inside a larger file unchanged.
|
|
875
|
+
|
|
876
|
+
Two of these — the `tEXt "ZIP"` chunk CRC and the universal payload — can only be
|
|
877
|
+
computed when the file is otherwise final, which fixes the last steps of the writer's
|
|
878
|
+
order (§6).
|
|
879
|
+
|
|
880
|
+
### 5.5 The character round trip
|
|
881
|
+
|
|
882
|
+
Universal mode recovers the ZIP region from characters rather than bytes (§2.1). The
|
|
883
|
+
inverse mapping the extractor applies is, for the declared charset:
|
|
884
|
+
|
|
885
|
+
1. **A code point equal to a byte value that decodes to itself → that byte.** Under
|
|
886
|
+
windows-1252 this covers 229 of the 256 values.
|
|
887
|
+
2. **Every other code point → a fixed reverse table.** The table is the inverse of the
|
|
888
|
+
encoding's index in the WHATWG standard, restricted to the byte values it does not
|
|
889
|
+
map to themselves: 27 entries for windows-1252 — the printable characters it places
|
|
890
|
+
in the 0x80–0x9F range (typographic quotes, dashes, the euro sign and so on). That
|
|
891
|
+
count is this rule's table alone; an implementation that folds rule 3 into the same
|
|
892
|
+
lookup, as the reference extractor does, has 28.
|
|
893
|
+
Deriving this table is mechanical, so a reader supports any qualifying charset the
|
|
894
|
+
same way — but it MUST be derived from the WHATWG index and not from the platform's
|
|
895
|
+
codec of the same name, which is usually not the same mapping. The WHATWG index
|
|
896
|
+
assigns every byte a code point; most platform codecs leave five positions of
|
|
897
|
+
windows-1252 undefined:
|
|
898
|
+
|
|
899
|
+
| Byte | 0x81 | 0x8D | 0x8F | 0x90 | 0x9D |
|
|
900
|
+
|---|---|---|---|---|---|
|
|
901
|
+
| WHATWG | U+0081 | U+008D | U+008F | U+0090 | U+009D |
|
|
902
|
+
| Python `cp1252`, Java `windows-1252` | undefined | undefined | undefined | undefined | undefined |
|
|
903
|
+
|
|
904
|
+
Those five bytes occur in ordinary compressed data, so a strict platform decode
|
|
905
|
+
raises on essentially every archive. Configuring the decoder to replace what it
|
|
906
|
+
cannot map is worse: it emits U+FFFD, which rule 3 below turns into NUL, corrupting
|
|
907
|
+
one byte per occurrence. Measured over ten specimen archives, 42 to 1468 bytes per
|
|
908
|
+
file would be lost this way. The payload checksum catches it.
|
|
909
|
+
3. **U+FFFD → 0x00.** No byte decodes to U+FFFD under a qualifying encoding (§2.1), so
|
|
910
|
+
the replacement character can only have come from a NUL byte. This holds because
|
|
911
|
+
the payload is inside a wrapper: in every tokenizer state the ladder of §5.1
|
|
912
|
+
produces — comment, raw text, script data, plaintext — the parser replaces NUL with
|
|
913
|
+
U+FFFD.
|
|
914
|
+
4. **Newlines from the payload.** The parser normalizes CR and CR LF to LF, so the
|
|
915
|
+
original byte sequence is unrecoverable from the text alone; each newline consumes
|
|
916
|
+
the next 2-bit code (0 = LF, 1 = CR, 2 = CR LF).
|
|
917
|
+
|
|
918
|
+
The payload itself is a sequence of little-endian 32-bit words — checksum, recovered
|
|
919
|
+
range length, newline count, then the codes packed 16 per word, least-significant pair
|
|
920
|
+
first — raw-deflated and base64-encoded with the standard alphabet and padding.
|
|
921
|
+
|
|
922
|
+
### 5.6 Password scope
|
|
923
|
+
|
|
924
|
+
A password encrypts the *contents* of ZIP entries with AES, and nothing else. A reader
|
|
925
|
+
gets no protection beyond that. Four consequences follow:
|
|
926
|
+
|
|
927
|
+
- **`page.pdf` is never encrypted** and never compressed: its bytes double as the PDF
|
|
928
|
+
face, which a viewer reads directly from the entry's data region (§4.3).
|
|
929
|
+
- **The PNG and PDF faces stay in the clear.** They render the page, and a viewer
|
|
930
|
+
reads their bytes directly (§4.3, §4.4), so they cannot be encrypted without
|
|
931
|
+
destroying the face. A password on an archive that also has one of them protects the
|
|
932
|
+
archived resources, not the page's visible content.
|
|
933
|
+
- **Entry metadata is never encrypted.** Names, uncompressed sizes and dates remain
|
|
934
|
+
readable in the central directory, so the resource list of an encrypted archive is
|
|
935
|
+
public. This is standard ZIP behavior, not a property of this format; §7 restates it.
|
|
936
|
+
- **What the writer withholds instead.** Three things are not forced into the clear by
|
|
937
|
+
the format and so are withheld when a password is set: the entry comments, which
|
|
938
|
+
would otherwise publish every resource's source URL (§4.2); the `<title>` element's
|
|
939
|
+
text, whose value `manifest.json` carries as an encrypted entry, leaving an empty
|
|
940
|
+
`<title></title>` in the prologue; and the optional
|
|
941
|
+
text body, which repeats the whole page text outside the archive (§4.6). Unlike the
|
|
942
|
+
PNG and PDF faces, none of the three is load-bearing for a reader, so a writer that
|
|
943
|
+
emits them in a password-protected archive publishes what the password is meant to
|
|
944
|
+
cover for no gain.
|
|
945
|
+
|
|
946
|
+
Encrypted entries are stamped AE-2, so their CRC-32 field is zero (§5.4). `page.pdf`
|
|
947
|
+
stays unencrypted, so in a password-protected archive its checksum is the only one a
|
|
948
|
+
ZIP tool can verify.
|
|
949
|
+
|
|
950
|
+
### 5.7 zip64
|
|
951
|
+
|
|
952
|
+
The archive uses the zip64 structures whenever the ordinary records cannot express
|
|
953
|
+
it: a central directory starting beyond 4 GiB (the prefix counts toward the offset,
|
|
954
|
+
§5.3), a directory 4 GiB or longer, or 65535 entries or more. The reference writer
|
|
955
|
+
never requests zip64 explicitly, so it appears only when reached, and given how large
|
|
956
|
+
that is, effectively never in a saved page.
|
|
957
|
+
|
|
958
|
+
When it is reached, the EOCD record carries the sentinel values `0xFFFF` and
|
|
959
|
+
`0xFFFFFFFF` in the fields that overflowed, preceded by a zip64 end of central
|
|
960
|
+
directory record and its locator. The `page.pdf` record injection then applies its
|
|
961
|
+
accounting to the zip64 record instead — entry counts and directory size there, and
|
|
962
|
+
the locator's pointer moved by the record's length — while leaving each saturated
|
|
963
|
+
field at its sentinel. A writer MUST NOT let the injection push a 16-bit or 32-bit
|
|
964
|
+
field to its sentinel value without emitting the corresponding zip64 record: a count
|
|
965
|
+
of `0xFFFF` sends readers looking for a zip64 record that does not exist.
|
|
966
|
+
|
|
967
|
+
This combination has been verified on a forced-zip64 build (§8): `page.pdf` is listed
|
|
968
|
+
first by both Info-ZIP and the reference reader, the central directory offset in the
|
|
969
|
+
zip64 record points at the injected record, and extraction produces the same page as
|
|
970
|
+
the non-zip64 build.
|
|
971
|
+
|
|
972
|
+
## 6. Writer algorithm
|
|
973
|
+
|
|
974
|
+
This section specifies the reference writer's build order. It is normative in the
|
|
975
|
+
sense that a file produced differently but satisfying every rule above is a valid
|
|
976
|
+
SingleFile archive; the order matters because several values can only be computed
|
|
977
|
+
once later bytes exist.
|
|
978
|
+
|
|
979
|
+
Most of that difficulty is optional, and a writer should know how much of it each face
|
|
980
|
+
buys. The cost is not evenly spread:
|
|
981
|
+
|
|
982
|
+
| To produce | The writer needs |
|
|
983
|
+
|---|---|
|
|
984
|
+
| The ZIP face alone | Nothing from this section. Write an ordinary archive with `index.html` first and a `manifest.json`; no wrapper, no retry, no patching |
|
|
985
|
+
| Plus the HTML face | The prologue and bootstrap, and the wrapper ladder of §5.1 — one scan of the finished archive, and a rebuild if the rung changes |
|
|
986
|
+
| Plus universal mode | The recovery payload, the character round trip of §5.5, the appended-data budget of §5.2, and the retry loops of §6.2. This is where the real complexity lives, and it buys opening the file from `file:` with no cooperation |
|
|
987
|
+
| Plus the PDF or PNG face | The header window of §4.3 or the chunk patching of §5.3, plus a second wrapper choice for the embedded payload |
|
|
988
|
+
|
|
989
|
+
Only the third row needs the retry loops, and only the fourth needs a value that cannot
|
|
990
|
+
be computed until the file is otherwise complete. A writer that only wants durable saved
|
|
991
|
+
pages can stop at the first row; the files it produces are accepted by every reader in
|
|
992
|
+
§8.1.
|
|
993
|
+
|
|
994
|
+
### 6.1 Build order
|
|
995
|
+
|
|
996
|
+
1. **PNG head.** With the PNG face, copy the signature and `IHDR` from the source
|
|
997
|
+
image unchanged. Without the HTML face but with the PDF face, emit the
|
|
998
|
+
`tEXt "PDF"` chunk holding the PDF document here, so its header falls inside the
|
|
999
|
+
PDF scan window (§4.3).
|
|
1000
|
+
2. **HTML prologue.** With the HTML face, emit the doctype (omitted under the PNG
|
|
1001
|
+
face, which owns the start of the file), the root element start tag, any comment the
|
|
1002
|
+
implementation adds, the `<meta charset>` required by §2.1, the head elements (the
|
|
1003
|
+
`<title>` among them, unless a password is set (§5.6)), the CSS and `<body hidden>`,
|
|
1004
|
+
the wait and error messages, the optional table of contents and text body, and the
|
|
1005
|
+
bootstrap script. With the PNG face the head of this region,
|
|
1006
|
+
through `<body hidden>`, is the data of the `tEXt "PNG"` chunk and the remainder is
|
|
1007
|
+
emitted after the `tEXt "ZIP"` chunk header in step 12; with the PDF face the
|
|
1008
|
+
region is interrupted by step 3 as well.
|
|
1009
|
+
|
|
1010
|
+
Whatever a writer puts in the prologue, closing every element it opens before the
|
|
1011
|
+
wrapper start tag is good practice but not a requirement: the extractor addresses
|
|
1012
|
+
the ZIP region by identifier (§4.5), so an element left open only makes the archive
|
|
1013
|
+
a descendant of it, and the lookup resolves the same way.
|
|
1014
|
+
3. **Embedded PDF.** With the PDF face and the HTML face, the prologue is *split*
|
|
1015
|
+
around the PDF, which MUST come early enough for `%PDF-` to start at offset 1024
|
|
1016
|
+
or lower (§4.3). Only what a parser needs first precedes it — the doctype, the
|
|
1017
|
+
root element, any leading comment and the charset declaration — and everything
|
|
1018
|
+
else in the head (title, link and meta elements, the stylesheet, `<body hidden>`,
|
|
1019
|
+
the messages, the optional table of contents and text body) follows it. Emit the
|
|
1020
|
+
wrapper start tag chosen for the PDF payload (§5.1), the hand-built `page.pdf`
|
|
1021
|
+
local file header, the PDF document, the wrapper end tag, and record the local
|
|
1022
|
+
header's absolute position; then resume the prologue.
|
|
1023
|
+
|
|
1024
|
+
The window is reachable but not structurally guaranteed, and it is the one place
|
|
1025
|
+
where the format depends on the writer rather than on its own layout. The
|
|
1026
|
+
irreducible part of the prefix is small: the root element start tag, the charset
|
|
1027
|
+
declaration, the wrapper start tag and the 38-byte local file header for
|
|
1028
|
+
`page.pdf`, plus a minimal doctype — 100 bytes in the reference layout, and a few
|
|
1029
|
+
more with a longer charset label or a wrapper past the first rung (§5.1). But two
|
|
1030
|
+
regions ahead of the header have no length the format controls: the doctype, which
|
|
1031
|
+
is copied from the saved page and carries its public and system identifiers
|
|
1032
|
+
verbatim, and any comment the implementation chooses to write there. Real doctypes
|
|
1033
|
+
are small; the longest in common use, XHTML 1.1 with MathML and SVG, is about 140
|
|
1034
|
+
bytes. But nothing caps either region, so a writer MUST cap them itself, keeping
|
|
1035
|
+
everything before the local file header inside the remaining budget of roughly 924
|
|
1036
|
+
bytes (879 with the PNG face, whose signature, `IHDR` and first chunk header take
|
|
1037
|
+
the first 45 bytes of the same window, and whose variant drops the doctype in
|
|
1038
|
+
exchange), shortening, dropping or relocating that content instead of emitting a
|
|
1039
|
+
header outside the window. A writer that places nothing of unbounded length before
|
|
1040
|
+
the PDF block satisfies the rule by construction and needs no check at all.
|
|
1041
|
+
|
|
1042
|
+
The reference writer does both. Its provenance comment is emitted after the PDF
|
|
1043
|
+
block, so the page URL it carries cannot reach the window at all, and the prefix is
|
|
1044
|
+
measured before the header is written: when the page's own doctype would push
|
|
1045
|
+
`%PDF-` past 1024, `<!DOCTYPE html>` is emitted in its place. Substituting the
|
|
1046
|
+
doctype changes the bootstrap document's rendering mode, which costs nothing here:
|
|
1047
|
+
the extracted page is written into the document with its own doctype (§4.1). A
|
|
1048
|
+
writer that must keep the page doctype has to find the room elsewhere.
|
|
1049
|
+
|
|
1050
|
+
Without the HTML face, the PDF is simply the first thing in the file and the
|
|
1051
|
+
question does not arise.
|
|
1052
|
+
4. **Reserved extra-data.** In universal mode, when a previous pass determined that
|
|
1053
|
+
the payload must be relocated (§5.2), emit an empty `<sfz-extra-data>` element
|
|
1054
|
+
followed by enough spaces to fill the reservation. The padding sits **outside** the
|
|
1055
|
+
element, so the element's text stays exactly the payload.
|
|
1056
|
+
5. **Wrapper start tag** for the ZIP region, carrying the identifier (§5.1).
|
|
1057
|
+
6. **The archive.** Create the ZIP writer, telling it the number of bytes already
|
|
1058
|
+
written so that its offsets are absolute (§5.3). Add `index.html` first, then
|
|
1059
|
+
`manifest.json`, then the page's resources, preserving that order in the central
|
|
1060
|
+
directory; STORE entries whose content is already compressed and deflate the rest;
|
|
1061
|
+
put each resource's source URL in its entry comment; encrypt entry contents if a
|
|
1062
|
+
password was given.
|
|
1063
|
+
7. **PDF central record.** With the PDF face and the HTML face, write the record for
|
|
1064
|
+
`page.pdf`, with the local header offset from step 3, immediately before closing
|
|
1065
|
+
the writer.
|
|
1066
|
+
8. **Close and patch.** Close the archive, then correct the end of central directory
|
|
1067
|
+
record for the injected record: entry counts, directory size, and the zip64
|
|
1068
|
+
record and locator when present (§5.7).
|
|
1069
|
+
9. **Universal payload.** In universal mode, read back the ZIP region and check it
|
|
1070
|
+
against the current wrapper (§5.1); on a collision, restart (§6.2). Otherwise
|
|
1071
|
+
compute the region's CRC-32 and its newline codes, build and compress the payload,
|
|
1072
|
+
and decide its placement against the budget (§5.2), restarting if the decision
|
|
1073
|
+
differs from the current pass.
|
|
1074
|
+
10. **Appended run.** Unless appended data is prevented, emit the wrapper end tag,
|
|
1075
|
+
the extra-data element when it is appended, and `</body></html>` — the end tags
|
|
1076
|
+
are omitted under the PNG face, which must end with `IEND`.
|
|
1077
|
+
11. **Fill the reservation.** In the relocated placement, write the payload into the
|
|
1078
|
+
space reserved in step 4; if it no longer fits, restart (§6.2).
|
|
1079
|
+
12. **PNG tail.** With the PNG face, patch the `tEXt "ZIP"` chunk's length field, now
|
|
1080
|
+
that the total size is known, compute that chunk's CRC over everything from its
|
|
1081
|
+
type to the last byte written, and append the CRC and the `IEND` chunk.
|
|
1082
|
+
|
|
1083
|
+
### 6.2 The retry loops
|
|
1084
|
+
|
|
1085
|
+
Four conditions restart the build from step 1, and each restart carries forward what
|
|
1086
|
+
the failed pass learned. The first three terminate because each of them advances a
|
|
1087
|
+
monotone quantity:
|
|
1088
|
+
|
|
1089
|
+
- **Wrapper collision** (§5.1): the next pass starts at the next rung of the ladder.
|
|
1090
|
+
The ladder is finite and its last rung, `<plaintext>`, is exempt from both selection
|
|
1091
|
+
tests, so it always fits.
|
|
1092
|
+
- **Payload does not fit the appended budget** (§5.2): the next pass reserves room
|
|
1093
|
+
ahead of the archive, sized at the measured payload length plus a margin.
|
|
1094
|
+
- **Reservation too small**: relocating the payload changes the file's layout, hence
|
|
1095
|
+
its offsets, hence the payload, which can grow past the room reserved for it. The
|
|
1096
|
+
next pass reserves the new length plus the same margin. For the loop to terminate,
|
|
1097
|
+
each reservation MUST be strictly larger than the payload that sized it: a margin
|
|
1098
|
+
that can round down to zero lets two passes measure the same length and reserve the
|
|
1099
|
+
same room forever.
|
|
1100
|
+
|
|
1101
|
+
The fourth is the converse of the second: a pass that reserved room but then found the
|
|
1102
|
+
payload would fit in the appended window discards the reservation and rebuilds without
|
|
1103
|
+
it, so the writer does not leave dead padding in the file. This step is not monotone,
|
|
1104
|
+
and it is the only one that could keep the build alive forever: dropping the reservation
|
|
1105
|
+
moves
|
|
1106
|
+
the archive back, which changes the offsets, which changes the payload that made the
|
|
1107
|
+
reservation necessary. A payload lying on the 65535-byte boundary can therefore be too
|
|
1108
|
+
large appended and small enough relocated, and the build oscillates. A writer MUST
|
|
1109
|
+
break that cycle: **the reservation is discarded at most once per build**, and a payload
|
|
1110
|
+
that fits the appended window on a later pass stays in the reservation it already has.
|
|
1111
|
+
The file then keeps at most the reservation's own margin of dead padding.
|
|
1112
|
+
|
|
1113
|
+
Given identical inputs, modification date and archive time, the process is
|
|
1114
|
+
deterministic: the same page produces the same bytes, retries included. The archive
|
|
1115
|
+
time is a separate input because `manifest.json` records when the archive was made
|
|
1116
|
+
(§7.1), so two builds of one page at two moments differ in that entry and in the entry
|
|
1117
|
+
sizes around it. A consumer MUST NOT treat the byte identity of two archives of the
|
|
1118
|
+
same page as meaningful.
|
|
1119
|
+
|
|
1120
|
+
## 7. Consuming SingleFile archives safely
|
|
1121
|
+
|
|
1122
|
+
This section addresses software that reads SingleFile archives it did not produce.
|
|
1123
|
+
The ZIP face is the interoperable one, and a reader that follows the rules below
|
|
1124
|
+
handles every variant of §2 without knowing which one it has.
|
|
1125
|
+
|
|
1126
|
+
### 7.1 Reading
|
|
1127
|
+
|
|
1128
|
+
- **Read through the central directory.** Locate the End Of Central Directory record
|
|
1129
|
+
by scanning backward from the end of the file, then follow its offset. A reader that
|
|
1130
|
+
streams local headers from offset 0 will not find an archive: the file starts with
|
|
1131
|
+
the HTML, PDF or PNG face (§1.2, and §8.1 measures what such readers actually do).
|
|
1132
|
+
- **Tolerate bytes before and after the archive.** They are the other faces, not
|
|
1133
|
+
corruption. Offsets are absolute, so no compensation is needed (§5.3).
|
|
1134
|
+
- **Accept both forms of appended data.** The bytes after the EOCD record may be raw
|
|
1135
|
+
or declared as the archive comment; both are valid (§4.2). A reader MUST NOT treat
|
|
1136
|
+
undeclared trailing bytes as a defect.
|
|
1137
|
+
- **Do not identify the format by file name.** Extensions are conventions (§2.2). A
|
|
1138
|
+
SingleFile archive is identifiable from its content, and recognition and extraction
|
|
1139
|
+
use different tests: an `index.html` entry accompanied by a `manifest.json` entry in
|
|
1140
|
+
the same directory is the positive signal for recognizing the format, while
|
|
1141
|
+
`index.html` alone is enough to *extract* from, since `manifest.json` is informative
|
|
1142
|
+
and MUST NOT be required (below). Neither test distinguishes this format from an
|
|
1143
|
+
arbitrary ZIP file laid out the same way, and none is offered, because nothing in the
|
|
1144
|
+
format depends on recognizing it. A reader that treats any archive containing a page
|
|
1145
|
+
entry as a saved page loses nothing.
|
|
1146
|
+
- **Resolve the page entry in this order.** The archive's internal layout is
|
|
1147
|
+
implementation-defined, and two properties of the reference layout matter to a
|
|
1148
|
+
reader. Every entry MAY sit under a single root directory, which the reference
|
|
1149
|
+
writer names from a timestamp when asked to create one; the page is then
|
|
1150
|
+
`<root>/index.html`. And a page's nested frames are stored as complete pages of
|
|
1151
|
+
their own under `frames/<n>/`, recursively, so an archive normally holds several
|
|
1152
|
+
`index.html` entries and only the outermost one is the page. Since neither property
|
|
1153
|
+
is guaranteed, a reader resolves the entry point in three steps, stopping at the
|
|
1154
|
+
first that succeeds:
|
|
1155
|
+
|
|
1156
|
+
1. `manifest.json`'s `indexFilename`, resolved against the root directory, when the
|
|
1157
|
+
entry exists. This is the only authoritative answer, so a writer that departs from
|
|
1158
|
+
the reference layout SHOULD emit the manifest even though a reader MUST NOT
|
|
1159
|
+
require it.
|
|
1160
|
+
2. Otherwise the `index.html` entry at the smallest directory depth.
|
|
1161
|
+
3. If several `index.html` entries tie at that depth, the archive does not name its
|
|
1162
|
+
page: a reader MUST NOT pick one arbitrarily. Report the ambiguity, or treat the
|
|
1163
|
+
file as a plain ZIP archive.
|
|
1164
|
+
|
|
1165
|
+
The reference writer never produces a tie, since it creates at most one root
|
|
1166
|
+
directory and nests every other page under `frames/<n>/`; step 3 exists for archives
|
|
1167
|
+
from other writers.
|
|
1168
|
+
- **Treat `manifest.json` as informative.** The reference writer records the original
|
|
1169
|
+
URL as `originalUrl`, the title as `title`, the save time as `archiveTime` (an ISO
|
|
1170
|
+
8601 string), the entry name of the page as `indexFilename` and the resource-to-URL
|
|
1171
|
+
map as `resources`. The page displays without any of it, and a reader MUST NOT require
|
|
1172
|
+
the entry or any field of it. `indexFilename` names the page relative to the root
|
|
1173
|
+
directory, not as a full entry name. The set of fields is not closed: a reader
|
|
1174
|
+
MUST ignore what it does not recognize.
|
|
1175
|
+
- **Expect a `page.pdf` entry whose data lies outside the archive proper** (§4.2). It
|
|
1176
|
+
is an ordinary STORE entry at an ordinary offset, so nothing special is needed to
|
|
1177
|
+
read it, but a reader that assumes every entry sits between the first local header
|
|
1178
|
+
and the central directory will reject or mislocate it.
|
|
1179
|
+
|
|
1180
|
+
### 7.2 Modifying
|
|
1181
|
+
|
|
1182
|
+
Do not rewrite the file in place. Adding, removing or recompressing entries moves the
|
|
1183
|
+
ZIP region and invalidates the other faces: the PNG chunk length and CRC that span the
|
|
1184
|
+
archive, the recovery payload's checksum, and the wrapper choice that depends on the
|
|
1185
|
+
archive's exact bytes (§1.2). Most ZIP rewriters also drop the prepended and appended
|
|
1186
|
+
regions, which discards every face but ZIP.
|
|
1187
|
+
|
|
1188
|
+
A tool that wants to produce a modified archive MUST rebuild it through the writer
|
|
1189
|
+
rules of §6. A tool that only wants the page content SHOULD extract rather than
|
|
1190
|
+
rewrite.
|
|
1191
|
+
|
|
1192
|
+
The hazard is automated handling: a deduplicating store, a backup system that
|
|
1193
|
+
recompresses, a mail or chat service that repacks attachments, or any pipeline that
|
|
1194
|
+
round-trips the file through a ZIP library will return an archive whose entries are all
|
|
1195
|
+
intact and whose other faces are gone, with no error at any step and no record of it in
|
|
1196
|
+
the file. Software that stores these archives SHOULD treat them as opaque bytes, and a
|
|
1197
|
+
preservation workflow that cannot guarantee that SHOULD keep a checksum of the original
|
|
1198
|
+
alongside it.
|
|
1199
|
+
|
|
1200
|
+
### 7.3 Security considerations
|
|
1201
|
+
|
|
1202
|
+
- **Entry names are untrusted.** They derive from a captured page's resource URLs. A
|
|
1203
|
+
reader MUST sanitize them before writing to a filesystem: reject absolute paths and
|
|
1204
|
+
`..` segments, and be aware that names may be long, may collide after case folding,
|
|
1205
|
+
and may contain characters the local filesystem rejects.
|
|
1206
|
+
- **Declared sizes are untrusted.** Do not pre-allocate from the declared uncompressed
|
|
1207
|
+
size, and enforce a limit on the expansion ratio; the archive can be crafted like
|
|
1208
|
+
any other ZIP file.
|
|
1209
|
+
- **The archived page is untrusted web content**, and the HTML face contains a script.
|
|
1210
|
+
Software that displays either MUST do so in a sandboxed context, and MUST NOT run
|
|
1211
|
+
the bootstrap in a privileged one. The format's own display path replaces the
|
|
1212
|
+
document with the extracted page, which is not an isolation boundary by itself.
|
|
1213
|
+
- **A password protects entry contents only** (§5.6). Entry names, sizes, dates and
|
|
1214
|
+
source URLs stay readable, and the PNG, PDF and text-body faces render the page
|
|
1215
|
+
regardless. Software MUST NOT present a password-protected archive as an encrypted
|
|
1216
|
+
document.
|
|
1217
|
+
- **Sniffing disagrees with itself on these files.** `file(1)` reports HTML, PNG, PDF
|
|
1218
|
+
or "data" depending on the variant (§8.1), so a server that guesses the media type
|
|
1219
|
+
from content may serve a saved page as an image. Software that serves SingleFile
|
|
1220
|
+
archives SHOULD set the media type explicitly — `text/html` for the self-extracting
|
|
1221
|
+
variants, `application/zip` otherwise.
|
|
1222
|
+
|
|
1223
|
+
### 7.4 What to reject and what to tolerate
|
|
1224
|
+
|
|
1225
|
+
A reader of a polyglot file meets conditions that look like corruption and are not, and
|
|
1226
|
+
others that look harmless but leave the extracted page wrong. A condition matters if and
|
|
1227
|
+
only if it affects the bytes the page is built from:
|
|
1228
|
+
|
|
1229
|
+
| Condition | Reader behavior |
|
|
1230
|
+
|---|---|
|
|
1231
|
+
| A recovery payload field disagrees with the reconstruction — length, newline count or checksum | **MUST** fail (§4.5). The reconstruction is wrong and nothing built from it can be trusted |
|
|
1232
|
+
| An entry's CRC-32 or AES authentication code does not match | **SHOULD** fail for that entry, and MUST NOT present a page rebuilt from it as intact |
|
|
1233
|
+
| `page.pdf` was reconstructed from the parsed page and its CRC-32 does not match | **MUST** discard the reconstruction (§4.5). The bytes are a guess about newlines the recovery payload does not describe, and the checksum is the only thing that tests it — unlike the row above, there is no read to have gone wrong, only an inference |
|
|
1234
|
+
| Bytes before the first local file header, or after the EOCD record | **MUST** tolerate: they are the other faces (§7.1) |
|
|
1235
|
+
| The appended run exceeds the 65535-byte budget (§5.2) | Not a reader's problem: if the EOCD record was found, the archive is readable. Readers MAY warn |
|
|
1236
|
+
| A `tEXt` chunk CRC does not match, or a chunk holds bytes PNG does not permit (§4.4) | Irrelevant to extraction; a reader of the archive MAY ignore both |
|
|
1237
|
+
| `page.pdf` is present but its data does not begin with `%PDF-` | Not an error. The entry is data like any other |
|
|
1238
|
+
| `index.html` is present without `manifest.json` | **MUST** still extract (§7.1) |
|
|
1239
|
+
| The recovered region (universal mode) disagrees with the same bytes read directly, in the EOCD's two comment-length bytes only | Expected, not an error. A recovered region always declares a zero-length comment (§4.5), so it differs here from any archive written in the declared form (§4.2). Compare the two only up to those bytes |
|
|
1240
|
+
| The recovered region (universal mode) disagrees with the same bytes read directly, anywhere else | The file is not well-formed, whichever side is at fault, and a reader that has both MUST NOT silently merge them or pick per entry. Prefer the direct read — it is the writer's own output, where the recovered region is a reconstruction of it — and surface the disagreement rather than displaying either as intact |
|
|
1241
|
+
|
|
1242
|
+
Anything the format does not constrain, a reader MUST NOT reject: entries may carry
|
|
1243
|
+
any extra fields, timestamps, data descriptors or name-encoding flags a ZIP writer
|
|
1244
|
+
would ordinarily emit, and none of it is specified here.
|
|
1245
|
+
|
|
1246
|
+
## 8. Appendices
|
|
1247
|
+
|
|
1248
|
+
### 8.1 Tool compatibility
|
|
1249
|
+
|
|
1250
|
+
Measured on macOS 26 with the specimens of §8.3 (Info-ZIP UnZip 6.00, libarchive
|
|
1251
|
+
3.7.4, Python 3.14, OpenJDK 21, 7-Zip 25.01, poppler `pdftotext`, macOS `ditto`,
|
|
1252
|
+
`sips` and Quick Look). Every result is predicted by two structural properties, so
|
|
1253
|
+
the variants are grouped by them:
|
|
1254
|
+
|
|
1255
|
+
| Class | Bytes before the archive | Bytes after the EOCD | Variants |
|
|
1256
|
+
|---|---|---|---|
|
|
1257
|
+
| A | — | — | pure zip |
|
|
1258
|
+
| B | yes | — | relocated (`preventAppendedData`), zip-pdf |
|
|
1259
|
+
| C | yes | yes | plain, universal, ladder, password, pdf, png, png-pdf, zip-png, zip-png-pdf, zip64 |
|
|
1260
|
+
|
|
1261
|
+
The classes follow the bytes, not the options. `preventAppendedData` puts the
|
|
1262
|
+
relocated variant in class B, but it suppresses *markup* after the archive, not the PNG
|
|
1263
|
+
face's tail: an archive combining that option with a PNG face has bytes after the EOCD
|
|
1264
|
+
and is class C.
|
|
1265
|
+
|
|
1266
|
+
| ZIP reader | A | B | C | Behavior |
|
|
1267
|
+
|---|---|---|---|---|
|
|
1268
|
+
| Info-ZIP `unzip`, `zipinfo` | ✔ | ✔ | ✔ | Lists and extracts every variant. AES entries are skipped — `need PK compat. v5.1 (can do v4.5)` — a limitation of the tool, not of the file; `page.pdf` still extracts because it is never encrypted |
|
|
1269
|
+
| Python `zipfile` | ✔ | ✔ | ✔ | Lists and extracts every variant |
|
|
1270
|
+
| 7-Zip (`7zz`) | ✔ | ✔ | ✔ | Lists and extracts every variant, AES included |
|
|
1271
|
+
| libarchive `bsdtar`, seekable input | ✔ | ✔ | ✔ | Lists and extracts every variant |
|
|
1272
|
+
| libarchive `bsdtar`, piped input | ✔ | ✘ | ✘ | `Unrecognized archive format` — the forward-only case of §1.2, measured |
|
|
1273
|
+
| Java `java.util.zip` (`jar tf`) | ✔ | ✔ | ✘ | `zip END header not found` whenever bytes follow the EOCD undeclared. Declaring them as the archive comment makes the same file open, measured on every class-C variant (§4.2) |
|
|
1274
|
+
| macOS `ditto -x -k` | ✔ | ✘ | ✘ | `Couldn't read PKZip signature` — requires a local file header at offset 0, so prepended data alone defeats it |
|
|
1275
|
+
|
|
1276
|
+
The cost of the declared form was measured on the same tools: it is a display cost, not
|
|
1277
|
+
a compatibility one. An archive whose trailing bytes are declared as the comment has
|
|
1278
|
+
them printed back on ordinary listings; `unzip -l` reproduces the whole run — in
|
|
1279
|
+
universal mode that is the `-->`, the entire `<sfz-extra-data>` element and the end tags
|
|
1280
|
+
— under the archive's own header. The raw form is the default for that reason (§4.2).
|
|
1281
|
+
|
|
1282
|
+
The other faces were exercised on the variants that carry them, and all succeeded:
|
|
1283
|
+
`pdftotext` extracts the page text from every PDF-face variant, including the
|
|
1284
|
+
password-protected one (`page.pdf` is never encrypted, §5.6), and `sips` reports the
|
|
1285
|
+
screenshot's true dimensions for every PNG-face variant. macOS Quick Look renders the
|
|
1286
|
+
PNG face of the self-extracting PNG specimen, and on the all-four-faces specimen it
|
|
1287
|
+
renders both faces of the same bytes: renamed to `.png` it yields the screenshot,
|
|
1288
|
+
renamed to `.pdf` the rendered document.
|
|
1289
|
+
|
|
1290
|
+
The PDF header window of §4.3 was measured rather than assumed, by moving `%PDF-`
|
|
1291
|
+
progressively later in otherwise identical archives. PDFium (build 153.0.7999.0, the
|
|
1292
|
+
engine of every Chromium-based browser) loads the document while the header starts at
|
|
1293
|
+
offset 1024 or less and fails with a data-format error from 1025 on — the documented
|
|
1294
|
+
1024-byte figure, enforced exactly. poppler `pdftotext` and macOS PDFKit render the
|
|
1295
|
+
same files with the header at 1000222 bytes, so they impose no window at all. A
|
|
1296
|
+
writer that keeps the header inside 1024 bytes satisfies every engine tested.
|
|
1297
|
+
|
|
1298
|
+
`file(1)` disagrees with itself across the variants. What it reports depends on
|
|
1299
|
+
where the archive falls relative to the fixed buffer it sniffs, not on the variant as
|
|
1300
|
+
such: a small self-extracting file whose archive starts within that buffer is
|
|
1301
|
+
reported as `data` where a large one is reported as HTML. On these specimens:
|
|
1302
|
+
`HTML document text` for the plain, universal and ladder specimens, `PNG image data`
|
|
1303
|
+
for every PNG-face variant, `PDF document` for a
|
|
1304
|
+
PDF-first archive, `Zip archive data` for a pure archive **and for the relocated
|
|
1305
|
+
variant**, whose first bytes are a doctype, and `data` where the HTML head carries
|
|
1306
|
+
the embedded PDF.
|
|
1307
|
+
|
|
1308
|
+
Two of these results bear on §1.1's customary tolerances. The EOCD backward scan and
|
|
1309
|
+
the tolerance of undeclared trailing bytes are near-universal but not unanimous: Java
|
|
1310
|
+
is the measured exception, and not a niche one, since `java.util.zip` is what
|
|
1311
|
+
Android and most JVM tooling use. Prepended data is tolerated by every ZIP reader
|
|
1312
|
+
measured except Apple's `ditto`.
|
|
1313
|
+
|
|
1314
|
+
### 8.2 Anatomy of a small archive
|
|
1315
|
+
|
|
1316
|
+
Offsets in `universal.sfz.html` (123077 bytes, two entries, saved from `example.com`
|
|
1317
|
+
with the §8.3 command, against core 1.5.108).
|
|
1318
|
+
The layout is the *universal* row of the byte map (§3).
|
|
1319
|
+
|
|
1320
|
+
| Offset | Bytes | Region |
|
|
1321
|
+
|---|---|---|
|
|
1322
|
+
| 0 | `<!DOCTYPE html>` | `html-prologue` begins |
|
|
1323
|
+
| 16 | `<html data-sfz>` | root element start tag; the attribute is the reference implementation's own marker (§1.3) |
|
|
1324
|
+
| 31 | `<meta charset=windows-1252>` | the charset rule, inside the first 1024 bytes (§2.1) |
|
|
1325
|
+
| 58 | `<!--` … `-->` (ends at 200) | comment written by the implementation, not part of the format; it follows the charset declaration so it cannot push it out of the prescan window |
|
|
1326
|
+
| 200 | `<title>` … `</title>` (ends at 229) | the page title, as numeric character references (§4.6) |
|
|
1327
|
+
| 677 | `<style>` | the stylesheet of the blank-page backstop (§4.1) |
|
|
1328
|
+
| 855 | `<body hidden>` | start of the blank-page backstop (§4.1) |
|
|
1329
|
+
| 868, 913 | wait and error messages | the two visible elements |
|
|
1330
|
+
| 1134 | `<script>` … `</script>` (ends at 121993) | `bootstrap`: ZIP reader, extractor, display, acquisition |
|
|
1331
|
+
| 121993 | `<!--sfz-data` | wrapper start tag, opening the ZIP region and carrying its identifier (§5.1) |
|
|
1332
|
+
| 122005 | `PK\3\4` | first local file header, `index.html` (1110 bytes, deflated to 593) — the ZIP region begins |
|
|
1333
|
+
| 122647 | `PK\3\4` | local file header, `manifest.json` (168 bytes, deflated to 130) |
|
|
1334
|
+
| 122829 | `PK\1\2` | central directory: `index.html`, then `manifest.json` |
|
|
1335
|
+
| 122981 | `PK\5\6` | EOCD: 2 entries, directory size 152, directory offset 122829 — an absolute file position (§5.3) — comment length 0 |
|
|
1336
|
+
| 123003 | `-->` | wrapper close tag; the ZIP region ends here |
|
|
1337
|
+
| 123006 | `<sfz-extra-data>` … `</sfz-extra-data>` | recovery payload, appended placement (§5.2); 24 base64 characters for this archive |
|
|
1338
|
+
| 123063 | `</body></html>` | end tags; end of file at 123077 |
|
|
1339
|
+
|
|
1340
|
+
The appended run is 74 bytes, well inside the 65535-byte budget (§5.2). The ZIP region
|
|
1341
|
+
is the 998 bytes from 122005 to 123003; the universal extractor reproduces the first 996
|
|
1342
|
+
of them and supplies the last two itself (§1.3).
|
|
1343
|
+
|
|
1344
|
+
### 8.3 Specimens
|
|
1345
|
+
|
|
1346
|
+
Generated with the command-line client running `single-file-core` against
|
|
1347
|
+
`example.com`. The anatomy of §8.2 was regenerated against 1.5.108; the compatibility
|
|
1348
|
+
results of §8.1 were measured on the 1.5.107 build of the same specimen set, which
|
|
1349
|
+
differs only inside the prologue and so falls in the same classes: those are grouped
|
|
1350
|
+
by whether bytes precede the archive and follow the EOCD, which no prologue change
|
|
1351
|
+
alters. `--compress-content` makes the output an archive; `extract-data-from-page`
|
|
1352
|
+
defaults to true there, so the plain variant has to switch it off:
|
|
1353
|
+
|
|
1354
|
+
| Specimen | Command |
|
|
1355
|
+
|---|---|
|
|
1356
|
+
| pure zip | `single-file --compress-content --self-extracting-archive=false <url> pure.zip` |
|
|
1357
|
+
| plain | `single-file --compress-content --extract-data-from-page=false <url> plain.sfz.html` |
|
|
1358
|
+
| universal | `single-file --compress-content <url> universal.sfz.html` |
|
|
1359
|
+
| relocated | add `--prevent-appended-data` |
|
|
1360
|
+
| pdf | add `--embed-pdf` |
|
|
1361
|
+
| png | add `--embed-screenshot` |
|
|
1362
|
+
| png-pdf | add `--embed-screenshot --embed-pdf` |
|
|
1363
|
+
| zip-pdf, zip-png, zip-png-pdf | add `--self-extracting-archive=false` to the pdf, png and png-pdf rows |
|
|
1364
|
+
| password | add `--password=<password>` |
|
|
1365
|
+
|
|
1366
|
+
These specimens are deliberately small, and a reader tested only against them is
|
|
1367
|
+
undertested: they are all flat archives of two or three entries. None
|
|
1368
|
+
exercises a root directory, `frames/<n>/` nesting, a second `index.html`, a `data:`-URL
|
|
1369
|
+
entry comment, the optional text body or table of contents (§4.6), a UTF-8 BOM, zip64
|
|
1370
|
+
(§5.7), a payload past the 64 KB budget, or a relocated reservation with padding left
|
|
1371
|
+
in it. Two omissions matter more than the rest, because they are the parts of §5.1 a
|
|
1372
|
+
writer is most likely to get wrong: no specimen defeats a rung by its **start**
|
|
1373
|
+
pattern, and none defeats one with an **upper-case** pattern. A writer that tested only
|
|
1374
|
+
end patterns, or matched them case-sensitively, produces every specimen here unchanged
|
|
1375
|
+
— and the first of those two mistakes is one the reference writer actually shipped
|
|
1376
|
+
(§8.5).
|
|
1377
|
+
|
|
1378
|
+
Two specimens cannot be produced from a URL alone. The **ladder** specimen, which
|
|
1379
|
+
forces the second rung of §5.1, needs a page referencing an image whose stored bytes
|
|
1380
|
+
contain `-->`; the archive then wraps in `<script type=sfz-data>`. The **zip64** specimen requires
|
|
1381
|
+
an archive past the thresholds of §5.7, so it is produced by calling the writer
|
|
1382
|
+
directly with zip64 forced on the ZIP writer.
|
|
1383
|
+
|
|
1384
|
+
The measurements quoted elsewhere in this document come from the same harness: the
|
|
1385
|
+
payload growth rate of §5.2 (86 KB → 181 bytes, 283 KB → 465, 1.07 MB → 1645, 4.2 MB
|
|
1386
|
+
→ 6497, i.e. one byte of element per 650 bytes of archive) and the zip64 verification
|
|
1387
|
+
of §5.7.
|
|
1388
|
+
|
|
1389
|
+
### 8.4 The charset round trip, measured
|
|
1390
|
+
|
|
1391
|
+
Two claims of §2.1 were verified.
|
|
1392
|
+
|
|
1393
|
+
**Which encodings qualify.** Decoding all 256 byte values through each encoding
|
|
1394
|
+
defined by the WHATWG standard shows 20 that are injective and never produce U+FFFD:
|
|
1395
|
+
`windows-1252` (and its `iso-8859-1` labels), `iso-8859-2`, `-4`, `-5`, `-10`, `-13`,
|
|
1396
|
+
`-14`, `-15`, `-16`, `koi8-r`, `koi8-u`, `macintosh`, `windows-1250`, `-1251`,
|
|
1397
|
+
`-1254`, `-1256`, `-1258`, `x-mac-cyrillic`, `ibm866` and `x-user-defined`. The last of
|
|
1398
|
+
those qualifies on the criterion but is a poor choice in practice: it maps 0x80–0xFF
|
|
1399
|
+
into the Private Use Area, U+F780–U+F7FF, so the payload's characters have no meaning
|
|
1400
|
+
outside this round trip and any tool that touches the text sees private-use code points.
|
|
1401
|
+
The remaining single-byte encodings have undefined positions in their index —
|
|
1402
|
+
`iso-8859-3`, `-6`, `-7`, `-8`, `windows-874`, `-1253`, `-1255`, `-1257` — and the
|
|
1403
|
+
multi-byte ones (`utf-8`, `utf-16le`, `utf-16be`, `gbk`, `gb18030`, `big5`, `euc-jp`,
|
|
1404
|
+
`shift_jis`, `euc-kr`, `iso-2022-jp`) decode a lone byte sequence to U+FFFD or to fewer
|
|
1405
|
+
than 256 characters.
|
|
1406
|
+
The reverse table each one needs ranges from 8 entries (`iso-8859-15`) to 128
|
|
1407
|
+
(`koi8-r`, `koi8-u` and `ibm866`); windows-1252 needs 27.
|
|
1408
|
+
|
|
1409
|
+
**That the round trip is charset-independent.** The mechanism of §5.5 — parse, then
|
|
1410
|
+
re-encode with the reverse table, restoring newlines from the 2-bit codes and NUL from
|
|
1411
|
+
U+FFFD — was run in a browser on a 4364-byte payload containing every byte value, 49
|
|
1412
|
+
newlines covering LF, CR, CR LF and a trailing CR, and NUL bytes. It recovers the
|
|
1413
|
+
payload byte for byte under `windows-1252`, `iso-8859-15`, `iso-8859-5`, `koi8-r`,
|
|
1414
|
+
`ibm866` and `macintosh`, and fails under `utf-8`, as the injectivity requirement
|
|
1415
|
+
predicts.
|
|
1416
|
+
|
|
1417
|
+
### 8.5 Format history
|
|
1418
|
+
|
|
1419
|
+
| When | Change |
|
|
1420
|
+
|---|---|
|
|
1421
|
+
| before 2023 | The format originates in SingleFileZ, a separate extension pairing a self-extracting HTML page with a ZIP archive. The `data-sfz` and `<sfz-extra-data>` identifiers date from there (§1.3) |
|
|
1422
|
+
| October 2023 | SingleFileZ's core is merged into single-file-core; universal mode (`extractDataFromPage`) and the wrapper ladder arrive with it |
|
|
1423
|
+
| November 2023 | `preventAppendedData`: archives that end exactly at the EOCD record, with the payload relocated ahead of the archive |
|
|
1424
|
+
| January 2024 | The PNG face: a screenshot's chunks wrap the archive, the HTML riding in a `tEXt` chunk |
|
|
1425
|
+
| September 2024 | The PDF face: an embedded PDF document placed so its header falls in the scan window, first with the HTML face, then for archives without it |
|
|
1426
|
+
| August 2026 | The embedded PDF becomes the `page.pdf` ZIP entry, listed first, so ZIP tools see the document as an ordinary entry (core 1.5.93) |
|
|
1427
|
+
| August 2026 | Core 1.5.107: page-text extraction skips `page.pdf` (§4.5), the appended-data budget accounts for the PNG tail (§5.2), and archives served with an HTTP error status fall back to page-text extraction (§4.1) |
|
|
1428
|
+
| August 2026 | Core 1.5.108: `<noscript>` leaves the wrapper ladder (§5.1), the title is emitted as character references instead of being dropped in universal mode (§4.6), the doctype is capped when it would push `%PDF-` out of the scan window (§6.1), and password-protected archives withhold the entry comments, the title and the text body (§5.6) |
|
|
1429
|
+
| August 2026 | Core 1.5.108: the writer enforces the two remaining HTML restrictions on comment text — a payload may not end with `<!-`, and the PNG face may not open its comment with the `>` or `->` its chunk checksum lands on once in 256 archives, which closed the wrapper and left the image data to the parser (§5.1) |
|
|
1430
|
+
| August 2026 | Core 1.5.108: the wrapper ladder escalates to `<script type=sfz-data>` and `<style type=sfz-data>` before the raw-text rungs, the only two whose content local text extractors drop the way they drop a comment (§5.1) |
|
|
1431
|
+
| August 2026 | Core 1.5.108: the ZIP region carries the identifier `sfz-data` and the extractor addresses it with that instead of deducing it from its position beside `<sfz-extra-data>` (§4.5). This fixes universal extraction on the `<style type=sfz-data>` rung, where the reference extractor's own relocation of `style` elements into the head moved the region out from under the positional rule |
|
|
1432
|
+
| August 2026 | Core 1.5.108: the recovery payload stops two bytes short of the End Of Central Directory record, excluding its comment-length field (§1.3), which lets universal-mode archives declare their appended data as the archive comment — a writer option, for `java.util.zip` and the readers that reject undeclared trailing bytes (§4.2) |
|
|
1433
|
+
| August 2026 | Core 1.5.108: the PDF and PNG faces test a wrapper rung's start pattern as well as its end pattern, closing the same script-data escape hole the ZIP region was already guarded against — a face payload holding `<!--` and then `<script` took the `<script type=sfz-data>` rung and swallowed the rest of the document (§5.1) |
|
|
1434
|
+
| August 2026 | Core 1.5.108: the retry loop discards a relocation reservation at most once per build, so a payload sitting on the appended-data boundary cannot oscillate between the two placements forever (§6.2) |
|
|
1435
|
+
|
|
1436
|
+
This document was itself revised in August 2026, against core 1.5.108, after several
|
|
1437
|
+
independent reviews. One of them was a reader built from this specification alone, with
|
|
1438
|
+
no access to the reference code. It read every specimen correctly, which is the best
|
|
1439
|
+
evidence available that the format is implementable from this text, and it found
|
|
1440
|
+
thirteen defects in the text. The load-bearing corrections are in §5.1 (the
|
|
1441
|
+
start-pattern test is necessary, not conservative: script data's escape states let a
|
|
1442
|
+
payload defeat the end-tag test), §5.5 (the WHATWG index is not the platform codec of
|
|
1443
|
+
the same name), §4.5 (the offset shift is derivable from the recovered region, and
|
|
1444
|
+
`page.pdf` is reachable, just not by offset) and §1.3 (the appended data may be a
|
|
1445
|
+
declared archive comment). A second pass by the same implementation caught a regression
|
|
1446
|
+
the revision had introduced: giving `<plaintext>` a start pattern would have let 55
|
|
1447
|
+
bytes of ASCII defeat all eight rungs, which the reference writer never did and which
|
|
1448
|
+
§6.2's termination argument forbids. Later passes added a normative order for resolving
|
|
1449
|
+
the page entry, since "the shallowest `index.html`" had no tiebreak (§7.1); the
|
|
1450
|
+
distinction between the logical archive and the ZIP region, which `page.pdf` is the one
|
|
1451
|
+
entry to fall outside (§1.3); the layer convention now stated at the head of the
|
|
1452
|
+
document; the limits of the reconstructed-`page.pdf` CRC check (§4.5); the durability
|
|
1453
|
+
ranking of the faces (§1.1); what each face costs a writer (§6); and the silent loss of
|
|
1454
|
+
the other faces to a pipeline that repacks the file (§7.2). One review found a live
|
|
1455
|
+
defect rather than a documentation one, the non-monotone retry step recorded above.
|