@sanning/proof 0.16.0 → 0.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -165,17 +165,62 @@ Every event also appears in `result.events`, in body order, with a status that n
165
165
 
166
166
  Bytes come back as `Uint8Array`, never a string: the spec commits to **raw bytes**, and a UTF-8 decode is lossy for any payload that is not valid UTF-8, re-encoding such a string would change the bytes and break the hash. Decode to text yourself if you know your payload is text.
167
167
 
168
+ ## Check the files that came with a pack
169
+
170
+ A pack proves what its records signed. To ask whether it vouches for the files
171
+ you were handed beside it, such as a screenshot, a report or an export, name
172
+ them after `--files`. Each file is hashed with SHA-256 and looked up among what
173
+ the pack's signed records commit to. A folder stands for every regular file
174
+ inside it, so one argument covers a whole download on any operating system,
175
+ the Windows command prompt and PowerShell included (they do not expand a
176
+ wildcard such as `evidence/*`, and this command does not either):
177
+
178
+ ```bash
179
+ npx @sanning/proof verify pack/bundle.json --logs pack/logs --files evidence/
180
+ ```
181
+
182
+ The report lists the files the pack vouches for, the files that match no record
183
+ and the commitments no file was offered for. A match is by hash alone: file and
184
+ folder names are never compared, and the verdict and the exit code never move.
185
+
186
+ - A folder is read recursively, and hidden files count. Its files are listed as
187
+ `<folder>/<path inside>` in UTF-16 code unit order of that path, so both
188
+ kernels print one list in one order. Files and folders can be mixed in one
189
+ command.
190
+ - A symbolic link inside a folder is never followed. It is listed as skipped,
191
+ with the reason, as is anything that is not a regular file (a pipe, a socket)
192
+ and a file or folder the command cannot read. Nothing is dropped without
193
+ being named, and a skipped file is not a verdict about it.
194
+ - Two files with one hash are both listed, each under its own name. An empty
195
+ file is an ordinary file with the empty digest. A file is read in
196
+ pieces and hashed, so it is never held in memory. A file you name yourself is
197
+ read at any size; the files found inside folders share 1 GiB per run, the cap
198
+ the `.zip` reader uses, and one that would push past it is listed as skipped
199
+ (`too_large`) and not read, so a sparse file of a terabyte cannot hold the
200
+ command. `case` bounds its other reads of files on disk as well; the section on `case` lists them.
201
+ - A folder nested more than 32 levels deep, one holding more than 50,000 files
202
+ and folders, or one with a name that is not valid UTF-8, exits 2, as `case`
203
+ does. A path that is not a readable file or folder exits 2.
204
+
168
205
  ## Read several packs together: a case reading
169
206
 
170
207
  When a case spans more than one pack (two agents, two organizations, one hand-off between them), check the packs together. This is `readCase` from a terminal, with no page in the way:
171
208
 
172
209
  ```bash
210
+ npx @sanning/proof case audit-file.zip # a whole download, one command
173
211
  npx @sanning/proof case assessor-pack/ challenger-pack/
174
- unzip packs.zip -d case/ && npx @sanning/proof case case/ # a zip of several packs
175
- npx @sanning/proof case case/ --json # the full reading, for a script
212
+ npx @sanning/proof case case/ --json # the full reading, for a script
213
+ ```
214
+
215
+ Each argument is a folder, or a case file named as a `.zip`. A folder is one pack (`bundle.json` plus `logs/`), or a folder that holds several packs, such as an unzipped folder of packs. A `.zip` is read as the folder `unzip <name>.zip -d <name>` would make, so the archive and its unpacked folder give one reading, with no unzip step and on any operating system. A `.zip` inside a folder is a file like any other: it is hashed, never unzipped. Packs are found by the verify page's splitting rule, so the page and this command apply the same rule to the same unpacked tree: every `bundle.json` is a pack, a file belongs to the deepest pack whose `bundle.json` or `logs/` it sits in, and every other file is listed as outside every pack and is never read as part of a pack. Several folders are read as one folder holding them, each under its own name. Each `bundle.json` is read with `verify`'s own reader (`readBundleBytes`), so the two commands classify one bundle the same way.
216
+
217
+ **The files outside every pack are checked too, with no flag.** A download usually holds more than packs: a screenshot, a report, an export. `case` hashes each file that sits outside every pack, with SHA-256 over its exact bytes, and looks the digest up among what every pack's signed records commit to, the same check `verify --files` makes. The report lists each as vouched for, naming the record and its pack, or as matching no record:
218
+
219
+ ```bash
220
+ npx @sanning/proof case audit-file.zip
176
221
  ```
177
222
 
178
- Each argument is a folder: one pack (`bundle.json` plus `logs/`), or a folder that holds several packs, such as an unzipped folder of packs. `case` does not open archives: given a `.zip`, it exits `2` and says to unzip it first and pass the folder. Packs are found by the verify page's splitting rule, so the page and this command apply the same rule to the same unpacked tree: every `bundle.json` is a pack, a file belongs to the deepest pack whose `bundle.json` or `logs/` it sits in, and every other file is listed as outside every pack and never opened. A `.zip` inside the folder is a file like any other. Several folders are read as one folder holding them, each under its own name. Each `bundle.json` is read with `verify`'s own reader (`readBundleBytes`), so the two commands classify one bundle the same way.
223
+ This is information beside the case reading and never inside it: the verdict, the exit code and the summary are the same whether a file matches or not, and a changed file reads as no match, nothing more. A match is by hash alone, so no name or folder is compared. Files after `--files` are offered as well. Each file that cannot be hashed, such as a link, a pipe, a zip entry the reader refused (with the reader's own words) or a file that cannot be read, is listed as skipped with the reason, so every file outside a pack is matched, unmatched or skipped. Two files with one hash are both listed; an empty file is an ordinary file; a file is read in pieces. The files share 1 GiB in all, the cap the `.zip` reader uses: one that would push past it is listed as skipped (`too_large`) and not read, so a sparse file of a terabyte in a download cannot hold `case` at that step. Every other read `case` makes of a file on disk is bounded too, so a sparse file of a terabyte can neither stall `case` nor exhaust its memory. The index comparison hashes every file under `packs/` and `report/` in pieces, within its own 1 GiB, the files the index lists first, and a listed file it has no room left for is reported as `case_index_file_too_large`, never as a match and never as absent. A pack's own files (`bundle.json`, `logs/`), `contents.json` and `witness-cache.json` are read whole within one 1 GiB total across the run: a pack's own file with no room left exits 2 at once, naming the file (the words say the run's total is used up, not that the file is itself over 1 GiB), a `contents.json` with no room left reads as an unreadable index (`case_index_unreadable`), and a `witness-cache.json` with no room left reads as no cache, with a `note` line saying so. A `.zip` larger than 1.25 GiB on disk exits 2 before it is read. A file outside every pack, `contents.json` and `witness-cache.json` included, therefore never keeps `case` from its verdict, and one a pack owns, or an archive that large, is refused promptly instead. Without `--files` the report lists each file read and then one line, `N recorded commitment(s) matched no loose file and no log in their own pack`, with no line for each record; `--files` and `--json` list every commitment no file matched, those the count leaves out included. The section is printed when a file sits outside every pack or `--files` is given, so a case with neither prints what it printed before, and the `files` member of `--json` follows the same rule.
179
224
 
180
225
  The walk is bounded and follows no link. A symbolic link is never followed: it is listed as outside every pack and not read, even under a pack's `logs/`, and a link named on the command line exits `2`. A folder nested more than 32 levels deep, more than 50,000 files and folders in all, or a name that is not valid UTF-8 exits `2`; a bad name is refused rather than escaped, because two different names would then read as one. A `bundle.json` nested more than 512 levels deep exits `2`, from `case` and from `verify` alike. The rules are pinned for both kernels by `vectors/case-split.json`, shipped in this tarball, as directory trees.
181
226
 
@@ -238,7 +283,7 @@ The primary signature covers `JCS(envelope minus signature minus co_signatures)`
238
283
 
239
284
  ## Why you can trust it (without trusting us)
240
285
 
241
- Three **independent implementations** of this kernel exist: this package, the Python [`sanning-proof`](https://pypi.org/project/sanning-proof/) (PyPI), and and the MIT-licensed Go reference. The TypeScript and Python kernels are conformance-gated **byte-for-byte** in CI against one shared corpus, pinned per file by SHA-256 at cut tag `test-vectors-v4.4`, and a cross-kernel agreement gate re-runs the corpus plus adversarial negatives through both, asserting identical verdicts on identical bytes. The Go reference runs as a third leg of that same gate, built at the commit its vendoring pins.
286
+ Three **independent implementations** of this kernel exist: this package, the Python [`sanning-proof`](https://pypi.org/project/sanning-proof/) (PyPI), and the MIT-licensed Go reference. The TypeScript and Python kernels are conformance-gated **byte-for-byte** in CI against one shared corpus, pinned per file by SHA-256 at the newest cut tag, named in `test-vectors/CORPUS-v4.md`, and a cross-kernel agreement gate re-runs the corpus plus adversarial negatives through both, asserting identical verdicts on identical bytes. The Go reference runs as a third leg of that same gate, built at the commit its vendoring pins.
242
287
 
243
288
  That mutual gate is the point: anyone can verify these provenance records with **no Sanning code in the trust path**, write your own verifier against the specs that ship inside the [`sanning-proof`](https://pypi.org/project/sanning-proof/) sdist, and the corpus will tell you if it is conformant.
244
289
 
@@ -14,12 +14,12 @@
14
14
  // 4. Confirm the reconstructed root == checkpoint_envelope.payload.merkle_root.
15
15
  // 5. (optional) Re-fetch checkpoint_tx_id from a gateway and byte-compare.
16
16
  import { hexToBytes, utf8 } from "./crypto.js";
17
+ import { safeInteger } from "./grading.js";
17
18
  import { leafHash, verifyInclusion } from "./merkle.js";
18
19
  import { MAX_CANONICAL_DEPTH, exceedsDepth, jcs, jcsParse, verifyEnvelope } from "./verifier.js";
19
20
  export const AGENT_PROOF_SPEC_VERSION = "sanning.agent.proof/v1";
20
- function stringifyErr(e) {
21
- return e instanceof Error ? e.message : String(e);
22
- }
21
+ /** The sentence `agent_proof.py` writes when the proof's own fields are not usable. */
22
+ const MALFORMED_PROOF = "merkle_root / audit_path / leaf_index malformed";
23
23
  function toHex(bytes) {
24
24
  let out = "";
25
25
  for (const x of bytes)
@@ -44,26 +44,28 @@ export async function verifyAgentProof(bundle, options = {}) {
44
44
  errors: [error],
45
45
  });
46
46
  if (bundle === null || typeof bundle !== "object" || Array.isArray(bundle)) {
47
- return fail("agent proof bundle is not a JSON object");
47
+ return fail("bundle is not a JSON object");
48
48
  }
49
49
  // Bound nesting before any canonicalization (checkpoint envelope + Merkle
50
50
  // leaves are canonicalized recursively) — a fixed shared limit keeps this a
51
51
  // `malformed` verdict in every kernel. See MAX_CANONICAL_DEPTH.
52
52
  if (exceedsDepth(bundle, MAX_CANONICAL_DEPTH)) {
53
- return fail(`agent proof bundle nesting exceeds the ${MAX_CANONICAL_DEPTH}-level canonicalization depth bound`);
53
+ return fail(`bundle nesting exceeds the ${MAX_CANONICAL_DEPTH}-level canonicalization depth bound`);
54
54
  }
55
55
  const b = bundle;
56
56
  if (!isAgentProofSpec(b.spec_version)) {
57
- return fail(`unsupported agent-proof spec_version: ${JSON.stringify(b.spec_version)}`);
57
+ // An absent one reads `null`, as in `agent_proof.py` (`dict.get` gives
58
+ // None); `JSON.stringify(undefined)` would write the word `undefined`.
59
+ return fail(`unsupported bundle spec_version: ${JSON.stringify(b.spec_version ?? null)}`);
58
60
  }
59
61
  if (b.checkpoint_envelope === null || typeof b.checkpoint_envelope !== "object") {
60
- return fail("agent proof bundle missing checkpoint_envelope");
62
+ return fail("bundle missing checkpoint_envelope");
61
63
  }
62
64
  if (b.leaf === null || typeof b.leaf !== "object") {
63
- return fail("agent proof bundle missing leaf");
65
+ return fail("bundle missing leaf");
64
66
  }
65
67
  if (!Array.isArray(b.audit_path)) {
66
- return fail("agent proof bundle missing audit_path[]");
68
+ return fail(MALFORMED_PROOF);
67
69
  }
68
70
  const errors = [];
69
71
  // Step 1: verify the checkpoint envelope (inline payload — no payloadBytes).
@@ -85,27 +87,50 @@ export async function verifyAgentProof(bundle, options = {}) {
85
87
  const payload = (b.checkpoint_envelope.payload ?? {});
86
88
  if (typeof payload.merkle_root === "string")
87
89
  merkleRoot = payload.merkle_root;
88
- if (typeof payload.leaf_count === "number")
89
- leafCount = payload.leaf_count;
90
+ // `leaf_count` and `leaf_index` are read BY VALUE, the rule every
91
+ // inclusion reader in both kernels follows (evidence-bundle A38): `3.0` is
92
+ // 3, and a fraction, a value past 2**53 - 1, a string or a boolean is not
93
+ // an integer. `agent_proof.py` reads them through `_safe_integer`.
94
+ leafCount = safeInteger(payload.leaf_count) ?? undefined;
90
95
  if (merkleRoot === undefined)
91
96
  errors.push("checkpoint payload missing merkle_root");
92
97
  if (leafCount === undefined)
93
- errors.push("checkpoint payload missing leaf_count");
98
+ errors.push("checkpoint payload leaf_count is not an integer");
94
99
  }
95
- catch (e) {
96
- errors.push(`checkpoint verify error: ${stringifyErr(e)}`);
100
+ catch {
101
+ // The sentence `agent_proof.py` writes for the same failure. The thrown
102
+ // text is the host's, and a public result carries the kernel's words only
103
+ // (proof#150).
104
+ errors.push("checkpoint envelope verification failed");
97
105
  }
98
106
  // Steps 2-4: leaf hash, audit-path walk, root match.
99
107
  let inclusionOk = false;
100
108
  if (merkleRoot !== undefined && leafCount !== undefined) {
109
+ let lh;
101
110
  try {
102
- const lh = await leafHash(utf8(jcs(b.leaf)));
103
- inclusionOk = await verifyInclusion(lh, b.leaf_index, leafCount, b.audit_path.map((h) => hexToBytes(h)), hexToBytes(merkleRoot));
104
- if (!inclusionOk)
105
- errors.push("RFC 9162 inclusion proof did not reconstruct merkle_root");
111
+ lh = await leafHash(utf8(jcs(b.leaf)));
112
+ }
113
+ catch {
114
+ errors.push("leaf is not JCS-canonicalizable");
115
+ }
116
+ const leafIndex = safeInteger(b.leaf_index);
117
+ if (lh !== undefined && leafIndex === null) {
118
+ // `agent_proof.py` refuses a leaf_index that is not an integer with this
119
+ // sentence. Without the check a string, `null` or `1.5` reaches the walk
120
+ // and reads as a failed inclusion proof instead (proof#150).
121
+ errors.push(MALFORMED_PROOF);
106
122
  }
107
- catch (e) {
108
- errors.push(`inclusion verify error: ${stringifyErr(e)}`);
123
+ else if (lh !== undefined) {
124
+ try {
125
+ inclusionOk = await verifyInclusion(lh, leafIndex, leafCount, b.audit_path.map((h) => hexToBytes(h)), hexToBytes(merkleRoot));
126
+ if (!inclusionOk)
127
+ errors.push("audit_path does not reconstruct merkle_root");
128
+ }
129
+ catch {
130
+ // `agent_proof.py`'s sentence for the same failure; the thrown text is
131
+ // the host's (proof#150).
132
+ errors.push(MALFORMED_PROOF);
133
+ }
109
134
  }
110
135
  }
111
136
  // Step 5: optional on-chain re-fetch.
@@ -144,7 +169,9 @@ export async function verifyAgentProof(bundle, options = {}) {
144
169
  inclusionOk,
145
170
  onChainOk,
146
171
  onChainChecked,
147
- leafIndex: typeof b.leaf_index === "number" ? b.leaf_index : -1,
172
+ // Read by value, as the walk above read it: a `2.5` or `1e+21` index is
173
+ // reported as no index, never printed beside a verdict that refused it.
174
+ leafIndex: safeInteger(b.leaf_index) ?? -1,
148
175
  leafCount: leafCount ?? null,
149
176
  checkpointTxId: typeof b.checkpoint_tx_id === "string" ? b.checkpoint_tx_id : null,
150
177
  errors,
@@ -155,8 +182,8 @@ async function refetch(envelope, txId, gateways, fetchImpl) {
155
182
  try {
156
183
  expectedHex = toHex(utf8(jcs(envelope)));
157
184
  }
158
- catch (e) {
159
- return { ok: false, unavailable: false, error: `cannot canonicalize checkpoint envelope: ${stringifyErr(e)}` };
185
+ catch {
186
+ return { ok: false, unavailable: false, error: "checkpoint envelope is not JCS-canonicalizable" };
160
187
  }
161
188
  let anyReachable = false;
162
189
  for (const gw of gateways) {
@@ -42,7 +42,7 @@ export declare const MAX_INDEX_DEPTH = 512;
42
42
  * Every finding this comparison can report, in the order it reports them
43
43
  * (5.6.3). None of them is a verdict word and none contributes to one.
44
44
  */
45
- export declare const CASE_FILE_FINDING_KINDS: readonly ["case_index_unreadable", "case_index_pack_missing", "case_index_pack_unlisted", "case_index_file_missing", "case_index_file_unlisted", "case_index_file_differs", "case_index_names_absent"];
45
+ export declare const CASE_FILE_FINDING_KINDS: readonly ["case_index_unreadable", "case_index_pack_missing", "case_index_pack_unlisted", "case_index_file_missing", "case_index_file_unlisted", "case_index_file_differs", "case_index_file_too_large", "case_index_names_absent"];
46
46
  export type CaseFileFindingKind = (typeof CASE_FILE_FINDING_KINDS)[number];
47
47
  /**
48
48
  * Whether `ref` is a legal pack folder name in a case file. For a producer; a
@@ -98,17 +98,36 @@ export interface CaseFileIndexReading {
98
98
  * The CLI's `bundle.json` reader uses it too.
99
99
  */
100
100
  export declare function jsonDepthExceeds(text: string | Uint8Array, limit: number): boolean;
101
+ /**
102
+ * The reading of an index the caller could not hold at all (it is over the
103
+ * run's whole-read total, or it cannot be read): one `case_index_unreadable`
104
+ * finding with the caller's `reason`, and no comparison. The index is never an
105
+ * input to the verdict, so a file that cannot be read keeps nothing from it.
106
+ */
107
+ export declare function unreadableCaseFileIndex(reason: string): CaseFileIndexReading;
108
+ /**
109
+ * Every path the index lists (each pack's files and `report`), or `null` where
110
+ * the index is unreadable. A caller that charges a budget for hashing charges
111
+ * these files first, so a file the index does not list, which needs no digest,
112
+ * cannot turn a real mismatch into `too_large`.
113
+ */
114
+ export declare function caseFileListedPaths(index: Uint8Array): ReadonlySet<string> | null;
101
115
  /**
102
116
  * Compare a case file's index with the folder it sits at the root of.
103
117
  *
104
118
  * `index` is the bytes of `contents.json`, or `null` where the folder holds
105
119
  * none. `files` is every regular file in the folder, as `{ path, sha256 }`
106
- * with `path` from the folder's root, `/`-separated; only the files under
120
+ * with `path` from the folder's root, `/`-separated, or `{ path, too_large: true }`
121
+ * for a file the caller did not hash because it is over its cap (reported as
122
+ * `case_index_file_too_large` where the index lists it, never as a match or as
123
+ * absent); only the files under
107
124
  * `packs/` and `report/` are compared, so a caller may list the rest or not. A
108
125
  * folder with no index is a folder of packs, with no finding.
109
126
  *
110
127
  * Findings come in the order of `CASE_FILE_FINDING_KINDS`, and within a kind in
111
- * the order of their `ref` or `path` by UTF-8 bytes; `case_index_names_absent`
128
+ * the order of their `ref` or `path` by UTF-8 bytes (the order a caller charges
129
+ * its hashing budget in is another, UTF-16 code units: evidence-bundle 5.6.3);
130
+ * `case_index_names_absent`
112
131
  * is in the index's own order, one per `absent[]` entry. An unreadable index is
113
132
  * one finding and no other comparison runs.
114
133
  *
@@ -116,7 +135,8 @@ export declare function jsonDepthExceeds(text: string | Uint8Array, limit: numbe
116
135
  */
117
136
  export declare function readCaseFileIndex(index: Uint8Array | null, files: readonly {
118
137
  path: string;
119
- sha256: string;
138
+ sha256?: string;
139
+ too_large?: boolean;
120
140
  }[]): CaseFileIndexReading;
121
141
  /**
122
142
  * The house wording for one finding: what the index says against what the
package/dist/case-file.js CHANGED
@@ -50,6 +50,7 @@ export const CASE_FILE_FINDING_KINDS = [
50
50
  "case_index_file_missing",
51
51
  "case_index_file_unlisted",
52
52
  "case_index_file_differs",
53
+ "case_index_file_too_large",
53
54
  "case_index_names_absent",
54
55
  ];
55
56
  // `<ref>`: 1 to 128 characters from A-Z, a-z, 0-9, `.`, `_` and `-`, and not
@@ -218,17 +219,59 @@ function readIndex(raw) {
218
219
  const report = fileList(own(value, "report"), "report");
219
220
  return { caseId, packs: listedPacks, absent: listedAbsent, report };
220
221
  }
222
+ /**
223
+ * The reading of an index the caller could not hold at all (it is over the
224
+ * run's whole-read total, or it cannot be read): one `case_index_unreadable`
225
+ * finding with the caller's `reason`, and no comparison. The index is never an
226
+ * input to the verdict, so a file that cannot be read keeps nothing from it.
227
+ */
228
+ export function unreadableCaseFileIndex(reason) {
229
+ return {
230
+ index: "unreadable",
231
+ caseId: null,
232
+ findings: [{ kind: "case_index_unreadable", reason }],
233
+ };
234
+ }
235
+ /**
236
+ * Every path the index lists (each pack's files and `report`), or `null` where
237
+ * the index is unreadable. A caller that charges a budget for hashing charges
238
+ * these files first, so a file the index does not list, which needs no digest,
239
+ * cannot turn a real mismatch into `too_large`.
240
+ */
241
+ export function caseFileListedPaths(index) {
242
+ let read;
243
+ try {
244
+ read = readIndex(index);
245
+ }
246
+ catch (e) {
247
+ if (!(e instanceof Unreadable))
248
+ throw e;
249
+ return null;
250
+ }
251
+ const paths = new Set();
252
+ for (const [, packFiles] of read.packs)
253
+ for (const [path] of packFiles)
254
+ paths.add(path);
255
+ for (const [path] of read.report)
256
+ paths.add(path);
257
+ return paths;
258
+ }
221
259
  /**
222
260
  * Compare a case file's index with the folder it sits at the root of.
223
261
  *
224
262
  * `index` is the bytes of `contents.json`, or `null` where the folder holds
225
263
  * none. `files` is every regular file in the folder, as `{ path, sha256 }`
226
- * with `path` from the folder's root, `/`-separated; only the files under
264
+ * with `path` from the folder's root, `/`-separated, or `{ path, too_large: true }`
265
+ * for a file the caller did not hash because it is over its cap (reported as
266
+ * `case_index_file_too_large` where the index lists it, never as a match or as
267
+ * absent); only the files under
227
268
  * `packs/` and `report/` are compared, so a caller may list the rest or not. A
228
269
  * folder with no index is a folder of packs, with no finding.
229
270
  *
230
271
  * Findings come in the order of `CASE_FILE_FINDING_KINDS`, and within a kind in
231
- * the order of their `ref` or `path` by UTF-8 bytes; `case_index_names_absent`
272
+ * the order of their `ref` or `path` by UTF-8 bytes (the order a caller charges
273
+ * its hashing budget in is another, UTF-16 code units: evidence-bundle 5.6.3);
274
+ * `case_index_names_absent`
232
275
  * is in the index's own order, one per `absent[]` entry. An unreadable index is
233
276
  * one finding and no other comparison runs.
234
277
  *
@@ -251,13 +294,20 @@ export function readCaseFileIndex(index, files) {
251
294
  findings: [{ kind: "case_index_unreadable", reason: e.reason }],
252
295
  };
253
296
  }
297
+ // `null` is a file that is here and was too large to hash (a file entry of
298
+ // `{ path, too_large: true }`): neither a match nor absent, reported as
299
+ // neither.
254
300
  const here = new Map();
255
301
  for (const entry of files) {
256
302
  const path = isObject(entry) ? own(entry, "path") : undefined;
257
303
  const digest = isObject(entry) ? own(entry, "sha256") : undefined;
258
- if (typeof path === "string" && typeof digest === "string" && !here.has(path)) {
304
+ const tooLarge = isObject(entry) && own(entry, "too_large") === true;
305
+ if (typeof path !== "string" || here.has(path))
306
+ continue;
307
+ if (typeof digest === "string")
259
308
  here.set(path, digest);
260
- }
309
+ else if (tooLarge)
310
+ here.set(path, null);
261
311
  }
262
312
  const listedRefs = new Set(read.packs.map(([ref]) => ref));
263
313
  // The first listing of a path wins, in index order: packs in order, each
@@ -305,7 +355,7 @@ export function readCaseFileIndex(index, files) {
305
355
  }
306
356
  for (const path of listedPaths) {
307
357
  const actual = here.get(path);
308
- if (actual !== undefined && actual !== listed.get(path)) {
358
+ if (actual !== undefined && actual !== null && actual !== listed.get(path)) {
309
359
  findings.push({
310
360
  kind: "case_index_file_differs",
311
361
  path,
@@ -314,6 +364,10 @@ export function readCaseFileIndex(index, files) {
314
364
  });
315
365
  }
316
366
  }
367
+ for (const path of listedPaths) {
368
+ if (here.get(path) === null)
369
+ findings.push({ kind: "case_index_file_too_large", path });
370
+ }
317
371
  for (const [reason, key] of read.absent) {
318
372
  findings.push(key === null
319
373
  ? { kind: "case_index_names_absent", reason }
@@ -341,6 +395,9 @@ export function renderCaseFileFinding(finding) {
341
395
  case "case_index_file_differs":
342
396
  return (`the index lists ${finding.path} with SHA-256 ${finding.listedSha256}; ` +
343
397
  `the file here hashes to ${finding.sha256}`);
398
+ case "case_index_file_too_large":
399
+ return (`the index lists ${finding.path}; hashing the file here would take case ` +
400
+ "past the 1 GiB it hashes in one run, so it was not compared");
344
401
  case "case_index_names_absent":
345
402
  if (finding.publicKey !== undefined) {
346
403
  return ("the assembler says a pack signed by key " +
package/dist/case.d.ts CHANGED
@@ -660,6 +660,41 @@ export type CaseNoteKind = (typeof CASE_NOTE_KINDS)[number];
660
660
  */
661
661
  export declare const CASE_EXCEPTION_KINDS: readonly ["counterpart_absent", "counterpart_absent_from_counterparty_records", "counterpart_absent_signer_unknown", "counterpart_ambiguous", "counterpart_malformed", "pair_failed", "record_malformed", "record_unclaimed_by_its_pack", "record_bytes_withheld", "record_bytes_unreadable", "pack_declaration_deviates", "pack_declaration_content_dependent", "pack_declaration_unreadable", "pack_undeclared", "pack_body_type_unknown", "pack_holds_no_records", "pack_key_unreadable", "pack_does_not_verify", "pack_grade_not_verified", "pack_bundle_content_dependent", "record_envelope_does_not_verify", "record_does_not_verify_in_its_pack", "record_content_mismatch", "record_content_dependent", "record_profile_quarantined", "case_holds_no_packs"];
662
662
  export type CaseExceptionKind = (typeof CASE_EXCEPTION_KINDS)[number];
663
+ /**
664
+ * The status every record in every pack carries (evidence-bundle 5.5.4
665
+ * v1.35, A43, proof#204), as a closed set. A record is listed for each element
666
+ * of its pack's `body.events` as `packRecords` reads them, or, for a
667
+ * `sanning.evidence.export/v2` pack, of the events of the inline source
668
+ * bundle it wraps (`resolveTraceEvents`), and gets exactly one:
669
+ *
670
+ * - `read_clean`: the record went into the pile, no exception carries its
671
+ * `packIndex` and `recordIndex`, and its pack's declaration check
672
+ * names it by nothing (`declarationFindings` is empty). It says nothing
673
+ * about its PACK: the pack's own grade and exceptions are on the pack, and a
674
+ * surface MUST show that grade beside the status.
675
+ * - `read_flagged`: the record went into the pile, and at least one
676
+ * exception carries its `packIndex` and `recordIndex`, or its pack's
677
+ * declaration check names its `eventId` by a deviation or a content
678
+ * finding (`declarationFindings` is not empty; under an `eventId` the
679
+ * pack carries twice, every record with that id). An exception about the
680
+ * pack alone flags no record.
681
+ * - `not_read`: the record never went into the pile. `reason` is the kind
682
+ * of the exception that says why, on the record
683
+ * (`record_profile_quarantined`, `record_envelope_does_not_verify`,
684
+ * `record_bytes_withheld`, `record_bytes_unreadable`) or on its pack
685
+ * (`pack_key_unreadable`, `pack_does_not_verify`,
686
+ * `pack_body_type_unknown`, `pack_grade_not_verified` where the body
687
+ * could not be read or the verifier graded a different number of records).
688
+ * An export's wrapped records carry the reason of the exception their pack
689
+ * actually got: `pack_body_type_unknown` where its wrapper holds, and
690
+ * otherwise `pack_key_unreadable`, `pack_does_not_verify` or
691
+ * `pack_grade_not_verified`.
692
+ *
693
+ * Before A43 the reading listed only the records it read, so a surface had to
694
+ * work out which records were missing, and why, by a rule of its own.
695
+ */
696
+ export declare const CASE_RECORD_STATUSES: readonly ["read_clean", "read_flagged", "not_read"];
697
+ export type CaseRecordStatus = (typeof CASE_RECORD_STATUSES)[number];
663
698
  /**
664
699
  * One pack in a case, as a reader was handed it.
665
700
  *
@@ -682,10 +717,25 @@ export interface CasePack {
682
717
  * `renderException`. */
683
718
  export interface CaseException {
684
719
  kind: CaseExceptionKind;
685
- /** Which pack this is about, by the reader's own name for it. */
720
+ /**
721
+ * Which pack this is about, by the reader's own name for it. Two packs can
722
+ * carry one `ref`, so it is a label and never the join: `packIndex` is.
723
+ */
686
724
  packRef?: string;
725
+ /**
726
+ * Which pack this is about, by its position in `CaseReading.packs` (0
727
+ * first), present wherever `packRef` is (A43). Unique in the reading.
728
+ */
729
+ packIndex?: number;
687
730
  /** Which record, by its envelope `event_id`. */
688
731
  eventId?: string;
732
+ /**
733
+ * Which record, by its position in its pack's `body.events` (0 first),
734
+ * present wherever `eventId` is (A43). One pack can carry one `event_id`
735
+ * twice, so `packIndex` and `recordIndex` together are the join to
736
+ * `CaseReading.records`.
737
+ */
738
+ recordIndex?: number;
689
739
  /**
690
740
  * The counterparty key the record named, for the two counterpart kinds.
691
741
  * Normalized (`normalizedSigningKey`), so a reader compares what this module
@@ -717,20 +767,54 @@ export interface CaseException {
717
767
  reasons?: string[];
718
768
  }
719
769
  /**
720
- * 4.1's reading of one record the case holds, with the pack it came from.
770
+ * One record of one pack, with its status and, where it was read, 4.1's
771
+ * reading of it.
721
772
  *
722
- * EVERY record every pack supplied and this reader could read gets one of
723
- * these, in pack order and then in bundle order, including the ones that are
724
- * not hand-off records at all (`findings` carries `not_a_handoff_record`). 4.1
725
- * (v1.9, A14) requires that and names the reason: the one thing an attacker
726
- * wants is for a record to be absent from the output.
773
+ * EVERY record of every pack gets one of these (A43), in pack order and then
774
+ * in bundle order: the ones this reader read, including the ones that are not
775
+ * hand-off records at all (`findings` carries `not_a_handoff_record`), and the
776
+ * ones it did not, with `status` `not_read` and the `reason`. 4.1 (v1.9, A14)
777
+ * names why: the one thing an attacker wants is for a record to be absent from
778
+ * the output. An export's wrapped records are listed, `not_read`. A pack whose
779
+ * body carries no `events` array, and wraps none, lists none, and its own
780
+ * exception says why.
727
781
  */
728
782
  export interface CaseRecordReading {
729
783
  packRef: string;
784
+ /** The pack, by its position in `CaseReading.packs` (A43). */
785
+ packIndex: number;
730
786
  eventId: string;
787
+ /**
788
+ * The record, by its position in the list of records its pack lists (A43):
789
+ * `body.events`, or for a `sanning.evidence.export/v2` pack the events of the
790
+ * source bundle it wraps.
791
+ */
792
+ recordIndex: number;
793
+ /** One of `CASE_RECORD_STATUSES` (A43). */
794
+ status: CaseRecordStatus;
795
+ /**
796
+ * For `not_read`, the kind of the exception that says why; `null` for a
797
+ * record that was read.
798
+ */
799
+ reason: CaseExceptionKind | null;
800
+ /**
801
+ * The kinds its pack's declaration check names this record by, deviations
802
+ * (`DECLARATION_DEVIATION_KINDS`) then content findings
803
+ * (`DECLARATION_CONTENT_KINDS`), each in the check's own order. Matched on
804
+ * `eventId`, so under an `event_id` the pack carries twice every record with
805
+ * that id carries it. A record that went into the pile with any is
806
+ * `read_flagged` (A43).
807
+ */
808
+ declarationFindings: string[];
809
+ /** 4.1's verdict, or `null` where 4.1 gave none or the record was not read. */
731
810
  verdict: string | null;
732
811
  findings: string[];
733
812
  failedChecks: string[];
813
+ /**
814
+ * 4.1's candidate counterparts, each as `<pack ref>/<event_id>`. A LABEL for
815
+ * a person to read and never the join: two packs can carry one ref, and one
816
+ * pack one `event_id` twice (A43).
817
+ */
734
818
  counterparts: string[];
735
819
  /**
736
820
  * The role the pack's v2 declaration signs for this record, `handoff` or
@@ -747,6 +831,8 @@ export interface CaseNote {
747
831
  kind: CaseNoteKind;
748
832
  /** Which pack, by the reader's own name for it. */
749
833
  packRef: string;
834
+ /** Which pack, by its position in `CaseReading.packs` (A43). */
835
+ packIndex: number;
750
836
  /**
751
837
  * For `CONTENT_WITHHELD_BY_DECLARATION`, how many of the pack's records its
752
838
  * declaration withholds.
@@ -759,6 +845,13 @@ export interface CaseNote {
759
845
  * its wrapper, and never by an organization (D10).
760
846
  */
761
847
  export interface CasePackReading {
848
+ /**
849
+ * The pack's position in the order the reader was given the packs, 0 first
850
+ * (A43): the one name for a pack that is unique in the reading. Every
851
+ * exception, note and record names its pack by it as `packIndex`.
852
+ */
853
+ index: number;
854
+ /** The reader's own name for the pack. Two packs can carry the same one. */
762
855
  ref: string;
763
856
  /** The wrapper key, normalized, or `null` where this reader could not read it. */
764
857
  signingKey: string | null;
@@ -790,7 +883,10 @@ export interface CaseReading {
790
883
  * reachable only together with `verdict === "verified"`.
791
884
  */
792
885
  exceptions: CaseException[];
793
- /** One reading per record the case holds and this reader could read. */
886
+ /**
887
+ * One entry per record of every pack, read or not, in pack order and then
888
+ * bundle order, each with its status (A43).
889
+ */
794
890
  records: CaseRecordReading[];
795
891
  /**
796
892
  * Every `[contributor, verdict]` the case verdict was computed from, in