@sanning/proof 0.15.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +49 -4
- package/dist/agent-proof.js +50 -23
- package/dist/case-file.d.ts +24 -4
- package/dist/case-file.js +62 -5
- package/dist/case.d.ts +104 -8
- package/dist/case.js +195 -53
- package/dist/cli.d.ts +105 -2
- package/dist/cli.js +664 -123
- package/dist/compose.js +32 -0
- package/dist/declaration.d.ts +1 -1
- package/dist/declaration.js +182 -3
- package/dist/evidence.js +108 -38
- package/dist/file-match.d.ts +64 -5
- package/dist/file-match.js +113 -22
- package/dist/grading.d.ts +7 -0
- package/dist/grading.js +14 -2
- package/dist/index.d.ts +4 -4
- package/dist/index.js +2 -2
- package/dist/merkle.js +13 -7
- package/dist/verifier.js +24 -19
- package/dist/zip-reader.d.ts +7 -0
- package/dist/zip-reader.js +7 -0
- package/package.json +2 -2
- package/src/agent-proof.ts +53 -27
- package/src/case-file.ts +61 -8
- package/src/case.ts +293 -64
- package/src/cli.ts +703 -132
- package/src/compose.ts +33 -0
- package/src/declaration.ts +186 -3
- package/src/evidence.ts +109 -43
- package/src/file-match.ts +160 -26
- package/src/grading.ts +15 -2
- package/src/index.ts +6 -0
- package/src/merkle.ts +13 -7
- package/src/verifier.ts +26 -22
- package/src/zip-reader.ts +7 -0
- package/vectors/README.md +5 -1
- package/vectors/case-file-index.json +88 -2
- package/vectors/file-match.json +135 -1
package/README.md
CHANGED
|
@@ -165,17 +165,62 @@ Every event also appears in `result.events`, in body order, with a status that n
|
|
|
165
165
|
|
|
166
166
|
Bytes come back as `Uint8Array`, never a string: the spec commits to **raw bytes**, and a UTF-8 decode is lossy for any payload that is not valid UTF-8, re-encoding such a string would change the bytes and break the hash. Decode to text yourself if you know your payload is text.
|
|
167
167
|
|
|
168
|
+
## Check the files that came with a pack
|
|
169
|
+
|
|
170
|
+
A pack proves what its records signed. To ask whether it vouches for the files
|
|
171
|
+
you were handed beside it, such as a screenshot, a report or an export, name
|
|
172
|
+
them after `--files`. Each file is hashed with SHA-256 and looked up among what
|
|
173
|
+
the pack's signed records commit to. A folder stands for every regular file
|
|
174
|
+
inside it, so one argument covers a whole download on any operating system,
|
|
175
|
+
the Windows command prompt and PowerShell included (they do not expand a
|
|
176
|
+
wildcard such as `evidence/*`, and this command does not either):
|
|
177
|
+
|
|
178
|
+
```bash
|
|
179
|
+
npx @sanning/proof verify pack/bundle.json --logs pack/logs --files evidence/
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
The report lists the files the pack vouches for, the files that match no record
|
|
183
|
+
and the commitments no file was offered for. A match is by hash alone: file and
|
|
184
|
+
folder names are never compared, and the verdict and the exit code never move.
|
|
185
|
+
|
|
186
|
+
- A folder is read recursively, and hidden files count. Its files are listed as
|
|
187
|
+
`<folder>/<path inside>` in UTF-16 code unit order of that path, so both
|
|
188
|
+
kernels print one list in one order. Files and folders can be mixed in one
|
|
189
|
+
command.
|
|
190
|
+
- A symbolic link inside a folder is never followed. It is listed as skipped,
|
|
191
|
+
with the reason, as is anything that is not a regular file (a pipe, a socket)
|
|
192
|
+
and a file or folder the command cannot read. Nothing is dropped without
|
|
193
|
+
being named, and a skipped file is not a verdict about it.
|
|
194
|
+
- Two files with one hash are both listed, each under its own name. An empty
|
|
195
|
+
file is an ordinary file with the empty digest. A file is read in
|
|
196
|
+
pieces and hashed, so it is never held in memory. A file you name yourself is
|
|
197
|
+
read at any size; the files found inside folders share 1 GiB per run, the cap
|
|
198
|
+
the `.zip` reader uses, and one that would push past it is listed as skipped
|
|
199
|
+
(`too_large`) and not read, so a sparse file of a terabyte cannot hold the
|
|
200
|
+
command. `case` bounds its other reads of files on disk as well; the section on `case` lists them.
|
|
201
|
+
- A folder nested more than 32 levels deep, one holding more than 50,000 files
|
|
202
|
+
and folders, or one with a name that is not valid UTF-8, exits 2, as `case`
|
|
203
|
+
does. A path that is not a readable file or folder exits 2.
|
|
204
|
+
|
|
168
205
|
## Read several packs together: a case reading
|
|
169
206
|
|
|
170
207
|
When a case spans more than one pack (two agents, two organizations, one hand-off between them), check the packs together. This is `readCase` from a terminal, with no page in the way:
|
|
171
208
|
|
|
172
209
|
```bash
|
|
210
|
+
npx @sanning/proof case audit-file.zip # a whole download, one command
|
|
173
211
|
npx @sanning/proof case assessor-pack/ challenger-pack/
|
|
174
|
-
|
|
175
|
-
|
|
212
|
+
npx @sanning/proof case case/ --json # the full reading, for a script
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
Each argument is a folder, or a case file named as a `.zip`. A folder is one pack (`bundle.json` plus `logs/`), or a folder that holds several packs, such as an unzipped folder of packs. A `.zip` is read as the folder `unzip <name>.zip -d <name>` would make, so the archive and its unpacked folder give one reading, with no unzip step and on any operating system. A `.zip` inside a folder is a file like any other: it is hashed, never unzipped. Packs are found by the verify page's splitting rule, so the page and this command apply the same rule to the same unpacked tree: every `bundle.json` is a pack, a file belongs to the deepest pack whose `bundle.json` or `logs/` it sits in, and every other file is listed as outside every pack and is never read as part of a pack. Several folders are read as one folder holding them, each under its own name. Each `bundle.json` is read with `verify`'s own reader (`readBundleBytes`), so the two commands classify one bundle the same way.
|
|
216
|
+
|
|
217
|
+
**The files outside every pack are checked too, with no flag.** A download usually holds more than packs: a screenshot, a report, an export. `case` hashes each file that sits outside every pack, with SHA-256 over its exact bytes, and looks the digest up among what every pack's signed records commit to, the same check `verify --files` makes. The report lists each as vouched for, naming the record and its pack, or as matching no record:
|
|
218
|
+
|
|
219
|
+
```bash
|
|
220
|
+
npx @sanning/proof case audit-file.zip
|
|
176
221
|
```
|
|
177
222
|
|
|
178
|
-
|
|
223
|
+
This is information beside the case reading and never inside it: the verdict, the exit code and the summary are the same whether a file matches or not, and a changed file reads as no match, nothing more. A match is by hash alone, so no name or folder is compared. Files after `--files` are offered as well. Each file that cannot be hashed, such as a link, a pipe, a zip entry the reader refused (with the reader's own words) or a file that cannot be read, is listed as skipped with the reason, so every file outside a pack is matched, unmatched or skipped. Two files with one hash are both listed; an empty file is an ordinary file; a file is read in pieces. The files share 1 GiB in all, the cap the `.zip` reader uses: one that would push past it is listed as skipped (`too_large`) and not read, so a sparse file of a terabyte in a download cannot hold `case` at that step. Every other read `case` makes of a file on disk is bounded too, so a sparse file of a terabyte can neither stall `case` nor exhaust its memory. The index comparison hashes every file under `packs/` and `report/` in pieces, within its own 1 GiB, the files the index lists first, and a listed file it has no room left for is reported as `case_index_file_too_large`, never as a match and never as absent. A pack's own files (`bundle.json`, `logs/`), `contents.json` and `witness-cache.json` are read whole within one 1 GiB total across the run: a pack's own file with no room left exits 2 at once, naming the file (the words say the run's total is used up, not that the file is itself over 1 GiB), a `contents.json` with no room left reads as an unreadable index (`case_index_unreadable`), and a `witness-cache.json` with no room left reads as no cache, with a `note` line saying so. A `.zip` larger than 1.25 GiB on disk exits 2 before it is read. A file outside every pack, `contents.json` and `witness-cache.json` included, therefore never keeps `case` from its verdict, and one a pack owns, or an archive that large, is refused promptly instead. Without `--files` the report lists each file read and then one line, `N recorded commitment(s) matched no loose file and no log in their own pack`, with no line for each record; `--files` and `--json` list every commitment no file matched, those the count leaves out included. The section is printed when a file sits outside every pack or `--files` is given, so a case with neither prints what it printed before, and the `files` member of `--json` follows the same rule.
|
|
179
224
|
|
|
180
225
|
The walk is bounded and follows no link. A symbolic link is never followed: it is listed as outside every pack and not read, even under a pack's `logs/`, and a link named on the command line exits `2`. A folder nested more than 32 levels deep, more than 50,000 files and folders in all, or a name that is not valid UTF-8 exits `2`; a bad name is refused rather than escaped, because two different names would then read as one. A `bundle.json` nested more than 512 levels deep exits `2`, from `case` and from `verify` alike. The rules are pinned for both kernels by `vectors/case-split.json`, shipped in this tarball, as directory trees.
|
|
181
226
|
|
|
@@ -238,7 +283,7 @@ The primary signature covers `JCS(envelope minus signature minus co_signatures)`
|
|
|
238
283
|
|
|
239
284
|
## Why you can trust it (without trusting us)
|
|
240
285
|
|
|
241
|
-
Three **independent implementations** of this kernel exist: this package, the Python [`sanning-proof`](https://pypi.org/project/sanning-proof/) (PyPI), and
|
|
286
|
+
Three **independent implementations** of this kernel exist: this package, the Python [`sanning-proof`](https://pypi.org/project/sanning-proof/) (PyPI), and the MIT-licensed Go reference. The TypeScript and Python kernels are conformance-gated **byte-for-byte** in CI against one shared corpus, pinned per file by SHA-256 at the newest cut tag, named in `test-vectors/CORPUS-v4.md`, and a cross-kernel agreement gate re-runs the corpus plus adversarial negatives through both, asserting identical verdicts on identical bytes. The Go reference runs as a third leg of that same gate, built at the commit its vendoring pins.
|
|
242
287
|
|
|
243
288
|
That mutual gate is the point: anyone can verify these provenance records with **no Sanning code in the trust path**, write your own verifier against the specs that ship inside the [`sanning-proof`](https://pypi.org/project/sanning-proof/) sdist, and the corpus will tell you if it is conformant.
|
|
244
289
|
|
package/dist/agent-proof.js
CHANGED
|
@@ -14,12 +14,12 @@
|
|
|
14
14
|
// 4. Confirm the reconstructed root == checkpoint_envelope.payload.merkle_root.
|
|
15
15
|
// 5. (optional) Re-fetch checkpoint_tx_id from a gateway and byte-compare.
|
|
16
16
|
import { hexToBytes, utf8 } from "./crypto.js";
|
|
17
|
+
import { safeInteger } from "./grading.js";
|
|
17
18
|
import { leafHash, verifyInclusion } from "./merkle.js";
|
|
18
19
|
import { MAX_CANONICAL_DEPTH, exceedsDepth, jcs, jcsParse, verifyEnvelope } from "./verifier.js";
|
|
19
20
|
export const AGENT_PROOF_SPEC_VERSION = "sanning.agent.proof/v1";
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
}
|
|
21
|
+
/** The sentence `agent_proof.py` writes when the proof's own fields are not usable. */
|
|
22
|
+
const MALFORMED_PROOF = "merkle_root / audit_path / leaf_index malformed";
|
|
23
23
|
function toHex(bytes) {
|
|
24
24
|
let out = "";
|
|
25
25
|
for (const x of bytes)
|
|
@@ -44,26 +44,28 @@ export async function verifyAgentProof(bundle, options = {}) {
|
|
|
44
44
|
errors: [error],
|
|
45
45
|
});
|
|
46
46
|
if (bundle === null || typeof bundle !== "object" || Array.isArray(bundle)) {
|
|
47
|
-
return fail("
|
|
47
|
+
return fail("bundle is not a JSON object");
|
|
48
48
|
}
|
|
49
49
|
// Bound nesting before any canonicalization (checkpoint envelope + Merkle
|
|
50
50
|
// leaves are canonicalized recursively) — a fixed shared limit keeps this a
|
|
51
51
|
// `malformed` verdict in every kernel. See MAX_CANONICAL_DEPTH.
|
|
52
52
|
if (exceedsDepth(bundle, MAX_CANONICAL_DEPTH)) {
|
|
53
|
-
return fail(`
|
|
53
|
+
return fail(`bundle nesting exceeds the ${MAX_CANONICAL_DEPTH}-level canonicalization depth bound`);
|
|
54
54
|
}
|
|
55
55
|
const b = bundle;
|
|
56
56
|
if (!isAgentProofSpec(b.spec_version)) {
|
|
57
|
-
|
|
57
|
+
// An absent one reads `null`, as in `agent_proof.py` (`dict.get` gives
|
|
58
|
+
// None); `JSON.stringify(undefined)` would write the word `undefined`.
|
|
59
|
+
return fail(`unsupported bundle spec_version: ${JSON.stringify(b.spec_version ?? null)}`);
|
|
58
60
|
}
|
|
59
61
|
if (b.checkpoint_envelope === null || typeof b.checkpoint_envelope !== "object") {
|
|
60
|
-
return fail("
|
|
62
|
+
return fail("bundle missing checkpoint_envelope");
|
|
61
63
|
}
|
|
62
64
|
if (b.leaf === null || typeof b.leaf !== "object") {
|
|
63
|
-
return fail("
|
|
65
|
+
return fail("bundle missing leaf");
|
|
64
66
|
}
|
|
65
67
|
if (!Array.isArray(b.audit_path)) {
|
|
66
|
-
return fail(
|
|
68
|
+
return fail(MALFORMED_PROOF);
|
|
67
69
|
}
|
|
68
70
|
const errors = [];
|
|
69
71
|
// Step 1: verify the checkpoint envelope (inline payload — no payloadBytes).
|
|
@@ -85,27 +87,50 @@ export async function verifyAgentProof(bundle, options = {}) {
|
|
|
85
87
|
const payload = (b.checkpoint_envelope.payload ?? {});
|
|
86
88
|
if (typeof payload.merkle_root === "string")
|
|
87
89
|
merkleRoot = payload.merkle_root;
|
|
88
|
-
|
|
89
|
-
|
|
90
|
+
// `leaf_count` and `leaf_index` are read BY VALUE, the rule every
|
|
91
|
+
// inclusion reader in both kernels follows (evidence-bundle A38): `3.0` is
|
|
92
|
+
// 3, and a fraction, a value past 2**53 - 1, a string or a boolean is not
|
|
93
|
+
// an integer. `agent_proof.py` reads them through `_safe_integer`.
|
|
94
|
+
leafCount = safeInteger(payload.leaf_count) ?? undefined;
|
|
90
95
|
if (merkleRoot === undefined)
|
|
91
96
|
errors.push("checkpoint payload missing merkle_root");
|
|
92
97
|
if (leafCount === undefined)
|
|
93
|
-
errors.push("checkpoint payload
|
|
98
|
+
errors.push("checkpoint payload leaf_count is not an integer");
|
|
94
99
|
}
|
|
95
|
-
catch
|
|
96
|
-
|
|
100
|
+
catch {
|
|
101
|
+
// The sentence `agent_proof.py` writes for the same failure. The thrown
|
|
102
|
+
// text is the host's, and a public result carries the kernel's words only
|
|
103
|
+
// (proof#150).
|
|
104
|
+
errors.push("checkpoint envelope verification failed");
|
|
97
105
|
}
|
|
98
106
|
// Steps 2-4: leaf hash, audit-path walk, root match.
|
|
99
107
|
let inclusionOk = false;
|
|
100
108
|
if (merkleRoot !== undefined && leafCount !== undefined) {
|
|
109
|
+
let lh;
|
|
101
110
|
try {
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
111
|
+
lh = await leafHash(utf8(jcs(b.leaf)));
|
|
112
|
+
}
|
|
113
|
+
catch {
|
|
114
|
+
errors.push("leaf is not JCS-canonicalizable");
|
|
115
|
+
}
|
|
116
|
+
const leafIndex = safeInteger(b.leaf_index);
|
|
117
|
+
if (lh !== undefined && leafIndex === null) {
|
|
118
|
+
// `agent_proof.py` refuses a leaf_index that is not an integer with this
|
|
119
|
+
// sentence. Without the check a string, `null` or `1.5` reaches the walk
|
|
120
|
+
// and reads as a failed inclusion proof instead (proof#150).
|
|
121
|
+
errors.push(MALFORMED_PROOF);
|
|
106
122
|
}
|
|
107
|
-
|
|
108
|
-
|
|
123
|
+
else if (lh !== undefined) {
|
|
124
|
+
try {
|
|
125
|
+
inclusionOk = await verifyInclusion(lh, leafIndex, leafCount, b.audit_path.map((h) => hexToBytes(h)), hexToBytes(merkleRoot));
|
|
126
|
+
if (!inclusionOk)
|
|
127
|
+
errors.push("audit_path does not reconstruct merkle_root");
|
|
128
|
+
}
|
|
129
|
+
catch {
|
|
130
|
+
// `agent_proof.py`'s sentence for the same failure; the thrown text is
|
|
131
|
+
// the host's (proof#150).
|
|
132
|
+
errors.push(MALFORMED_PROOF);
|
|
133
|
+
}
|
|
109
134
|
}
|
|
110
135
|
}
|
|
111
136
|
// Step 5: optional on-chain re-fetch.
|
|
@@ -144,7 +169,9 @@ export async function verifyAgentProof(bundle, options = {}) {
|
|
|
144
169
|
inclusionOk,
|
|
145
170
|
onChainOk,
|
|
146
171
|
onChainChecked,
|
|
147
|
-
|
|
172
|
+
// Read by value, as the walk above read it: a `2.5` or `1e+21` index is
|
|
173
|
+
// reported as no index, never printed beside a verdict that refused it.
|
|
174
|
+
leafIndex: safeInteger(b.leaf_index) ?? -1,
|
|
148
175
|
leafCount: leafCount ?? null,
|
|
149
176
|
checkpointTxId: typeof b.checkpoint_tx_id === "string" ? b.checkpoint_tx_id : null,
|
|
150
177
|
errors,
|
|
@@ -155,8 +182,8 @@ async function refetch(envelope, txId, gateways, fetchImpl) {
|
|
|
155
182
|
try {
|
|
156
183
|
expectedHex = toHex(utf8(jcs(envelope)));
|
|
157
184
|
}
|
|
158
|
-
catch
|
|
159
|
-
return { ok: false, unavailable: false, error:
|
|
185
|
+
catch {
|
|
186
|
+
return { ok: false, unavailable: false, error: "checkpoint envelope is not JCS-canonicalizable" };
|
|
160
187
|
}
|
|
161
188
|
let anyReachable = false;
|
|
162
189
|
for (const gw of gateways) {
|
package/dist/case-file.d.ts
CHANGED
|
@@ -42,7 +42,7 @@ export declare const MAX_INDEX_DEPTH = 512;
|
|
|
42
42
|
* Every finding this comparison can report, in the order it reports them
|
|
43
43
|
* (5.6.3). None of them is a verdict word and none contributes to one.
|
|
44
44
|
*/
|
|
45
|
-
export declare const CASE_FILE_FINDING_KINDS: readonly ["case_index_unreadable", "case_index_pack_missing", "case_index_pack_unlisted", "case_index_file_missing", "case_index_file_unlisted", "case_index_file_differs", "case_index_names_absent"];
|
|
45
|
+
export declare const CASE_FILE_FINDING_KINDS: readonly ["case_index_unreadable", "case_index_pack_missing", "case_index_pack_unlisted", "case_index_file_missing", "case_index_file_unlisted", "case_index_file_differs", "case_index_file_too_large", "case_index_names_absent"];
|
|
46
46
|
export type CaseFileFindingKind = (typeof CASE_FILE_FINDING_KINDS)[number];
|
|
47
47
|
/**
|
|
48
48
|
* Whether `ref` is a legal pack folder name in a case file. For a producer; a
|
|
@@ -98,17 +98,36 @@ export interface CaseFileIndexReading {
|
|
|
98
98
|
* The CLI's `bundle.json` reader uses it too.
|
|
99
99
|
*/
|
|
100
100
|
export declare function jsonDepthExceeds(text: string | Uint8Array, limit: number): boolean;
|
|
101
|
+
/**
|
|
102
|
+
* The reading of an index the caller could not hold at all (it is over the
|
|
103
|
+
* run's whole-read total, or it cannot be read): one `case_index_unreadable`
|
|
104
|
+
* finding with the caller's `reason`, and no comparison. The index is never an
|
|
105
|
+
* input to the verdict, so a file that cannot be read keeps nothing from it.
|
|
106
|
+
*/
|
|
107
|
+
export declare function unreadableCaseFileIndex(reason: string): CaseFileIndexReading;
|
|
108
|
+
/**
|
|
109
|
+
* Every path the index lists (each pack's files and `report`), or `null` where
|
|
110
|
+
* the index is unreadable. A caller that charges a budget for hashing charges
|
|
111
|
+
* these files first, so a file the index does not list, which needs no digest,
|
|
112
|
+
* cannot turn a real mismatch into `too_large`.
|
|
113
|
+
*/
|
|
114
|
+
export declare function caseFileListedPaths(index: Uint8Array): ReadonlySet<string> | null;
|
|
101
115
|
/**
|
|
102
116
|
* Compare a case file's index with the folder it sits at the root of.
|
|
103
117
|
*
|
|
104
118
|
* `index` is the bytes of `contents.json`, or `null` where the folder holds
|
|
105
119
|
* none. `files` is every regular file in the folder, as `{ path, sha256 }`
|
|
106
|
-
* with `path` from the folder's root, `/`-separated
|
|
120
|
+
* with `path` from the folder's root, `/`-separated, or `{ path, too_large: true }`
|
|
121
|
+
* for a file the caller did not hash because it is over its cap (reported as
|
|
122
|
+
* `case_index_file_too_large` where the index lists it, never as a match or as
|
|
123
|
+
* absent); only the files under
|
|
107
124
|
* `packs/` and `report/` are compared, so a caller may list the rest or not. A
|
|
108
125
|
* folder with no index is a folder of packs, with no finding.
|
|
109
126
|
*
|
|
110
127
|
* Findings come in the order of `CASE_FILE_FINDING_KINDS`, and within a kind in
|
|
111
|
-
* the order of their `ref` or `path` by UTF-8 bytes
|
|
128
|
+
* the order of their `ref` or `path` by UTF-8 bytes (the order a caller charges
|
|
129
|
+
* its hashing budget in is another, UTF-16 code units: evidence-bundle 5.6.3);
|
|
130
|
+
* `case_index_names_absent`
|
|
112
131
|
* is in the index's own order, one per `absent[]` entry. An unreadable index is
|
|
113
132
|
* one finding and no other comparison runs.
|
|
114
133
|
*
|
|
@@ -116,7 +135,8 @@ export declare function jsonDepthExceeds(text: string | Uint8Array, limit: numbe
|
|
|
116
135
|
*/
|
|
117
136
|
export declare function readCaseFileIndex(index: Uint8Array | null, files: readonly {
|
|
118
137
|
path: string;
|
|
119
|
-
sha256
|
|
138
|
+
sha256?: string;
|
|
139
|
+
too_large?: boolean;
|
|
120
140
|
}[]): CaseFileIndexReading;
|
|
121
141
|
/**
|
|
122
142
|
* The house wording for one finding: what the index says against what the
|
package/dist/case-file.js
CHANGED
|
@@ -50,6 +50,7 @@ export const CASE_FILE_FINDING_KINDS = [
|
|
|
50
50
|
"case_index_file_missing",
|
|
51
51
|
"case_index_file_unlisted",
|
|
52
52
|
"case_index_file_differs",
|
|
53
|
+
"case_index_file_too_large",
|
|
53
54
|
"case_index_names_absent",
|
|
54
55
|
];
|
|
55
56
|
// `<ref>`: 1 to 128 characters from A-Z, a-z, 0-9, `.`, `_` and `-`, and not
|
|
@@ -218,17 +219,59 @@ function readIndex(raw) {
|
|
|
218
219
|
const report = fileList(own(value, "report"), "report");
|
|
219
220
|
return { caseId, packs: listedPacks, absent: listedAbsent, report };
|
|
220
221
|
}
|
|
222
|
+
/**
|
|
223
|
+
* The reading of an index the caller could not hold at all (it is over the
|
|
224
|
+
* run's whole-read total, or it cannot be read): one `case_index_unreadable`
|
|
225
|
+
* finding with the caller's `reason`, and no comparison. The index is never an
|
|
226
|
+
* input to the verdict, so a file that cannot be read keeps nothing from it.
|
|
227
|
+
*/
|
|
228
|
+
export function unreadableCaseFileIndex(reason) {
|
|
229
|
+
return {
|
|
230
|
+
index: "unreadable",
|
|
231
|
+
caseId: null,
|
|
232
|
+
findings: [{ kind: "case_index_unreadable", reason }],
|
|
233
|
+
};
|
|
234
|
+
}
|
|
235
|
+
/**
|
|
236
|
+
* Every path the index lists (each pack's files and `report`), or `null` where
|
|
237
|
+
* the index is unreadable. A caller that charges a budget for hashing charges
|
|
238
|
+
* these files first, so a file the index does not list, which needs no digest,
|
|
239
|
+
* cannot turn a real mismatch into `too_large`.
|
|
240
|
+
*/
|
|
241
|
+
export function caseFileListedPaths(index) {
|
|
242
|
+
let read;
|
|
243
|
+
try {
|
|
244
|
+
read = readIndex(index);
|
|
245
|
+
}
|
|
246
|
+
catch (e) {
|
|
247
|
+
if (!(e instanceof Unreadable))
|
|
248
|
+
throw e;
|
|
249
|
+
return null;
|
|
250
|
+
}
|
|
251
|
+
const paths = new Set();
|
|
252
|
+
for (const [, packFiles] of read.packs)
|
|
253
|
+
for (const [path] of packFiles)
|
|
254
|
+
paths.add(path);
|
|
255
|
+
for (const [path] of read.report)
|
|
256
|
+
paths.add(path);
|
|
257
|
+
return paths;
|
|
258
|
+
}
|
|
221
259
|
/**
|
|
222
260
|
* Compare a case file's index with the folder it sits at the root of.
|
|
223
261
|
*
|
|
224
262
|
* `index` is the bytes of `contents.json`, or `null` where the folder holds
|
|
225
263
|
* none. `files` is every regular file in the folder, as `{ path, sha256 }`
|
|
226
|
-
* with `path` from the folder's root, `/`-separated
|
|
264
|
+
* with `path` from the folder's root, `/`-separated, or `{ path, too_large: true }`
|
|
265
|
+
* for a file the caller did not hash because it is over its cap (reported as
|
|
266
|
+
* `case_index_file_too_large` where the index lists it, never as a match or as
|
|
267
|
+
* absent); only the files under
|
|
227
268
|
* `packs/` and `report/` are compared, so a caller may list the rest or not. A
|
|
228
269
|
* folder with no index is a folder of packs, with no finding.
|
|
229
270
|
*
|
|
230
271
|
* Findings come in the order of `CASE_FILE_FINDING_KINDS`, and within a kind in
|
|
231
|
-
* the order of their `ref` or `path` by UTF-8 bytes
|
|
272
|
+
* the order of their `ref` or `path` by UTF-8 bytes (the order a caller charges
|
|
273
|
+
* its hashing budget in is another, UTF-16 code units: evidence-bundle 5.6.3);
|
|
274
|
+
* `case_index_names_absent`
|
|
232
275
|
* is in the index's own order, one per `absent[]` entry. An unreadable index is
|
|
233
276
|
* one finding and no other comparison runs.
|
|
234
277
|
*
|
|
@@ -251,13 +294,20 @@ export function readCaseFileIndex(index, files) {
|
|
|
251
294
|
findings: [{ kind: "case_index_unreadable", reason: e.reason }],
|
|
252
295
|
};
|
|
253
296
|
}
|
|
297
|
+
// `null` is a file that is here and was too large to hash (a file entry of
|
|
298
|
+
// `{ path, too_large: true }`): neither a match nor absent, reported as
|
|
299
|
+
// neither.
|
|
254
300
|
const here = new Map();
|
|
255
301
|
for (const entry of files) {
|
|
256
302
|
const path = isObject(entry) ? own(entry, "path") : undefined;
|
|
257
303
|
const digest = isObject(entry) ? own(entry, "sha256") : undefined;
|
|
258
|
-
|
|
304
|
+
const tooLarge = isObject(entry) && own(entry, "too_large") === true;
|
|
305
|
+
if (typeof path !== "string" || here.has(path))
|
|
306
|
+
continue;
|
|
307
|
+
if (typeof digest === "string")
|
|
259
308
|
here.set(path, digest);
|
|
260
|
-
|
|
309
|
+
else if (tooLarge)
|
|
310
|
+
here.set(path, null);
|
|
261
311
|
}
|
|
262
312
|
const listedRefs = new Set(read.packs.map(([ref]) => ref));
|
|
263
313
|
// The first listing of a path wins, in index order: packs in order, each
|
|
@@ -305,7 +355,7 @@ export function readCaseFileIndex(index, files) {
|
|
|
305
355
|
}
|
|
306
356
|
for (const path of listedPaths) {
|
|
307
357
|
const actual = here.get(path);
|
|
308
|
-
if (actual !== undefined && actual !== listed.get(path)) {
|
|
358
|
+
if (actual !== undefined && actual !== null && actual !== listed.get(path)) {
|
|
309
359
|
findings.push({
|
|
310
360
|
kind: "case_index_file_differs",
|
|
311
361
|
path,
|
|
@@ -314,6 +364,10 @@ export function readCaseFileIndex(index, files) {
|
|
|
314
364
|
});
|
|
315
365
|
}
|
|
316
366
|
}
|
|
367
|
+
for (const path of listedPaths) {
|
|
368
|
+
if (here.get(path) === null)
|
|
369
|
+
findings.push({ kind: "case_index_file_too_large", path });
|
|
370
|
+
}
|
|
317
371
|
for (const [reason, key] of read.absent) {
|
|
318
372
|
findings.push(key === null
|
|
319
373
|
? { kind: "case_index_names_absent", reason }
|
|
@@ -341,6 +395,9 @@ export function renderCaseFileFinding(finding) {
|
|
|
341
395
|
case "case_index_file_differs":
|
|
342
396
|
return (`the index lists ${finding.path} with SHA-256 ${finding.listedSha256}; ` +
|
|
343
397
|
`the file here hashes to ${finding.sha256}`);
|
|
398
|
+
case "case_index_file_too_large":
|
|
399
|
+
return (`the index lists ${finding.path}; hashing the file here would take case ` +
|
|
400
|
+
"past the 1 GiB it hashes in one run, so it was not compared");
|
|
344
401
|
case "case_index_names_absent":
|
|
345
402
|
if (finding.publicKey !== undefined) {
|
|
346
403
|
return ("the assembler says a pack signed by key " +
|
package/dist/case.d.ts
CHANGED
|
@@ -660,6 +660,41 @@ export type CaseNoteKind = (typeof CASE_NOTE_KINDS)[number];
|
|
|
660
660
|
*/
|
|
661
661
|
export declare const CASE_EXCEPTION_KINDS: readonly ["counterpart_absent", "counterpart_absent_from_counterparty_records", "counterpart_absent_signer_unknown", "counterpart_ambiguous", "counterpart_malformed", "pair_failed", "record_malformed", "record_unclaimed_by_its_pack", "record_bytes_withheld", "record_bytes_unreadable", "pack_declaration_deviates", "pack_declaration_content_dependent", "pack_declaration_unreadable", "pack_undeclared", "pack_body_type_unknown", "pack_holds_no_records", "pack_key_unreadable", "pack_does_not_verify", "pack_grade_not_verified", "pack_bundle_content_dependent", "record_envelope_does_not_verify", "record_does_not_verify_in_its_pack", "record_content_mismatch", "record_content_dependent", "record_profile_quarantined", "case_holds_no_packs"];
|
|
662
662
|
export type CaseExceptionKind = (typeof CASE_EXCEPTION_KINDS)[number];
|
|
663
|
+
/**
|
|
664
|
+
* The status every record in every pack carries (evidence-bundle 5.5.4
|
|
665
|
+
* v1.35, A43, proof#204), as a closed set. A record is listed for each element
|
|
666
|
+
* of its pack's `body.events` as `packRecords` reads them, or, for a
|
|
667
|
+
* `sanning.evidence.export/v2` pack, of the events of the inline source
|
|
668
|
+
* bundle it wraps (`resolveTraceEvents`), and gets exactly one:
|
|
669
|
+
*
|
|
670
|
+
* - `read_clean`: the record went into the pile, no exception carries its
|
|
671
|
+
* `packIndex` and `recordIndex`, and its pack's declaration check
|
|
672
|
+
* names it by nothing (`declarationFindings` is empty). It says nothing
|
|
673
|
+
* about its PACK: the pack's own grade and exceptions are on the pack, and a
|
|
674
|
+
* surface MUST show that grade beside the status.
|
|
675
|
+
* - `read_flagged`: the record went into the pile, and at least one
|
|
676
|
+
* exception carries its `packIndex` and `recordIndex`, or its pack's
|
|
677
|
+
* declaration check names its `eventId` by a deviation or a content
|
|
678
|
+
* finding (`declarationFindings` is not empty; under an `eventId` the
|
|
679
|
+
* pack carries twice, every record with that id). An exception about the
|
|
680
|
+
* pack alone flags no record.
|
|
681
|
+
* - `not_read`: the record never went into the pile. `reason` is the kind
|
|
682
|
+
* of the exception that says why, on the record
|
|
683
|
+
* (`record_profile_quarantined`, `record_envelope_does_not_verify`,
|
|
684
|
+
* `record_bytes_withheld`, `record_bytes_unreadable`) or on its pack
|
|
685
|
+
* (`pack_key_unreadable`, `pack_does_not_verify`,
|
|
686
|
+
* `pack_body_type_unknown`, `pack_grade_not_verified` where the body
|
|
687
|
+
* could not be read or the verifier graded a different number of records).
|
|
688
|
+
* An export's wrapped records carry the reason of the exception their pack
|
|
689
|
+
* actually got: `pack_body_type_unknown` where its wrapper holds, and
|
|
690
|
+
* otherwise `pack_key_unreadable`, `pack_does_not_verify` or
|
|
691
|
+
* `pack_grade_not_verified`.
|
|
692
|
+
*
|
|
693
|
+
* Before A43 the reading listed only the records it read, so a surface had to
|
|
694
|
+
* work out which records were missing, and why, by a rule of its own.
|
|
695
|
+
*/
|
|
696
|
+
export declare const CASE_RECORD_STATUSES: readonly ["read_clean", "read_flagged", "not_read"];
|
|
697
|
+
export type CaseRecordStatus = (typeof CASE_RECORD_STATUSES)[number];
|
|
663
698
|
/**
|
|
664
699
|
* One pack in a case, as a reader was handed it.
|
|
665
700
|
*
|
|
@@ -682,10 +717,25 @@ export interface CasePack {
|
|
|
682
717
|
* `renderException`. */
|
|
683
718
|
export interface CaseException {
|
|
684
719
|
kind: CaseExceptionKind;
|
|
685
|
-
/**
|
|
720
|
+
/**
|
|
721
|
+
* Which pack this is about, by the reader's own name for it. Two packs can
|
|
722
|
+
* carry one `ref`, so it is a label and never the join: `packIndex` is.
|
|
723
|
+
*/
|
|
686
724
|
packRef?: string;
|
|
725
|
+
/**
|
|
726
|
+
* Which pack this is about, by its position in `CaseReading.packs` (0
|
|
727
|
+
* first), present wherever `packRef` is (A43). Unique in the reading.
|
|
728
|
+
*/
|
|
729
|
+
packIndex?: number;
|
|
687
730
|
/** Which record, by its envelope `event_id`. */
|
|
688
731
|
eventId?: string;
|
|
732
|
+
/**
|
|
733
|
+
* Which record, by its position in its pack's `body.events` (0 first),
|
|
734
|
+
* present wherever `eventId` is (A43). One pack can carry one `event_id`
|
|
735
|
+
* twice, so `packIndex` and `recordIndex` together are the join to
|
|
736
|
+
* `CaseReading.records`.
|
|
737
|
+
*/
|
|
738
|
+
recordIndex?: number;
|
|
689
739
|
/**
|
|
690
740
|
* The counterparty key the record named, for the two counterpart kinds.
|
|
691
741
|
* Normalized (`normalizedSigningKey`), so a reader compares what this module
|
|
@@ -717,20 +767,54 @@ export interface CaseException {
|
|
|
717
767
|
reasons?: string[];
|
|
718
768
|
}
|
|
719
769
|
/**
|
|
720
|
-
*
|
|
770
|
+
* One record of one pack, with its status and, where it was read, 4.1's
|
|
771
|
+
* reading of it.
|
|
721
772
|
*
|
|
722
|
-
* EVERY record every pack
|
|
723
|
-
*
|
|
724
|
-
*
|
|
725
|
-
*
|
|
726
|
-
* wants is for a record to be absent from
|
|
773
|
+
* EVERY record of every pack gets one of these (A43), in pack order and then
|
|
774
|
+
* in bundle order: the ones this reader read, including the ones that are not
|
|
775
|
+
* hand-off records at all (`findings` carries `not_a_handoff_record`), and the
|
|
776
|
+
* ones it did not, with `status` `not_read` and the `reason`. 4.1 (v1.9, A14)
|
|
777
|
+
* names why: the one thing an attacker wants is for a record to be absent from
|
|
778
|
+
* the output. An export's wrapped records are listed, `not_read`. A pack whose
|
|
779
|
+
* body carries no `events` array, and wraps none, lists none, and its own
|
|
780
|
+
* exception says why.
|
|
727
781
|
*/
|
|
728
782
|
export interface CaseRecordReading {
|
|
729
783
|
packRef: string;
|
|
784
|
+
/** The pack, by its position in `CaseReading.packs` (A43). */
|
|
785
|
+
packIndex: number;
|
|
730
786
|
eventId: string;
|
|
787
|
+
/**
|
|
788
|
+
* The record, by its position in the list of records its pack lists (A43):
|
|
789
|
+
* `body.events`, or for a `sanning.evidence.export/v2` pack the events of the
|
|
790
|
+
* source bundle it wraps.
|
|
791
|
+
*/
|
|
792
|
+
recordIndex: number;
|
|
793
|
+
/** One of `CASE_RECORD_STATUSES` (A43). */
|
|
794
|
+
status: CaseRecordStatus;
|
|
795
|
+
/**
|
|
796
|
+
* For `not_read`, the kind of the exception that says why; `null` for a
|
|
797
|
+
* record that was read.
|
|
798
|
+
*/
|
|
799
|
+
reason: CaseExceptionKind | null;
|
|
800
|
+
/**
|
|
801
|
+
* The kinds its pack's declaration check names this record by, deviations
|
|
802
|
+
* (`DECLARATION_DEVIATION_KINDS`) then content findings
|
|
803
|
+
* (`DECLARATION_CONTENT_KINDS`), each in the check's own order. Matched on
|
|
804
|
+
* `eventId`, so under an `event_id` the pack carries twice every record with
|
|
805
|
+
* that id carries it. A record that went into the pile with any is
|
|
806
|
+
* `read_flagged` (A43).
|
|
807
|
+
*/
|
|
808
|
+
declarationFindings: string[];
|
|
809
|
+
/** 4.1's verdict, or `null` where 4.1 gave none or the record was not read. */
|
|
731
810
|
verdict: string | null;
|
|
732
811
|
findings: string[];
|
|
733
812
|
failedChecks: string[];
|
|
813
|
+
/**
|
|
814
|
+
* 4.1's candidate counterparts, each as `<pack ref>/<event_id>`. A LABEL for
|
|
815
|
+
* a person to read and never the join: two packs can carry one ref, and one
|
|
816
|
+
* pack one `event_id` twice (A43).
|
|
817
|
+
*/
|
|
734
818
|
counterparts: string[];
|
|
735
819
|
/**
|
|
736
820
|
* The role the pack's v2 declaration signs for this record, `handoff` or
|
|
@@ -747,6 +831,8 @@ export interface CaseNote {
|
|
|
747
831
|
kind: CaseNoteKind;
|
|
748
832
|
/** Which pack, by the reader's own name for it. */
|
|
749
833
|
packRef: string;
|
|
834
|
+
/** Which pack, by its position in `CaseReading.packs` (A43). */
|
|
835
|
+
packIndex: number;
|
|
750
836
|
/**
|
|
751
837
|
* For `CONTENT_WITHHELD_BY_DECLARATION`, how many of the pack's records its
|
|
752
838
|
* declaration withholds.
|
|
@@ -759,6 +845,13 @@ export interface CaseNote {
|
|
|
759
845
|
* its wrapper, and never by an organization (D10).
|
|
760
846
|
*/
|
|
761
847
|
export interface CasePackReading {
|
|
848
|
+
/**
|
|
849
|
+
* The pack's position in the order the reader was given the packs, 0 first
|
|
850
|
+
* (A43): the one name for a pack that is unique in the reading. Every
|
|
851
|
+
* exception, note and record names its pack by it as `packIndex`.
|
|
852
|
+
*/
|
|
853
|
+
index: number;
|
|
854
|
+
/** The reader's own name for the pack. Two packs can carry the same one. */
|
|
762
855
|
ref: string;
|
|
763
856
|
/** The wrapper key, normalized, or `null` where this reader could not read it. */
|
|
764
857
|
signingKey: string | null;
|
|
@@ -790,7 +883,10 @@ export interface CaseReading {
|
|
|
790
883
|
* reachable only together with `verdict === "verified"`.
|
|
791
884
|
*/
|
|
792
885
|
exceptions: CaseException[];
|
|
793
|
-
/**
|
|
886
|
+
/**
|
|
887
|
+
* One entry per record of every pack, read or not, in pack order and then
|
|
888
|
+
* bundle order, each with its status (A43).
|
|
889
|
+
*/
|
|
794
890
|
records: CaseRecordReading[];
|
|
795
891
|
/**
|
|
796
892
|
* Every `[contributor, verdict]` the case verdict was computed from, in
|