@crewhaus/provability 0.0.0-stage → 0.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,201 @@
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "[]"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright [yyyy] [name of copyright owner]
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
package/NOTICE ADDED
@@ -0,0 +1,9 @@
1
+ CrewHaus Factory
2
+ Copyright 2026 Max Meier
3
+
4
+ This product includes software developed by Max Meier
5
+ (operating as StudioMax — https://studiomax.io) and contributors.
6
+
7
+ The CrewHaus name and logo are trademarks of Max Meier. See
8
+ https://github.com/crewhaus/factory/blob/main/TRADEMARK.md for the
9
+ use-of-marks policy.
package/README.md CHANGED
@@ -1,3 +1,301 @@
1
- # Temporary Holding Version
1
+ # @crewhaus/provability
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ The provability ladder: `cited`, `grounded`, `checked`, `proven`. A spec sets a floor, and every
4
+ claim in an answer must bind to evidence at or above it. This package is the deterministic part
5
+ of that: the evidence index built from the run's own event log, the four verifiers behind one
6
+ function (`verifyDocument`), the proof bundle they produce, the ways to render it, and where
7
+ bundles and proof artifacts are stored.
8
+
9
+ It makes no model call and imports no model adapter (`crewhaus doctor --philosophy-alignment`
10
+ checks that). The one model-backed step in the feature, the opt-in paraphrase waiver, is a
11
+ callback the caller injects, and so are the other two things that reach outside the process: the
12
+ HEAD request behind `sources.resolve: head` and the replay of a pure tool. The in-loop gate, the
13
+ `ProvabilityPreflight` tool, `crewhaus provability check` and the research report gate all call
14
+ the same `verifyDocument`, so a draft the model preflights is graded exactly as the gate will
15
+ grade it. Every text operation (claims, markers, definitions, the evidence-reference grammar,
16
+ locating a claim in a body, numbers, quotations, coverage) is
17
+ [`@crewhaus/claim-ledger`](../claim-ledger)'s; nothing here re-implements one.
18
+
19
+ ```ts
20
+ import {
21
+ buildEvidenceIndex,
22
+ createBundleStore,
23
+ createFileArtifactStore,
24
+ evidenceFromMessages,
25
+ evidenceFromSessionLog,
26
+ proofsRoot,
27
+ renderFooter,
28
+ verifyDocument,
29
+ } from "@crewhaus/provability";
30
+
31
+ const root = proofsRoot(process.cwd(), "support-bot"); // <cwd>/.crewhaus/proofs/support-bot
32
+ const evidence = buildEvidenceIndex(
33
+ [
34
+ ...(await evidenceFromSessionLog({ sessionId })), // the log, sub-agent children included
35
+ ...evidenceFromMessages(turnMessages), // the turn still in flight
36
+ ],
37
+ { sessionId },
38
+ );
39
+ const bundle = await verifyDocument(answerText, {
40
+ config: ir.provability, // the lowered, fully resolved IrProvability
41
+ evidence,
42
+ artifacts: createFileArtifactStore({ rootDir: root }),
43
+ replay, // optional: re-run a tool by name with its logged input
44
+ isPureTool, // optional: (toolName) => whether replay may run it
45
+ resolveHead, // optional: a guarded HEAD, for `sources.resolve: head`
46
+ judgeWaiver, // optional: the paraphrase waiver
47
+ retryIndex,
48
+ });
49
+ const path = await createBundleStore({ rootDir: root }).put(bundle, { sessionId, turnNumber });
50
+ console.log(renderFooter(bundle, { path }));
51
+ // ── provability · grounded ✓ · 6/6 claims · 3 sources · 0 assumptions · .crewhaus/proofs/…
52
+ ```
53
+
54
+ ## API
55
+
56
+ **Verifying.** `verifyDocument(text, opts: VerifyOptions): Promise<ProofBundle>`. Besides
57
+ `config` and `evidence`, the options are `workspaceRoot` and `baseDir` (where path sources
58
+ resolve), `stateRoots` (the run's own state directories, never a source), `artifacts` (a
59
+ `ProofArtifactStore`; absent, every `proof:` binding is `proof_missing`), `recheckProof`,
60
+ `trustStoredProofs`, `replay` and `isPureTool`, `resolveHead` and `headDeadlineMs`,
61
+ `judgeWaiver`, `turnNumbers` (`scope: turn`), `retryIndex` and `now`. Beside it:
62
+ `unverifiedBundle(config, error, retryIndex?, now?)` for a caller that caught a throw,
63
+ `bundleHash(bundle)`, `claimReasons(claim)`, `isToolAllowed(toolName, allow)`,
64
+ `statementMismatch(proved, sentence, bar)`, `independenceKey(origin)`, and the bounds
65
+ `MAX_HEAD_REQUESTS` (50), `HEAD_CONCURRENCY` (6), `HEAD_DEADLINE_MS` (15 000) and
66
+ `MAX_GROUNDING_TOKENS`. The errors this package raises are `ProvabilityError`s (a
67
+ `CrewhausError`): `code: "config"` for a malformed argument (a config without a valid `level`, an
68
+ invalid session id, spec name, turn number or artifact), `"runtime"` for a file the store
69
+ refused or could not read.
70
+
71
+ **Evidence.** `evidenceFromSessionLog({ sessionId, rootDir?, includeChildren?, maxDepth?,
72
+ workspaceRoot?, toolResultsRoot?, inputSchemaOf?, onUnreadable? })` and
73
+ `evidenceFromMessages(messages, opts?)` produce `EvidenceRecord`s; `recordFromCall(call, opts?)`
74
+ builds one; `buildEvidenceIndex(records, { sessionId? }): EvidenceIndex` merges them;
75
+ `restrictToTurns(index, turnNumbers)` scopes one. `evidenceFromSessionLog` throws a
76
+ `ProvabilityError` for a session id that is not `sess_<16 hex>`; a log it cannot read is
77
+ reported, not thrown (below). The readers behind them are exported too: `groundingBody`,
78
+ `servedFrom`, `servesPage`, `spilledOutput`, `sha256Of`.
79
+
80
+ **Rendering.** `renderFooter(bundle, { path? })` (the one-line footer), `renderBundleJson`,
81
+ `renderMarkdownSection` (the research report's `## Provability`), `renderNudge` (the retry
82
+ message) and `summarizeBundle(bundle): ProvabilityGradedSummary` (the `provability_graded` and
83
+ `provability_evaluated` payload).
84
+
85
+ **Storage.** `proofsRoot(cwd, specName)`, `createBundleStore({ rootDir }): BundleStore`
86
+ (`put`, `get`, `list`, `prune`), `createFileArtifactStore({ rootDir })`, and one definition of a
87
+ proof id and an artifact's shape: `PROOF_ID_REGEX`, `newProofId`, `isProofId`,
88
+ `isProofArtifact`.
89
+
90
+ **Levels, origins, purity.** `levelRank` (none 0, cited 1, grounded 2, checked 3, proven 4),
91
+ `rungOfRank`, `meetsFloor`, `maxRung`, `isProvabilityLevel`, `PROVABILITY_LEVELS`;
92
+ `normalizeUrl`, `normalizePath`, `originKey`, `hostOf`, `hostMatches`, `registrableDomain`,
93
+ `isPrivateHost`, `hasUserinfo`, `doiUrl`; and `isPureTool(tool)`, which tools replay may re-run
94
+ (read-only, concurrency-safe, not destructive, no sandbox, internal scope, no io capability).
95
+
96
+ **Types** (`types.ts`): `ProofBundle`, `ProofClaim`, `ProofBinding`, `UnmetReason`,
97
+ `EvidenceRecord`, `EvidenceIndex`, `ProofArtifact`, `ProofArtifactStore`, `VerifyOptions`,
98
+ `ProvabilityGradedSummary`, and `ProvabilityConfig` (the lowered `IrProvability`). The bundle's
99
+ wire schema is the zod `ProofBundle` in `@crewhaus/gateway-protocol`, held to this type by its
100
+ tests. `private-address.ts` is the repo's synchronised private-address classifier, a copy that
101
+ `apps/cli/src/tool-registry.test.ts` sweeps and executes: never edit it here alone.
102
+
103
+ ## Who calls it
104
+
105
+ - `@crewhaus/evaluation-service`'s gate, which builds the index from the session log and the
106
+ transcript, wires the stores, replay through the catalog, the guarded HEAD and the waiver judge,
107
+ and hands runtime-core the closures.
108
+ - `@crewhaus/tool-proof`'s `ProvabilityPreflight`, which skips the HEAD, the replay and the judge.
109
+ - `crewhaus provability check`, offline over any document (with `trustStoredProofs` when it has
110
+ no session log to read).
111
+ - `@crewhaus/report-writer`'s research report gate, over the citation tracker's cache.
112
+
113
+ ## The rungs, as checked
114
+
115
+ - **cited.** Every marker in the claim has a definition, and the definition names something
116
+ locatable: an `http(s)` URL or a `doi:` whose host passes `sources.allow` / `sources.deny`
117
+ (deny wins; a DOI is checked as `doi.org`) and is not a private address, or a workspace path
118
+ that stays inside the workspace with every symlink followed, and exists. A URL that puts a user
119
+ name before its host (`https://acme.example@evil.example/`) is `host_denied`. Free text counts
120
+ only under `sources.bibliographic: true`, and can never be grounded. With `sources.resolve:
121
+ head` the URL must answer one HEAD with 2xx or 3xx: at most `MAX_HEAD_REQUESTS` per
122
+ verification, `HEAD_CONCURRENCY` at once, all within `headDeadlineMs`; a URL past the cap or the
123
+ deadline is `unresolved`. A dangling `[2]` beside a good `[1]` leaves the claim unmet
124
+ (`no_definition`). A defined second marker that fails does not unmeet a claim another binding
125
+ meets; the bundle still shows it.
126
+ - **grounded.** The cited origin was fetched or read in this run (not an error result, and not a
127
+ fetch that answered outside 2xx), and claim-ledger's `locateClaim` finds the claim in that body
128
+ at `grounded.coverage` within `grounded.window_tokens`. A quotation must appear verbatim; a
129
+ figure or a negation is never excused by a partial bar, at `coverage: 1` too. The binding
130
+ freezes an excerpt (240 characters at most, the claim at its tightest) and the body's digest,
131
+ so the bundle stays checkable after the transcript is evicted. A path under the harness's own
132
+ state (`.crewhaus/`, `.crewhaus-shared/`, and the `stateRoots` the caller names) is never a
133
+ source, and neither is a copy of one (`CopyPath`, `MovePath`, `ConcatFiles`, the archive tools,
134
+ followed back up to four steps): a session log holds every answer verbatim (`not_fetched`). A
135
+ body a redirect brought from a host `sources.deny` or the private-address rule refuses does not
136
+ ground the URL that led there. One verification reads at most `MAX_GROUNDING_TOKENS` of fetched
137
+ text; a source past the budget reads `below_floor`, with a detail naming the cap.
138
+ - **checked.** `tool:<toolUseId>` resolves to a recorded, non-error call of a tool on
139
+ `execution.allow`: the graded session's own call first, then a sub-agent's; an id only several
140
+ sub-agents share is `tool_missing`. Every number the claim states is in the result as an exact
141
+ decimal, and every quotation word for word; a claim with neither must share
142
+ `grounded.coverage` of its words with the call. The result is read with the claim's words the
143
+ input handed over blanked out where the tool repeats them, so a tool that only echoes the draft
144
+ (NormalizeText of the sentence, a lint run over the answer) checks nothing, while a value
145
+ selected from a document (JsonQuery over a record) does. A `FactCrossCheck` result counts only
146
+ where it found this sentence `supported`. A figure's polarity is read word by word
147
+ (claim-ledger's `governingNegation`) in the statement it sits in: a figure the claim denies
148
+ must be denied in the result too, and "91 is not prime" or "91 (not prime)" does not check
149
+ "91 is prime". At a `checked` floor, with `execution.replay` on and a pure tool, the gate
150
+ re-runs it once per call (at most `max_replays` per verification) and compares digests: a
151
+ mismatch is `replay_mismatch`; a replay that could not run leaves the binding `checked` with
152
+ `replayed: false`.
153
+ - **proven.** `proof:prf_<16 hex>` names a stored artifact a checker on `formal.checkers`
154
+ accepted, and the run's evidence holds the `ProofCheck` call that wrote it: accepted, with the
155
+ assumptions and axioms the file lists, given exactly the checker, claim, statement and proof
156
+ the file holds. The file sits in the workspace, where the model's file tools could forge one;
157
+ the run's record of the check is what counts. Where `recheckProof` is wired (the gate and the
158
+ preflight wire tool-proof's, the builtin checker), the checker accepts the stored proof again.
159
+ The sentence must state the artifact's recorded `claim` (`statementMismatch`): its words cover
160
+ the claim's at `grounded.coverage`, both state the same figures, and each word of the proof's
161
+ claim is denied in both or in neither, so a proof that no two meetings overlap does not prove
162
+ "two meetings overlap". The checker, the formal statement, the hypotheses the proof rests on
163
+ (builtin `given` steps, SMT asserts and definitions) and the axioms go into the bundle beside
164
+ the prose. Hypotheses are assumptions: counted in the footer, refused under `assumptions:
165
+ forbid` and capped by `max_assumptions`.
166
+
167
+ ## How each outcome is represented
168
+
169
+ - A claim with no marker is `unmet` with zero bindings; `claimReasons(claim)` reports it as
170
+ `no_citation`.
171
+ - An allowed assumption binding is `ok: false` with no `reason`: not evidence, not a failure. A
172
+ claim whose only bindings are allowed assumptions has status `assumption`, is left out of the
173
+ score on both sides, and is always counted in the bundle and the footer. A refused assumption
174
+ (`assumptions: forbid`, or past `max_assumptions` in document order) is `assumption_forbidden`
175
+ and fails the verdict whatever the score, with `achieved: "none"`; the footer shows every
176
+ declared one (`3 assumptions (2 refused)`).
177
+ - Under `grounded.independent: n` at a `grounded` floor, a claim met only through sources needs n
178
+ distinct origins (`independenceKey`: registrable domain, DOI registrant prefix, or file), each
179
+ read off the host that served the body. Short of that, each of its source bindings becomes
180
+ `not_independent`. A claim met by an execution or formal binding is met regardless.
181
+ - A source the run wrote itself (a `Write` or `Edit` of the file, a PUT to the URL, or a copy of
182
+ one) still grounds, because the file does say it now, but the binding carries
183
+ `writtenBy: { toolUseId, toolName }`, the bundle counts it in `selfWritten`, and the footer
184
+ reads `1 source (1 written by this run)`. A `FactCrossCheck` that found the claim only in such
185
+ files reaches `grounded` at most. An MCP result that only hands back text an earlier call of
186
+ the run sent (a stored row read back) is `not_fetched`.
187
+
188
+ ## Things worth knowing
189
+
190
+ **The outside calls happen only where the floor needs them.** Rungs above the floor are still
191
+ computed, because `achieved` is the highest floor the answer would have passed, but without the
192
+ outside calls: the HEAD runs only at `cited` and `grounded`, the waiver judge only at `grounded`
193
+ and only for a miss claim-ledger calls waivable (`isWaivableMiss`: words partly found, none of
194
+ the missing ones a figure or a negation), and only while the claim still needs it. Replay runs
195
+ only at `checked`.
196
+
197
+ **Evidence comes from the log, merged with the transcript.** `evidenceFromSessionLog` walks child
198
+ sessions through their `sub_agent_start` brackets and places a child's calls where the bracket
199
+ sits. An id logged twice pairs its last `tool_use` with its last `tool_result`, the rule
200
+ continuity-store applies. A result the runtime spilled to a file is read whole from this call's
201
+ own file under the spill root, never through a link. A log that cannot be read to its end keeps
202
+ the records before that point and reports it through `onUnreadable`: fewer records fails closed.
203
+ A call is its session and its id together, because an OpenAI-compatible server numbers ids per
204
+ conversation and a parent and its child can share one.
205
+
206
+ **Origins are read off the call, and only where the result is that origin's content.** Among
207
+ builtins: `WebFetch`'s and `Fetch`'s `url` (a `Fetch` only when its method is GET, HEAD or
208
+ unset) and `Read`'s `path`, plus the final URL a WebFetch header names after a redirect. An MCP
209
+ tool's `url`/`uri`/`href`/`path`/`file`/`file_path`/`filePath` names an origin only when the
210
+ tool's declared schema lists that field (`inputSchemaOf`, default `defaultCatalog`'s), and, where no
211
+ schema is known, only when the input carries nothing else a call could have been pointed at.
212
+ The runtime's own tools whose results are model output (`Task`, `Consult`, `Escalate`, `Skill`,
213
+ the memory, plan and goal tools) name no origin. Hits are indexed under their own origins:
214
+ `Retrieve`'s `[n] id=… doc=…` blocks in exactly the structure Retrieve prints, and JSON
215
+ `url`/`uri`/`href`/`link`/`source` hits from an MCP tool whose input names no URL or path. When a
216
+ body is grounded against, WebFetch's header and the `[user prompt: …]` line it prepends, and
217
+ Fetch's status line and headers, are removed.
218
+
219
+ **The text a definition line shows is read.** `[8]: https://acme.example/x Payments clear
220
+ within 5 days` renders as that sentence, so what it shows past its destination
221
+ (`definitionGloss`) is verified as claims citing that definition (`definition: "[8]"`). A link
222
+ title (`"…"`, `'…'`, `(…)`) renders as nothing and is not read, which is how to keep a page
223
+ title out.
224
+
225
+ **A vacuous pass is backstopped.** An answer with no claims passes every floor, which is right
226
+ for a greeting or a clarifying question and wrong for an answer whose claim the extractor did
227
+ not see. So when nothing counted is extracted (`vacuous.ts`), what a reader sees, code,
228
+ definition lines, URLs and list-item numbers aside, is cut far more bluntly: at every line break
229
+ (U+2028, `<br>` and the rest), every sentence terminator, every semicolon, em dash, en dash (not
230
+ one between figures) and table pipe, never at a comma. The answer is vacuous only when no segment
231
+ carries a marker and no segment outside a question states a figure; otherwise each such segment
232
+ is a claim, verified like any other.
233
+
234
+ **The nudge crosses no trust boundary.** `renderNudge` is built from the model's own sentences,
235
+ the identifiers it cited and fixed reasons, ASCII apart from the model's own words. It never
236
+ includes a fetched body, an excerpt or judge text; the one checker output it repeats is a
237
+ rejected proof's reason and counterexample.
238
+
239
+ **`verifyDocument` does not throw for anything a document can contain.** An unreadable artifact,
240
+ a failing judge or replay and a refused HEAD each become a binding's reason; it throws a
241
+ `ProvabilityError` only for a malformed config. A caller that catches anything else serves
242
+ `unverifiedBundle`, so the footer reads `? unverified`, never `✓`. `bundleHash` is a sha256 over
243
+ the canonical JSON of every other field, keys sorted at every depth.
244
+
245
+ **Storage.** Bundles go to `.crewhaus/proofs/<spec>/<sessionId>/turn-<n>.json`, and a bundle put
246
+ without a session id under `no-session/` (`NO_SESSION`). A later pass of the same turn (a retry)
247
+ goes beside it as `turn-<n>.<k>.json`, never over it, so every `bundleHash` an audit record
248
+ carries names a bundle still on disk. `get` reads a turn's last pass and `list` each turn's last
249
+ pass unless asked for all. Artifacts go to
250
+ `.crewhaus/proofs/<spec>/artifacts/<prf_id>.json` and are never replaced. Files are mode 0600,
251
+ written atomically through `@crewhaus/tool-safety/fs`, never through a link; spec names, session
252
+ ids, turn numbers and proof ids are validated before any path is built from them. Bundles have
253
+ no TTL; `prune` removes old ones.
254
+
255
+ ## Limits
256
+
257
+ Deliberate, or known and not closed in this release:
258
+
259
+ - A fabricated but well-formed URL passes `cited`: that rung checks the reference, not the page.
260
+ `isPrivateHost` is offline, so a public-looking name that resolves privately is caught by the
261
+ fetch that reads it, not here.
262
+ - `grounded` is faithfulness, not truth: a source that is wrong grounds a wrong claim.
263
+ - `registrableDomain` is a pragmatic eTLD+1 table, not the Public Suffix List: every
264
+ `*.github.io` site counts as one origin under `grounded.independent`.
265
+ - At `grounded`, a negation before the span the claim is found in counts only when it governs
266
+ the claim's first word, so "It is not the case that the meetings overlap on Tuesday afternoon"
267
+ grounds "The meetings overlap on Tuesday afternoon".
268
+ - At `checked`, a result's negation counts only where it governs the figure or one of the
269
+ claim's own words (`governingNegation`: before it, at most two words between): "1,284.50 is
270
+ not correct" and "it is not the case that the total is 1,284.50" both check "The total is
271
+ 1,284.50". A question answered by a bare "No." is not read either. An identifier the claim
272
+ restates unquoted (a UUID, a hash) is compared by value only where the number layer reads a
273
+ figure in it, and otherwise falls back to word coverage: quote identifiers. A claim a tool
274
+ reassembles from several input strings, or a draft padded past twice the claim's length, is
275
+ not caught as an echo.
276
+ - At `proven`, the coverage runs from the proof's claim into the sentence, so a sentence that
277
+ adds a clause of its own to the proved statement passes. That residual sits with the
278
+ formalization gap, shown beside the claim (the formal statement), not closed.
279
+ - Self-written text goes unmarked when `Bash` or `RunCommand` wrote it (a command line names no
280
+ place), when a write names the place in a form that cannot be mapped to the cited one (a
281
+ repository path behind a raw-file URL), or when an earlier session wrote it. An MCP store row
282
+ written before the evidence begins is read as the server's own text.
283
+ - A reflection a tool rewrites on the way out (an SQL function applied to the literal, an address
284
+ printed re-spelled) still grounds; closing that needs the spec to say whose hits are trusted.
285
+ A search hit that repeats a phrase the model searched for, word for word, is refused instead,
286
+ and fetching the page grounds it.
287
+ - A forged hit header after a blank line inside the last Retrieve hit's preview still reads as
288
+ one more hit; closing that needs Retrieve to print a delimiter a preview cannot contain.
289
+
290
+ ## Tests
291
+
292
+ ```sh
293
+ bun test src
294
+ ```
295
+
296
+ No test needs a network, a model or Docker. The suites pin each rung's reasons, the evidence
297
+ readers against the formats the real tools print, the self-written trace (`verify-provenance`,
298
+ `verify-self-written`), the backstop (`vacuous`), the renderers and the store's file rules.
299
+
300
+ Design: [design/0.8.0-provability-release.md](../../design/0.8.0-provability-release.md) §2, §5,
301
+ §6, §10.
@@ -0,0 +1,9 @@
1
+ import type { ProofArtifact } from "./types";
2
+ /** `prf_` and 16 lowercase hex: the only shape a proof id has. */
3
+ export declare const PROOF_ID_REGEX: RegExp;
4
+ /** Is `id` a proof id (`prf_<16 lowercase hex>`)? */
5
+ export declare function isProofId(id: unknown): id is string;
6
+ /** A fresh proof id: `prf_` and 16 hex characters from the system CSPRNG. */
7
+ export declare function newProofId(): string;
8
+ /** Does `value` have every field a `ProofArtifact` must, with the right types? */
9
+ export declare function isProofArtifact(value: unknown): value is ProofArtifact;
@@ -0,0 +1,38 @@
1
+ /**
2
+ * Proof-artifact identity and shape (design §6.4): one definition of what a
3
+ * `prf_` id is and what a stored artifact must hold, shared by the store
4
+ * (which refuses anything else before a path is built from it) and the
5
+ * `proven` verifier (which refuses anything else before reading a field).
6
+ */
7
+ import { randomBytes } from "node:crypto";
8
+ /** `prf_` and 16 lowercase hex: the only shape a proof id has. */
9
+ export const PROOF_ID_REGEX = /^prf_[0-9a-f]{16}$/;
10
+ /** Is `id` a proof id (`prf_<16 lowercase hex>`)? */
11
+ export function isProofId(id) {
12
+ return typeof id === "string" && PROOF_ID_REGEX.test(id);
13
+ }
14
+ /** A fresh proof id: `prf_` and 16 hex characters from the system CSPRNG. */
15
+ export function newProofId() {
16
+ return `prf_${randomBytes(8).toString("hex")}`;
17
+ }
18
+ const strings = (value) => Array.isArray(value) && value.every((item) => typeof item === "string");
19
+ /** Does `value` have every field a `ProofArtifact` must, with the right types? */
20
+ export function isProofArtifact(value) {
21
+ if (typeof value !== "object" || value === null)
22
+ return false;
23
+ const v = value;
24
+ return (v["version"] === 1 &&
25
+ isProofId(v["id"]) &&
26
+ typeof v["checker"] === "string" &&
27
+ typeof v["claim"] === "string" &&
28
+ typeof v["statement"] === "string" &&
29
+ typeof v["proof"] === "string" &&
30
+ typeof v["ok"] === "boolean" &&
31
+ strings(v["assumptions"]) &&
32
+ strings(v["axioms"]) &&
33
+ typeof v["output"] === "string" &&
34
+ typeof v["outputDigest"] === "string" &&
35
+ typeof v["checkedAt"] === "string" &&
36
+ (v["counterexample"] === undefined || typeof v["counterexample"] === "string") &&
37
+ (v["reason"] === undefined || typeof v["reason"] === "string"));
38
+ }
@@ -0,0 +1,32 @@
1
+ /** tool-web's fetcher: its result opens with a `URL: …\nStatus: …` header. */
2
+ export declare const WEB_FETCH = "WebFetch";
3
+ /** tool-fetch's fetcher: its result opens with `HTTP <status>` and the headers. */
4
+ export declare const FETCH = "Fetch";
5
+ /** tool-fs's reader: its result is the file's text. */
6
+ export declare const READ = "Read";
7
+ /** tool-retrieve's tool (pipeline and every `knowledge:` corpus): `[n] id=… doc=… score=…` hits. */
8
+ export declare const RETRIEVE = "Retrieve";
9
+ /** tool-proof's checker tool: its result is the JSON report of the artifact it wrote. */
10
+ export declare const PROOF_CHECK = "ProofCheck";
11
+ /** tool-verify's claim checker: its result is a JSON verdict per claim, with the source text behind it. */
12
+ export declare const FACT_CROSS_CHECK = "FactCrossCheck";
13
+ /** tool-proof's preflight: its input is the model's draft, which it only reads (design §7). */
14
+ export declare const PROVABILITY_PREFLIGHT = "ProvabilityPreflight";
15
+ /**
16
+ * Is `name` a tool an MCP server serves (`mcp__<server>__<tool>`, tool-mcp's
17
+ * naming)? Its result is what that server returned — classified at the
18
+ * `mcp` origin on the way in, and not the runtime's own model output: the
19
+ * runtime's other non-builtin tools are that — `Task` returns a sub-agent's
20
+ * final message, `Consult`/`Escalate` a sibling model's reply, `Skill` a
21
+ * skill body, the memory, plan and goal tools what a model stored.
22
+ *
23
+ * Not "never text the model wrote", though: a server with storage (SQL, a
24
+ * key-value or notes store, a vector index) hands back what was written to
25
+ * it, by this run or an earlier one. The verifier refuses a claim whose
26
+ * words an MCP result carries only as text the run itself sent earlier
27
+ * (verify.ts `echoCheck`); rows stored by an earlier SESSION are the
28
+ * residual the README names.
29
+ */
30
+ export declare function isMcpTool(name: string): boolean;
31
+ /** The builtin spec key a recorded tool name maps to (case-insensitively), or `undefined`. */
32
+ export declare function builtinKey(name: string): string | undefined;
@@ -0,0 +1,46 @@
1
+ /**
2
+ * Which recorded tool names are builtins (`@crewhaus/tool-categories`), which
3
+ * an MCP server serves, and the ones whose result formats the evidence index
4
+ * and the verifiers read.
5
+ */
6
+ import { builtinKeyForName } from "@crewhaus/tool-categories";
7
+ /** tool-web's fetcher: its result opens with a `URL: …\nStatus: …` header. */
8
+ export const WEB_FETCH = "WebFetch";
9
+ /** tool-fetch's fetcher: its result opens with `HTTP <status>` and the headers. */
10
+ export const FETCH = "Fetch";
11
+ /** tool-fs's reader: its result is the file's text. */
12
+ export const READ = "Read";
13
+ /** tool-retrieve's tool (pipeline and every `knowledge:` corpus): `[n] id=… doc=… score=…` hits. */
14
+ export const RETRIEVE = "Retrieve";
15
+ /** tool-proof's checker tool: its result is the JSON report of the artifact it wrote. */
16
+ export const PROOF_CHECK = "ProofCheck";
17
+ /** tool-verify's claim checker: its result is a JSON verdict per claim, with the source text behind it. */
18
+ export const FACT_CROSS_CHECK = "FactCrossCheck";
19
+ /** tool-proof's preflight: its input is the model's draft, which it only reads (design §7). */
20
+ export const PROVABILITY_PREFLIGHT = "ProvabilityPreflight";
21
+ /**
22
+ * Is `name` a tool an MCP server serves (`mcp__<server>__<tool>`, tool-mcp's
23
+ * naming)? Its result is what that server returned — classified at the
24
+ * `mcp` origin on the way in, and not the runtime's own model output: the
25
+ * runtime's other non-builtin tools are that — `Task` returns a sub-agent's
26
+ * final message, `Consult`/`Escalate` a sibling model's reply, `Skill` a
27
+ * skill body, the memory, plan and goal tools what a model stored.
28
+ *
29
+ * Not "never text the model wrote", though: a server with storage (SQL, a
30
+ * key-value or notes store, a vector index) hands back what was written to
31
+ * it, by this run or an earlier one. The verifier refuses a claim whose
32
+ * words an MCP result carries only as text the run itself sent earlier
33
+ * (verify.ts `echoCheck`); rows stored by an earlier SESSION are the
34
+ * residual the README names.
35
+ */
36
+ export function isMcpTool(name) {
37
+ return name.startsWith("mcp__");
38
+ }
39
+ /** Builtin keys by recorded name, memoized: the table is 500+ entries long. */
40
+ const keys = new Map();
41
+ /** The builtin spec key a recorded tool name maps to (case-insensitively), or `undefined`. */
42
+ export function builtinKey(name) {
43
+ if (!keys.has(name))
44
+ keys.set(name, builtinKeyForName(name));
45
+ return keys.get(name);
46
+ }