assertledger 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.fr.md +3 -3
- package/README.md +3 -3
- package/SECURITY.md +11 -6
- package/conformance/schema-extensions.json +36 -0
- package/dist/build-info.d.ts +16 -0
- package/dist/build-info.d.ts.map +1 -0
- package/dist/build-info.js +18 -0
- package/dist/build-info.js.map +1 -0
- package/dist/build-info.json +7 -0
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +117 -19
- package/dist/cli.js.map +1 -1
- package/dist/contracts/index.d.ts +943 -0
- package/dist/contracts/index.d.ts.map +1 -1
- package/dist/contracts/index.js +561 -21
- package/dist/contracts/index.js.map +1 -1
- package/dist/core/index.d.ts +10 -1
- package/dist/core/index.d.ts.map +1 -1
- package/dist/core/index.js +473 -4
- package/dist/core/index.js.map +1 -1
- package/dist/diagnostics.d.ts.map +1 -1
- package/dist/diagnostics.js +42 -2
- package/dist/diagnostics.js.map +1 -1
- package/dist/engine/adapters/node-test-runtime.d.ts +17 -0
- package/dist/engine/adapters/node-test-runtime.d.ts.map +1 -1
- package/dist/engine/adapters/node-test-runtime.js +45 -23
- package/dist/engine/adapters/node-test-runtime.js.map +1 -1
- package/dist/engine/container.d.ts +54 -0
- package/dist/engine/container.d.ts.map +1 -0
- package/dist/engine/container.js +464 -0
- package/dist/engine/container.js.map +1 -0
- package/dist/engine/git-regression.d.ts +12 -2
- package/dist/engine/git-regression.d.ts.map +1 -1
- package/dist/engine/git-regression.js +63 -16
- package/dist/engine/git-regression.js.map +1 -1
- package/dist/engine/index.d.ts +7 -1
- package/dist/engine/index.d.ts.map +1 -1
- package/dist/engine/index.js +300 -74
- package/dist/engine/index.js.map +1 -1
- package/dist/mcp/index.d.ts.map +1 -1
- package/dist/mcp/index.js +53 -1
- package/dist/mcp/index.js.map +1 -1
- package/dist/sdk/index.d.ts +16 -6
- package/dist/sdk/index.d.ts.map +1 -1
- package/dist/sdk/index.js +63 -5
- package/dist/sdk/index.js.map +1 -1
- package/docs/agentic-test-profile.md +5 -0
- package/docs/architecture.md +8 -3
- package/docs/ci.md +7 -1
- package/docs/conformance-v1.md +11 -2
- package/docs/container-isolation.md +156 -0
- package/docs/evidence-export.md +146 -0
- package/docs/git-regression.md +7 -3
- package/docs/migration-verification-v2.md +55 -0
- package/docs/project-intent.md +3 -2
- package/docs/proof-model.md +8 -4
- package/docs/reference.md +56 -11
- package/docs/roadmap.md +12 -4
- package/examples/agentic-profile/profile-manifest.mjs +5 -1
- package/examples/evidence-export/consumer.mjs +35 -0
- package/package.json +2 -2
- package/schemas/evidence-export-replay-result.v1.json +49 -0
- package/schemas/evidence-export-request.v1.json +681 -0
- package/schemas/evidence-export.v1.json +1479 -0
- package/schemas/evidence-manifest.v2.json +886 -0
- package/schemas/evidence-provider-manifest.v1.json +269 -0
- package/schemas/verification-request.v2.json +457 -0
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# Container isolation
|
|
2
|
+
|
|
3
|
+
[Back to the reference](reference.md) · [Security policy](../SECURITY.md)
|
|
4
|
+
|
|
5
|
+
Verification request v2 adds a `container` isolation backend. Every control and candidate
|
|
6
|
+
execution then runs in a fresh Linux container created from a digest-pinned image that is already
|
|
7
|
+
present on an operator-administered Docker-compatible daemon. `trusted-local` stays available in v1
|
|
8
|
+
and v2, remains explicitly `UNSANDBOXED`, and still requires external authorization.
|
|
9
|
+
|
|
10
|
+
Container isolation is a containment layer around an execution, not a proof that a campaign is
|
|
11
|
+
correct. The daemon, its host kernel, the image and the AssertLedger engine remain trusted.
|
|
12
|
+
|
|
13
|
+
## Select the backend
|
|
14
|
+
|
|
15
|
+
The Git workflow selects the backend explicitly. Without `--container-image` or
|
|
16
|
+
`--allow-unsafe-execution`, `check` stops before reading the repository and names both options:
|
|
17
|
+
|
|
18
|
+
```sh
|
|
19
|
+
assertledger check . --before BEFORE --after AFTER --neutral NEUTRAL --neutral-reason "explicit control reason" --test tests/regression.test.js --base-test tests/base.test.js --out .assertledger/evidence --container-image node@sha256:DIGEST
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
`verify` runs a v2 request whose `isolation.kind` is `container` without
|
|
23
|
+
`--allow-unsafe-execution`:
|
|
24
|
+
|
|
25
|
+
```json
|
|
26
|
+
{
|
|
27
|
+
"schemaVersion": "2.0.0",
|
|
28
|
+
"isolation": {
|
|
29
|
+
"kind": "container",
|
|
30
|
+
"image": "node@sha256:DIGEST",
|
|
31
|
+
"environment": [{ "name": "TZ", "value": "UTC" }],
|
|
32
|
+
"limits": {
|
|
33
|
+
"memoryBytes": 1073741824,
|
|
34
|
+
"cpuMillicores": 2000,
|
|
35
|
+
"pids": 256,
|
|
36
|
+
"temporaryDirectoryBytes": 67108864
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
}
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
The other request fields keep their v1 meaning; `assertledger schema verification-request-v2 --json`
|
|
43
|
+
prints the complete contract. `check` uses the limits shown above and a 30-second timeout per
|
|
44
|
+
execution.
|
|
45
|
+
|
|
46
|
+
Combining a container image or request with `--allow-unsafe-execution` fails with
|
|
47
|
+
`ISOLATION_MODE_CONFLICT` before any runtime call. Combining `--container-runtime` with a
|
|
48
|
+
trusted-local request fails the same way.
|
|
49
|
+
|
|
50
|
+
The runtime command belongs to the operator, never to the request. It defaults to `["docker"]` and
|
|
51
|
+
is passed as a JSON argument array; it is never interpreted by a shell:
|
|
52
|
+
|
|
53
|
+
```sh
|
|
54
|
+
assertledger verify request.json --container-runtime '["docker"]' --json
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
On Windows with Docker Engine inside WSL, use
|
|
58
|
+
`--container-runtime '["wsl.exe","-d","Ubuntu","--exec","docker"]'` from a POSIX shell. Shells that
|
|
59
|
+
strip inner double quotes from native arguments, such as Windows PowerShell 5.1, need escaped
|
|
60
|
+
quotes. The SDK takes the same values through its v2 entry points, which return v2 manifests:
|
|
61
|
+
|
|
62
|
+
```js
|
|
63
|
+
const ledger = new AssertLedger();
|
|
64
|
+
await ledger.verifyV2(request, { containerRuntime: { command: ["docker"] } });
|
|
65
|
+
await ledger.checkGitRegressionV2({ ...options, container: { image: "node@sha256:DIGEST" } });
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
`verify()` and `checkGitRegression()` keep their v1 contracts and result types.
|
|
69
|
+
|
|
70
|
+
The MCP `check` and `verify` tools still execute only trusted-local v1 requests.
|
|
71
|
+
|
|
72
|
+
## Backend checks before execution
|
|
73
|
+
|
|
74
|
+
AssertLedger queries the runtime before creating a workspace container. Each refusal happens before
|
|
75
|
+
any repository code runs and has an [`explain`](diagnostics.md) entry:
|
|
76
|
+
|
|
77
|
+
| Reason code | Meaning |
|
|
78
|
+
| --- | --- |
|
|
79
|
+
| `CONTAINER_RUNTIME_COMMAND_INVALID` | The runtime argv is not a JSON array of 1 to 16 non-empty strings. |
|
|
80
|
+
| `CONTAINER_RUNTIME_NOT_FOUND` | The runtime executable cannot be started. |
|
|
81
|
+
| `CONTAINER_RUNTIME_UNAVAILABLE` | The runtime reports no reachable daemon. |
|
|
82
|
+
| `CONTAINER_RUNTIME_PLATFORM_UNSUPPORTED` | The daemon does not run Linux containers. |
|
|
83
|
+
| `CONTAINER_IMAGE_REFERENCE_INVALID` | The `check` image is not pinned by `@sha256:`; a v2 request with such an image fails request validation. |
|
|
84
|
+
| `CONTAINER_IMAGE_NOT_PRESENT` | The pinned image is absent locally. AssertLedger never pulls. |
|
|
85
|
+
| `CONTAINER_IMAGE_DIGEST_MISMATCH` | The local image does not carry the pinned digest. |
|
|
86
|
+
| `CONTAINER_IMAGE_PLATFORM_UNSUPPORTED` | The image is not a Linux image. |
|
|
87
|
+
| `CONTAINER_CLEANUP_FAILED` | A finished container could not be removed; no result is produced. |
|
|
88
|
+
|
|
89
|
+
Review an image, then pull it yourself by the same digest, for example
|
|
90
|
+
`docker pull node@sha256:DIGEST`. For `node:test`, the engine also probes `node` inside the image
|
|
91
|
+
twice and records its path, version and SHA-256 digest, as it does locally.
|
|
92
|
+
|
|
93
|
+
## Controls applied to every execution
|
|
94
|
+
|
|
95
|
+
| Boundary | Control |
|
|
96
|
+
| --- | --- |
|
|
97
|
+
| Files | No host path is mounted. The prepared workspace is streamed as an archive into an anonymous volume; the root file system is read-only; `/tmp` is a `tmpfs` bounded by `temporaryDirectoryBytes`. |
|
|
98
|
+
| Result | Only a regular file named `result.json`, no larger than the report bound, is read back as an archive in memory. A link, a renamed entry or a malformed archive yields `INFRA_ERROR`. |
|
|
99
|
+
| Identity | User and group `65534`, all capabilities dropped, `no-new-privileges`. |
|
|
100
|
+
| Network | `--network=none`: only the loopback interface exists. |
|
|
101
|
+
| Environment | The image's own variables, the variables the runtime sets itself such as `HOSTNAME` and `HOME`, the declared `environment` entries and the adapter protocol variables. The host environment is never forwarded. |
|
|
102
|
+
| Resources | `pids`, `memoryBytes` with swap disabled, and `cpuMillicores` limits. |
|
|
103
|
+
| Time | On timeout the container is killed, which ends every process in its PID namespace, including detached descendants. |
|
|
104
|
+
| Output | Standard output and error are bounded by `maximumOutputBytes`; the runtime log driver is disabled. |
|
|
105
|
+
| Lifetime | One container per execution, labeled `assertledger.execution`, removed with its volume afterwards. |
|
|
106
|
+
|
|
107
|
+
A timeout is recorded as `TIMEOUT`. A process stopped by the memory or process limit exits without a
|
|
108
|
+
valid report and is recorded as an infrastructure or process failure. Neither can count as target
|
|
109
|
+
detection: only an attributed `ASSERTION_FAILURE` kills a target.
|
|
110
|
+
|
|
111
|
+
## Evidence
|
|
112
|
+
|
|
113
|
+
A v2 manifest binds the backend facts to the decision digest in
|
|
114
|
+
`evidenceContext.execution.backend`: image reference, identifier, OS and architecture; runtime
|
|
115
|
+
client and server versions, server OS and architecture, cgroup version and reported security
|
|
116
|
+
options; and the controls, declared environment and limits above. The top-level
|
|
117
|
+
`isolation` summary records `kind: "container"`, `level: "CONTAINER"` and the runtime argv; it is
|
|
118
|
+
covered by the artifact digest. The manifest's `limitations` state the main limits of this backend.
|
|
119
|
+
|
|
120
|
+
Declared environment values are recorded verbatim. Never place a secret in `isolation.environment`.
|
|
121
|
+
|
|
122
|
+
Replay validates v1 and v2 manifests. Evidence export, Agentic Test Profiles and benchmarks still
|
|
123
|
+
accept only v1 manifests. The v1 request and manifest contracts are unchanged.
|
|
124
|
+
|
|
125
|
+
## Limits and non-claims
|
|
126
|
+
|
|
127
|
+
- Containers share the daemon host's kernel; this is not a virtual machine boundary.
|
|
128
|
+
- Access to the Docker daemon is equivalent to administrative access on its host. Run it on a host
|
|
129
|
+
without secrets, as you would a CI runner.
|
|
130
|
+
- AssertLedger does not bound the writable workspace volume. Storage quotas remain an
|
|
131
|
+
operator-administered daemon setting.
|
|
132
|
+
- `/dev/shm` keeps the runtime default size (64 MiB on Docker Engine), charged to the memory limit.
|
|
133
|
+
- The image and declared environment values are operator inputs. AssertLedger verifies the image
|
|
134
|
+
digest, not its contents. With no network, the image must already contain every runtime the
|
|
135
|
+
adapter needs.
|
|
136
|
+
- An interrupted AssertLedger process can leave a labeled container. Remove it with
|
|
137
|
+
`docker ps --all --filter label=assertledger.execution` followed by `docker rm --force --volumes`.
|
|
138
|
+
- Only Linux daemons are supported. The host side is platform-neutral Node.js: CI runs the backend
|
|
139
|
+
selection, diagnostic, archive and cleanup tests against a fake runtime on Linux, Windows and
|
|
140
|
+
macOS. The hostile scenario suite runs against real Docker Engine on Linux in CI and was run
|
|
141
|
+
locally against Docker Engine in WSL on Windows. GitHub-hosted Windows and macOS runners provide
|
|
142
|
+
no Linux daemon, so no real daemon is exercised from a macOS host.
|
|
143
|
+
|
|
144
|
+
## Real-daemon test suite
|
|
145
|
+
|
|
146
|
+
`tests/container-isolation-docker.test.ts` runs hostile scenarios against a real daemon: network
|
|
147
|
+
egress, root file system writes, host paths, environment leakage, privileges, temporary storage
|
|
148
|
+
and process count, a detached descendant after timeout, a result link, a memory limit, an absent
|
|
149
|
+
image and an end-to-end `check`. It is skipped unless both variables below are set, and
|
|
150
|
+
`ASSERTLEDGER_REQUIRE_CONTAINER_TESTS=1` turns a missing configuration into a failure:
|
|
151
|
+
|
|
152
|
+
```sh
|
|
153
|
+
ASSERTLEDGER_CONTAINER_RUNTIME='["docker"]' ASSERTLEDGER_CONTAINER_IMAGE='node@sha256:DIGEST' pnpm test
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
The suite never pulls; provide the image first.
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
# Interoperable evidence export
|
|
2
|
+
|
|
3
|
+
AssertLedger can hand its results to an external engine, such as a change-governance tool, as
|
|
4
|
+
typed evidence. The export is a thin, deterministic projection of a replay-valid
|
|
5
|
+
[evidence manifest](proof-model.md). It adds no gate, no authority, and no runtime dependency on
|
|
6
|
+
any consumer: AssertLedger behaves identically whether a consumer exists, accepts the evidence,
|
|
7
|
+
degrades it, or ignores it.
|
|
8
|
+
|
|
9
|
+
Three public contracts are involved, each with a versioned JSON Schema:
|
|
10
|
+
|
|
11
|
+
| Contract | Schema | Produced by |
|
|
12
|
+
| --- | --- | --- |
|
|
13
|
+
| Provider manifest | [`evidence-provider-manifest.v1.json`](../schemas/evidence-provider-manifest.v1.json) | `assertledger provider`, `AssertLedger.providerManifest()`, `assertledger_provider` |
|
|
14
|
+
| Export request | [`evidence-export-request.v1.json`](../schemas/evidence-export-request.v1.json) | The consumer or operator |
|
|
15
|
+
| Evidence export | [`evidence-export.v1.json`](../schemas/evidence-export.v1.json) | `assertledger export`, `AssertLedger.exportEvidence()`, `assertledger_export` |
|
|
16
|
+
| Export replay result | [`evidence-export-replay-result.v1.json`](../schemas/evidence-export-replay-result.v1.json) | `assertledger export-replay`, `AssertLedger.replayEvidenceExport()`, `assertledger_export_replay` |
|
|
17
|
+
|
|
18
|
+
## Provider manifest: what is announced
|
|
19
|
+
|
|
20
|
+
The provider manifest describes the installed provider: its version, the source revision recorded
|
|
21
|
+
at build time, its scope, the formats it accepts and emits, its capabilities, its supported
|
|
22
|
+
adapters, its cost model, and its limits. It carries a `manifestDigest` over its canonical
|
|
23
|
+
projection.
|
|
24
|
+
|
|
25
|
+
The source revision is `RECORDED` with the commit and a `CLEAN` or `DIRTY` worktree only when the
|
|
26
|
+
package build captured it (`dist/build-info.json`). A source checkout or a build without Git reports
|
|
27
|
+
`UNKNOWN`; the current checkout never fills in a missing value.
|
|
28
|
+
|
|
29
|
+
Capabilities are announcements. `CONTROL_WITHOUT_CANDIDATE`, `REFERENCE_PASS`,
|
|
30
|
+
`REGRESSION_DETECTION`, `NEUTRAL_PASS`, and `STABILITY_REPETITION` are `SUPPORTED` test-observed
|
|
31
|
+
controls. `GIT_REVISION_PROVENANCE` is `SUPPORTED_WHEN_RECORDED`. `EXECUTION_FRESHNESS`,
|
|
32
|
+
`PRODUCER_AUTHENTICATION`, and `SANDBOXED_EXECUTION` are `UNSUPPORTED`. A capability never proves
|
|
33
|
+
that a control ran: only the recorded observations of an exported manifest do.
|
|
34
|
+
|
|
35
|
+
## Export request
|
|
36
|
+
|
|
37
|
+
```json
|
|
38
|
+
{
|
|
39
|
+
"schemaVersion": "1.0.0",
|
|
40
|
+
"manifest": { "...": "a replay-valid evidence manifest v1" },
|
|
41
|
+
"consumerRequest": {
|
|
42
|
+
"reference": "change-42",
|
|
43
|
+
"profileId": null,
|
|
44
|
+
"obligations": [
|
|
45
|
+
{ "id": "detect", "control": "REGRESSION_DETECTION" },
|
|
46
|
+
{ "id": "sandbox", "control": "SANDBOXED_EXECUTION" }
|
|
47
|
+
]
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
`consumerRequest` may be `null`. Obligation identifiers must be unique. The request is not a
|
|
53
|
+
policy for AssertLedger: it only lets the export state which requested controls were executed,
|
|
54
|
+
not executed, or unsupported. No particular plan format is required.
|
|
55
|
+
|
|
56
|
+
The export refuses a manifest that fails replay with `EVIDENCE_EXPORT_SOURCE_INVALID` (CLI exit
|
|
57
|
+
code `4`, no output). A malformed request fails with `EVIDENCE_EXPORT_REQUEST_INVALID`, as does a
|
|
58
|
+
request embedding a v2 manifest from [container isolation](container-isolation.md): this export
|
|
59
|
+
version accepts only v1 manifests.
|
|
60
|
+
|
|
61
|
+
## Evidence export: what was observed
|
|
62
|
+
|
|
63
|
+
The export embeds its `sourceManifest` and is self-contained. Its sections are deliberately
|
|
64
|
+
separate:
|
|
65
|
+
|
|
66
|
+
- `result`: the verbatim campaign decision, a per-candidate and per-world view, and a single
|
|
67
|
+
conservative `detection` with a stable `reasonCode`.
|
|
68
|
+
- `integrity`: the replay rails that were verified before export and the bound digests (artifact,
|
|
69
|
+
decision, repository, policy, and world digests).
|
|
70
|
+
- `authenticity`: always `UNAUTHENTICATED` with attestation `NONE`; the producer name and version
|
|
71
|
+
are only declared by the manifest.
|
|
72
|
+
- `environment`: `trusted-local`/`UNSANDBOXED` isolation, the environment allowlist names, and the
|
|
73
|
+
adapter. The framework is `RECORDED` only for a `node:test` manifest that recorded its official
|
|
74
|
+
adapter profile; otherwise it stays `UNKNOWN`.
|
|
75
|
+
- `confidence`: the level `REPLAY_CONSISTENT_UNAUTHENTICATED`, with the properties that replay
|
|
76
|
+
established and those it did not (producer authenticity, observation truthfulness, execution
|
|
77
|
+
isolation, execution freshness, and the semantic relevance of the declared worlds).
|
|
78
|
+
- `scope`: attempts, candidate and observation counts, and each world with its digest, declared
|
|
79
|
+
provenance, and Git revision. Git commits and trees are exported only when the provenance is the
|
|
80
|
+
exact canonical record written by [Git regression qualification](git-regression.md);
|
|
81
|
+
`gitRevisions` is `RECORDED`, `PARTIAL`, or `NOT_RECORDED`.
|
|
82
|
+
- `controls`: executed controls with their observation counts, requested obligations with
|
|
83
|
+
`EXECUTED`, `NOT_EXECUTED`, or `UNSUPPORTED` coverage, and gates that did not run.
|
|
84
|
+
- `profile`: `NOT_REQUESTED`, or `UNKNOWN_PROFILE` for any requested profile identifier. No usage
|
|
85
|
+
profile is defined yet, so none is ever applied or inferred.
|
|
86
|
+
- `policy`: the effective AssertLedger policy version and digest.
|
|
87
|
+
- `cost`: the estimated process executions derived from the recorded campaign shape, the observed
|
|
88
|
+
executions and recorded wall time with its coverage, and execution freshness `UNKNOWN` with cache
|
|
89
|
+
provenance `NOT_RECORDED`, because evidence manifest v1 does not record them.
|
|
90
|
+
|
|
91
|
+
### Detection mapping
|
|
92
|
+
|
|
93
|
+
The first matching row applies.
|
|
94
|
+
|
|
95
|
+
| Situation | `detection` | `modality` | `reasonCode` |
|
|
96
|
+
| --- | --- | --- | --- |
|
|
97
|
+
| Campaign `VERIFIED` | `OBSERVED` | `TEST_OBSERVED` | `REGRESSION_ASSERTION_OBSERVED` |
|
|
98
|
+
| Campaign `ENGINE_ERROR` | `NOT_ESTABLISHED` | `NONE` | `ENGINE_ERROR` |
|
|
99
|
+
| Invalid controls | `NOT_ESTABLISHED` | `NONE` | `CONTROL_EVIDENCE_INVALID` |
|
|
100
|
+
| An `UNSTABLE` or `INCONCLUSIVE` candidate | `NOT_ESTABLISHED` | `NONE` | `CANDIDATE_EVIDENCE_INCONCLUSIVE` |
|
|
101
|
+
| An `INVALID` candidate | `NOT_ESTABLISHED` | `NONE` | `CANDIDATE_EVIDENCE_INVALID` |
|
|
102
|
+
| A target outcome that is neither a stable attributed `PASS` nor a stable attributed assertion | `NOT_ESTABLISHED` | `NONE` | `OPERATIONAL_OUTCOME_NOT_DETECTION` |
|
|
103
|
+
| At least one target observed passing | `NOT_OBSERVED` | `TEST_OBSERVED` | `TARGET_PASSED_WITHOUT_DETECTION` |
|
|
104
|
+
| Otherwise, such as every target killed without meeting the policy | `NOT_ESTABLISHED` | `NONE` | `TARGET_STRENGTH_INSUFFICIENT` |
|
|
105
|
+
|
|
106
|
+
The targeted test is the candidate: its `id` and content `digest`, the SHA-256 of the canonical JSON
|
|
107
|
+
of its `[{ "path", "content" }]` overlay files. Evidence manifest v1 does not record the file paths
|
|
108
|
+
themselves; a consumer holding the verification request (for example `executed-request.json`
|
|
109
|
+
written by `assertledger check`) can recompute the digest to bind paths to the exported evidence.
|
|
110
|
+
|
|
111
|
+
Per world, `signal` is `RED` only for complete, stable, attributed `ASSERTION_FAILURE` runs and
|
|
112
|
+
`GREEN` only for complete, stable, attributed `PASS` runs. A target world's `detection` is
|
|
113
|
+
`OBSERVED` or `NOT_OBSERVED` only when controls are valid and the candidate is `ELIGIBLE` or
|
|
114
|
+
`WEAK_ORACLE`. Compilation, collection, crash, timeout, infrastructure, and no-test outcomes are
|
|
115
|
+
never promoted to detection evidence in either direction.
|
|
116
|
+
|
|
117
|
+
## Determinism and replay
|
|
118
|
+
|
|
119
|
+
The same manifest and consumer request always produce the same export bytes after canonical
|
|
120
|
+
serialization, independent of JSON key order; obligations are ordered by identifier. The
|
|
121
|
+
`exportDigest` covers the whole export except itself.
|
|
122
|
+
|
|
123
|
+
`assertledger export-replay` validates the export schema, replays the embedded source manifest,
|
|
124
|
+
recomputes `exportDigest`, and rebuilds the export from the embedded manifest and consumer request.
|
|
125
|
+
`valid` requires all four rails. It exits with `0` when valid and `4` otherwise. A re-digested
|
|
126
|
+
forgery keeps `exportDigestValid` true but fails `semanticsValid`.
|
|
127
|
+
|
|
128
|
+
## Consumer responsibilities
|
|
129
|
+
|
|
130
|
+
A consumer decides admissibility with its own obligations. The
|
|
131
|
+
[`consumer example`](../examples/evidence-export/consumer.mjs) rejects an export that fails replay,
|
|
132
|
+
ignores exports without observed detection, and degrades observed detection to advisory when
|
|
133
|
+
controls or Git revisions are missing or the evidence is unauthenticated, which is always the case
|
|
134
|
+
for this version. Rejecting, degrading, or ignoring an export changes nothing in AssertLedger.
|
|
135
|
+
|
|
136
|
+
## Non-claims
|
|
137
|
+
|
|
138
|
+
The export does not rerun tests, authenticate the producer, attest isolation, prove that recorded
|
|
139
|
+
observations were truthful, prove freshness, or make declared worlds semantically relevant. It
|
|
140
|
+
covers only the recorded worlds, candidates, and attempts.
|
|
141
|
+
|
|
142
|
+
## Versioning
|
|
143
|
+
|
|
144
|
+
These four schemas are additive to the frozen [conformance v1](conformance-v1.md) set and are
|
|
145
|
+
locked by `conformance/schema-extensions.json`. Any change to their bytes, their mapping, or their
|
|
146
|
+
digest projections requires a new schema version, a migration note, and compatibility tests.
|
package/docs/git-regression.md
CHANGED
|
@@ -11,14 +11,18 @@ assertledger check . --before BEFORE --after AFTER --neutral NEUTRAL --neutral-r
|
|
|
11
11
|
|
|
12
12
|
`--after` utilise `HEAD` par défaut. `--base-test` peut être répété. Le dossier donné à `--out` doit être un nouveau chemin relatif au dépôt. AssertLedger le réserve de manière exclusive, écrit `executed-request.json` et `summary.md`, puis publie `manifest.json` en dernier par renommage atomique. La présence de `manifest.json` est le marqueur de complétion ; un lecteur doit ignorer un dossier qui ne le contient pas.
|
|
13
13
|
|
|
14
|
-
|
|
14
|
+
Le mode d’exécution est toujours choisi explicitement. Sans `--container-image` ni `--allow-unsafe-execution`, `check` refuse de s’exécuter ; les deux ensemble sont refusés avec `ISOLATION_MODE_CONFLICT`.
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
Pour isoler l’exécution, remplacez `--allow-unsafe-execution` par `--container-image NOM@sha256:DIGEST`. Chaque contrôle et chaque candidat s’exécute alors dans un conteneur Linux neuf, sans réseau ni montage de l’hôte, avec les limites par défaut et un délai de 30 secondes par exécution. L’image doit déjà être présente sur le démon et contenir `node` : AssertLedger ne la télécharge jamais. `--container-runtime` accepte la commande du runtime sous forme de tableau JSON, `["docker"]` par défaut. Le manifeste produit est alors en version `2.0.0`. Voir [l’isolation par conteneur](container-isolation.md).
|
|
17
|
+
|
|
18
|
+
L’exécution `trusted-local` est volontairement **UNSANDBOXED**. Le drapeau `--allow-unsafe-execution` constitue l’autorisation distincte de l’opérateur. Côté SDK, `new AssertLedger().checkGitRegression(options)` exige `allowUnsafeExecution: true` et renvoie un manifeste v1 ; `checkGitRegressionV2(options)` exige `container: { image }` et renvoie un manifeste v2.
|
|
19
|
+
|
|
20
|
+
MCP expose le parcours `trusted-local` avec `assertledger_check` (alias `testforge_check`) seulement si
|
|
17
21
|
l’opérateur a démarré le serveur avec `--allow-unsafe-execution`. L’entrée reprend les options
|
|
18
22
|
du SDK, sans le champ de permission : `repository`, `before`, `after` facultatif, `neutral`,
|
|
19
23
|
`neutralReason`, `test`, `baseTests` et `out`. La racine est confinée aux dépôts autorisés ; le
|
|
20
24
|
dossier de sortie suit les mêmes contrôles que la CLI. Un client ne peut pas s’accorder cette
|
|
21
|
-
permission dans son message.
|
|
25
|
+
permission dans son message. Le mode conteneur n’est pas encore exposé par MCP.
|
|
22
26
|
|
|
23
27
|
## Limites de cette première tranche
|
|
24
28
|
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Verification request and evidence manifest v2
|
|
2
|
+
|
|
3
|
+
Version `2.0.0` of the verification request and evidence manifest adds
|
|
4
|
+
[container isolation](container-isolation.md). It is a new major contract beside v1, not a change
|
|
5
|
+
to v1.
|
|
6
|
+
|
|
7
|
+
## What stays the same
|
|
8
|
+
|
|
9
|
+
- The 34 v1 schema bytes, the conformance-v1 lock root, v1 decision and artifact digest
|
|
10
|
+
projections, outcomes, gates and reason-code semantics are unchanged.
|
|
11
|
+
- A v1 request still produces a v1 manifest, and v1 manifests replay exactly as before.
|
|
12
|
+
- `parseVerificationRequest()` and `parseEvidenceManifest()` still accept only v1 and reject v2.
|
|
13
|
+
- The SDK methods `verify()` and `checkGitRegression()` keep their v1 inputs, results and TypeScript
|
|
14
|
+
types; `verify()` still refuses a v2 request with `SCHEMA_VERSION_UNSUPPORTED`.
|
|
15
|
+
- `trusted-local` keeps its fields, stays `UNSANDBOXED` and still requires explicit authorization.
|
|
16
|
+
- MCP tool schemas, evidence export, Agentic Test Profiles and benchmarks still accept only v1.
|
|
17
|
+
|
|
18
|
+
## What v2 adds
|
|
19
|
+
|
|
20
|
+
- `schemas/verification-request.v2.json`: the v1 request with `schemaVersion: "2.0.0"` and an
|
|
21
|
+
`isolation` union of the unchanged `trusted-local` object and a `container` object with a
|
|
22
|
+
digest-pinned image, declared environment and limits. A request cannot name the runtime command
|
|
23
|
+
or forward host variables.
|
|
24
|
+
- `schemas/evidence-manifest.v2.json`: the v1 manifest with `evidenceContext.execution.backend`,
|
|
25
|
+
which the decision digest covers, and a top-level `isolation` union whose container form records
|
|
26
|
+
the runtime argv. A v2 `trusted-local` manifest records `{ "kind": "trusted-local", "level":
|
|
27
|
+
"UNSANDBOXED" }` as its backend.
|
|
28
|
+
- Both schemas join the additive schema-extension lock, whose published digest changes
|
|
29
|
+
accordingly.
|
|
30
|
+
- `parseVerificationRequestV2()` and `parseEvidenceManifestV2()` accept only v2;
|
|
31
|
+
`parseVersionedVerificationRequest()` and `parseVersionedEvidenceManifest()` accept either version.
|
|
32
|
+
- The SDK adds `verifyV2(request, options)` and `checkGitRegressionV2(options)`, which return
|
|
33
|
+
`EvidenceManifestV2Contract`, and the `GitRegressionV2Options` type.
|
|
34
|
+
|
|
35
|
+
## Changes visible to existing callers
|
|
36
|
+
|
|
37
|
+
- A request declaring `schemaVersion: "2.0.0"` previously failed with
|
|
38
|
+
`SCHEMA_VERSION_UNSUPPORTED` everywhere. The CLI `verify` command, `verifyV2()` and the exported
|
|
39
|
+
engine function `verifyCampaign()` now validate it as v2. Other unknown versions still fail with
|
|
40
|
+
that code.
|
|
41
|
+
- CLI and SDK container failures use new `CONTAINER_*` and `ISOLATION_MODE_CONFLICT` reason codes,
|
|
42
|
+
with CLI exit code `4`, before any repository code runs.
|
|
43
|
+
- `assertledger schema` also prints `verification-request-v2` and `evidence-manifest-v2`.
|
|
44
|
+
|
|
45
|
+
## Compatibility witnesses
|
|
46
|
+
|
|
47
|
+
- `tests/verification-v2.test.ts`: v2 acceptance and rejection, digest binding of the backend,
|
|
48
|
+
replay tampering, v1 parsers and export closed to v2, and published schema identities.
|
|
49
|
+
- `tests/conformance-v1.test.ts` and `tests/schema-registry.test.ts`: unchanged v1 bytes and lock,
|
|
50
|
+
registered v2 schemas.
|
|
51
|
+
- `tests/container-backend.test.ts` and `tests/container-facades.test.ts`: backend selection,
|
|
52
|
+
hardened runtime arguments, archive handling and CLI/SDK behavior against a fake runtime.
|
|
53
|
+
- `tests/container-isolation-docker.test.ts`: hostile scenarios against a real Docker Engine.
|
|
54
|
+
- `tests/engine.test.ts` and `tests/core.test.ts` now use `3.0.0` as their unsupported-version
|
|
55
|
+
witness, because `2.0.0` became a supported version.
|
package/docs/project-intent.md
CHANGED
|
@@ -50,8 +50,9 @@ référence, un monde neutre rouge, une observation instable ou une attribution
|
|
|
50
50
|
|
|
51
51
|
Le socle local comprend les contrats versionnés, le noyau déterministe, les digests, le replay
|
|
52
52
|
sémantique, le moteur d’exécution, les façades CLI/SDK/MCP, l’audit statique et l’initialisation.
|
|
53
|
-
Le registre contient
|
|
54
|
-
|
|
53
|
+
Le registre contient 38 schémas JSON : les 34 figés par le verrou de conformance v1 et les 4
|
|
54
|
+
schémas d’export d’évidence, verrouillés par une extension additive qui ne modifie pas le bundle v1.
|
|
55
|
+
L’ancien décompte de 30 dans la roadmap était périmé.
|
|
55
56
|
|
|
56
57
|
L’adaptateur officiel intégré est `node:test`. Le protocole `testforge-command` permet d’intégrer
|
|
57
58
|
un autre framework avec un adaptateur fourni par l’opérateur. Détecter Vitest, Jest, Bun ou pytest
|
package/docs/proof-model.md
CHANGED
|
@@ -98,10 +98,12 @@ observations were truthful, authenticate the producer, or replace a signed exter
|
|
|
98
98
|
|
|
99
99
|
The manifest alone is not a self-contained reproduction bundle. It stores candidate and world
|
|
100
100
|
digests rather than their file bodies, stdout/stderr digests rather than raw logs, and environment
|
|
101
|
-
allowlist names rather than effective values.
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
101
|
+
allowlist names rather than effective values. A v2 container manifest also binds the image identity,
|
|
102
|
+
runtime facts, declared environment values, and limits to the decision. The built-in `node:test`
|
|
103
|
+
adapter records its resolved executable real path, version, and SHA-256 digest; the
|
|
104
|
+
structured-command adapter records only its configured command. Preserve the original request,
|
|
105
|
+
repository snapshot, dependencies, missing executable identities, and raw logs separately when
|
|
106
|
+
independent audit matters.
|
|
105
107
|
|
|
106
108
|
## Explicit non-claims
|
|
107
109
|
|
|
@@ -113,4 +115,6 @@ AssertLedger does not prove:
|
|
|
113
115
|
- semantic relevance or correctness of operator-supplied worlds;
|
|
114
116
|
- resistance to a candidate designed to recognize the worlds;
|
|
115
117
|
- containment of hostile code under `trusted-local`;
|
|
118
|
+
- containment beyond the recorded [container controls](container-isolation.md), which share the
|
|
119
|
+
daemon host's kernel and trust the daemon, its host, and the image;
|
|
116
120
|
- provenance authenticity without a separate signed attestation.
|
package/docs/reference.md
CHANGED
|
@@ -18,7 +18,11 @@ assertledger audit . --json
|
|
|
18
18
|
assertledger analyze . --json
|
|
19
19
|
assertledger schema verification-request --json
|
|
20
20
|
assertledger verify assertledger.request.json --allow-unsafe-execution --json
|
|
21
|
+
assertledger verify assertledger.container-request.json --container-runtime '["docker"]' --json
|
|
21
22
|
assertledger replay assertledger.manifest.json --json
|
|
23
|
+
assertledger provider --json
|
|
24
|
+
assertledger export assertledger.export-request.json --json
|
|
25
|
+
assertledger export-replay assertledger.evidence-export.json --json
|
|
22
26
|
assertledger profile assertledger.profile-request.json --json
|
|
23
27
|
assertledger profile-replay assertledger.profile-report.json --json
|
|
24
28
|
assertledger profile-v2 assertledger.profile-v2-request.json --json
|
|
@@ -45,15 +49,35 @@ directory or in a colocated `*.test.*`/`*.spec.*` source file. Comments, string
|
|
|
45
49
|
documentation-like config filenames do not count; no evidence produces an empty list. Detection is
|
|
46
50
|
intentionally incomplete: an unrecognized manifest layout or test-file convention yields no claim.
|
|
47
51
|
|
|
48
|
-
`verify`, `replay`, `
|
|
52
|
+
`verify`, `replay`, `export`, `export-replay`, `profile`, `profile-replay`, `profile-v2`,
|
|
53
|
+
`profile-v2-replay`, `benchmark`, and
|
|
49
54
|
`benchmark-replay`, `benchmark-acquire`, and `benchmark-acquire-replay` also accept `-`
|
|
50
55
|
or an omitted file argument and
|
|
51
56
|
then read JSON from stdin. The CLI rejects file and stdin JSON inputs larger than 16 MiB. JSON
|
|
52
57
|
results go to stdout. Diagnostics go to stderr. `assertledger mcp` reserves stdout for JSON-RPC.
|
|
53
58
|
|
|
54
59
|
`--allow-unsafe-execution` is an external authorization signal. The CLI requires it for every
|
|
55
|
-
campaign and sets the request's local acknowledgement before validation. The flag does
|
|
56
|
-
sandbox.
|
|
60
|
+
trusted-local campaign and sets the request's local acknowledgement before validation. The flag does
|
|
61
|
+
not create a sandbox.
|
|
62
|
+
|
|
63
|
+
A v2 request with `container` isolation runs without that flag: each execution uses a fresh
|
|
64
|
+
container from a digest-pinned local image through the operator's `--container-runtime` JSON argv,
|
|
65
|
+
which defaults to `["docker"]`. `check` selects the same backend with `--container-image`. Combining
|
|
66
|
+
the two modes fails with `ISOLATION_MODE_CONFLICT`. See [container isolation](container-isolation.md).
|
|
67
|
+
|
|
68
|
+
`profile` exits with `0` for `QUALIFIED`, `2` for `NOT_QUALIFIED` or `BUDGET_MISSED`, and `3` for
|
|
69
|
+
`INSUFFICIENT_TIMING_EVIDENCE`. A malformed request or a replay-invalid source manifest writes no
|
|
70
|
+
report, prints its reason code such as `AGENTIC_PROFILE_SOURCE_INVALID` to stderr, and exits with
|
|
71
|
+
`4`. `profile-replay` exits with `0` only when every replay rail is valid and with `4` otherwise.
|
|
72
|
+
`profile-v2` has no `NOT_QUALIFIED` status; it uses the same codes for the statuses it shares with
|
|
73
|
+
v1 and `4` for `OBSERVED_BENCHMARK_FAILURE` and `COMPARISON_SCOPE_MISMATCH`, and
|
|
74
|
+
`profile-v2-replay` follows `profile-replay`. Unexpected engine errors exit with `5`.
|
|
75
|
+
|
|
76
|
+
`provider` prints the evidence provider manifest and exits with `0`. `export` exits with `0` after
|
|
77
|
+
writing an [evidence export](evidence-export.md), whatever its detection result; a malformed request
|
|
78
|
+
or a replay-invalid source manifest writes no export, prints `EVIDENCE_EXPORT_REQUEST_INVALID` or
|
|
79
|
+
`EVIDENCE_EXPORT_SOURCE_INVALID` to stderr, and exits with `4`. `export-replay` exits with `0` only
|
|
80
|
+
when every replay rail is valid and with `4` otherwise.
|
|
57
81
|
|
|
58
82
|
Versioned JSON Schemas are published for the
|
|
59
83
|
[`verification request`](../schemas/verification-request.v1.json),
|
|
@@ -78,8 +102,12 @@ Versioned JSON Schemas are published for the
|
|
|
78
102
|
[`corpus trust policy`](../schemas/agentic-corpus-trust-policy.v1.json) and paired
|
|
79
103
|
[`corpus provenance`](../schemas/agentic-corpus-provenance.v1.json), six corpus allocation and
|
|
80
104
|
two-party [`commitment/reveal`](../schemas/agentic-corpus-allocation-commitment.v1.json) contracts,
|
|
81
|
-
plus six pre-declared [`H3 experiment`](../schemas/agentic-corpus-experiment-plan.v1.json) contracts
|
|
82
|
-
|
|
105
|
+
plus six pre-declared [`H3 experiment`](../schemas/agentic-corpus-experiment-plan.v1.json) contracts,
|
|
106
|
+
and the [`evidence provider manifest`](../schemas/evidence-provider-manifest.v1.json),
|
|
107
|
+
[`evidence export request`](../schemas/evidence-export-request.v1.json),
|
|
108
|
+
[`evidence export`](../schemas/evidence-export.v1.json), and
|
|
109
|
+
[`evidence export replay result`](../schemas/evidence-export-replay-result.v1.json).
|
|
110
|
+
The main CLI `schema` command prints the thirty-six facade schemas by name; the corpus evaluator consumes
|
|
83
111
|
the two trust/provenance schemas directly.
|
|
84
112
|
A complete runnable verification request
|
|
85
113
|
is available at
|
|
@@ -96,8 +124,8 @@ campaign wall-time proxy with an exact scoped warm-total-wall p95 cost basis. It
|
|
|
96
124
|
artifacts and commands remain supported.
|
|
97
125
|
|
|
98
126
|
The checked-in [`conformance v1 bundle`](conformance-v1.md) locks autonomous inputs, complete
|
|
99
|
-
expected outputs, negative replay witnesses,
|
|
100
|
-
|
|
127
|
+
expected outputs, negative replay witnesses, the v1 schema bytes, and selected public digests; an
|
|
128
|
+
additive lock covers the schemas published after v1. `pnpm check` validates this static oracle without regenerating it.
|
|
101
129
|
|
|
102
130
|
## TypeScript SDK
|
|
103
131
|
|
|
@@ -128,6 +156,14 @@ const profile = assertLedger.profile({
|
|
|
128
156
|
});
|
|
129
157
|
const profileIntegrity = assertLedger.replayProfile(profile);
|
|
130
158
|
|
|
159
|
+
const provider = assertLedger.providerManifest();
|
|
160
|
+
const evidenceExport = assertLedger.exportEvidence({
|
|
161
|
+
schemaVersion: "1.0.0",
|
|
162
|
+
manifest,
|
|
163
|
+
consumerRequest: null,
|
|
164
|
+
});
|
|
165
|
+
const exportIntegrity = assertLedger.replayEvidenceExport(evidenceExport);
|
|
166
|
+
|
|
131
167
|
const benchmarkRequest = JSON.parse(await readFile("assertledger.benchmark-request.json", "utf8"));
|
|
132
168
|
const benchmark = assertLedger.benchmark(benchmarkRequest);
|
|
133
169
|
const benchmarkIntegrity = assertLedger.replayBenchmark(benchmark);
|
|
@@ -160,6 +196,10 @@ surface, for consumers migrating from the prior name.
|
|
|
160
196
|
The SDK accepts plain JSON-compatible values and validates them against the same contracts as the
|
|
161
197
|
CLI. Unlike the CLI and MCP tool, `AssertLedger.verify()` has no separate authorization parameter: the
|
|
162
198
|
caller must set `isolation.acknowledgedUnsafeExecution` to `true` after applying its own policy.
|
|
199
|
+
`verify()` and `checkGitRegression()` accept only v1 inputs and return v1 manifests.
|
|
200
|
+
`verifyV2(request, { containerRuntime: { command } })` and `checkGitRegressionV2(options)` run
|
|
201
|
+
[container isolation](container-isolation.md) and return v2 manifests; a container request needs no
|
|
202
|
+
acknowledgement, and the runtime argv never comes from the request.
|
|
163
203
|
|
|
164
204
|
`AssertLedger.replay()` reports schema validity, both digest checks, and deterministic
|
|
165
205
|
decision-semantic validity. Its aggregate `valid` field is true only when all four checks pass. Replay
|
|
@@ -233,7 +273,7 @@ server name reported to clients is `assertledger`. Every tool is registered twic
|
|
|
233
273
|
`assertledger_*` name and a legacy `testforge_*` name bound to the same handler and the same tool
|
|
234
274
|
configuration. The schema lookup pair has no single fixed output schema because its selected JSON
|
|
235
275
|
Schema document varies; every other pair shares the same output-schema object. The default server
|
|
236
|
-
exposes
|
|
276
|
+
exposes thirty-four read-only tools:
|
|
237
277
|
|
|
238
278
|
| Preferred tool | Legacy alias | Purpose |
|
|
239
279
|
| --- | --- | --- |
|
|
@@ -247,6 +287,9 @@ exposes twenty-eight read-only tools:
|
|
|
247
287
|
| `assertledger_profile_v2` | `testforge_profile_v2` | Derive a benchmark-backed strength and warm-cost profile |
|
|
248
288
|
| `assertledger_profile_v2_replay` | `testforge_profile_v2_replay` | Replay a self-contained Profile v2 report |
|
|
249
289
|
| `assertledger_schema` | `testforge_schema` | Return any of the published JSON Schemas |
|
|
290
|
+
| `assertledger_provider` | `testforge_provider` | Describe the evidence provider, its announced capabilities, cost model, and limits |
|
|
291
|
+
| `assertledger_export` | `testforge_export` | Export replay-valid evidence for an external consumer |
|
|
292
|
+
| `assertledger_export_replay` | `testforge_export_replay` | Replay a self-contained evidence export |
|
|
250
293
|
| `assertledger_replay` | `testforge_replay` | Validate and replay a manifest's schema, digests, and decision semantics |
|
|
251
294
|
| `assertledger_benchmark_acquire_replay` | `testforge_benchmark_acquire_replay` | Replay acquisition source, artifact, context, digest, and status bindings |
|
|
252
295
|
| `assertledger_corpus_allocate` | `testforge_corpus_allocate` | Create a deterministic calibration/holdout allocation |
|
|
@@ -280,8 +323,9 @@ evidence manifest. The operator's capability is required for both tools.
|
|
|
280
323
|
## Continuous integration
|
|
281
324
|
|
|
282
325
|
Run `pnpm check` on every change. The included GitHub Actions workflow runs this gate on Node.js 22
|
|
283
|
-
and 24 on Ubuntu and
|
|
284
|
-
|
|
326
|
+
and 24 on Ubuntu, Windows and macOS. A separate matrix installs and exercises the packed artifact
|
|
327
|
+
on Ubuntu and Windows with Node.js 22.15.0 and 24. On Ubuntu, the gate also runs the real-daemon
|
|
328
|
+
[container isolation](container-isolation.md) suite. A CI job that executes campaigns must
|
|
285
329
|
also treat `trusted-local` as `UNSANDBOXED`: use an isolated runner without secrets or host
|
|
286
330
|
credentials, and pass `--allow-unsafe-execution` only from reviewed CI configuration.
|
|
287
331
|
|
|
@@ -328,7 +372,8 @@ deterministic core and protocols are framework-independent; `node:test` is the f
|
|
|
328
372
|
framework adapter. Other frameworks integrate through the structured-command protocol described in
|
|
329
373
|
[docs/adapter-protocol.md](adapter-protocol.md).
|
|
330
374
|
|
|
331
|
-
Planned work is not shipped behavior. Priorities include
|
|
375
|
+
Planned work is not shipped behavior. Priorities include VM isolation, container evidence in
|
|
376
|
+
derived reports, additional
|
|
332
377
|
framework reporters with runtime attribution, signed provenance, cross-runtime conformance
|
|
333
378
|
fixtures, and more built-in adapters. See [docs/roadmap.md](roadmap.md).
|
|
334
379
|
|
package/docs/roadmap.md
CHANGED
|
@@ -7,7 +7,9 @@ regression-test qualification workflow, acceptance criteria, and evidence bounda
|
|
|
7
7
|
## Implemented locally in v0.1
|
|
8
8
|
|
|
9
9
|
- deterministic repository analysis;
|
|
10
|
-
-
|
|
10
|
+
- forty versioned public JSON Schemas: thirty-four frozen by conformance v1, and four
|
|
11
|
+
evidence-export schemas plus the v2 verification request and evidence manifest locked by the
|
|
12
|
+
additive schema-extension lock;
|
|
11
13
|
- reference, target, and neutral overlay worlds;
|
|
12
14
|
- one campaign snapshot and disposable execution workspaces;
|
|
13
15
|
- built-in `node:test` runtime reporter and structured-command adapter;
|
|
@@ -28,11 +30,16 @@ regression-test qualification workflow, acceptance criteria, and evidence bounda
|
|
|
28
30
|
planned-process and planned-timeout equality, content-addressed suites, per-source
|
|
29
31
|
non-inferiority, and a shared frozen candidate universe;
|
|
30
32
|
- a static conformance-v1 oracle locking canonicalization, decisions, replay witnesses, Profile v1,
|
|
31
|
-
Benchmark v1, and all 34
|
|
33
|
+
Benchmark v1, and all 34 v1 schema bytes, plus an additive lock for later schemas;
|
|
32
34
|
- a pinned 24-case, three-source empirical corpus plan with signed admission and holdout rules,
|
|
33
35
|
including eight receipt-linked TestExplora cases admitted only for curated calibration;
|
|
34
36
|
- JSON CLI, TypeScript SDK, MCP v2 stdio server, and integration skill;
|
|
35
|
-
-
|
|
37
|
+
- an interoperable evidence export with a provider manifest, separate result, integrity,
|
|
38
|
+
authenticity, environment, confidence, control, and cost sections, and deterministic replay;
|
|
39
|
+
- `trusted-local`, explicitly recorded as `UNSANDBOXED`;
|
|
40
|
+
- a v2 [container isolation backend](container-isolation.md): a fresh digest-pinned Linux container
|
|
41
|
+
per execution, no network or host mounts, bounded resources, and backend facts bound to the
|
|
42
|
+
decision.
|
|
36
43
|
|
|
37
44
|
## Release 1.0 priority
|
|
38
45
|
|
|
@@ -55,7 +62,8 @@ scoped deterministic qualification workflow.
|
|
|
55
62
|
tranche is curated calibration evidence and cannot satisfy the holdout requirement.
|
|
56
63
|
2. Add faithful framework-specific phase adapters. The framework-neutral acquisition path is
|
|
57
64
|
shipped, but the built-in `node:test` reporter cannot attribute all four phases.
|
|
58
|
-
3.
|
|
65
|
+
3. Extend container isolation to evidence export, profiles, benchmarks and MCP, and add a VM
|
|
66
|
+
boundary for threats that a shared kernel does not contain.
|
|
59
67
|
4. Add framework reporters beyond `node:test` that derive discovery and attribution from runtime
|
|
60
68
|
events.
|
|
61
69
|
5. Publish cross-runtime conformance fixtures for canonicalization, decisions, and both digests.
|
|
@@ -24,5 +24,9 @@ if (manifestPath === undefined) {
|
|
|
24
24
|
},
|
|
25
25
|
});
|
|
26
26
|
process.stdout.write(`${JSON.stringify(report)}\n`);
|
|
27
|
-
|
|
27
|
+
// Same mapping as `assertledger profile`.
|
|
28
|
+
process.exitCode =
|
|
29
|
+
{ QUALIFIED: 0, NOT_QUALIFIED: 2, BUDGET_MISSED: 2, INSUFFICIENT_TIMING_EVIDENCE: 3 }[
|
|
30
|
+
report.status
|
|
31
|
+
] ?? 5;
|
|
28
32
|
}
|