@agentskit/harness 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/CHANGELOG.md +77 -0
  2. package/CODE_OF_CONDUCT.md +5 -0
  3. package/CONTRIBUTING.md +26 -0
  4. package/LICENSE +21 -0
  5. package/README.md +473 -0
  6. package/SECURITY.md +11 -0
  7. package/dist/cli.js +1308 -0
  8. package/dist/cli.js.map +1 -0
  9. package/dist/index.d.ts +968 -0
  10. package/dist/index.js +1828 -0
  11. package/dist/index.js.map +1 -0
  12. package/docs/ADR-0001-extensible-kernel.md +41 -0
  13. package/docs/ADR-0002-profiles-and-context.md +22 -0
  14. package/docs/ADR-0003-doc-bridge-context-binding.md +36 -0
  15. package/docs/ADR-0004-run-metrics.md +29 -0
  16. package/docs/ADR-0005-benchmark-manifest.md +27 -0
  17. package/docs/ADR-0006-agent-session-protocol.md +36 -0
  18. package/docs/ADR-0007-policy-gate.md +33 -0
  19. package/docs/ADR-0008-runtime-executor.md +33 -0
  20. package/docs/ADR-0009-process-runtime-boundary.md +34 -0
  21. package/docs/ADR-0010-docker-sandbox-runtime.md +32 -0
  22. package/docs/ADR-0011-runtime-attestation.md +30 -0
  23. package/docs/ADR-0012-controlled-baseline-observations.md +27 -0
  24. package/docs/ADR-0013-honest-benchmark-comparability.md +27 -0
  25. package/docs/ADR-0014-criterion-level-benchmark-evidence.md +24 -0
  26. package/docs/ADR-0015-directional-benchmark-outcomes.md +23 -0
  27. package/docs/ADR-0016-baseline-evidence-digests.md +21 -0
  28. package/docs/ADR-0017-event-log-integrity.md +25 -0
  29. package/docs/ADR-0018-verification-projection-attestation.md +23 -0
  30. package/docs/ADR-0019-human-decision-attestation.md +27 -0
  31. package/docs/ADR-0020-terminal-reconciliation.md +26 -0
  32. package/docs/ADR-0021-event-lock-recovery.md +25 -0
  33. package/docs/ADR-0022-signed-evidence-bundle.md +27 -0
  34. package/docs/ADR-0023-safe-action-recovery.md +30 -0
  35. package/docs/ADR-0024-controlled-completion-metrics.md +27 -0
  36. package/docs/ADR-0025-ci-dogfood.md +22 -0
  37. package/docs/ADR-0026-ci-evidence-artifact.md +22 -0
  38. package/docs/ADR-0027-portable-evidence.md +19 -0
  39. package/docs/ADR-0028-effective-metrics.md +20 -0
  40. package/docs/ADR-0029-honest-ci-preparation.md +20 -0
  41. package/docs/ADR-0030-agentskit-os-benchmark-bridge.md +20 -0
  42. package/docs/ADR-0031-real-provider-baseline.md +18 -0
  43. package/docs/ADR-0032-harness-equivalent-benchmark.md +25 -0
  44. package/docs/ADR-0033-portable-agent-gate.md +25 -0
  45. package/docs/ADR-0034-measurement-quality-gates.md +25 -0
  46. package/docs/ADR-0035-reproducible-benchmark-samples.md +20 -0
  47. package/docs/ADR-0036-comparable-baseline-samples.md +20 -0
  48. package/docs/ADR-0037-replicated-baseline-collection.md +27 -0
  49. package/docs/ADR-0038-end-to-end-benchmark-boundary.md +28 -0
  50. package/docs/ADR-0039-artifact-and-protocol-metrics.md +39 -0
  51. package/docs/ADR-0040-benchmark-corpus-surfaces.md +32 -0
  52. package/package.json +68 -0
@@ -0,0 +1,41 @@
1
+ # ADR-0001: Extensible kernel with append-only run events
2
+
3
+ - Status: accepted
4
+ - Date: 2026-08-29
5
+
6
+ ## Context
7
+
8
+ The harness must work with different coding agents, repositories, browsers,
9
+ databases, documentation systems, and tracking providers without moving its
10
+ completion guarantees into optional integrations. A mutable summary alone is
11
+ also insufficient for diagnosing retries and stale evidence.
12
+
13
+ ## Decision
14
+
15
+ Keep the contract, lifecycle transitions, evidence validation, source binding,
16
+ staleness rules, approval boundaries, and security checks in the kernel. Expose
17
+ an in-process, typed plugin registry for optional capabilities. Plugins declare
18
+ their API version and dependencies, contribute through named typed slots, and
19
+ are cleaned up deterministically.
20
+
21
+ Persist lifecycle facts as append-only NDJSON under each run. `run.json` remains
22
+ the compatible projection used by the current CLI; event records carry the run
23
+ ID, source revision, and contract hash so projections and diagnostics can be
24
+ checked against the same execution.
25
+
26
+ ## Constraints
27
+
28
+ - No remote plugin installation, hot reload, marketplace, or distributed event
29
+ store is part of this decision.
30
+ - A plugin cannot approve a run or bypass kernel evidence validation.
31
+ - Event records are appended by the harness and rejected when malformed or out
32
+ of order.
33
+ - Doc Bridge, Playbook profiles, browser adapters, and SCM integrations are
34
+ consumers of this seam, not kernel dependencies.
35
+
36
+ ## Consequences
37
+
38
+ The public API can grow by adding typed slots and event payloads while the
39
+ existing CLI and JSON projection remain stable. The current event log records
40
+ run creation and state transitions; check-level event types and remote stores
41
+ can be added when a measured diagnostic need justifies them.
@@ -0,0 +1,22 @@
1
+ # ADR-0002: Profiles and optional context providers
2
+
3
+ - Status: accepted
4
+ - Date: 2026-08-29
5
+
6
+ ## Decision
7
+
8
+ Verification contracts may define named profiles. A selected profile resolves
9
+ its parent chain deterministically, then overlays budgets, cleanup roots, and
10
+ existing checks by ID before validation and contract hashing. Missing profiles,
11
+ cycles, and unknown check IDs fail closed.
12
+
13
+ Context retrieval is an optional typed plugin slot. A provider returns a query,
14
+ references, content provenance, and a snapshot hash. Doc Bridge and Playbook
15
+ adapters can use the seam without becoming kernel dependencies.
16
+
17
+ ## Boundaries
18
+
19
+ Profiles cannot bypass required checks or human approval. Providers cannot
20
+ approve runs, alter evidence validation, or silently replace the frozen
21
+ contract. Remote provider discovery, credential management, and context cache
22
+ policy remain integration concerns.
@@ -0,0 +1,36 @@
1
+ # ADR-0003: Bind optional Doc Bridge context to a run
2
+
3
+ - Status: accepted
4
+ - Date: 2026-08-29
5
+
6
+ ## Decision
7
+
8
+ The harness provides a dependency-free `createDocBridgeContextProvider` adapter
9
+ that reads the local `.doc-bridge/index.json` contract. It returns at most
10
+ eight deterministic references, carries the index `contentHash` as source
11
+ provenance, and computes a stable snapshot hash.
12
+
13
+ `planRun` accepts resolved context snapshots and freezes them into `run.json`
14
+ with a `contextHash`. The lifecycle log records one `context.attached` event per
15
+ snapshot. Resolution timestamps remain audit metadata and are excluded from the
16
+ reproducibility hash.
17
+
18
+ Shell-based agents use the same wire boundary with `context resolve` and
19
+ `plan --context-file`. A context file contains one snapshot or an array of
20
+ snapshots, allowing another provider to participate without a dynamic runtime
21
+ loader.
22
+
23
+ The file loader verifies the snapshot hash before planning. This makes the
24
+ portable handoff tamper-evident while leaving provider-specific source
25
+ provenance and authorization at the provider boundary.
26
+
27
+ The kernel applies the same verification to snapshots supplied directly to
28
+ `planRun`, so typed callers cannot bypass the integrity boundary.
29
+
30
+ ## Boundaries
31
+
32
+ The adapter does not install or import Doc Bridge, access remote services, or
33
+ refresh context after planning. A caller owns query selection and must resolve
34
+ context before the contract is verified. Index schema validation and richer
35
+ semantic search remain responsibilities of Doc Bridge; malformed or missing
36
+ indexes fail the adapter closed.
@@ -0,0 +1,29 @@
1
+ # ADR-0004: Derive benchmark metrics from immutable run projections
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ The harness needs to measure improvement across repeated development runs without
10
+ introducing a database or a remote telemetry service. Each run already stores its
11
+ source, contract, checks, evidence, lifecycle state, approval, retry lineage,
12
+ and duration.
13
+
14
+ ## Decision
15
+
16
+ Expose `benchmarkRuns(stateDir)` and the `ak-harness benchmark` command. They
17
+ read the historical `run.json` projections, sort by run ID, and emit a versioned
18
+ JSON report with per-run facts and aggregate rates for checks, outcomes, evidence,
19
+ approvals, retries, stale runs, and duration.
20
+
21
+ ## Consequences
22
+
23
+ - Metrics are local, reproducible, portable, and usable without a service.
24
+ - Historical reports can be committed or exported by CI when the project wants a
25
+ trend record.
26
+ - The report measures harness execution and review friction; it does not claim
27
+ developer productivity or causality without a controlled task benchmark.
28
+ - A future remote metrics store can consume this versioned report without changing
29
+ the verification kernel.
@@ -0,0 +1,27 @@
1
+ # ADR-0005: Use explicit benchmark manifests and task bindings
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ Run IDs identify executions, not the task being measured. Comparing harness
10
+ performance requires a stable task corpus and an honest baseline. A missing
11
+ baseline must not be silently inferred from harness history.
12
+
13
+ ## Decision
14
+
15
+ Add a versioned benchmark manifest containing task IDs, titles, acceptance
16
+ criteria, and optional baseline observations. Verification contracts may bind a
17
+ run to a suite and task ID. The benchmark report compares only bound harness runs
18
+ with explicit, non-`not-run` baseline observations and reports the rest as
19
+ non-comparable.
20
+
21
+ ## Consequences
22
+
23
+ - Multiple runs can be grouped as attempts of one task.
24
+ - Baseline provenance is visible and reviewable.
25
+ - The first corpus can be committed without fabricating baseline data.
26
+ - Controlled task execution and baseline collection remain a later phase; this
27
+ schema provides the stable seam now.
@@ -0,0 +1,36 @@
1
+ # ADR-0006: Record agent sessions through a protocol-first seam
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ The harness must make agent work observable without coupling the kernel to a
10
+ specific model, runtime, tool host, or provider. It also must not turn prompts,
11
+ tool arguments, results, or chain-of-thought into persisted application data.
12
+
13
+ ## Decision
14
+
15
+ Expose a small `AgentAdapter` descriptor and `createSessionRecorder` API. The
16
+ recorder accepts hashes and metadata, writes correlated session, turn, and tool
17
+ events to the existing append-only run log, and rejects invalid local ordering.
18
+ It can start only from `IMPLEMENTING`; with `resume: true`, it reconstructs
19
+ unfinished actions from the event log and emits a `session.resumed` event. A
20
+ persisted `tool.execution.started` event marks an unfinished action as
21
+ ambiguous; resumption requires an explicit human `retry` or `abandon` decision
22
+ through `recoverTool` before the runtime can be entered again. It does not
23
+ execute tools or make policy decisions.
24
+
25
+ ## Consequences
26
+
27
+ - Any coding agent can implement the descriptor without importing an agent SDK.
28
+ - Session evidence is reproducible and correlated by run, session, turn, and
29
+ action IDs.
30
+ - Raw content stays outside the harness event log.
31
+ - An interrupted process can resume pending approvals and actions without
32
+ replaying completed tools or bypassing the policy gate.
33
+ - An interrupted action that may have reached an external runtime is fail-closed
34
+ until a human explicitly chooses retry or abandonment.
35
+ - A later policy gate or runtime adapter can consume this protocol without
36
+ changing the verification kernel.
@@ -0,0 +1,33 @@
1
+ # ADR-0007: Gate tool actions with ordered deny-by-default policy
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ The session protocol observes tool calls, but observation alone cannot prevent
10
+ an unsafe or out-of-scope action from reaching a runtime. The kernel needs a
11
+ portable control point that works before provider-specific tool execution.
12
+
13
+ ## Decision
14
+
15
+ Require every `SessionRecorder` to receive a typed `PolicyGate`. The built-in
16
+ `createPolicyGate` evaluates ordered exact-tool rules, uses the first match,
17
+ and blocks by default when no rule allows the tool. Rules may allow, block, or
18
+ require human approval. Each decision is recorded; approval-required attempts
19
+ cannot enter the pending execution lifecycle until a human approves them.
20
+
21
+ The gate receives hashes and identifiers only. It does not execute tools,
22
+ inspect raw arguments, or approve the overall task.
23
+
24
+ ## Consequences
25
+
26
+ - A tool cannot be requested through the public session API without a policy
27
+ decision.
28
+ - Sensitive tools can pause for an explicit human decision before execution.
29
+ - Policy behavior is deterministic, inspectable, and provider-independent.
30
+ - Rejected or unresolved approvals remain fail-closed and generate an audit
31
+ trail without storing raw prompts, arguments, or results.
32
+ - Existing integrations must provide an explicit allowlist or their tools are
33
+ blocked by default.
@@ -0,0 +1,33 @@
1
+ # ADR-0008: Execute allowed actions through a bounded runtime seam
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ The policy gate can decide whether a tool action is allowed, but the session
10
+ protocol still needs a portable way to execute that action and close it with
11
+ reproducible evidence. Tool providers differ widely, so the kernel must not
12
+ depend on a shell, MCP host, model SDK, or container runtime.
13
+
14
+ ## Decision
15
+
16
+ Require every `SessionRecorder` to receive a `ToolRuntime`. The built-in
17
+ runtime dispatches to explicitly registered handlers, propagates an
18
+ `AbortSignal`, applies a timeout, hashes successful results, and returns
19
+ structured failures for missing tools, exceptions, and timeouts. The recorder
20
+ invokes it only after a policy allow decision and converts the result into the
21
+ existing `tool.completed` or `tool.failed` event.
22
+
23
+ The built-in runtime is in-process. It is an execution boundary, not a hard
24
+ security sandbox; isolated process or container runtimes remain provider
25
+ adapters for a later phase.
26
+
27
+ ## Consequences
28
+
29
+ - Runtime handlers are portable and independently replaceable.
30
+ - Blocked actions cannot reach a handler through the public session API.
31
+ - Raw arguments and results remain in memory only; the event log stores hashes.
32
+ - Timeouts bound the recorder's wait, but a handler that ignores abort cannot be
33
+ force-killed by an in-process runtime.
@@ -0,0 +1,34 @@
1
+ # ADR-0009: Add a bounded shell-free process runtime
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ An in-process handler is useful for adapters, but it cannot be killed if it
10
+ ignores abort and is not an isolation boundary for untrusted code. The harness
11
+ needs a portable process seam for commands that require stronger lifecycle
12
+ control without pretending that every host has a container runtime.
13
+
14
+ ## Decision
15
+
16
+ Expose `createProcessToolRuntime`. Each tool has a fixed executable, argument
17
+ list, optional working directory, and explicit environment. The runtime uses
18
+ `spawn` with `shell: false`, sends one JSON request over stdin, kills timed-out
19
+ or oversized children, and returns only hashed stdout or structured failure
20
+ metadata.
21
+
22
+ Recovery closes the failed action and requires a fresh policy-approved action;
23
+ the harness does not automatically replay potentially side-effecting tools.
24
+
25
+ This is a bounded child-process boundary, not a complete security sandbox.
26
+ Container, OS policy, filesystem, network, and credential isolation remain
27
+ provider-specific responsibilities.
28
+
29
+ ## Consequences
30
+
31
+ - Shell interpolation is removed from the portable process path.
32
+ - Timeouts and output limits prevent common runaway-process failures.
33
+ - Explicit environments reduce accidental secret inheritance.
34
+ - Hard isolation and resource quotas require a later sandbox provider.
@@ -0,0 +1,32 @@
1
+ # ADR-0010: Optional Docker sandbox runtime
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ The process runtime bounds execution but does not isolate an agent tool from
10
+ the host. Coding tasks sometimes need a stronger boundary while the harness
11
+ must remain portable and must not require Docker for its core API.
12
+
13
+ ## Decision
14
+
15
+ Expose `createDockerToolRuntime` as an optional provider built on the existing
16
+ process runtime. Each tool declares a fixed image and argv; the provider never
17
+ constructs or invokes a shell. By default it uses a cached image only, disables
18
+ container networking, makes the root filesystem read-only, drops Linux
19
+ capabilities, enables `no-new-privileges`, runs as an unprivileged UID, and
20
+ applies PIDs, memory, CPU, timeout, and output limits. Host mounts are absent
21
+ unless explicitly declared and default to read-only.
22
+
23
+ ## Consequences
24
+
25
+ - Consumers without Docker can keep using the portable process or in-process
26
+ providers; Docker failures are structured runtime failures.
27
+ - A Docker image is a trust boundary input and should be pinned by consumers
28
+ when reproducibility matters. `pull: 'never'` prevents an implicit network
29
+ fetch by default.
30
+ - The provider is not a VM or a guarantee against a compromised Docker daemon.
31
+ Stronger VM, rootless, Windows, and remote sandbox providers can implement
32
+ the same `ToolRuntime` seam later without changing the kernel.
@@ -0,0 +1,30 @@
1
+ # ADR-0011: Runtime attestation in tool evidence
2
+
3
+ ## Status
4
+
5
+ Accepted
6
+
7
+ ## Context
8
+
9
+ A sandbox result hash proves what the tool returned, but not which image or
10
+ limits produced it. Enterprise review needs to correlate terminal tool events
11
+ with the exact runtime profile used for that action.
12
+
13
+ ## Decision
14
+
15
+ Runtime providers may attach typed runtime evidence to completed or failed tool
16
+ results. The Docker provider resolves the local image ID before execution,
17
+ hashes the effective profile plus that ID, and carries the attestation into the
18
+ terminal event. Missing images or an unavailable daemon fail before execution.
19
+
20
+ The evidence is descriptive and bound to the harness event's run, source
21
+ revision, and configuration hash; it is not a claim that the Docker daemon or
22
+ host kernel is trustworthy.
23
+
24
+ ## Consequences
25
+
26
+ - Reviewers can identify the provider, image digest, security profile, and
27
+ resource limits for each action without storing raw command output.
28
+ - Custom runtimes remain compatible because runtime evidence is optional.
29
+ - Future VM or remote providers can publish their own typed evidence without
30
+ changing the session protocol.
@@ -0,0 +1,27 @@
1
+ # ADR-0012: Record controlled baseline observations through the public interface
2
+
3
+ ## Context
4
+
5
+ The harness already aggregates run history, but a benchmark comparison is only
6
+ meaningful when the baseline is real, attributable, and tied to the same task
7
+ identity. Manual JSON editing is easy to get wrong and makes the measurement
8
+ process hard to reproduce.
9
+
10
+ ## Decision
11
+
12
+ Expose `recordBenchmarkObservation` and `ak-harness benchmark baseline` as the
13
+ single supported write path for one baseline observation per task. The API
14
+ validates the manifest and observation, rejects unknown tasks and duplicates,
15
+ and writes through a temporary file followed by an atomic rename. It also
16
+ checks that the source manifest did not change during the operation.
17
+
18
+ `not-run` remains an explicit non-comparable status. The benchmark report must
19
+ not infer a baseline from harness history or report improvement without both a
20
+ valid baseline and a bound harness run.
21
+
22
+ ## Consequences
23
+
24
+ - Baselines have a reproducible CLI/API entry point and visible provenance.
25
+ - Concurrent or stale writes fail instead of silently overwriting newer data.
26
+ - A controlled task run is still required; this API records evidence but does
27
+ not manufacture experimental results.
@@ -0,0 +1,27 @@
1
+ # ADR-0013: Require completed evidence for benchmark comparability
2
+
3
+ ## Context
4
+
5
+ An explicit baseline is necessary but not sufficient for a meaningful
6
+ comparison. A blocked or still-running harness attempt must not be presented as
7
+ an achieved result, and aggregate pass rates alone do not explain why a task is
8
+ or is not comparable.
9
+
10
+ ## Decision
11
+
12
+ `benchmarkRuns` reports a task as comparable only when it has a baseline whose
13
+ status is not `not-run` and the latest bound harness run is `COMPLETE`. Every
14
+ other case has a typed reason: missing baseline, not-run baseline, harness not
15
+ run, or harness not complete.
16
+
17
+ The comparison also exposes the latest harness check, outcome, evidence,
18
+ duration, attempt, and human-review metrics. Human-review duration is derived
19
+ only when approval is recorded. The report remains descriptive; it does not
20
+ claim causality or productivity improvement.
21
+
22
+ ## Consequences
23
+
24
+ - A blocked or partial attempt cannot inflate the comparison dataset.
25
+ - Reviewers can distinguish missing evidence from failed execution.
26
+ - Future statistical analysis has stable inputs without changing the run
27
+ protocol.
@@ -0,0 +1,24 @@
1
+ # ADR-0014: Criterion-level benchmark evidence
2
+
3
+ ## Context
4
+
5
+ Scalar baseline numbers do not prove that a benchmark task exercised its
6
+ acceptance criteria. Treating those numbers as comparable can make an
7
+ incomplete delivery look like an improvement.
8
+
9
+ ## Decision
10
+
11
+ Baseline observations may include an `evidence` array. Each entry names one
12
+ acceptance criterion exactly, records its status, and points to a source. The
13
+ manifest validator rejects unknown or duplicate criteria and invalid evidence
14
+ metadata. A comparison is `comparable` only when every criterion is covered,
15
+ the baseline is not `not-run`, and the latest bound harness run is `COMPLETE`.
16
+ The report exposes `baselineEvidenceCoverageRate` and returns
17
+ `baseline-evidence-missing` when coverage is incomplete.
18
+
19
+ ## Consequences
20
+
21
+ Baseline collection has a small additional input requirement, but benchmark
22
+ reports can distinguish missing proof from a real completed comparison. The
23
+ evidence source remains a reference; this phase does not claim causality or
24
+ replace criterion-level harness evidence for the current run.
@@ -0,0 +1,23 @@
1
+ # ADR-0015: Directional benchmark outcomes
2
+
3
+ ## Context
4
+
5
+ The harness already reports raw deltas, but a negative duration delta is easy
6
+ to misread and non-comparable tasks can be mistaken for missing data. Operators
7
+ need a small, explicit interpretation layer for the measured resources.
8
+
9
+ ## Decision
10
+
11
+ For comparable tasks, report a rate for duration, attempts, and human review:
12
+ `(baseline - harness) / baseline`. Positive means the harness used less of the
13
+ resource and is labelled `improved`; negative is `regressed`; zero is
14
+ `unchanged`. A missing value, zero baseline, or non-comparable task is labelled
15
+ `unavailable`. These are per-resource signals, not a composite productivity
16
+ score or a causal claim.
17
+
18
+ ## Consequences
19
+
20
+ Benchmark consumers can read the direction without reimplementing arithmetic,
21
+ while trade-offs remain visible instead of being hidden in one score. A real
22
+ baseline with complete criterion evidence is still required before any outcome
23
+ is reported as comparable.
@@ -0,0 +1,21 @@
1
+ # ADR-0016: Baseline evidence digests
2
+
3
+ ## Context
4
+
5
+ Criterion coverage and directional metrics can still be detached from the
6
+ exact JSON input used during baseline collection. Without a digest, a later
7
+ review cannot distinguish the recorded evidence file from a changed copy.
8
+
9
+ ## Decision
10
+
11
+ When `ak-harness benchmark baseline` reads `--evidence-file`, it computes and
12
+ stores the file's lowercase SHA-256 digest in `evidenceDigest`. Manifest
13
+ validation accepts only a 64-character lowercase SHA-256 value. The digest is
14
+ of the evidence file content, not of the external artifact named by each
15
+ criterion source.
16
+
17
+ ## Consequences
18
+
19
+ The baseline record is tamper-evident with respect to its input file and can be
20
+ replayed from the same bytes. External systems remain an explicit integration
21
+ boundary; their content verification is not claimed by the local harness.
@@ -0,0 +1,25 @@
1
+ # ADR-0017: Hash chained lifecycle event logs
2
+
3
+ ## Context
4
+
5
+ The lifecycle log already preserves order and binds events to a run revision
6
+ and contract hash, but a later edit to an existing record would not be
7
+ detected. Enterprise review needs a deterministic integrity signal without
8
+ persisting prompts, tool arguments, or adding a remote service.
9
+
10
+ ## Decision
11
+
12
+ New events include `previousHash` and `eventHash`. The hash covers the exact
13
+ serialized event body, including the preceding hash, so changing or reordering
14
+ records makes the chain invalid. `FileEventStore.read` validates the chain and
15
+ `FileEventStore.verify` exposes a typed `verified` or `legacy` result. Existing
16
+ logs without integrity fields remain readable and are never reported as
17
+ verified; appending to them preserves their legacy status.
18
+
19
+ ## Consequences
20
+
21
+ The log is tamper-evident for new records and can be checked locally or through
22
+ `ak-harness events verify`. This is not a signature or external notarization:
23
+ an attacker able to rewrite both the log and its consumer can recompute the
24
+ chain. A future signed export can add that trust boundary without changing the
25
+ event lifecycle API.
@@ -0,0 +1,23 @@
1
+ # ADR-0018: Attest verification projections before approval
2
+
3
+ ## Context
4
+
5
+ `run.json` is a convenient projection of checks and outcomes, but it is not
6
+ itself an immutable store. Approval must not trust a projection that was
7
+ changed after verification while the source and contract remain unchanged.
8
+
9
+ ## Decision
10
+
11
+ After every verification pass, the harness hashes the checks, outcomes, and
12
+ metrics projection, persists that `verificationDigest` in `run.json`, and
13
+ appends a `verification.completed` event carrying the same digest. Human
14
+ approval recomputes the digest, verifies the audit log, and requires the latest
15
+ completion event to match it.
16
+
17
+ ## Consequences
18
+
19
+ Manual edits to the verification projection are rejected before approval while
20
+ the existing source and contract freshness checks remain in force. The digest
21
+ protects the local projection through the audit log; it is not a digital
22
+ signature or external notarization. Dogfood uses the public API, and prior
23
+ Docker sandbox evidence remains covered by the earlier runtime phase.
@@ -0,0 +1,27 @@
1
+ # ADR-0019: Record human decisions as attested lifecycle events
2
+
3
+ ## Context
4
+
5
+ The verification result is now attested before approval, but the approval and
6
+ tracking authorization themselves only existed in the mutable `run.json`
7
+ projection. An audit consumer could see that a run reached `COMPLETE` without
8
+ an immutable record of which human decision accepted the exact evidence.
9
+
10
+ ## Decision
11
+
12
+ Record `approval.recorded` and `authorization.recorded` events in the existing
13
+ hash-chained lifecycle log. Each event contains the decision, resulting state,
14
+ verification digest, source revision, contract hash, and human actor; tracking
15
+ authorization also contains its declared target. Rejections are recorded too,
16
+ so every human terminal decision is auditable.
17
+
18
+ The existing freshness and verification-attestation checks run before either
19
+ decision is accepted. The `run.json` fields remain a convenient projection,
20
+ but the event is the audit record.
21
+
22
+ ## Consequences
23
+
24
+ Consumers can independently prove which verified result was approved and which
25
+ external tracking target was authorized. Legacy logs remain readable and are
26
+ reported as legacy until they contain the new hashed event protocol. This is an
27
+ audit attestation, not a digital signature or external notarization.
@@ -0,0 +1,26 @@
1
+ # ADR-0020: Reconcile terminal projections before reporting completion
2
+
3
+ ## Context
4
+
5
+ The event log and human decision attestations protect the audit trail, but
6
+ `run.json` is still a mutable projection. Reading it directly after approval
7
+ could report `COMPLETE` even if its digest, decision projection, or decision
8
+ event had been removed or changed.
9
+
10
+ ## Decision
11
+
12
+ Expose `reconcileRun` in the public API and `ak-harness audit` in the CLI. The
13
+ reconciliation verifies the event hash chain, event-to-run binding, current
14
+ verification digest, and the required approval/authorization event and
15
+ projection for terminal states. `ak-harness status` uses this same gate.
16
+
17
+ Event appends use an atomic per-run lock and fail closed when another writer is
18
+ active, preventing concurrent lifecycle operations from duplicating a sequence
19
+ number or corrupting the append-only log.
20
+
21
+ ## Consequences
22
+
23
+ Post-approval projection tampering and removal of a decision event fail closed
24
+ with a harness error. Historical non-terminal projections remain inspectable;
25
+ terminal runs from before the attestation protocol must be reverified rather
26
+ than being treated as enterprise-grade evidence.
@@ -0,0 +1,25 @@
1
+ # ADR-0021: Explicit recovery for orphaned event-log locks
2
+
3
+ ## Context
4
+
5
+ The per-run event-log lock prevents concurrent append and read operations from
6
+ duplicating sequence numbers or observing a partial write. A process can still
7
+ terminate after acquiring the lock, leaving a file that must not be removed
8
+ automatically because another writer may still own it.
9
+
10
+ ## Decision
11
+
12
+ Write the owning process ID and creation timestamp into every lock. Expose lock
13
+ inspection and an explicit `human`-authorized recovery operation. Recovery is
14
+ allowed only when the lock is older than the requested threshold and its owner
15
+ process is no longer alive. Malformed, young, or live locks remain fail-closed.
16
+
17
+ The CLI exposes this maintenance operation as `events unlock`; it never runs as
18
+ part of normal status, verification, or append flows.
19
+
20
+ ## Consequences
21
+
22
+ An interrupted local run can be recovered without manually deleting an audit
23
+ artifact, while active or ambiguous locks cannot be silently bypassed. A lock
24
+ owner PID is local-process evidence; distributed writers still require a shared
25
+ coordination primitive outside this file store.
@@ -0,0 +1,27 @@
1
+ # ADR-0022: Signed portable evidence bundles
2
+
3
+ ## Context
4
+
5
+ Hash chaining protects the local event log from accidental or casual edits,
6
+ but a reviewer or CI system needs a portable artifact whose contents and
7
+ provenance can be checked outside the originating workspace.
8
+
9
+ ## Decision
10
+
11
+ `exportEvidenceBundle` packages a reconciled `COMPLETE` run, its projection,
12
+ event log, and referenced check outputs. Each file is hashed with SHA-256 and
13
+ the canonical bundle payload is signed with an Ed25519 private key. The key ID
14
+ and signature travel with the bundle; `verifyEvidenceBundle` independently
15
+ validates every file hash, the payload hash, and the signature. Callers can
16
+ additionally provide an explicit trust store to require an active key with the
17
+ expected identity instead of trusting the embedded public key.
18
+
19
+ The bundle contains check outputs as captured, so operators must treat it as a
20
+ potentially sensitive artifact and apply their existing retention controls.
21
+
22
+ ## Consequences
23
+
24
+ Evidence can cross machine and CI boundaries without adding a remote service.
25
+ Key custody remains an operator responsibility; this is a signed artifact, not
26
+ a replacement for an enterprise KMS, certificate authority, or notarization
27
+ service. Trust-store entries support explicit rotation and revocation.