@agentskit/harness 0.1.0 → 0.3.0

package/docs/ADR-0025-portable-orchestration-controls.md

@@ -0,0 +1,27 @@
+# ADR-0025: Portable orchestration controls
+
+## Status
+
+Accepted
+
+## Context
+
+The SDLC reference project contained useful dispatch, machine, recovery, and
+retro mechanisms, but also hard-coded repositories, people, branches, provider
+CLIs, and dashboard infrastructure. The Harness must remain reusable and
+provider-neutral.
+
+## Decision
+
+Add only deterministic kernel seams: issue/worktree claims, an idempotent
+dispatch ledger, failure classification with bounded recovery, file-scoped
+preflight planning, portable block/status/learning records, model bindings,
+machine thresholds, and provider-neutral Orca/tracking adapters. External
+adapters own network effects and credentials. Human decisions remain required
+for material ambiguity, recovery, and learning promotion.
+
+## Consequences
+
+The same controls can be used with Orca, another orchestrator, Linear, GitHub,
+or a local process. A dashboard, watcher, provider SDK, or repository-specific
+shell workflow can be added later without weakening the kernel gates.

package/docs/ORGANIZATION.md

@@ -0,0 +1,37 @@
+# Repository organization
+
+This is a TypeScript library and CLI, not an Angular application. We adopt the
+Angular team's Conventional Commits standard and a capability-first layout so
+the repository stays navigable without speculative layers.
+
+```text
+.
+├── src/
+│   ├── index.ts          # supported package API
+│   ├── cli.ts            # ak-harness / ak-verify commands
+│   ├── adapters/         # optional integrations
+│   └── <capability>.ts   # contract, evidence, delivery, runtime, metrics...
+├── test/                 # deterministic tests and fixtures
+├── scripts/              # real CLI, packaging, and repository checks
+├── docs/                 # ADRs and this organization contract
+├── .codex/               # verification contract and local run state
+└── .github/              # CI and release workflows
+```
+
+## Boundaries
+
+- Import internal modules with explicit relative paths; only `src/index.ts`
+  is supported for consumers.
+- Add a directory when a capability has multiple cohesive modules. Do not add
+  a layer for one file.
+- Adapters may depend on the kernel; the kernel must not depend on adapters or
+  external providers.
+- Tests may use internal modules when exercising a boundary, but consumer
+  checks must import the package entry point.
+- Generated output belongs in `dist/` and is ignored by Git; npm generates it
+  during the release gate.
+
+## Naming
+
+Use kebab-case filenames, PascalCase exported types/classes, camelCase
+functions, and descriptive test names that state behavior and condition.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agentskit/harness",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "Portable, evidence-backed development harness for coding agents.",
   "type": "module",
   "bin": { "ak-harness": "dist/cli.js", "ak-verify": "dist/cli.js" },
@@ -12,38 +12,44 @@
       "import": "./dist/index.js"
     }
   },
-  "files": ["dist", "README.md", "CHANGELOG.md", "CONTRIBUTING.md", "CODE_OF_CONDUCT.md", "SECURITY.md", "LICENSE", "docs"],
+  "files": ["dist", "README.md", "CHANGELOG.md", "CONTRIBUTING.md", "CODE_OF_CONDUCT.md", "SECURITY.md", "MANIFESTO.md", "LICENSE", "docs"],
   "engines": { "node": ">=22" },
   "scripts": {
     "build": "tsup && node -e \"console.log(JSON.stringify({status:'passed',criteria:['package']}))\"",
     "typecheck": "tsc -p tsconfig.json --noEmit",
-    "test": "pnpm typecheck && vitest run --config ../../vitest.config.ts test && node -e \"console.log(JSON.stringify({status:'passed',criteria:['package']}))\"",
-    "test:contract": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/contract.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['protocol']}))\"",
-    "test:extensibility": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/extensibility.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['extensibility']}))\"",
-    "test:event-log": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/extensibility.test.ts test/harness.test.ts -t \"ordered append-only lifecycle log|complete typed task through human approval|authorization only when tracking|human rejection\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['event-log','verification-attestation','approval-attestation']}))\"",
-    "test:reconciliation": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/harness.test.ts -t \"reconciles terminal decisions\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['terminal-reconciliation']}))\"",
-    "test:lock-recovery": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/lock.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['lock-recovery']}))\"",
-    "test:signed-bundle": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/bundle.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['signed-evidence']}))\"",
-    "test:key-trust": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/bundle.test.ts -t \"trust store|revoked|rotation\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['key-trust']}))\"",
-    "test:human-tool-approval": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/agent.test.ts -t \"human approval|approval\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['human-tool-approval']}))\"",
-    "test:session-recovery": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/agent.test.ts -t \"resume|recovery\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['session-recovery']}))\"",
-    "test:safe-action-recovery": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/agent.test.ts -t \"possibly started action\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['safe-action-recovery']}))\"",
-    "test:profiles": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/profiles.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['profiles']}))\"",
-    "test:context": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/context.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context']}))\"",
-    "test:doc-bridge": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/doc-bridge.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['doc-bridge-adapter']}))\"",
-    "test:context-binding": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/context.test.ts -t \"binds context\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context-binding']}))\"",
-    "test:context-cli": "pnpm typecheck && node ../../scripts/verify-harness-context-cli.mjs && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context-cli']}))\"",
-    "test:context-integrity": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/context.test.ts -t \"rejects\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context-integrity']}))\"",
-    "test:metrics": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/metrics.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['metrics']}))\"",
-    "test:ci-approval": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/harness.test.ts -t \"CI prepare|CI attempts|CI preparation across\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['ci-approval']}))\"",
-    "test:benchmark-record": "pnpm typecheck && node ../../scripts/verify-harness-benchmark-record.mjs",
-    "test:benchmark-bridge": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/coding-benchmark.test.ts test/metrics.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['benchmark-integrity','benchmark-bridge']}))\"",
-    "test:agent-protocol": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/agent.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['agent-protocol']}))\"",
-    "test:policy-gate": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/policy.test.ts test/agent.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['policy-gate','agent-protocol']}))\"",
-    "test:runtime": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/runtime.test.ts test/agent.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['runtime','agent-protocol']}))\"",
-    "test:process-runtime": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/runtime.test.ts test/agent.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['process-runtime','runtime']}))\"",
-    "test:docker-runtime": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/runtime.test.ts -t \"Docker sandbox\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['docker-runtime']}))\"",
-    "test:runtime-attestation": "pnpm typecheck && vitest run --config ../../vitest.config.ts test/runtime.test.ts test/agent.test.ts -t \"attestation|Docker sandbox\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['runtime-attestation']}))\"",
+    "test": "pnpm typecheck && vitest run --config vitest.config.ts test && node -e \"console.log(JSON.stringify({status:'passed',criteria:['package']}))\"",
+    "test:contract": "pnpm typecheck && vitest run --config vitest.config.ts test/contract.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['protocol']}))\"",
+    "test:extensibility": "pnpm typecheck && vitest run --config vitest.config.ts test/extensibility.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['extensibility']}))\"",
+    "test:event-log": "pnpm typecheck && vitest run --config vitest.config.ts test/extensibility.test.ts test/harness.test.ts -t \"ordered append-only lifecycle log|complete typed task through human approval|authorization only when tracking|human rejection\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['event-log','verification-attestation','approval-attestation']}))\"",
+    "test:reconciliation": "pnpm typecheck && vitest run --config vitest.config.ts test/harness.test.ts -t \"reconciles terminal decisions\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['terminal-reconciliation']}))\"",
+    "test:lock-recovery": "pnpm typecheck && vitest run --config vitest.config.ts test/lock.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['lock-recovery']}))\"",
+    "test:signed-bundle": "pnpm typecheck && vitest run --config vitest.config.ts test/bundle.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['signed-evidence']}))\"",
+    "test:key-trust": "pnpm typecheck && vitest run --config vitest.config.ts test/bundle.test.ts -t \"trust store|revoked|rotation\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['key-trust']}))\"",
+    "test:human-tool-approval": "pnpm typecheck && vitest run --config vitest.config.ts test/agent.test.ts -t \"human approval|approval\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['human-tool-approval']}))\"",
+    "test:session-recovery": "pnpm typecheck && vitest run --config vitest.config.ts test/agent.test.ts -t \"resume|recovery\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['session-recovery']}))\"",
+    "test:safe-action-recovery": "pnpm typecheck && vitest run --config vitest.config.ts test/agent.test.ts -t \"possibly started action\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['safe-action-recovery']}))\"",
+    "test:profiles": "pnpm typecheck && vitest run --config vitest.config.ts test/profiles.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['profiles']}))\"",
+    "test:context": "pnpm typecheck && vitest run --config vitest.config.ts test/context.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context']}))\"",
+    "test:doc-bridge": "pnpm typecheck && vitest run --config vitest.config.ts test/doc-bridge.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['doc-bridge-adapter']}))\"",
+    "test:context-binding": "pnpm typecheck && vitest run --config vitest.config.ts test/context.test.ts -t \"binds context\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context-binding']}))\"",
+    "test:context-cli": "pnpm typecheck && node scripts/verify-harness-context-cli.mjs && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context-cli']}))\"",
+    "test:context-integrity": "pnpm typecheck && vitest run --config vitest.config.ts test/context.test.ts -t \"rejects\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['context-integrity']}))\"",
+    "test:discovery": "pnpm typecheck && vitest run --config vitest.config.ts test/discovery.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['discovery']}))\"",
+    "test:wip": "pnpm typecheck && vitest run --config vitest.config.ts test/wip.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['wip']}))\"",
+    "test:experiment": "pnpm typecheck && vitest run --config vitest.config.ts test/experiment.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['runtime-experiment']}))\"",
+    "test:delivery": "pnpm typecheck && vitest run --config vitest.config.ts test/delivery.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['delivery-gates']}))\"",
+    "test:pilot": "pnpm typecheck && vitest run --config vitest.config.ts test/pilot.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['pilot']}))\"",
+    "test:cycle": "pnpm typecheck && vitest run --config vitest.config.ts test/cycle.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['cycle']}))\"",
+    "test:optimization": "pnpm typecheck && vitest run --config vitest.config.ts test/optimization.test.ts test/issues-010-014.eval.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['optimization-contracts','eval','cache','parallelism']}))\"",
+    "test:source": "pnpm typecheck && vitest run --config vitest.config.ts test/source.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['source-current']}))\"",
+    "test:metrics": "pnpm typecheck && vitest run --config vitest.config.ts test/metrics.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['metrics']}))\"",
+    "test:benchmark-record": "pnpm typecheck && node scripts/verify-harness-benchmark-record.mjs",
+    "test:agent-protocol": "pnpm typecheck && vitest run --config vitest.config.ts test/agent.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['agent-protocol']}))\"",
+    "test:policy-gate": "pnpm typecheck && vitest run --config vitest.config.ts test/policy.test.ts test/agent.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['policy-gate','agent-protocol']}))\"",
+    "test:runtime": "pnpm typecheck && vitest run --config vitest.config.ts test/runtime.test.ts test/agent.test.ts -t \"registered tool|missing and timed-out|invalid runtime\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['runtime']}))\"",
+    "test:process-runtime": "pnpm typecheck && vitest run --config vitest.config.ts test/runtime.test.ts -t \"child process|configured process runtime|timed-out and oversized|non-zero child\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['process-runtime']}))\"",
+    "test:docker-runtime": "pnpm typecheck && HARNESS_REQUIRE_DOCKER=1 vitest run --config vitest.config.ts test/runtime.test.ts -t \"Docker sandbox\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['docker-runtime']}))\"",
+    "test:runtime-attestation": "pnpm typecheck && vitest run --config vitest.config.ts test/runtime.test.ts test/agent.test.ts -t \"attestation|Docker sandbox\" && node -e \"console.log(JSON.stringify({status:'passed',criteria:['runtime-attestation']}))\"",
     "prepublishOnly": "pnpm typecheck && pnpm build && pnpm test"
   },
   "dependencies": {
@@ -56,13 +62,14 @@
     "vitest": "^4.1.10"
   },
   "license": "MIT",
+  "packageManager": "pnpm@10.34.4",
   "repository": {
     "type": "git",
-    "url": "git+https://github.com/AgentsKit-io/agents-playbook.git",
-    "directory": "packages/harness"
+    "url": "git+https://github.com/AgentsKit-io/harness.git"
   },
-  "bugs": { "url": "https://github.com/AgentsKit-io/agents-playbook/issues" },
-  "homepage": "https://github.com/AgentsKit-io/agents-playbook/tree/main/packages/harness",
+  "bugs": { "url": "https://github.com/AgentsKit-io/harness/issues" },
+  "homepage": "https://github.com/AgentsKit-io/harness",
   "author": "AgentsKit Contributors",
-  "sideEffects": false
+  "sideEffects": false,
+  "publishConfig": { "access": "public", "provenance": true }
 }

package/docs/ADR-0025-ci-dogfood.md

@@ -1,22 +0,0 @@
-# ADR-0025: Dogfood the harness in Playbook CI
-
-## Context
-
-The package can validate itself locally, but the Playbook CI previously ran
-repository checks without explicitly building, invoking, or installing the
-harness as a consumer. A release could therefore regress the harness path
-without failing the repository gate.
-
-## Decision
-
-Run `harness:test`, `harness:cli`, and the clean packed-consumer probe in the
-quality job immediately after dependency installation. Keep the checks
-explicit and local; the harness lifecycle still owns human approval for
-task-level completion.
-
-## Consequences
-
-The Playbook continuously exercises the package and both CLI entrypoints in
-the same Node 22 environment used by CI. The CI job does a small amount of
-duplicate work with package-level checks, which is intentional release-path
-coverage.

package/docs/ADR-0026-ci-evidence-artifact.md

@@ -1,22 +0,0 @@
-# ADR-0026: Preserve CI harness evidence for human review
-
-## Context
-
-Running checks in CI without retaining the harness run leaves reviewers with
-only a green job and no portable evidence bundle to inspect. The harness still
-requires human approval, so the CI result must remain reviewable rather than
-being treated as completion.
-
-## Decision
-
-The Playbook quality job runs the current harness contract, keeps its
-`.codex/verification/harness-phase-24` directory, and uploads it as a
-short-lived artifact. The run must reach `AWAITING_HUMAN_APPROVAL`; approval is
-still a separate human action.
-
-## Consequences
-
-Reviewers can inspect the run projection, event log, and check outputs from the
-same CI execution. Artifact retention is limited to 14 days; long-term audit
-retention belongs to the release or compliance system that consumes the
-artifact.

package/docs/ADR-0027-portable-evidence.md

@@ -1,19 +0,0 @@
-# ADR-0027: Verify signed evidence outside the source workspace
-
-## Context
-
-An evidence artifact is only useful to CI reviewers and release systems if it
-does not depend on the original checkout. Existing bundle tests verified
-signatures and trust, but did not prove path independence.
-
-## Decision
-
-The CLI flow copies the signed bundle and trust store into a separate temporary
-directory and verifies them there. The bundle remains self-contained and the
-existing trusted-key model is unchanged.
-
-## Consequences
-
-The portability guarantee is exercised without a new transport or dependency.
-The probe remains local and deterministic; CI or a release system can upload
-the same bundle as an artifact when its completed run is available.

package/docs/ADR-0028-effective-metrics.md

@@ -1,20 +0,0 @@
-# ADR-0028: Separate effective delivery metrics from retry history
-
-## Context
-
-Aggregate pass rates include blocked or stale attempts that were later
-superseded. Those facts are useful for measuring friction, but they can make a
-successful current delivery look like a failed one.
-
-## Decision
-
-Keep historical attempt metrics unchanged and add `effective*` summary metrics
-computed from runs that are not superseded by another run in the same local
-history. Report effective run count, completion count/rate, and check,
-outcome, and evidence rates.
-
-## Consequences
-
-Reviewers can distinguish process friction from the current delivery state
-without losing retry history. The lineage rule is local and deterministic; a
-future remote aggregation system must preserve `supersedes` relationships.

package/docs/ADR-0029-honest-ci-preparation.md

@@ -1,20 +0,0 @@
-# ADR-0029: Separate CI preparation from human approval
-
-## Context
-
-The CI dogfood flow needs to create and verify a run, but CI cannot truthfully
-act as the human who approved the task contract. Recording `--by human` in CI
-would make the audit trail misleading.
-
-## Decision
-
-`plan prepared --by ci` creates a planned run with a `contractPreparation`
-record. It may proceed through verification and produce evidence, but only the
-existing human approval path can transition the run to `COMPLETE`. Human
-planning continues to use `plan approved --by human`.
-
-## Consequences
-
-CI evidence is honest and reviewable without weakening the completion gate.
-The run schema remains readable for older v1 records through the optional
-legacy `contractApproval` field.

package/docs/ADR-0030-agentskit-os-benchmark-bridge.md

@@ -1,20 +0,0 @@
-# ADR-0030: Provenance-bearing external coding benchmark bridge
-
-## Decision
-
-Keep the harness benchmark schema portable and add optional task provenance:
-the source repository, immutable revision, prompt path and digest, and read /
-write scope. Validate the existing AgentsKit OS coding benchmark report through
-a small dependency-free adapter.
-
-The adapter validates report integrity only. Provider status, completeness
-heuristics, and `successPassed` are observations; they do not grant human
-acceptance or make a benchmark comparable without a controlled baseline,
-criterion evidence, and a `COMPLETE` harness run.
-
-## Consequences
-
-The same manifest can be checked from another agent or repository, while the
-AgentsKit OS benchmark remains responsible for running providers. Phase 28
-ships the seed corpus and bridge. Real baseline collection across repeated,
-reviewed tasks is the next measurement step; no improvement claim is made here.

package/docs/ADR-0031-real-provider-baseline.md

@@ -1,18 +0,0 @@
-# ADR-0031: Real-provider baseline before efficacy claims
-
-## Decision
-
-Collect the baseline by running the existing AgentsKit OS coding benchmark on
-disposable, git-initialized copies of its fixture workspace. Record one report
-per manifest task with provider output, duration, validation command, criterion
-evidence, and the pinned source revision.
-
-The baseline runner is separate from verification. Verification validates
-recorded observations; it does not invoke a paid or nondeterministic provider on
-every harness run.
-
-## Measurement rule
-
-Baseline data is not comparable by itself. A directional improvement requires
-the same task, scope, provider conditions, acceptance evidence, and a completed
-harness run. Missing harness-equivalent runs remain `comparableTaskCount: 0`.

package/docs/ADR-0032-harness-equivalent-benchmark.md

@@ -1,25 +0,0 @@
-# ADR-0032: Harness-equivalent benchmark runs
-
-## Status
-
-Accepted for the benchmark pilot.
-
-## Decision
-
-Run the frozen benchmark corpus once per task with a task-specific harness
-contract and `benchmark: { suiteId, taskId, mode: "harness" }` binding. Keep each
-task's lifecycle state isolated so a blocked provider result cannot be mistaken
-for another task's approval. Collection delegates directional calculations to
-the existing `benchmarkRuns` implementation.
-
-The runner prepares and executes plans but never records human approval. A task
-is comparable only after the real checks pass and a human moves it to
-`COMPLETE`; pending and blocked runs remain visible and non-comparable.
-
-## Consequences
-
-- Baseline and harness runs share task identity and provenance.
-- Human approval remains an auditable gate rather than an automation shortcut.
-- The first collection may report no improvement when the provider is blocked;
-  that is a valid result, not missing data to be filled in.
-- Temporary untracked directories no longer crash source freshness checks.

package/docs/ADR-0033-portable-agent-gate.md

@@ -1,25 +0,0 @@
-# ADR-0033: Portable agent-side verification gate
-
-## Status
-
-Accepted for the benchmark pilot.
-
-## Decision
-
-Disposable coding-agent fixtures expose the built `ak-verify` command through a
-fixture-local wrapper and PATH entry. The fixture contract is prepared with
-`plan prepared --by ci` before the provider starts. The provider can therefore
-run the same real verification command from inside its working directory.
-
-The wrapper is a convenience for provider-side feedback, not the final trust
-boundary. The outer harness still validates the provider report and acceptance
-evidence, and only a human can move an outer run to `COMPLETE`.
-
-## Consequences
-
-- Missing `ak-verify` is observable as an environment/setup failure instead of
-  an unexplained provider partial result.
-- No human decision is fabricated inside the benchmark fixture.
-- The wrapper executes the pinned local harness build; it does not install
-  dependencies or grant network access.
-- Provider status and artifact validation remain separate signals.

package/docs/ADR-0034-measurement-quality-gates.md

@@ -1,25 +0,0 @@
-# ADR-0034: Make benchmark improvement claims fail closed
-
-## Context
-
-A benchmark can complete every acceptance criterion while still costing more
-time or retries. Aggregate pass rates alone hide that trade-off, and a small
-sample cannot support a reliable general claim.
-
-## Decision
-
-Benchmark manifests may declare a policy with `minComparableTasks`,
-`maxDurationRegressionRate`, `minCompletedRunsPerTask`, and
-`requireZeroEscapedIncomplete`.
-
-Reports expose `retryCount`, `completedRuns`, per-task confidence, and a
-`qualityGate`. The gate is `insufficient-data` when the task sample is too
-small, `failed` when policy detects a regression, and `passed` only when all
-configured quality conditions hold. Historical retries remain visible but do
-not make a superseded run the effective result.
-
-## Consequences
-
-The harness shows quality/cost trade-offs instead of claiming that every
-improvement is positive. Directional results support iteration; reliable
-samples are required for release claims.

package/docs/ADR-0035-reproducible-benchmark-samples.md

@@ -1,20 +0,0 @@
-# ADR-0035: Use independent samples for benchmark confidence
-
-## Context
-
-A single provider run is too noisy to support a reliable performance claim.
-Counting repeated measurements as retries also exaggerates delivery cost.
-
-## Decision
-
-The benchmark runner accepts `--repeats N` and creates an isolated contract and
-state directory for each sample. Reports count `COMPLETE` samples toward
-confidence, compare duration using their median, and count retries only from
-superseded run lineages.
-
-## Consequences
-
-Three or more completed samples can reach reliable confidence when the manifest
-policy requires it. Human approval remains required for every sample. A median
-reduces the influence of one slow provider run but does not prove causality or
-productivity improvement.

package/docs/ADR-0036-comparable-baseline-samples.md

@@ -1,20 +0,0 @@
-# ADR-0036: Require comparable baseline samples
-
-## Context
-
-Comparing three harness samples with one baseline observation makes the
-performance result look more certain than it is. Provider latency can vary
-between runs, so the baseline needs the same replication discipline.
-
-## Decision
-
-Benchmark policy includes `minBaselineSamplesPerTask`. Baseline observations
-can carry `durationSamplesMs`; reports use their median and keep a legacy
-single `durationMs` as one sample. A task is non-comparable until both the
-baseline and harness meet their configured sample minimums.
-
-## Consequences
-
-The quality gate becomes `insufficient-data` until a fair baseline exists.
-This delays performance claims but prevents a replicated harness result from
-being presented as a reliable improvement against a one-off baseline.

package/docs/ADR-0037-replicated-baseline-collection.md

@@ -1,27 +0,0 @@
-# ADR-0037: Replicated baseline collection
-
-## Decision
-
-The AgentsKit OS baseline runner accepts `--repeats N` and executes every task
-in an independent disposable fixture. It records the measured durations as
-`durationSamplesMs`, keeps the task failed when any sample fails, and can
-replace the complete baseline observation set with `--record-manifest`.
-
-The replacement is staged through the existing typed
-`recordBenchmarkObservation` API and committed with one atomic rename. A
-partial task selection cannot update a complete manifest. The benchmark report
-then compares the median baseline duration with the median harness duration.
-
-## Rationale
-
-A single baseline run is not a fair comparator for three harness replicas.
-Independent baseline samples reduce sensitivity to provider and machine
-variance without misclassifying replicas as retries. Atomic recording prevents
-an interrupted collection from leaving a partially refreshed manifest.
-
-## Limits
-
-This controls sample count and provenance, not all sources of variance. The
-same provider, task corpus, pinned source revision, and comparable runtime
-should be used. Three samples are the current minimum policy, not a universal
-statistical guarantee; larger studies belong in a later benchmark program.

package/docs/ADR-0038-end-to-end-benchmark-boundary.md

@@ -1,28 +0,0 @@
-# ADR-0038: Use an end-to-end benchmark boundary
-
-## Decision
-
-Baseline samples measure elapsed time from disposable fixture setup through
-provider execution, artifact validation, and the end of the sample. Harness
-comparisons continue to use the elapsed duration of the configured real check.
-The provider-reported duration is retained as diagnostic data, not as the
-comparable duration.
-
-Baseline duration is comparable only when the baseline status is `passed` and
-every criterion evidence entry is `passed`. Escape reduction is reported
-independently, so an incomplete baseline cannot create a false duration or
-resource improvement while still exposing the observed delivery gap.
-
-## Rationale
-
-Comparing provider-only baseline time with the harness check time mixes two
-different scopes and can manufacture a regression. A common boundary measures
-the delivery workflow the human actually waits for. It also exposes whether a
-slow result comes from the provider or from validation overhead.
-
-## Limits
-
-The boundary does not remove provider, machine, network, or human-approval
-variance. Independent samples and median aggregation remain required; deeper
-latency decomposition is a later optimization if the comparable end-to-end
-measurement still regresses.

package/docs/ADR-0039-artifact-and-protocol-metrics.md

@@ -1,39 +0,0 @@
-# ADR-0039: Separate artifact acceptance from protocol completion
-
-## Decision
-
-Repeated external benchmark samples record `artifactAcceptanceRate`: the
-fraction whose task-specific artifact validation passed. The observation
-`status` remains protocol-aware and only qualifies for comparable duration or
-resource metrics when the baseline delivery and criterion evidence are
-complete.
-
-Harness runs may expose the same metric through structured check evidence. The
-benchmark report aggregates the available run-level rates and reports their
-sample count; missing evidence is not converted to an acceptance failure.
-
-Baseline observations also record `protocolCompletionRate`: the fraction of
-samples whose provider completed the verification protocol. Harness reports
-expose the corresponding per-task rate and sample count. This makes protocol
-reliability measurable without treating an incomplete baseline as comparable.
-
-Relative improvement remains unavailable for a zero baseline. The comparison
-also records an absolute delta and derives its direction from that delta, so
-zero-to-positive outcomes remain visible without invalid division.
-
-## Rationale
-
-An agent can produce a correct artifact while failing to provide the required
-verification protocol, for example because the verifier is unavailable. That
-is valuable evidence about task outcome and protocol reliability, but treating
-it as a complete baseline would make timing and resource comparisons unsafe.
-Keeping the dimensions separate makes the failure visible without converting
-partial evidence into a completion claim.
-
-## Limits
-
-Artifact acceptance is only as strong as the real task validator supplied by
-the benchmark. It does not prove human approval, protocol compliance, or
-enterprise readiness. A future benchmark schema may add criterion-level sample
-rates when the corpus needs per-criterion variance rather than one aggregate
-rate.

package/docs/ADR-0040-benchmark-corpus-surfaces.md

@@ -1,32 +0,0 @@
-# ADR-0040: benchmark corpus surface coverage
-
-## Status
-
-Accepted — 2026-08-31
-
-## Context
-
-The AgentsKit OS bridge initially measured three real coding tasks, all backed
-by a small logic fixture. That was enough to prove the protocol binding, but it
-could not show that the harness preserved different delivery surfaces.
-
-## Decision
-
-Benchmark tasks may declare a typed `surfaces` list using the same surface names
-as the verification contract. The phase-45 corpus remains anchored to the
-AgentsKit OS benchmark definition and adds executable CLI and documentation
-tasks. The runner validates the CLI through real child-process invocations and
-validates the documentation contract against the resulting README.
-
-The surface list is descriptive metadata. It does not make a task comparable,
-complete, or human-approved; those claims still require the normal lifecycle,
-criterion evidence, and current-source verification.
-
-## Consequences
-
-- Corpus validators can require coverage of named surfaces without guessing
-  from file paths.
-- The bridge now exercises logic, CLI, and documentation delivery shapes.
-- Endpoint, database, MCP, and UI tasks remain future corpus additions and must
-  bring their real runtime checks when introduced.
-- No benchmark performance improvement is inferred from corpus expansion alone.