@agentskit/harness 0.3.0 → 0.5.0

package/CHANGELOG.md

@@ -1,5 +1,31 @@
 # Changelog
 
+## [0.5.0] - 2026-09-11
+
+- Added the keep-pushing SDLC loop foundation (`ak-harness loop validate|doctor`): `loop.config.yaml` schema (zod), provider detection with Orca usage/rate-limit awareness and cooldowns, role-based tiered model routing, machine slot assessment, Orca CLI and Linear-via-Orca adapters, and the loop doctor report.
+- Added loop phase 2 adapters: Orca worktree create/set/rm, terminal send/wait/read and automations argv; Linear issue detail, status/comment/label/attach writes and a Linear `TrackingAdapter`; GitHub PR snapshots, check assessment, self-edit path guard and optimistic squash-merge via `gh`.
+- Added loop phase 3 (`ak-harness loop tick|precheck|contract`): orchestrator-frozen task contracts with untrusted issue text, candidate fallback and provider cooldown on auth/quota failures, dispatch ledger claims, Orca worktree dispatch with `--linear-issue`, worker briefs, Linear In Progress transition and `needs-info` escalation.
+- Added loop phase 4 (`ak-harness loop deliver`, `precheck deliver`): per-issue delivery state, PR detection by branch, protected-path hold, conflict/CI/review fix rounds sent to the worker terminal with a bounded budget, `agentskit-review` at the current head, optimistic squash-merge, Linear attach/Done, worktree cleanup, stuck/abandoned escalation with slot release.
+- Added loop phase 5 (`ak-harness loop install|uninstall|status|hook`): idempotent Orca automations with `--precheck`, existing-workspace mode and session reuse; status with latest runs; a status-only SessionStart hook line; a cross-platform CI job (ubuntu/macos/windows); ADR-0027.
+- `loop install` is guided: doctor and environment checks (harness/review CLIs, `gh auth`, Orca repo registration), optional dry-run tick rehearsal, explicit confirmation; `--yes`, `--force`, `--skip-rehearsal`, `--dry-run`, `--plain`.
+- `loop install` offers to create the per-machine `loop.config.local.yaml` (queue owner from the Linear team, RAM reserve, worker ceiling) when it is missing; the CLI renders checks and prompts with Ink on TTYs and falls back to plain lines elsewhere. Grok is treated as a CLI subscription (`grok login`), no API key.
+- Orca automations now run the stage inside the `--precheck` command (`ak-harness loop stage tick|deliver`, always exit 1) so no agent session is opened per run; `schedule.runner: agent` keeps the previous behaviour. Fixes stuck bypass-permissions sessions leaking one terminal per run.
+- Added `ak-harness loop retro [--since 7d] [--json|--learnings]`: escalations by reason, dispatches by provider, merged/blocked/stuck, review outcomes, fix rounds, median lead time, cooldowns, Orca run summary, and rule-based calibration suggestions with the config knob to turn; the Markdown follows the harness retro grammar so `parseRetro`/`promoteLearnings` apply. Suggestions are split by target — `project` (config, issues, process) versus `harness` (library defects seen in production) — with `--target` to filter; the tick records `contract.failed` events.
+- Reviews run `agentskit-review --mode trusted-local` by default (`delivery.review.mode`); the isolated default gives claude/codex a temporary HOME without credentials and every lens fails with "Not logged in".
+- Review defaults fit an Orca stage: `profile: fast`, `votes: 1`, `concurrency: 4`, and the deadline is capped to the stage budget under `runner: precheck`; `full` profile stays available for long deadlines.
+- `delivery.review.transport` (`acp` \| `headless` \| `auto`) is passed through to `agentskit-review` so Grok can use headless when ACP is broken.
+- Added `ak-harness loop debrief`: read-only human explanation of in-flight work, holds, escalations and cooldowns (Markdown or `--json`).
+- Added `ak-harness loop watch`: TypeScript poller over `delivery.json` (+ optional live PR) emitting `DONE` / `FAILED` / `ACTION_REQUIRED` / `PROGRESS`.
+- Workers are launched with the configured TUI command in their own terminal (`orca worktree create` without `--agent`, then `orca terminal create --command <tui>`, wait for idle, send the brief). Orca's `--agent claude` starts in bypass-permissions mode and blocks on a human prompt. The tick has a wall-clock budget under `runner: precheck`; a failed dispatch removes its half-created worktree.
+- Add bounded agent eval, safe context/read-only LLM cache, deterministic workflow fan-out/fan-in, and validated optimization observation contracts for token, memory, cache, and parallelism measurements.
+
+## [0.4.0] - 2026-09-10
+
+- Added phase quality matrices, watchdog classification, and resource telemetry.
+- Added versioned eval and ecosystem compatibility manifests with fail-closed
+  evidence handling.
+- Added runnable consumer onboarding, adapter examples, and troubleshooting.
+
 ## [0.3.0] - 2026-09-10
 
 - Added portable issue/worktree claims and idempotent dispatch ledger.
@@ -9,9 +35,6 @@
   policies, and provider-neutral Orca/tracking adapters.
 - Added configurable machine pressure thresholds and adaptive workflow limits.
 
-## [Unreleased]
-
-- Add bounded agent eval, safe context/read-only LLM cache, deterministic workflow fan-out/fan-in, and validated optimization observation contracts for token, memory, cache, and parallelism measurements.
 
 All notable changes to `@agentskit/harness` are documented here.
 

package/README.md

@@ -6,7 +6,15 @@ docbridge:
 
 # @agentskit/harness
 
-Portable, evidence-backed development protocol for coding agents. The harness freezes a human-approved task contract, executes every configured check, binds evidence to the current source revision, detects stale results, and refuses completion without human approval.
+Portable, evidence-backed development protocol for coding agents. The harness freezes a task contract, executes every configured check, binds evidence to the current source revision, detects stale results, and applies the configured controlled or YOLO approval policy.
+
+## Keep-pushing loop (Orca)
+
+`ak-harness loop …` drains one person's Linear queue through Orca worktrees 24/7 — tiered model routing with
+usage-aware fallback, orchestrator-frozen contracts, adversarial review, squash-merge, and one deduplicated
+escalation when a ticket cannot be verified. Everything project-specific lives in `loop.config.yaml`
+(start from [`loop.config.example.yaml`](loop.config.example.yaml)). Guide: [`docs/LOOP.md`](docs/LOOP.md) ·
+decision record: [`docs/ADR-0027-keep-pushing-loop.md`](docs/ADR-0027-keep-pushing-loop.md).
 
 ## Install
 
@@ -14,6 +22,10 @@ Portable, evidence-backed development protocol for coding agents. The harness fr
 pnpm add -D @agentskit/harness
 ```
 
+New consumers can run [`examples/minimum-profile.mjs`](examples/minimum-profile.mjs)
+after `pnpm build`; the walkthrough is in [`docs/GETTING-STARTED.md`](docs/GETTING-STARTED.md).
+Common gate and runtime failures are documented in [`docs/TROUBLESHOOTING.md`](docs/TROUBLESHOOTING.md).
+
 The package requires Node.js 22 or newer and exposes both `ak-harness` and the common-protocol alias `ak-verify`.
 
 ## Workflow
@@ -44,7 +56,7 @@ Endpoint, database, CLI, MCP, and UI checks must declare `execution: "real"`. UI
 
 ## API
 
-The public TypeScript API is exported from `src/index.ts` and includes configuration loading, lifecycle operations, state transitions, evidence verification, approvals, cancellation, retries, and task-owned cleanup. Internal modules are not part of the supported API.
+The public TypeScript API is exported from `src/index.ts` and includes configuration loading, lifecycle operations, state transitions, evidence verification, approvals, cancellation, retries, task-owned cleanup, versioned capability manifests, event-envelope validation, deterministic phase execution, and stable error classification. Internal modules are not part of the supported API. The checked-in [capability manifest](./capabilities/public-surface.json) is generated from this entry point; run `pnpm test:capabilities` to detect drift.
 
 ## Extensibility
 
@@ -85,6 +97,29 @@ the current state, so a post-approval edit cannot appear as `COMPLETE`.
 Concurrent event writers are serialized by an atomic per-run lock and fail
 closed if the log is busy.
 
+Structured plans, findings, decisions, repairs, blockers, approvals, and phase
+results can be persisted as provenance-bound `ArtifactEnvelope` records. Each
+artifact has a version, run/issue/source/contract/config/context hashes, a
+content digest, and both JSON and Markdown representations. `FileArtifactStore`
+is idempotent: retrying the same write does not duplicate the event-log record.
+Use `resumeStateFromArtifacts` to rebuild completed phase outputs after an
+interruption, and inspect records with `ak-harness artifacts inspect <path>` or
+`ak-harness artifacts list [run-id]`.
+
+The legacy event-log record remains schema version 1 for compatibility. New
+provider-neutral integrations can exchange the schema-versioned v2
+`HarnessEventEnvelope`, which requires event identity, correlation, source
+revision, idempotency, and provenance metadata. `classifyHarnessError` maps
+stable Harness error codes to `retry`, `block`, or `escalate` dispositions.
+
+Replaceable integrations use the shared `AdapterMetadata` contract: every
+adapter declares an assurance level (`unverified`, `contract-tested`, or
+`runtime-attested`) and measured/unknown telemetry. Coding agents return
+structured output, diff, usage, timeout/cancellation status, and failure
+classification; Doc Bridge reports relevance and context cost; Orca exposes
+lease/lock/worktree/SHA projections; and tracking adapters deduplicate effects
+by idempotency key (with a dry-run mode).
+
 Each harness event may also carry an optional `correlation` envelope. Its
 `operationId` is the stable identity used when a lifecycle crosses into
 AgentsKit, Chat, Doc Bridge, or Code Review; the optional `runId`, `sessionId`,
@@ -123,6 +158,40 @@ contract is frozen:
 
 `autonomy: "yolo"` removes the generic final review only after every applicable check passes, tracking is disabled, and the frozen contract has no ambiguity. It never auto-approves a material decision, external tracking, or a tool rule that requires approval.
 
+The phase executor applies the same rule to a declarative SDLC profile. A profile
+declares dependencies, inputs/outputs, gates, bounded retries, budgets, and an
+effect class (`read`, `write`, or `external`). `safe`, `yolo`, and `dry-run`
+profiles share the engine; only the effect policy changes:
+
+```ts
+const profile = createPhaseProfile({
+  id: 'feature', mode: 'yolo',
+  phases: [
+    { id: 'discover', outputs: ['plan'], effect: 'read' },
+    { id: 'implement', inputs: ['plan'], dependsOn: ['discover'], effect: 'write' },
+  ],
+})
+const result = await executePhaseProfile(profile, {
+  preflight: grillMeAndPreflight,
+  handlers: { discover, implement },
+})
+```
+
+Preflight runs for all mutating phases before any effect. Material ambiguities
+are returned as one structured decision packet; dry-run previews mutating phases
+without invoking their handlers. `planPhaseProfile` exposes the deterministic
+route without executing it.
+
+`runAdversarialReview` executes independent review lenses with bounded
+concurrency/retries and blocks empty or non-reproducible verdicts. Delivery
+helpers hash-bind the approved PR body/metadata and only emit a QA transition
+after feature validation and G5 acceptance; failed QA returns to verification.
+
+`createQualityMatrix` aggregates phase evidence, outcomes, duration, token/cache,
+machine, and concurrency signals into bounded 0–100 dimensions with baseline
+deltas. Missing measurements remain `unknown`; `evaluateWatchdog` emits typed
+budget/resource/contention blockers instead of treating absent data as success.
+
 Use named profiles to make the operational choice explicit:
 
 ```json
@@ -242,6 +311,17 @@ in sorted, bounded fan-out/fan-in batches while serializing nodes that share a
 optional token, memory, cache, and parallelism measurements and refuses
 incomparable provider/model/configuration bindings.
 
+`evals/manifest.json` is the versioned evaluation battery. `validateEvalManifest`
+requires contract, deterministic, integration, quality, regression, and
+resource layers plus coverage for every supported component. `runEvalBattery`
+repeats each case and reports min/median/max scores; unknown, stale, critical,
+subjective, or unapproved regression results block the gate.
+
+`compatibility/manifest.json` pins the AgentsKit ecosystem revisions and the
+upstream test/eval commands. `assessCompatibility` accepts only complete,
+evidence-bound real-adapter observations and blocks unknown or failed upstream
+results; migration and rollback procedures are kept beside the manifest.
+
 These are seams, not replacements for AgentsKit packages. An integration may
 adapt `@agentskit/memory` and `@agentskit/eval` into them while keeping the
 Harness provider-neutral. Missing measurements remain missing; they are never
@@ -524,6 +604,9 @@ Releases are published by `.github/workflows/release-harness.yml` after a merge
 or require an `NPM_TOKEN`. Configure the npm trusted publisher once for
 `AgentsKit-io/harness`, workflow `release-harness.yml`, and package
 `@agentskit/harness`; version changes remain the release trigger.
+The 0.4.0 candidate checklist and explicit blockers live in
+[`release/manifest.json`](release/manifest.json) and
+[`release/notes.md`](release/notes.md).
 
 ## License