@agentskit/harness 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +32 -0
- package/README.md +1 -1
- package/capabilities/public-surface.json +107 -84
- package/dist/cli.js +407 -73
- package/dist/cli.js.map +1 -1
- package/dist/index.d.ts +211 -56
- package/dist/index.js +373 -85
- package/dist/index.js.map +1 -1
- package/docs/ADR-0003-doc-bridge-context-binding.md +10 -3
- package/docs/ADR-0031-loop-observability.md +36 -0
- package/docs/LOOP.md +7 -0
- package/docs/MODULE-BOUNDARIES.md +1 -0
- package/loop.config.example.yaml +1 -1
- package/package.json +2 -2
- package/release/manifest.json +3 -3
- package/release/notes.md +19 -0
|
@@ -6,9 +6,16 @@
|
|
|
6
6
|
## Decision
|
|
7
7
|
|
|
8
8
|
The harness provides a dependency-free `createDocBridgeContextProvider` adapter
|
|
9
|
-
that reads the local `.doc-bridge/index.json` contract. It
|
|
10
|
-
|
|
11
|
-
|
|
9
|
+
that reads the local `.doc-bridge/index.json` contract. It searches both the
|
|
10
|
+
knowledge corpus and `lookup.ownership` records, deduplicates references by
|
|
11
|
+
path with ownership tie-breaking, returns at most eight deterministic
|
|
12
|
+
references, carries the index `contentHash` as source provenance, and computes
|
|
13
|
+
a stable snapshot hash.
|
|
14
|
+
|
|
15
|
+
When `contract.docBridgeMaxAgeHours` is greater than zero, the adapter rejects
|
|
16
|
+
an index older than that budget before attaching context. This is an age guard,
|
|
17
|
+
not a source-content proof; the exact Doc Bridge hash gate remains the required
|
|
18
|
+
check for CI and release evidence.
|
|
12
19
|
|
|
13
20
|
`planRun` accepts resolved context snapshots and freezes them into `run.json`
|
|
14
21
|
with a `contextHash`. The lifecycle log records one `context.attached` event per
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# ADR-0031: Read-only loop observability
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
The keep-pushing loop already records durable events and exposes doctor, debrief,
|
|
10
|
+
and retro reports, but operators still have to correlate them manually to find
|
|
11
|
+
stalled workers, missing delivery state, or an idle queue. A daemon or dashboard
|
|
12
|
+
would add another runtime to operate.
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
Add a read-only `loop observe` command and a pure `assessObservability` function.
|
|
17
|
+
The collector reuses existing doctor/debrief/event-log data and read-only Orca
|
|
18
|
+
queries. It reports queue, delivery, machine, provider, memory, cache, and token
|
|
19
|
+
metrics, plus deterministic anomaly rules for:
|
|
20
|
+
|
|
21
|
+
- connected terminals with no output;
|
|
22
|
+
- active claims without `delivery.json`;
|
|
23
|
+
- completed worktrees with uncommitted files;
|
|
24
|
+
- ready work with a free slot but no recent dispatch; and
|
|
25
|
+
- in-flight worker/review phases past `delivery.workerIdleTimeoutMin`.
|
|
26
|
+
|
|
27
|
+
`--precheck` provides scheduler-compatible exit semantics (0 actionable, 1
|
|
28
|
+
healthy). The event log remains the source of truth; no new persistence or
|
|
29
|
+
background process is introduced.
|
|
30
|
+
|
|
31
|
+
## Consequences
|
|
32
|
+
|
|
33
|
+
Operators and the existing observer can consume one stable JSON/Markdown report,
|
|
34
|
+
while tests exercise the anomaly rules without Orca or network fixtures. A future
|
|
35
|
+
dashboard or MCP read-only surface can consume the same report without changing
|
|
36
|
+
the loop execution path.
|
package/docs/LOOP.md
CHANGED
|
@@ -25,6 +25,7 @@ ak-harness loop uninstall [--dry-run] # remove them
|
|
|
25
25
|
ak-harness loop status # what Orca knows: enabled, trigger, provider, latest run
|
|
26
26
|
ak-harness loop hook # one status line for a SessionStart hook; never mutates
|
|
27
27
|
ak-harness loop debrief [--issue ENG-123] [--since 24h] # human-facing: what is in flight, held, escalated
|
|
28
|
+
ak-harness loop observe [--since 24h] [--json] # anomaly scan + queue, delivery, machine, memory/cache metrics
|
|
28
29
|
ak-harness loop watch [--issue ENG-123] [--once] [--interval 30] # poll delivery/PR; DONE|FAILED|ACTION_REQUIRED
|
|
29
30
|
ak-harness loop retro [--since 7d] [--json|--learnings] # weekly digest + calibration suggestions
|
|
30
31
|
```
|
|
@@ -47,6 +48,12 @@ so it is safe to run from a SessionStart hook or a chat agent that needs context
|
|
|
47
48
|
|
|
48
49
|
Use `--once` for a single snapshot; omit it to block until a terminal outcome (or `--timeout <seconds>`).
|
|
49
50
|
|
|
51
|
+
`loop observe` is the scheduler-friendly health view. It reuses the doctor, debrief and durable event log, then
|
|
52
|
+
checks for a connected terminal with no output, an active claim without `delivery.json`, a finalized dirty worktree,
|
|
53
|
+
a ready queue with an idle slot, and an in-flight review/worker past `delivery.workerIdleTimeoutMin`. It also reports
|
|
54
|
+
machine pressure, provider headroom, delivery counts, fix rounds, memory recalls, cached contracts and observed token
|
|
55
|
+
fields. It is read-only; `--precheck` uses exit 0 for an actionable anomaly and exit 1 when healthy.
|
|
56
|
+
|
|
50
57
|
## Running 24/7 with Orca
|
|
51
58
|
|
|
52
59
|
`loop install` is guided and rendered with Ink when stdin/stdout are terminals (plain lines otherwise). When no
|
|
@@ -100,6 +100,7 @@ modules, not a provider dependency.
|
|
|
100
100
|
| `src/loop/local-config.ts` | Composition | Per-machine overlay wizard: Linear team members via Orca, queue owner and machine tuning answers, YAML rendering and reload | `orca-cli`, `config`, `yaml` | Orca via runner; writes `loop.config.local.yaml` |
|
|
101
101
|
| `src/loop/retro.ts` | Composition | Retro digest over `events.ndjson`, per-issue state, cooldowns and Orca runs; rule-based calibration suggestions; Markdown that `parseRetro` can lift into learnings | `orca-cli`, `hash`, `learning`, `config`, `contract`, `cooldown`, `deliver`, `install`, `tick` | Orca via runner (optional); `<stateDir>` files |
|
|
102
102
|
| `src/loop/debrief.ts` | Composition | Human-facing read-only snapshot of what the loop is working on (in-flight, holds, escalations, cooldowns) | `config`, `contract`, `cooldown`, `deliver`, `retro`, `tick` | `<stateDir>` files only |
|
|
103
|
+
| `src/loop/observability.ts` | Composition | Read-only anomaly assessment and operating metrics built from doctor, debrief, Orca terminal/worktree state, and the durable loop event log | `config`, `doctor`, `debrief`, `deliver`, `retro`, `tick`, `coordination`, `orca-cli` | Orca/Git via injected runner; `<stateDir>` files |
|
|
103
104
|
| `src/loop/watch.ts` | Composition | Poll delivery (+ optional live PR) and emit DONE/FAILED/ACTION_REQUIRED/PROGRESS for agents or humans | `command`, `github-cli`, `config`, `deliver`, `tick` | optional `gh` via runner; `<stateDir>` files |
|
|
104
105
|
| `src/loop/ui/components.tsx` | Composition | Ink components: check rows, sections, banner, spinner, select/confirm/text prompts | `ink`, `react`, `doctor` (type-only) | Terminal |
|
|
105
106
|
| `src/loop/ui/terminal.tsx` | Composition | `createRichIO`: Ink-backed IO for TTYs with a plain-text fallback | `ink`, `react`, `components`, `guided-install` (type-only) | Terminal |
|
package/loop.config.example.yaml
CHANGED
|
@@ -32,7 +32,7 @@ linear:
|
|
|
32
32
|
rotation:
|
|
33
33
|
enabled: false # set true to advance through owners when the current dispatchable queue drains
|
|
34
34
|
owners: [my-linear-display-name] # ordered display names; the first entry is the initial owner
|
|
35
|
-
advanceWhenEmpty: true # never switch while an
|
|
35
|
+
advanceWhenEmpty: true # never switch while an implementation lease remains
|
|
36
36
|
states: [Todo, Ready] # dispatchable states; anything already started is left alone
|
|
37
37
|
excludeLabels: [blocked, needs-info]
|
|
38
38
|
requireLabels: []
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agentskit/harness",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.12.0",
|
|
4
4
|
"description": "Portable, evidence-backed development harness for coding agents.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -68,7 +68,7 @@
|
|
|
68
68
|
"test:phase-executor": "pnpm typecheck && vitest run --config vitest.config.ts test/phase-executor.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['phase-executor']}))\"",
|
|
69
69
|
"test:artifacts": "pnpm typecheck && vitest run --config vitest.config.ts test/artifacts.test.ts && node scripts/verify-artifact-cli.mjs",
|
|
70
70
|
"test:adapters": "pnpm typecheck && vitest run --config vitest.config.ts test/adapters.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['adapters']}))\"",
|
|
71
|
-
"test:loop": "pnpm typecheck && vitest run --config vitest.config.ts test/loop.test.ts test/loop-adapters.test.ts test/loop-tick.test.ts test/loop-deliver.test.ts test/loop-install.test.ts test/loop-guided-install.test.ts test/loop-retro.test.ts test/loop-debrief.test.ts test/loop-watch.test.ts test/loop-agent-registry.test.ts test/loop-memory.test.ts test/loop-contract-failure.test.ts test/loop-resilience-state.test.ts test/loop-skills.test.ts test/loop-github-intake.test.ts test/loop-event-bus.test.ts test/loop-progress.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['loop-config','loop-providers','loop-routing','loop-doctor','loop-adapters','loop-tick','loop-deliver','loop-install','loop-debrief','loop-watch','loop-agent-registry','loop-memory','loop-contract-failure','loop-resilience-state','loop-skills','loop-github-intake','loop-event-bus','loop-progress']}))\"",
|
|
71
|
+
"test:loop": "pnpm typecheck && vitest run --config vitest.config.ts test/loop.test.ts test/loop-adapters.test.ts test/loop-tick.test.ts test/loop-deliver.test.ts test/loop-install.test.ts test/loop-guided-install.test.ts test/loop-retro.test.ts test/loop-debrief.test.ts test/loop-watch.test.ts test/loop-observability.test.ts test/loop-agent-registry.test.ts test/loop-memory.test.ts test/loop-contract-failure.test.ts test/loop-resilience-state.test.ts test/loop-skills.test.ts test/loop-github-intake.test.ts test/loop-event-bus.test.ts test/loop-progress.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['loop-config','loop-providers','loop-routing','loop-doctor','loop-adapters','loop-tick','loop-deliver','loop-install','loop-debrief','loop-watch','loop-observability','loop-agent-registry','loop-memory','loop-contract-failure','loop-resilience-state','loop-skills','loop-github-intake','loop-event-bus','loop-progress']}))\"",
|
|
72
72
|
"test:review": "pnpm typecheck && vitest run --config vitest.config.ts test/review.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['review-delivery']}))\"",
|
|
73
73
|
"test:quality": "pnpm typecheck && vitest run --config vitest.config.ts test/quality.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['quality-matrix']}))\"",
|
|
74
74
|
"test:eval-battery": "pnpm typecheck && vitest run --config vitest.config.ts test/eval-battery.test.ts test/issues-010-014.eval.test.ts test/optimization.test.ts && pnpm build >/dev/null && node scripts/verify-harness-eval-manifest.mjs && node -e \"console.log(JSON.stringify({status:'passed',criteria:['eval-battery','eval-manifest','eval-coverage']}))\"",
|
package/release/manifest.json
CHANGED
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
"type": "agentskit-harness-release-manifest",
|
|
3
3
|
"schemaVersion": 1,
|
|
4
4
|
"package": "@agentskit/harness",
|
|
5
|
-
"version": "0.
|
|
5
|
+
"version": "0.12.0",
|
|
6
6
|
"channel": "latest",
|
|
7
|
-
"sourceRevision": "
|
|
7
|
+
"sourceRevision": "922d2444ef48bfc89e58ccbe0553efc06a09c877",
|
|
8
8
|
"requiredChecks": [
|
|
9
9
|
"typecheck",
|
|
10
10
|
"test",
|
|
@@ -32,5 +32,5 @@
|
|
|
32
32
|
"pilot-benchmark"
|
|
33
33
|
],
|
|
34
34
|
"nextBaseline": "benchmarks/harness-0.4.0-baseline.json",
|
|
35
|
-
"digest": "
|
|
35
|
+
"digest": "439805502a4fdd074491269cf733b4144f968ac2ef4b32a6271a4c21102ecc7a"
|
|
36
36
|
}
|
package/release/notes.md
CHANGED
|
@@ -1,3 +1,22 @@
|
|
|
1
|
+
# 0.12.0 release candidate
|
|
2
|
+
|
|
3
|
+
Reuses Orca instead of reinventing it, where Orca's own tooling gives a better answer than the harness's own
|
|
4
|
+
heuristics: `loop deliver` escalations now fold in the dispatched worker's own terminal output (best-effort,
|
|
5
|
+
`terminal read --screen`) so a human sees the worker's own diagnosis instead of a generic idle/no-PR message; slot
|
|
6
|
+
assessment prefers Orca's `diagnostics memory` (macOS's real memory-pressure reading) and real per-session RSS over
|
|
7
|
+
the harness's own `vm_stat` approximation and static per-agent guess, which were measured roughly 2x more
|
|
8
|
+
conservative at the same instant. Also ships an installable `ak-harness-loop` Orca Skill documenting the loop's
|
|
9
|
+
commands and the operational constraints this release's investigation confirmed (Orca's orchestration mutations —
|
|
10
|
+
`run-create`/`task-create`/`worker-start`/`send` — require a live coordinator terminal pane and cannot be called
|
|
11
|
+
from the loop's headless `tick`/`deliver`, verified from every angle tried). Publication remains gated on a merge
|
|
12
|
+
to `main` through npm Trusted Publishing.
|
|
13
|
+
|
|
14
|
+
# 0.11.0 release candidate
|
|
15
|
+
|
|
16
|
+
Read-only loop observability for the keep-pushing SDLC loop: deterministic anomaly detection and operating metrics
|
|
17
|
+
for queue, delivery, machine pressure, provider headroom, memory, cache, reviews, fix rounds, lead time, and
|
|
18
|
+
observed token fields. Publication remains gated on a merge to `main` through npm Trusted Publishing.
|
|
19
|
+
|
|
1
20
|
# 0.8.0 release candidate
|
|
2
21
|
|
|
3
22
|
Worker handoff: continue in-flight tickets on the same Orca worktree/branch with another provider when usage runs out.
|