@agentskit/harness 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +42 -0
- package/README.md +1 -1
- package/capabilities/public-surface.json +153 -80
- package/dist/cli.js +1000 -180
- package/dist/cli.js.map +1 -1
- package/dist/index.d.ts +340 -57
- package/dist/index.js +837 -120
- package/dist/index.js.map +1 -1
- package/docs/ADR-0003-doc-bridge-context-binding.md +10 -3
- package/docs/ADR-0030-loop-event-bus-orchestration-hooks.md +54 -0
- package/docs/ADR-0031-loop-observability.md +36 -0
- package/docs/LOOP.md +94 -2
- package/docs/MODULE-BOUNDARIES.md +4 -0
- package/loop.config.example.yaml +24 -1
- package/package.json +2 -2
- package/release/manifest.json +2 -2
- package/release/notes.md +6 -0
|
@@ -6,9 +6,16 @@
|
|
|
6
6
|
## Decision
|
|
7
7
|
|
|
8
8
|
The harness provides a dependency-free `createDocBridgeContextProvider` adapter
|
|
9
|
-
that reads the local `.doc-bridge/index.json` contract. It
|
|
10
|
-
|
|
11
|
-
|
|
9
|
+
that reads the local `.doc-bridge/index.json` contract. It searches both the
|
|
10
|
+
knowledge corpus and `lookup.ownership` records, deduplicates references by
|
|
11
|
+
path with ownership tie-breaking, returns at most eight deterministic
|
|
12
|
+
references, carries the index `contentHash` as source provenance, and computes
|
|
13
|
+
a stable snapshot hash.
|
|
14
|
+
|
|
15
|
+
When `contract.docBridgeMaxAgeHours` is greater than zero, the adapter rejects
|
|
16
|
+
an index older than that budget before attaching context. This is an age guard,
|
|
17
|
+
not a source-content proof; the exact Doc Bridge hash gate remains the required
|
|
18
|
+
check for CI and release evidence.
|
|
12
19
|
|
|
13
20
|
`planRun` accepts resolved context snapshots and freezes them into `run.json`
|
|
14
21
|
with a `contextHash`. The lifecycle log records one `context.attached` event per
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# ADR-0030: Loop event bus and orchestration hooks, not model-loop middleware
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-09-12
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Comparing the loop against frameworks that build a "custom agent harness" in-process (e.g. LangChain's
|
|
9
|
+
`create_agent` + middleware: hooks before/after every model call and every tool call) surfaced a real gap: this
|
|
10
|
+
harness had no deterministic extension point at all. `appendLoopEvent` (`src/loop/tick.ts`) only appends to
|
|
11
|
+
`<stateDir>/events.ndjson` — nothing can react in real time, and nothing can say "stop, don't do this" before a
|
|
12
|
+
consequential action.
|
|
13
|
+
|
|
14
|
+
The gap is real, but the shape of the fix is not "add middleware like LangChain's": this harness orchestrates
|
|
15
|
+
external CLI agents (claude/codex/grok) as opaque processes inside an Orca worktree. There is no in-process
|
|
16
|
+
model↔tool loop to hook into — the worker's own context window, its own tool calls, are invisible to us by design
|
|
17
|
+
(ADR-0027). Building a `before model call` hook here is not possible without controlling that loop, which is
|
|
18
|
+
explicitly out of scope.
|
|
19
|
+
|
|
20
|
+
What we *do* control, end to end, is the orchestration around that opaque worker: whether we dispatch it, whether
|
|
21
|
+
we act on its review, whether we merge its PR. That is the level at which a deterministic extension point is both
|
|
22
|
+
useful and achievable.
|
|
23
|
+
|
|
24
|
+
## Decision
|
|
25
|
+
|
|
26
|
+
1. **`src/loop/event-bus.ts`** (Composition): a local, in-process pub/sub over the loop's existing free-form event
|
|
27
|
+
vocabulary (`contract.failed`, `worker.dispatched`, `pr.reviewed`, …) plus a small, fixed set of **orchestration
|
|
28
|
+
lifecycle hooks**: `beforeDispatch`, `afterDispatch`, `beforeReview`, `afterReview`, `beforeMerge`, `afterMerge`,
|
|
29
|
+
`onPause`, `onEscalate`. A `before*` hook may return `{ block: true, reason }` to stop the action; every other
|
|
30
|
+
hook is notification-only. A listener or hook that throws is swallowed (logged as a note, never fatal) — a
|
|
31
|
+
broken local plugin must not take down tick or deliver.
|
|
32
|
+
2. **Not `kernel/plugins.ts`.** The kernel's plugin registry is tied to `HARNESS_EVENT_TYPES`, the harness's own
|
|
33
|
+
run-lifecycle vocabulary, and carries plugin ids/versions/dependency ordering appropriate for a shared kernel
|
|
34
|
+
contract. The loop's event vocabulary is an open set of strings owned by composition, not the kernel, so
|
|
35
|
+
`event-bus.ts` is a separate, simpler primitive with the same "who's listening" ergonomics rather than a reuse
|
|
36
|
+
of that contract.
|
|
37
|
+
3. **`plugins.modules`** (`loop.config.yaml`): local `.mjs` files, relative to `project.root`, loaded once at the
|
|
38
|
+
start of `tick`/`deliver`. Same trust level as `agents.registry.yaml` today — files already checked into the
|
|
39
|
+
project's own repo, never fetched over a network. Each module exports `{ id, apply(bus) }`.
|
|
40
|
+
4. **`appendLoopEvent` gains an optional third `bus` parameter.** Every existing call site (`retro.ts`, tests) is
|
|
41
|
+
unaffected; `tick.ts`/`deliver.ts` pass their bus so every event written to disk is also emitted live.
|
|
42
|
+
5. **`loop doctor` gains a `plugins.modules` check** using the same `loadLoopPlugins` the runtime uses, so a typo
|
|
43
|
+
or a broken module surfaces before a tick silently drops a plugin.
|
|
44
|
+
|
|
45
|
+
## Consequences
|
|
46
|
+
|
|
47
|
+
- No kernel change; `pnpm test:boundaries` stays green.
|
|
48
|
+
- `loop.config.yaml` gains one new block (`plugins: { modules: [] }`), empty by default — zero behavior change for
|
|
49
|
+
every existing config.
|
|
50
|
+
- This directly enables the human-approval gate (`delivery.merge.requireHumanApproval`, ADR follow-up in the same
|
|
51
|
+
release) and the PII-scan wiring as bus consumers, without a bespoke mechanism for each.
|
|
52
|
+
- Explicitly **not** delivered: hooking into the worker's own model/tool loop (structural — the worker is an
|
|
53
|
+
opaque CLI), a distributed/external event bus (webhook, message broker) as a first consumer, and dynamic module
|
|
54
|
+
loading from anywhere other than the local repo.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# ADR-0031: Read-only loop observability
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
The keep-pushing loop already records durable events and exposes doctor, debrief,
|
|
10
|
+
and retro reports, but operators still have to correlate them manually to find
|
|
11
|
+
stalled workers, missing delivery state, or an idle queue. A daemon or dashboard
|
|
12
|
+
would add another runtime to operate.
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
Add a read-only `loop observe` command and a pure `assessObservability` function.
|
|
17
|
+
The collector reuses existing doctor/debrief/event-log data and read-only Orca
|
|
18
|
+
queries. It reports queue, delivery, machine, provider, memory, cache, and token
|
|
19
|
+
metrics, plus deterministic anomaly rules for:
|
|
20
|
+
|
|
21
|
+
- connected terminals with no output;
|
|
22
|
+
- active claims without `delivery.json`;
|
|
23
|
+
- completed worktrees with uncommitted files;
|
|
24
|
+
- ready work with a free slot but no recent dispatch; and
|
|
25
|
+
- in-flight worker/review phases past `delivery.workerIdleTimeoutMin`.
|
|
26
|
+
|
|
27
|
+
`--precheck` provides scheduler-compatible exit semantics (0 actionable, 1
|
|
28
|
+
healthy). The event log remains the source of truth; no new persistence or
|
|
29
|
+
background process is introduced.
|
|
30
|
+
|
|
31
|
+
## Consequences
|
|
32
|
+
|
|
33
|
+
Operators and the existing observer can consume one stable JSON/Markdown report,
|
|
34
|
+
while tests exercise the anomaly rules without Orca or network fixtures. A future
|
|
35
|
+
dashboard or MCP read-only surface can consume the same report without changing
|
|
36
|
+
the loop execution path.
|
package/docs/LOOP.md
CHANGED
|
@@ -25,6 +25,7 @@ ak-harness loop uninstall [--dry-run] # remove them
|
|
|
25
25
|
ak-harness loop status # what Orca knows: enabled, trigger, provider, latest run
|
|
26
26
|
ak-harness loop hook # one status line for a SessionStart hook; never mutates
|
|
27
27
|
ak-harness loop debrief [--issue ENG-123] [--since 24h] # human-facing: what is in flight, held, escalated
|
|
28
|
+
ak-harness loop observe [--since 24h] [--json] # anomaly scan + queue, delivery, machine, memory/cache metrics
|
|
28
29
|
ak-harness loop watch [--issue ENG-123] [--once] [--interval 30] # poll delivery/PR; DONE|FAILED|ACTION_REQUIRED
|
|
29
30
|
ak-harness loop retro [--since 7d] [--json|--learnings] # weekly digest + calibration suggestions
|
|
30
31
|
```
|
|
@@ -47,6 +48,12 @@ so it is safe to run from a SessionStart hook or a chat agent that needs context
|
|
|
47
48
|
|
|
48
49
|
Use `--once` for a single snapshot; omit it to block until a terminal outcome (or `--timeout <seconds>`).
|
|
49
50
|
|
|
51
|
+
`loop observe` is the scheduler-friendly health view. It reuses the doctor, debrief and durable event log, then
|
|
52
|
+
checks for a connected terminal with no output, an active claim without `delivery.json`, a finalized dirty worktree,
|
|
53
|
+
a ready queue with an idle slot, and an in-flight review/worker past `delivery.workerIdleTimeoutMin`. It also reports
|
|
54
|
+
machine pressure, provider headroom, delivery counts, fix rounds, memory recalls, cached contracts and observed token
|
|
55
|
+
fields. It is read-only; `--precheck` uses exit 0 for an actionable anomaly and exit 1 when healthy.
|
|
56
|
+
|
|
50
57
|
## Running 24/7 with Orca
|
|
51
58
|
|
|
52
59
|
`loop install` is guided and rendered with Ink when stdin/stdout are terminals (plain lines otherwise). When no
|
|
@@ -113,6 +120,7 @@ For every issue the loop dispatched (`<stateDir>/issues/<id>/dispatch.json`) and
|
|
|
113
120
|
| No PR, idle ≥ `workerIdleTimeoutMin` | one check-in via `terminal send`; idle again after that → **stuck**: lease released, issue → `returnState` + `blocked`, worktree kept |
|
|
114
121
|
| No PR, terminal gone (> 5 min after dispatch) | **stuck** as above |
|
|
115
122
|
| PR touches `selfEditPaths` | **held**: one PR comment, no review, no merge |
|
|
123
|
+
| PR touches `secretFilePatterns` (`.env`, `*.pem`, `*.key`, `id_rsa`, `credentials.json`, … by default) | **held**: same as `selfEditPaths` — the loop cannot inspect diff content, only filenames, so this holds on the filename shape alone even if the content is innocuous |
|
|
116
124
|
| PR conflicting | rebase instruction to the worker, once per head (does not count as a fix round) |
|
|
117
125
|
| CI red | failing check names to the worker; counts as a fix round |
|
|
118
126
|
| CI pending / required check missing | wait |
|
|
@@ -120,6 +128,7 @@ For every issue the loop dispatched (`<stateDir>/issues/<id>/dispatch.json`) and
|
|
|
120
128
|
| Review findings ≥ floor | findings to the worker; counts as a fix round; same head is never re-reviewed |
|
|
121
129
|
| Fix rounds exhausted (`maxFixRounds`) | **blocked**: Linear comment + label + `returnState`, PR comment, lease released, worktree and PR kept |
|
|
122
130
|
| Review clean, `merge.auto` | `gh api PUT …/merge` with `sha=<reviewed head>` (GitHub refuses if the head moved) → Linear attach + comment + `doneState`, worktree removed when `cleanupWorktree` |
|
|
131
|
+
| Review clean, `merge.requireHumanApproval` set, no GitHub approval yet | **held**: reuses `pr.reviewDecision` already fetched with the PR snapshot — no extra GitHub call; merges automatically as soon as `reviewDecision` becomes `APPROVED` on a later run |
|
|
123
132
|
| PR merged outside the loop | same completion path |
|
|
124
133
|
| PR closed without merge | **abandoned**: lease released, issue → `returnState`, worktree kept |
|
|
125
134
|
|
|
@@ -145,6 +154,40 @@ two differences forced by having no Linear issue and no worker terminal:
|
|
|
145
154
|
PRs it dispatched itself, never one it was only asked to review. If the label is removed on GitHub before the
|
|
146
155
|
loop finishes, it stops tracking the PR the same way (held, no further comments).
|
|
147
156
|
|
|
157
|
+
## Event bus and orchestration hooks
|
|
158
|
+
|
|
159
|
+
`appendLoopEvent` writes every loop event to `<stateDir>/events.ndjson`, but nothing could react to one *while it
|
|
160
|
+
happens*, and there was no deterministic way to say "don't do this" before a consequential action. `plugins.modules`
|
|
161
|
+
(`loop.config.yaml`, empty by default — zero behavior change until configured) lists local `.mjs` files, relative to
|
|
162
|
+
`project.root` (same trust level as `agents.registry.yaml`: files already in this repo, never fetched over a
|
|
163
|
+
network), loaded once at the start of `tick`/`deliver`. Each exports `{ id, apply(bus) }`:
|
|
164
|
+
|
|
165
|
+
```js
|
|
166
|
+
export default {
|
|
167
|
+
id: 'slack-notify',
|
|
168
|
+
apply(bus) {
|
|
169
|
+
bus.on('worker.dispatched', (event) => { /* … */ })
|
|
170
|
+
bus.hook('beforeMerge', (payload) => {
|
|
171
|
+
if (isFrozeWindow()) return { block: true, reason: 'release freeze' }
|
|
172
|
+
})
|
|
173
|
+
},
|
|
174
|
+
}
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
- `bus.on(type | '*', listener)` subscribes to the loop's existing event vocabulary (`contract.failed`,
|
|
178
|
+
`worker.dispatched`, `pr.reviewed`, `provider.cooldown`, `issue.paused`, …) live, in addition to the ndjson log.
|
|
179
|
+
- `bus.hook(name, listener)` subscribes to an **orchestration lifecycle hook**: `beforeDispatch`, `afterDispatch`,
|
|
180
|
+
`beforeReview`, `afterReview`, `beforeMerge`, `afterMerge`, `onPause`, `onEscalate`. A `before*` listener may
|
|
181
|
+
return `{ block: true, reason }` to stop the action (surfaced as a `skipped`/`waiting`/`held` result with the
|
|
182
|
+
reason); every other hook is notification-only. This is deliberately **not** a hook into the worker's own
|
|
183
|
+
model/tool loop — the worker is an opaque external CLI (ADR-0027) and that loop is invisible to us. These hooks
|
|
184
|
+
fire around the orchestration decisions we actually make: dispatch, review, merge.
|
|
185
|
+
- A listener or hook that throws is swallowed (never fatal — a broken plugin must not stop tick or deliver) and,
|
|
186
|
+
for a hook, its error is reported back through `runHook`'s `errors`.
|
|
187
|
+
- `loop doctor` runs a `plugins.modules` check confirming every configured module exists and loads cleanly.
|
|
188
|
+
|
|
189
|
+
See `src/loop/event-bus.ts` for the full API (`createLoopEventBus`, `loadLoopPlugins`).
|
|
190
|
+
|
|
148
191
|
## One tick
|
|
149
192
|
|
|
150
193
|
1. **Intake** — `orca linear list-issues` once per configured state, filtered and ordered locally; issues already
|
|
@@ -191,6 +234,52 @@ tick forever. (The 2026-09-11/12 pilot logged 19 such retries across 4 issues in
|
|
|
191
234
|
Neither mechanism touches the existing `blocked`/`stuck` escalations (fix-round exhaustion, an idle worker with no
|
|
192
235
|
PR) — those already label the issue and route it out of the queue via `linear.excludeLabels`.
|
|
193
236
|
|
|
237
|
+
## PII/secret scanning
|
|
238
|
+
|
|
239
|
+
`security.pii.enabled` (default `false` — enabling it never changes behavior for a project that doesn't need it)
|
|
240
|
+
scans issue text before it enters the orchestrator prompt (`contract.ts`) and the worker brief (`brief.ts`) for
|
|
241
|
+
PII-shaped patterns: emails, common provider API-key prefixes (`sk-…`, `ghp_…`, `AKIA…`, Slack tokens), phone
|
|
242
|
+
numbers, card-number-shaped digit runs (`src/kernel/pii.ts`, pure and dependency-free). `security.pii.action`
|
|
243
|
+
controls what happens on a match:
|
|
244
|
+
|
|
245
|
+
- `redact` (default when enabled): each match is replaced with `[REDACTED:<kind>]` before the text is embedded.
|
|
246
|
+
- `warn`: the text is sent unchanged; a `security.pii-detected` event is still recorded (source `issue-text` or
|
|
247
|
+
`worker-brief`, with the matched kinds and count — never the matched value itself).
|
|
248
|
+
- `block`: the dispatch fails closed instead of ever sending the text anywhere, with a message naming the kinds
|
|
249
|
+
found (not the values). Recorded like any other dispatch failure, so `resilience.maxConsecutiveFailures` still
|
|
250
|
+
applies if it keeps happening.
|
|
251
|
+
|
|
252
|
+
This is a pattern scanner, not a claim of completeness — it catches common shapes, not every possible secret.
|
|
253
|
+
|
|
254
|
+
## Cost/time circuit breakers
|
|
255
|
+
|
|
256
|
+
The loop cannot count a worker CLI's own model or tool calls — it is an opaque process, not a loop we run
|
|
257
|
+
ourselves — so there is no way to cap "cost" the way an in-process agent harness would. Two proxies close most of
|
|
258
|
+
the gap, both unset (disabled) by default so an existing config is unaffected:
|
|
259
|
+
|
|
260
|
+
- **`delivery.maxDispatchMinutes`**: a hard wall-clock ceiling on one dispatch, independent of idle detection.
|
|
261
|
+
`delivery.workerIdleTimeoutMin` only catches a worker that stopped producing output; this catches one that is
|
|
262
|
+
still active but has run far longer than any real task on the project should. Past the ceiling, `deliver` stops
|
|
263
|
+
nudging/reviewing/merging the issue and escalates it exactly like a stuck worker (Linear comment + label +
|
|
264
|
+
`returnState`, worktree preserved for inspection, lease released) — recorded as a `max-duration.tripped` event.
|
|
265
|
+
- **`resilience.maxUsageDeltaPercent`**: a cost proxy from Orca's own usage reporting. The builder's remaining
|
|
266
|
+
usage percent (`RankedModel.remainingPercent`) is snapshotted at dispatch time (`dispatch.json`'s
|
|
267
|
+
`initialRemainingPercent`); every later `deliver` run compares it against that provider's *current* remaining
|
|
268
|
+
usage. If it dropped by at least this many percentage points while the issue was in flight, the dispatch is
|
|
269
|
+
stopped the same way — recorded as `cost-guard.tripped`. This is deliberately usage-delta, not call-count: it is
|
|
270
|
+
the only per-issue cost signal Orca actually reports for an opaque worker CLI.
|
|
271
|
+
|
|
272
|
+
## Dynamic outcome progress
|
|
273
|
+
|
|
274
|
+
The contract's outcome list (`brief.ts`) is a static plan frozen before dispatch — it cannot become a live todo
|
|
275
|
+
list without controlling the worker's own loop, which this harness deliberately does not do (ADR-0027). The brief
|
|
276
|
+
documents a lightweight, optional convention instead: as the worker finishes or starts an outcome, it writes
|
|
277
|
+
`progress.json` at the root of its own worktree, e.g. `{"o1": "done", "o2": "in-progress"}` (ids match the
|
|
278
|
+
outcome list). `loop debrief` reads it back best-effort (`readOutcomeProgress`, `src/loop/progress.ts`) — a
|
|
279
|
+
missing, unreadable, or malformed file is never an error, since nothing enforces the worker keeps it current and
|
|
280
|
+
older dispatches never wrote one at all. When present, it shows as `N/M outcome(s) done` per in-flight issue
|
|
281
|
+
instead of a flat "in flight".
|
|
282
|
+
|
|
194
283
|
## Skills pinned into the worker brief
|
|
195
284
|
|
|
196
285
|
`brief.skills` (default `[]`) lists Markdown files, relative to `project.root`, that every worker brief embeds
|
|
@@ -239,6 +328,8 @@ same shell commands. `project.setup.command` (unset by default; an argv array, e
|
|
|
239
328
|
| `machine.slots` | `sampleMachine` + `adaptiveConcurrency`, free RAM reserve, WSL cap, running worktrees | no — 0 free slots is a warning, not a failure |
|
|
240
329
|
| `linear.queue` | `orca linear list-issues` per configured state, filtered and ordered locally | yes — an unreachable Linear blocks |
|
|
241
330
|
| `brief.skills` | existence + readability of each `brief.skills` path under `project.root` | yes when any are unreadable — dispatch would fail closed anyway |
|
|
331
|
+
| `plugins.modules` | each configured module exists and `import()`s without throwing | yes when any fails to load |
|
|
332
|
+
| `mcp.allowlist` | only runs when `mcp.enabled`; builds the default-deny bridge from `mcp.allowTools` and self-tests it (no live MCP server involved — ADR-0028) | warning on an empty allowlist, failed if the allow/deny wiring itself misbehaves |
|
|
242
333
|
|
|
243
334
|
|
|
244
335
|
## Dynamic model routing
|
|
@@ -263,7 +354,8 @@ its binary is on PATH, its auth is not known to be missing, no Orca usage window
|
|
|
263
354
|
|
|
264
355
|
Providers authenticate through their own CLI login (`claude login`, `codex login`, `grok login`); only providers
|
|
265
356
|
declared `auth: api-key` need an environment variable, and the loop never reads its value. Orca has no per-run model flag; the chosen model is rendered into `providers.<id>.tui` (for example
|
|
266
|
-
`codex -m {model}
|
|
357
|
+
`codex -m {model} -s workspace-write -a never`) and launched in the worker terminal. The explicit
|
|
358
|
+
approval policy keeps YOLO runs non-interactive while the workspace sandbox limits changes to the assigned worktree.
|
|
267
359
|
|
|
268
360
|
When a provider runs out of usage the loop records a cooldown in `<stateDir>/provider-cooldowns.json`:
|
|
269
361
|
`initialMin` doubling up to `maxMin`, never earlier than the reset instant Orca reported.
|
|
@@ -306,7 +398,7 @@ the published dependency tree. `@agentskit/harness` stays dependency-light (`com
|
|
|
306
398
|
| Trigger | `.doc-bridge/index.json` exists under `project.root` |
|
|
307
399
|
| Knob | `contract.maxContextReferences` (default `6`; `0` disables) |
|
|
308
400
|
| Behaviour | Query is `"<issue id> <title>"`. Up to N deterministic references are appended to the orchestrator prompt. Missing or malformed index → **no refs** (loop continues). |
|
|
309
|
-
| Boundary | No `@agentskit/doc-bridge` import; the adapter only reads the local index contract ([ADR-0003](ADR-0003-doc-bridge-context-binding.md)). Index build/refresh stays with Doc Bridge (`pnpm docs:bridge:index` in repos that use it). |
|
|
401
|
+
| Boundary | No `@agentskit/doc-bridge` import; the adapter only reads the local index contract ([ADR-0003](ADR-0003-doc-bridge-context-binding.md)). Contract resolution rejects indexes older than `contract.docBridgeMaxAgeHours`; exact source freshness remains a Doc Bridge gate. Index build/refresh stays with Doc Bridge (`pnpm docs:bridge:index` in repos that use it). |
|
|
310
402
|
|
|
311
403
|
### Code Review (`agentskit-review`)
|
|
312
404
|
|
|
@@ -44,6 +44,7 @@ modules, not a provider dependency.
|
|
|
44
44
|
| `src/kernel/learning.ts` | Kernel | Retrospective parsing and human learning promotion | stdlib, `errors` | None |
|
|
45
45
|
| `src/kernel/status.ts` | Kernel | Deterministic status snapshot and digest | `errors`, `hash`, `block`, `types` (type-only) | None |
|
|
46
46
|
| `src/kernel/model-policy.ts` | Kernel | Role-to-model binding and validation | `errors`, `hash` | Provider is data, not an SDK |
|
|
47
|
+
| `src/kernel/pii.ts` | Kernel | Deterministic PII/secret pattern scan + redaction over a plain string | None | None |
|
|
47
48
|
| `src/execution/machine.ts` | Execution support | Machine sampling and adaptive concurrency | stdlib, `types`, `errors` | Host CPU/memory metrics |
|
|
48
49
|
| `src/execution/coordination.ts` | Execution support | Atomic issue/worktree claims and dispatch ledger | stdlib, `errors`, `hash` | Local state directory only |
|
|
49
50
|
| `src/kernel/resilience.ts` | Kernel | Failure classification and bounded retry/recovery policy | `errors` | Operation callback supplied by caller |
|
|
@@ -90,6 +91,8 @@ modules, not a provider dependency.
|
|
|
90
91
|
| `src/loop/resilience-state.ts` | Composition | Per-issue consecutive-failure counter + pause/resume, and per-stage (`tick`/`deliver`) crash-loop pause/resume | stdlib | `<stateDir>/issues/<id>/failures.json`, `<stateDir>/paused.json` |
|
|
91
92
|
| `src/loop/skills.ts` | Composition | `brief.skills` file loading: sha256 digest, truncation at `maxSkillChars`, fail-closed on a missing file | stdlib (`crypto`) | Local Markdown files under `project.root` |
|
|
92
93
|
| `src/loop/github-intake.ts` | Composition | GitHub label-based intake: discovers open PRs carrying `github.intakeLabel` and tracks them as `pr-<n>` (no Linear issue) | `github-cli` | GitHub via the injected runner; `<stateDir>/issues/pr-<n>/intake.json` |
|
|
94
|
+
| `src/loop/event-bus.ts` | Composition | Local pub/sub over the loop's event vocabulary plus orchestration lifecycle hooks (`beforeDispatch`, `beforeReview`, `beforeMerge`, …); loads local `plugins.modules` files | stdlib (`node:path`, `node:url`, dynamic `import()`) | Local `.mjs` files under `project.root`, never fetched over a network |
|
|
95
|
+
| `src/loop/progress.ts` | Composition | Best-effort read of `progress.json` from a worktree — the worker's optional, dynamic outcome-progress report | stdlib | The worktree filesystem only |
|
|
93
96
|
| `src/loop/tick.ts` | Composition | One keep-pushing tick: intake, admit (slots + dispatch ledger claim), contract, optional worktree setup command, skills pinning, dispatch into an Orca worktree, Linear transition, escalation, per-issue failure tracking, precheck | adapters, `coordination`, `errors`, `brief`, `config`, `contract`, `cooldown`, `doctor`, `resilience-state`, `routing`, `skills`, `slots` | Orca CLI + Linear via runner; `<stateDir>` files |
|
|
94
97
|
| `src/loop/deliver.ts` | Composition | Deliver stage per dispatched issue and per label-intake PR: PR detection, self-edit hold, conflict/CI/review fix rounds via terminal (or PR comment for intake), review at head, optimistic squash-merge (dispatched issues only — intake is review + comment, never merge), Linear Done, cleanup, stuck/abandoned escalation | adapters, `coordination`, `errors`, `config`, `cooldown`, `doctor`, `github-intake`, `routing`, `tick` | Orca, Linear, GitHub, agentskit-review via runner; `<stateDir>` files |
|
|
95
98
|
| `src/loop/install.ts` | Composition | Orca automation specs (`<prefix>-tick`, `<prefix>-deliver`) with read-only prechecks, idempotent create/edit by name, uninstall, status and the SessionStart hook line | `command`, `orca-cli`, `providers`, `errors`, `config`, `cooldown`, `doctor`, `routing` | Orca automations via runner |
|
|
@@ -97,6 +100,7 @@ modules, not a provider dependency.
|
|
|
97
100
|
| `src/loop/local-config.ts` | Composition | Per-machine overlay wizard: Linear team members via Orca, queue owner and machine tuning answers, YAML rendering and reload | `orca-cli`, `config`, `yaml` | Orca via runner; writes `loop.config.local.yaml` |
|
|
98
101
|
| `src/loop/retro.ts` | Composition | Retro digest over `events.ndjson`, per-issue state, cooldowns and Orca runs; rule-based calibration suggestions; Markdown that `parseRetro` can lift into learnings | `orca-cli`, `hash`, `learning`, `config`, `contract`, `cooldown`, `deliver`, `install`, `tick` | Orca via runner (optional); `<stateDir>` files |
|
|
99
102
|
| `src/loop/debrief.ts` | Composition | Human-facing read-only snapshot of what the loop is working on (in-flight, holds, escalations, cooldowns) | `config`, `contract`, `cooldown`, `deliver`, `retro`, `tick` | `<stateDir>` files only |
|
|
103
|
+
| `src/loop/observability.ts` | Composition | Read-only anomaly assessment and operating metrics built from doctor, debrief, Orca terminal/worktree state, and the durable loop event log | `config`, `doctor`, `debrief`, `deliver`, `retro`, `tick`, `coordination`, `orca-cli` | Orca/Git via injected runner; `<stateDir>` files |
|
|
100
104
|
| `src/loop/watch.ts` | Composition | Poll delivery (+ optional live PR) and emit DONE/FAILED/ACTION_REQUIRED/PROGRESS for agents or humans | `command`, `github-cli`, `config`, `deliver`, `tick` | optional `gh` via runner; `<stateDir>` files |
|
|
101
105
|
| `src/loop/ui/components.tsx` | Composition | Ink components: check rows, sections, banner, spinner, select/confirm/text prompts | `ink`, `react`, `doctor` (type-only) | Terminal |
|
|
102
106
|
| `src/loop/ui/terminal.tsx` | Composition | `createRichIO`: Ink-backed IO for TTYs with a plain-text fallback | `ink`, `react`, `components`, `guided-install` (type-only) | Terminal |
|
package/loop.config.example.yaml
CHANGED
|
@@ -29,6 +29,10 @@ linear:
|
|
|
29
29
|
person: my-linear-display-name # the ONLY queue this machine drains
|
|
30
30
|
people: # display name -> Linear user id (orca linear team members …)
|
|
31
31
|
my-linear-display-name: <linear-user-id>
|
|
32
|
+
rotation:
|
|
33
|
+
enabled: false # set true to advance through owners when the current dispatchable queue drains
|
|
34
|
+
owners: [my-linear-display-name] # ordered display names; the first entry is the initial owner
|
|
35
|
+
advanceWhenEmpty: true # never switch while an implementation lease remains
|
|
32
36
|
states: [Todo, Ready] # dispatchable states; anything already started is left alone
|
|
33
37
|
excludeLabels: [blocked, needs-info]
|
|
34
38
|
requireLabels: []
|
|
@@ -80,7 +84,8 @@ models:
|
|
|
80
84
|
bin: codex
|
|
81
85
|
auth: subscription
|
|
82
86
|
envKeys: [OPENAI_API_KEY]
|
|
83
|
-
|
|
87
|
+
# Codex 0.154+ uses explicit approval/sandbox flags; --full-auto is not a valid CLI option.
|
|
88
|
+
tui: "codex -m {model} -s workspace-write -a never"
|
|
84
89
|
headless: [codex, exec, -m, "{model}", -s, read-only, --skip-git-repo-check, "{prompt}"]
|
|
85
90
|
# effortFlag: "-c model_reasoning_effort={effort}"
|
|
86
91
|
opencode:
|
|
@@ -124,10 +129,15 @@ delivery:
|
|
|
124
129
|
auto: true
|
|
125
130
|
method: squash
|
|
126
131
|
requireChecks: true
|
|
132
|
+
requireHumanApproval: false # true: also require a human GitHub review approval before merging, even
|
|
133
|
+
# after a clean loop review + green checks
|
|
127
134
|
maxFixRounds: 2
|
|
128
135
|
workerIdleTimeoutMin: 45
|
|
136
|
+
# maxDispatchMinutes: 240 # hard ceiling on one dispatch regardless of activity; unset = disabled
|
|
129
137
|
handoff: { enabled: true, maxHandoffs: 2, onlyWhenProviderUnavailable: true }
|
|
130
138
|
selfEditPaths: [loop.config.yaml, ".github/**"] # PRs touching these never auto-merge
|
|
139
|
+
# secretFilePatterns: [...] # default already covers .env, *.pem, *.key, id_rsa, credentials.json, etc.
|
|
140
|
+
# PRs touching a matching filename are held, never reviewed or merged
|
|
131
141
|
ignoreChecks: [] # advisory check names that never block
|
|
132
142
|
requiredChecks: [] # empty = every reported check must be green
|
|
133
143
|
cleanupWorktree: true # remove the Orca worktree after merge
|
|
@@ -173,10 +183,23 @@ github:
|
|
|
173
183
|
# null disables intake entirely
|
|
174
184
|
reviewOnly: true # fixed: intake PRs are always review + comment, never auto-merged
|
|
175
185
|
|
|
186
|
+
plugins:
|
|
187
|
+
modules: [] # e.g. [scripts/loop-plugins/notify.mjs] — local .mjs files (relative to
|
|
188
|
+
# project.root, never fetched over the network) loaded once per tick/deliver;
|
|
189
|
+
# each exports { id, apply(bus) } and can subscribe to loop events or block a
|
|
190
|
+
# beforeDispatch/beforeReview/beforeMerge hook — see docs/LOOP.md
|
|
191
|
+
|
|
192
|
+
security:
|
|
193
|
+
pii:
|
|
194
|
+
enabled: false # scan issue text / worker brief for PII-shaped patterns before embedding them
|
|
195
|
+
action: redact # redact | warn | block
|
|
196
|
+
|
|
176
197
|
resilience:
|
|
177
198
|
maxConsecutiveFailures: 3 # pause an issue after this many consecutive contract/dispatch failures
|
|
178
199
|
pausedLabel: loop:paused # applied to the issue in Linear; remove it (or `loop resume <id>`) to retry
|
|
179
200
|
stagePauseAfterRuns: 3 # pause `loop stage tick|deliver` itself after this many consecutive thrown runs
|
|
201
|
+
# maxUsageDeltaPercent: 40 # stop a dispatch if its provider's remaining usage drops by this many
|
|
202
|
+
# points while the issue is in flight; unset = disabled (cost circuit breaker)
|
|
180
203
|
|
|
181
204
|
brief:
|
|
182
205
|
skills: [] # e.g. [AGENTS.md, CLAUDE.md, docs/for-agents/INDEX.md] — paths relative to project.root,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agentskit/harness",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "Portable, evidence-backed development harness for coding agents.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -68,7 +68,7 @@
|
|
|
68
68
|
"test:phase-executor": "pnpm typecheck && vitest run --config vitest.config.ts test/phase-executor.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['phase-executor']}))\"",
|
|
69
69
|
"test:artifacts": "pnpm typecheck && vitest run --config vitest.config.ts test/artifacts.test.ts && node scripts/verify-artifact-cli.mjs",
|
|
70
70
|
"test:adapters": "pnpm typecheck && vitest run --config vitest.config.ts test/adapters.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['adapters']}))\"",
|
|
71
|
-
"test:loop": "pnpm typecheck && vitest run --config vitest.config.ts test/loop.test.ts test/loop-adapters.test.ts test/loop-tick.test.ts test/loop-deliver.test.ts test/loop-install.test.ts test/loop-guided-install.test.ts test/loop-retro.test.ts test/loop-debrief.test.ts test/loop-watch.test.ts test/loop-agent-registry.test.ts test/loop-memory.test.ts test/loop-contract-failure.test.ts test/loop-resilience-state.test.ts test/loop-skills.test.ts test/loop-github-intake.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['loop-config','loop-providers','loop-routing','loop-doctor','loop-adapters','loop-tick','loop-deliver','loop-install','loop-debrief','loop-watch','loop-agent-registry','loop-memory','loop-contract-failure','loop-resilience-state','loop-skills','loop-github-intake']}))\"",
|
|
71
|
+
"test:loop": "pnpm typecheck && vitest run --config vitest.config.ts test/loop.test.ts test/loop-adapters.test.ts test/loop-tick.test.ts test/loop-deliver.test.ts test/loop-install.test.ts test/loop-guided-install.test.ts test/loop-retro.test.ts test/loop-debrief.test.ts test/loop-watch.test.ts test/loop-observability.test.ts test/loop-agent-registry.test.ts test/loop-memory.test.ts test/loop-contract-failure.test.ts test/loop-resilience-state.test.ts test/loop-skills.test.ts test/loop-github-intake.test.ts test/loop-event-bus.test.ts test/loop-progress.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['loop-config','loop-providers','loop-routing','loop-doctor','loop-adapters','loop-tick','loop-deliver','loop-install','loop-debrief','loop-watch','loop-observability','loop-agent-registry','loop-memory','loop-contract-failure','loop-resilience-state','loop-skills','loop-github-intake','loop-event-bus','loop-progress']}))\"",
|
|
72
72
|
"test:review": "pnpm typecheck && vitest run --config vitest.config.ts test/review.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['review-delivery']}))\"",
|
|
73
73
|
"test:quality": "pnpm typecheck && vitest run --config vitest.config.ts test/quality.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['quality-matrix']}))\"",
|
|
74
74
|
"test:eval-battery": "pnpm typecheck && vitest run --config vitest.config.ts test/eval-battery.test.ts test/issues-010-014.eval.test.ts test/optimization.test.ts && pnpm build >/dev/null && node scripts/verify-harness-eval-manifest.mjs && node -e \"console.log(JSON.stringify({status:'passed',criteria:['eval-battery','eval-manifest','eval-coverage']}))\"",
|
package/release/manifest.json
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"type": "agentskit-harness-release-manifest",
|
|
3
3
|
"schemaVersion": 1,
|
|
4
4
|
"package": "@agentskit/harness",
|
|
5
|
-
"version": "0.
|
|
5
|
+
"version": "0.11.0",
|
|
6
6
|
"channel": "latest",
|
|
7
7
|
"sourceRevision": "b8cfe596e5f30f2c373b89f00f271079b296feaf",
|
|
8
8
|
"requiredChecks": [
|
|
@@ -32,5 +32,5 @@
|
|
|
32
32
|
"pilot-benchmark"
|
|
33
33
|
],
|
|
34
34
|
"nextBaseline": "benchmarks/harness-0.4.0-baseline.json",
|
|
35
|
-
"digest": "
|
|
35
|
+
"digest": "9fd3e219da299dab39477062cb3c17296e18cc6b707bceb42436cdb75afd281b"
|
|
36
36
|
}
|
package/release/notes.md
CHANGED
|
@@ -1,3 +1,9 @@
|
|
|
1
|
+
# 0.11.0 release candidate
|
|
2
|
+
|
|
3
|
+
Read-only loop observability for the keep-pushing SDLC loop: deterministic anomaly detection and operating metrics
|
|
4
|
+
for queue, delivery, machine pressure, provider headroom, memory, cache, reviews, fix rounds, lead time, and
|
|
5
|
+
observed token fields. Publication remains gated on a merge to `main` through npm Trusted Publishing.
|
|
6
|
+
|
|
1
7
|
# 0.8.0 release candidate
|
|
2
8
|
|
|
3
9
|
Worker handoff: continue in-flight tickets on the same Orca worktree/branch with another provider when usage runs out.
|