@agentskit/harness 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +40 -0
- package/capabilities/public-surface.json +187 -76
- package/dist/cli.js +1171 -151
- package/dist/cli.js.map +1 -1
- package/dist/index.d.ts +342 -7
- package/dist/index.js +1002 -81
- package/dist/index.js.map +1 -1
- package/docs/ADR-0029-loop-resilience-pinning-intake.md +68 -0
- package/docs/ADR-0030-loop-event-bus-orchestration-hooks.md +54 -0
- package/docs/LOOP.md +185 -2
- package/docs/MODULE-BOUNDARIES.md +9 -3
- package/loop.config.example.yaml +54 -1
- package/package.json +2 -2
- package/release/manifest.json +2 -2
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# ADR-0029: Loop resilience, brief pinning, worktree setup, effort routing, and GitHub intake
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-09-12
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
The 2026-09-11/12 pilot ran the keep-pushing loop (ADR-0027) unattended for ~7 hours and surfaced two production
|
|
9
|
+
defects and, separately, a review of `ColeMurray/background-agents` (an open-source clone of Ramp's Inspect
|
|
10
|
+
background-agent system) suggested transferable ideas already partially covered by this harness's own mechanisms.
|
|
11
|
+
|
|
12
|
+
Measured from `events.ndjson`: `classifyProviderFailure` did not recognise real Claude/Codex usage-limit phrasing
|
|
13
|
+
("You've hit your session limit", "usage limit reached") or Grok's ("Overloaded", "temporarily limiting requests"),
|
|
14
|
+
so every such failure fell through to `other` and `onProviderFailure` never fired — no cooldown was ever recorded.
|
|
15
|
+
The same gap existed on the review path (`agentskit-review` exit classified without checking the same patterns).
|
|
16
|
+
Result: 19 unclassified `contract.failed` retries across 4 issues and 12 incomplete reviews over 7h, with no cap
|
|
17
|
+
anywhere — nothing counted consecutive failures on an issue or a stage, so a single misclassified error retried
|
|
18
|
+
forever.
|
|
19
|
+
|
|
20
|
+
## Decision
|
|
21
|
+
|
|
22
|
+
1. **Failure classification** (bug fix, no schema change): `classifyProviderFailure` (`src/loop/contract.ts`) checks
|
|
23
|
+
a `QUOTA_PATTERN` covering the phrasing above before falling back to the kernel's generic `classifyFailure`, and
|
|
24
|
+
`extractResetsAt` parses a relative (`resets in 3h`) or clock-time (`resets 10:40pm`) reset out of the message.
|
|
25
|
+
`deliver.ts`'s review-incomplete path applies the same classification and marks the **reviewer's own provider id**
|
|
26
|
+
(`ctx.reviewer.provider`, e.g. `codex`) cooling down — not the review-CLI transport id (`review.provider`, e.g.
|
|
27
|
+
`codex-cli`), which `rankModels`/`detectProviders` never look up. This was the second, related pilot bug.
|
|
28
|
+
2. **Per-issue and per-stage auto-pause** (new `src/loop/resilience-state.ts`, Composition, no kernel change):
|
|
29
|
+
`resilience.maxConsecutiveFailures` (default 3) pauses a single issue — one deduplicated Linear comment, the
|
|
30
|
+
`resilience.pausedLabel` — after that many consecutive `contract.failed`/`worker.dispatch-failed` events; a
|
|
31
|
+
successful dispatch clears the counter. `resilience.stagePauseAfterRuns` does the same for a `loop stage
|
|
32
|
+
tick|deliver` run that *throws* (not a normal idle/ok/blocked report) repeatedly. Both are separate from, and do
|
|
33
|
+
not replace, the existing `blocked`/`stuck` escalation via `linear.excludeLabels` (ADR-0027 §6) — those already
|
|
34
|
+
self-exclude a terminal outcome; this closes the two pre-dispatch paths that had no ceiling at all.
|
|
35
|
+
3. **Skills pinned into the worker brief** (new `src/loop/skills.ts`): `brief.skills` lists Markdown files, read
|
|
36
|
+
once at dispatch time, sha256-digested, truncated at `brief.maxSkillChars` with a visible note, and embedded in
|
|
37
|
+
a new brief section. A missing file fails the dispatch closed. This is deliberately a separate mechanism from
|
|
38
|
+
the existing Doc Bridge `contract.briefScopes` (path/title pointers resolved per issue) — skills are full pinned
|
|
39
|
+
content with a cryptographic digest recorded in `dispatch.json`, and a handoff never re-reads them, so editing a
|
|
40
|
+
skill file after dispatch cannot affect an in-flight worker.
|
|
41
|
+
4. **Worktree setup command**: `project.setup.command` (argv, no shell) runs once between `orca worktree create`
|
|
42
|
+
and opening the worker terminal — e.g. `pnpm install --frozen-lockfile` — bounded by `project.setup.timeoutSec`
|
|
43
|
+
(reserved out of the tick time budget so it cannot itself blow the budget). `project.setup.required` (default
|
|
44
|
+
true) routes a failing/timing-out setup through the same worktree-cleanup and consecutive-failure path as any
|
|
45
|
+
other dispatch failure.
|
|
46
|
+
5. **Reasoning effort per role**: `models.effort.<role>` is rendered into `tui`/`headless` only for a provider that
|
|
47
|
+
declares `providers.<id>.effortFlag`; a provider without one silently ignores it, so the feature is opt-in per
|
|
48
|
+
provider and safe to default-enable. `agentskit-review` has no such flag, so `models.effort.reviewer` is
|
|
49
|
+
recorded (for `loop retro` grouping and future use) without being wired into the review CLI call.
|
|
50
|
+
6. **GitHub label intake** (new `src/loop/github-intake.ts`): `github.intakeLabel` lets a human ask the loop to
|
|
51
|
+
review a PR it never dispatched (tracked as `pr-<n>`, no Linear issue). It reuses the same checks/review/fix-round
|
|
52
|
+
decisions as a normal dispatch, but every nudge becomes a PR comment (no worker terminal exists) and
|
|
53
|
+
`github.reviewOnly` is pinned `true` in the schema — this loop merges only PRs it dispatched itself, never one it
|
|
54
|
+
was only asked to review, however clean that review comes back.
|
|
55
|
+
|
|
56
|
+
## Consequences
|
|
57
|
+
|
|
58
|
+
- No kernel change; `pnpm test:boundaries` stays green. All six changes are Composition (`src/loop/*`) or a small
|
|
59
|
+
Adapter addition (`githubOpenPullRequests` gains `label?`, plus `githubLabelRemove`, in `src/adapters/github-cli.ts`).
|
|
60
|
+
- `loop.config.yaml` gains six new top-level/nested blocks (`resilience`, `brief`, `project.setup`, `models.effort`,
|
|
61
|
+
`providers.<id>.effortFlag`, `github`), all with safe defaults (`.prefault({})`) so an existing 0.8.0 config keeps
|
|
62
|
+
its exact prior behaviour unless a project opts in.
|
|
63
|
+
- New CLI surface: `ak-harness loop resume [issue] [--stage tick|deliver]` and `ak-harness loop paused`.
|
|
64
|
+
- `loop doctor` gained a `brief.skills` check; `loop retro`'s `dispatches.byProvider` now groups by
|
|
65
|
+
`provider/model@effort` when an effort was recorded.
|
|
66
|
+
- Explicitly out of scope (from `background-agents`): a remote control plane, sandboxed/remote worktrees
|
|
67
|
+
(Modal/E2B/Daytona), prebuilds/snapshots, a multiplayer web UI, and Slack/GitHub-App bots. This harness's runtime
|
|
68
|
+
stays Orca + local worktrees; none of the above changes that.
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# ADR-0030: Loop event bus and orchestration hooks, not model-loop middleware
|
|
2
|
+
|
|
3
|
+
- Status: Accepted
|
|
4
|
+
- Date: 2026-09-12
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Comparing the loop against frameworks that build a "custom agent harness" in-process (e.g. LangChain's
|
|
9
|
+
`create_agent` + middleware: hooks before/after every model call and every tool call) surfaced a real gap: this
|
|
10
|
+
harness had no deterministic extension point at all. `appendLoopEvent` (`src/loop/tick.ts`) only appends to
|
|
11
|
+
`<stateDir>/events.ndjson` — nothing can react in real time, and nothing can say "stop, don't do this" before a
|
|
12
|
+
consequential action.
|
|
13
|
+
|
|
14
|
+
The gap is real, but the shape of the fix is not "add middleware like LangChain's": this harness orchestrates
|
|
15
|
+
external CLI agents (claude/codex/grok) as opaque processes inside an Orca worktree. There is no in-process
|
|
16
|
+
model↔tool loop to hook into — the worker's own context window, its own tool calls, are invisible to us by design
|
|
17
|
+
(ADR-0027). Building a `before model call` hook here is not possible without controlling that loop, which is
|
|
18
|
+
explicitly out of scope.
|
|
19
|
+
|
|
20
|
+
What we *do* control, end to end, is the orchestration around that opaque worker: whether we dispatch it, whether
|
|
21
|
+
we act on its review, whether we merge its PR. That is the level at which a deterministic extension point is both
|
|
22
|
+
useful and achievable.
|
|
23
|
+
|
|
24
|
+
## Decision
|
|
25
|
+
|
|
26
|
+
1. **`src/loop/event-bus.ts`** (Composition): a local, in-process pub/sub over the loop's existing free-form event
|
|
27
|
+
vocabulary (`contract.failed`, `worker.dispatched`, `pr.reviewed`, …) plus a small, fixed set of **orchestration
|
|
28
|
+
lifecycle hooks**: `beforeDispatch`, `afterDispatch`, `beforeReview`, `afterReview`, `beforeMerge`, `afterMerge`,
|
|
29
|
+
`onPause`, `onEscalate`. A `before*` hook may return `{ block: true, reason }` to stop the action; every other
|
|
30
|
+
hook is notification-only. A listener or hook that throws is swallowed (logged as a note, never fatal) — a
|
|
31
|
+
broken local plugin must not take down tick or deliver.
|
|
32
|
+
2. **Not `kernel/plugins.ts`.** The kernel's plugin registry is tied to `HARNESS_EVENT_TYPES`, the harness's own
|
|
33
|
+
run-lifecycle vocabulary, and carries plugin ids/versions/dependency ordering appropriate for a shared kernel
|
|
34
|
+
contract. The loop's event vocabulary is an open set of strings owned by composition, not the kernel, so
|
|
35
|
+
`event-bus.ts` is a separate, simpler primitive with the same "who's listening" ergonomics rather than a reuse
|
|
36
|
+
of that contract.
|
|
37
|
+
3. **`plugins.modules`** (`loop.config.yaml`): local `.mjs` files, relative to `project.root`, loaded once at the
|
|
38
|
+
start of `tick`/`deliver`. Same trust level as `agents.registry.yaml` today — files already checked into the
|
|
39
|
+
project's own repo, never fetched over a network. Each module exports `{ id, apply(bus) }`.
|
|
40
|
+
4. **`appendLoopEvent` gains an optional third `bus` parameter.** Every existing call site (`retro.ts`, tests) is
|
|
41
|
+
unaffected; `tick.ts`/`deliver.ts` pass their bus so every event written to disk is also emitted live.
|
|
42
|
+
5. **`loop doctor` gains a `plugins.modules` check** using the same `loadLoopPlugins` the runtime uses, so a typo
|
|
43
|
+
or a broken module surfaces before a tick silently drops a plugin.
|
|
44
|
+
|
|
45
|
+
## Consequences
|
|
46
|
+
|
|
47
|
+
- No kernel change; `pnpm test:boundaries` stays green.
|
|
48
|
+
- `loop.config.yaml` gains one new block (`plugins: { modules: [] }`), empty by default — zero behavior change for
|
|
49
|
+
every existing config.
|
|
50
|
+
- This directly enables the human-approval gate (`delivery.merge.requireHumanApproval`, ADR follow-up in the same
|
|
51
|
+
release) and the PII-scan wiring as bus consumers, without a bespoke mechanism for each.
|
|
52
|
+
- Explicitly **not** delivered: hooking into the worker's own model/tool loop (structural — the worker is an
|
|
53
|
+
opaque CLI), a distributed/external event bus (webhook, message broker) as a first consumer, and dynamic module
|
|
54
|
+
loading from anywhere other than the local repo.
|
package/docs/LOOP.md
CHANGED
|
@@ -113,6 +113,7 @@ For every issue the loop dispatched (`<stateDir>/issues/<id>/dispatch.json`) and
|
|
|
113
113
|
| No PR, idle ≥ `workerIdleTimeoutMin` | one check-in via `terminal send`; idle again after that → **stuck**: lease released, issue → `returnState` + `blocked`, worktree kept |
|
|
114
114
|
| No PR, terminal gone (> 5 min after dispatch) | **stuck** as above |
|
|
115
115
|
| PR touches `selfEditPaths` | **held**: one PR comment, no review, no merge |
|
|
116
|
+
| PR touches `secretFilePatterns` (`.env`, `*.pem`, `*.key`, `id_rsa`, `credentials.json`, … by default) | **held**: same as `selfEditPaths` — the loop cannot inspect diff content, only filenames, so this holds on the filename shape alone even if the content is innocuous |
|
|
116
117
|
| PR conflicting | rebase instruction to the worker, once per head (does not count as a fix round) |
|
|
117
118
|
| CI red | failing check names to the worker; counts as a fix round |
|
|
118
119
|
| CI pending / required check missing | wait |
|
|
@@ -120,12 +121,66 @@ For every issue the loop dispatched (`<stateDir>/issues/<id>/dispatch.json`) and
|
|
|
120
121
|
| Review findings ≥ floor | findings to the worker; counts as a fix round; same head is never re-reviewed |
|
|
121
122
|
| Fix rounds exhausted (`maxFixRounds`) | **blocked**: Linear comment + label + `returnState`, PR comment, lease released, worktree and PR kept |
|
|
122
123
|
| Review clean, `merge.auto` | `gh api PUT …/merge` with `sha=<reviewed head>` (GitHub refuses if the head moved) → Linear attach + comment + `doneState`, worktree removed when `cleanupWorktree` |
|
|
124
|
+
| Review clean, `merge.requireHumanApproval` set, no GitHub approval yet | **held**: reuses `pr.reviewDecision` already fetched with the PR snapshot — no extra GitHub call; merges automatically as soon as `reviewDecision` becomes `APPROVED` on a later run |
|
|
123
125
|
| PR merged outside the loop | same completion path |
|
|
124
126
|
| PR closed without merge | **abandoned**: lease released, issue → `returnState`, worktree kept |
|
|
125
127
|
|
|
126
128
|
State lives in `<stateDir>/issues/<id>/delivery.json` (reviews per head, fix rounds, nudges) and every decision is
|
|
127
129
|
appended to `<stateDir>/events.ndjson`. `--dry-run` reports the decision for each issue without touching anything.
|
|
128
130
|
|
|
131
|
+
## GitHub label intake: reviewing PRs the loop never dispatched
|
|
132
|
+
|
|
133
|
+
The loop's normal queue is Linear issues; `github.intakeLabel` (default `loop:review`, set to `null` to disable)
|
|
134
|
+
lets a human ask it to review a PR it had nothing to do with — a contributor's PR, a manual branch, anything —
|
|
135
|
+
without filing a Linear issue for it. Every `deliver` run lists open PRs carrying the label
|
|
136
|
+
(`gh pr list --label <intakeLabel>`) and starts tracking any not seen before as `pr-<n>` under
|
|
137
|
+
`<stateDir>/issues/pr-<n>/intake.json` (`{ pr, headRef, source: 'github-label', addedAt }`); tracking is
|
|
138
|
+
idempotent, so discovery never re-adds a PR it already knows about.
|
|
139
|
+
|
|
140
|
+
An intake PR runs the same checks → review → fix-round decisions as a normal dispatch (see the table above), with
|
|
141
|
+
two differences forced by having no Linear issue and no worker terminal:
|
|
142
|
+
|
|
143
|
+
- Every nudge (conflict, red CI, review findings) is posted as a **PR comment** instead of sent to a worker
|
|
144
|
+
terminal — there is no worker to nudge.
|
|
145
|
+
- `github.reviewOnly` is a fixed guarantee, not a knob (the schema pins it to `true`): a clean review always ends
|
|
146
|
+
in **held**, commented as "merge is human", the label removed, and `finishedAt` recorded — this loop merges only
|
|
147
|
+
PRs it dispatched itself, never one it was only asked to review. If the label is removed on GitHub before the
|
|
148
|
+
loop finishes, it stops tracking the PR the same way (held, no further comments).
|
|
149
|
+
|
|
150
|
+
## Event bus and orchestration hooks
|
|
151
|
+
|
|
152
|
+
`appendLoopEvent` writes every loop event to `<stateDir>/events.ndjson`, but nothing could react to one *while it
|
|
153
|
+
happens*, and there was no deterministic way to say "don't do this" before a consequential action. `plugins.modules`
|
|
154
|
+
(`loop.config.yaml`, empty by default — zero behavior change until configured) lists local `.mjs` files, relative to
|
|
155
|
+
`project.root` (same trust level as `agents.registry.yaml`: files already in this repo, never fetched over a
|
|
156
|
+
network), loaded once at the start of `tick`/`deliver`. Each exports `{ id, apply(bus) }`:
|
|
157
|
+
|
|
158
|
+
```js
|
|
159
|
+
export default {
|
|
160
|
+
id: 'slack-notify',
|
|
161
|
+
apply(bus) {
|
|
162
|
+
bus.on('worker.dispatched', (event) => { /* … */ })
|
|
163
|
+
bus.hook('beforeMerge', (payload) => {
|
|
164
|
+
if (isFrozeWindow()) return { block: true, reason: 'release freeze' }
|
|
165
|
+
})
|
|
166
|
+
},
|
|
167
|
+
}
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
- `bus.on(type | '*', listener)` subscribes to the loop's existing event vocabulary (`contract.failed`,
|
|
171
|
+
`worker.dispatched`, `pr.reviewed`, `provider.cooldown`, `issue.paused`, …) live, in addition to the ndjson log.
|
|
172
|
+
- `bus.hook(name, listener)` subscribes to an **orchestration lifecycle hook**: `beforeDispatch`, `afterDispatch`,
|
|
173
|
+
`beforeReview`, `afterReview`, `beforeMerge`, `afterMerge`, `onPause`, `onEscalate`. A `before*` listener may
|
|
174
|
+
return `{ block: true, reason }` to stop the action (surfaced as a `skipped`/`waiting`/`held` result with the
|
|
175
|
+
reason); every other hook is notification-only. This is deliberately **not** a hook into the worker's own
|
|
176
|
+
model/tool loop — the worker is an opaque external CLI (ADR-0027) and that loop is invisible to us. These hooks
|
|
177
|
+
fire around the orchestration decisions we actually make: dispatch, review, merge.
|
|
178
|
+
- A listener or hook that throws is swallowed (never fatal — a broken plugin must not stop tick or deliver) and,
|
|
179
|
+
for a hook, its error is reported back through `runHook`'s `errors`.
|
|
180
|
+
- `loop doctor` runs a `plugins.modules` check confirming every configured module exists and loads cleanly.
|
|
181
|
+
|
|
182
|
+
See `src/loop/event-bus.ts` for the full API (`createLoopEventBus`, `loadLoopPlugins`).
|
|
183
|
+
|
|
129
184
|
## One tick
|
|
130
185
|
|
|
131
186
|
1. **Intake** — `orca linear list-issues` once per configured state, filtered and ordered locally; issues already
|
|
@@ -148,6 +203,114 @@ terminal handle, provider/model and lease for the deliver stage.
|
|
|
148
203
|
|
|
149
204
|
Start from [`loop.config.example.yaml`](../loop.config.example.yaml) at the package root.
|
|
150
205
|
|
|
206
|
+
## Resilience: auto-pause after repeated failures
|
|
207
|
+
|
|
208
|
+
Two failure paths have no natural ceiling elsewhere in the pipeline — contract generation failing on every
|
|
209
|
+
candidate, and a worktree/worker dispatch failing outright — because the issue never gets a worktree, a lease that
|
|
210
|
+
would otherwise expire, or a label that would exclude it from the queue. Left alone, a single misclassified or
|
|
211
|
+
persistent error (a quota message the classifier didn't recognise, a broken `orca worktree create`) retries every
|
|
212
|
+
tick forever. (The 2026-09-11/12 pilot logged 19 such retries across 4 issues in 7h before this existed.)
|
|
213
|
+
|
|
214
|
+
- `resilience.maxConsecutiveFailures` (default 3): after this many **consecutive** `contract.failed` or
|
|
215
|
+
`worker.dispatch-failed` events on the *same* issue, the loop stops retrying it: one deduplicated Linear comment
|
|
216
|
+
explaining why, the `resilience.pausedLabel` (default `loop:paused`), and the issue is skipped locally on every
|
|
217
|
+
later tick regardless of whether that label is in `linear.excludeLabels`. A successful dispatch clears the
|
|
218
|
+
counter. State lives in `<stateDir>/issues/<id>/failures.json` (`loop paused` lists every paused issue).
|
|
219
|
+
- **Resuming** an issue: remove the `loop:paused` label on Linear (the next tick notices via `list-issues` and
|
|
220
|
+
clears the local state itself) or run `ak-harness loop resume <issue>`, which also best-effort removes the label.
|
|
221
|
+
- `resilience.stagePauseAfterRuns` (default 3): a scheduled `loop stage tick|deliver` run that *throws* (a config or
|
|
222
|
+
adapter crash, not a normal idle/ok/blocked report) this many times in a row pauses that stage — `loop stage`
|
|
223
|
+
then short-circuits to a `{"status":"paused", ...}` report instead of running, so a crash loop cannot spend budget
|
|
224
|
+
or provider usage under Orca. `ak-harness loop resume --stage tick|deliver` clears it; a single successful run
|
|
225
|
+
clears it automatically. State lives in `<stateDir>/paused.json`.
|
|
226
|
+
|
|
227
|
+
Neither mechanism touches the existing `blocked`/`stuck` escalations (fix-round exhaustion, an idle worker with no
|
|
228
|
+
PR) — those already label the issue and route it out of the queue via `linear.excludeLabels`.
|
|
229
|
+
|
|
230
|
+
## PII/secret scanning
|
|
231
|
+
|
|
232
|
+
`security.pii.enabled` (default `false` — enabling it never changes behavior for a project that doesn't need it)
|
|
233
|
+
scans issue text before it enters the orchestrator prompt (`contract.ts`) and the worker brief (`brief.ts`) for
|
|
234
|
+
PII-shaped patterns: emails, common provider API-key prefixes (`sk-…`, `ghp_…`, `AKIA…`, Slack tokens), phone
|
|
235
|
+
numbers, card-number-shaped digit runs (`src/kernel/pii.ts`, pure and dependency-free). `security.pii.action`
|
|
236
|
+
controls what happens on a match:
|
|
237
|
+
|
|
238
|
+
- `redact` (default when enabled): each match is replaced with `[REDACTED:<kind>]` before the text is embedded.
|
|
239
|
+
- `warn`: the text is sent unchanged; a `security.pii-detected` event is still recorded (source `issue-text` or
|
|
240
|
+
`worker-brief`, with the matched kinds and count — never the matched value itself).
|
|
241
|
+
- `block`: the dispatch fails closed instead of ever sending the text anywhere, with a message naming the kinds
|
|
242
|
+
found (not the values). Recorded like any other dispatch failure, so `resilience.maxConsecutiveFailures` still
|
|
243
|
+
applies if it keeps happening.
|
|
244
|
+
|
|
245
|
+
This is a pattern scanner, not a claim of completeness — it catches common shapes, not every possible secret.
|
|
246
|
+
|
|
247
|
+
## Cost/time circuit breakers
|
|
248
|
+
|
|
249
|
+
The loop cannot count a worker CLI's own model or tool calls — it is an opaque process, not a loop we run
|
|
250
|
+
ourselves — so there is no way to cap "cost" the way an in-process agent harness would. Two proxies close most of
|
|
251
|
+
the gap, both unset (disabled) by default so an existing config is unaffected:
|
|
252
|
+
|
|
253
|
+
- **`delivery.maxDispatchMinutes`**: a hard wall-clock ceiling on one dispatch, independent of idle detection.
|
|
254
|
+
`delivery.workerIdleTimeoutMin` only catches a worker that stopped producing output; this catches one that is
|
|
255
|
+
still active but has run far longer than any real task on the project should. Past the ceiling, `deliver` stops
|
|
256
|
+
nudging/reviewing/merging the issue and escalates it exactly like a stuck worker (Linear comment + label +
|
|
257
|
+
`returnState`, worktree preserved for inspection, lease released) — recorded as a `max-duration.tripped` event.
|
|
258
|
+
- **`resilience.maxUsageDeltaPercent`**: a cost proxy from Orca's own usage reporting. The builder's remaining
|
|
259
|
+
usage percent (`RankedModel.remainingPercent`) is snapshotted at dispatch time (`dispatch.json`'s
|
|
260
|
+
`initialRemainingPercent`); every later `deliver` run compares it against that provider's *current* remaining
|
|
261
|
+
usage. If it dropped by at least this many percentage points while the issue was in flight, the dispatch is
|
|
262
|
+
stopped the same way — recorded as `cost-guard.tripped`. This is deliberately usage-delta, not call-count: it is
|
|
263
|
+
the only per-issue cost signal Orca actually reports for an opaque worker CLI.
|
|
264
|
+
|
|
265
|
+
## Dynamic outcome progress
|
|
266
|
+
|
|
267
|
+
The contract's outcome list (`brief.ts`) is a static plan frozen before dispatch — it cannot become a live todo
|
|
268
|
+
list without controlling the worker's own loop, which this harness deliberately does not do (ADR-0027). The brief
|
|
269
|
+
documents a lightweight, optional convention instead: as the worker finishes or starts an outcome, it writes
|
|
270
|
+
`progress.json` at the root of its own worktree, e.g. `{"o1": "done", "o2": "in-progress"}` (ids match the
|
|
271
|
+
outcome list). `loop debrief` reads it back best-effort (`readOutcomeProgress`, `src/loop/progress.ts`) — a
|
|
272
|
+
missing, unreadable, or malformed file is never an error, since nothing enforces the worker keeps it current and
|
|
273
|
+
older dispatches never wrote one at all. When present, it shows as `N/M outcome(s) done` per in-flight issue
|
|
274
|
+
instead of a flat "in flight".
|
|
275
|
+
|
|
276
|
+
## Skills pinned into the worker brief
|
|
277
|
+
|
|
278
|
+
`brief.skills` (default `[]`) lists Markdown files, relative to `project.root`, that every worker brief embeds
|
|
279
|
+
verbatim under a `## Skills (pinned)` section — house conventions the orchestrator's contract can reference but a
|
|
280
|
+
worker starting cold has no other way to see (e.g. `AGENTS.md`, `CLAUDE.md`, `docs/for-agents/INDEX.md`).
|
|
281
|
+
|
|
282
|
+
- Reading and hashing happens once, at dispatch time (`loadPinnedSkills`, `src/loop/skills.ts`): each file is
|
|
283
|
+
sha256-digested and truncated at `brief.maxSkillChars` (default 6000) with a visible `[truncated N chars]` note so
|
|
284
|
+
one large file cannot exhaust the brief budget. A configured path that does not exist or cannot be read **fails
|
|
285
|
+
the dispatch** (fail-closed) rather than silently sending a worker without guidance it was told it would have —
|
|
286
|
+
the same worktree-cleanup and consecutive-failure accounting as any other dispatch failure applies.
|
|
287
|
+
- The rendered brief is persisted to `<stateDir>/issues/<id>/brief.md`, and `dispatch.json` records `briefDigest`
|
|
288
|
+
(hash of the full brief) plus `skills: [{path, digest}]` — enough to prove after the fact exactly which revision
|
|
289
|
+
of a skill file a given worker saw.
|
|
290
|
+
- **Pinning is by design, not by accident:** a handoff (`renderHandoffBrief`) reuses the worktree/branch state, not
|
|
291
|
+
the original brief, and never re-reads `brief.skills` — so editing a skill file after dispatch affects only
|
|
292
|
+
*future* dispatches, never a worker (or its handoff) already in flight.
|
|
293
|
+
- `loop doctor` runs a `brief.skills` check confirming every configured file currently exists and is readable, so a
|
|
294
|
+
typo or a moved file surfaces before the next dispatch fails.
|
|
295
|
+
|
|
296
|
+
## Worktree setup command
|
|
297
|
+
|
|
298
|
+
A freshly created Orca worktree is a bare checkout — no `node_modules`, no build output, nothing a worker can run
|
|
299
|
+
tests against until it installs dependencies itself, wasting the first several minutes of every dispatch on the
|
|
300
|
+
same shell commands. `project.setup.command` (unset by default; an argv array, e.g.
|
|
301
|
+
`[pnpm, install, --frozen-lockfile]` — no shell, so no `&&`/`|`) runs once in the new worktree between
|
|
302
|
+
`orca worktree create` and opening the worker's terminal.
|
|
303
|
+
|
|
304
|
+
- `project.setup.timeoutSec` (default 600) bounds the run; the loop's per-tick time budget already reserves this
|
|
305
|
+
much time before attempting a dispatch, so a configured setup command cannot itself blow the tick budget.
|
|
306
|
+
- `project.setup.required` (default `true`): a non-zero exit or a timeout removes the just-created worktree, never
|
|
307
|
+
opens a terminal, and fails the dispatch — recorded as a `worker.dispatch-failed` event and counted by the
|
|
308
|
+
per-issue consecutive-failure tracker above, exactly like a contract or worktree-create failure. Set it to
|
|
309
|
+
`false` to have a failing setup only log a note and still hand the worker its terminal.
|
|
310
|
+
- Every run (pass or fail) is recorded as a `worker.setup` event and, when the dispatch succeeds, as `setup:
|
|
311
|
+
{command, exitCode, durationMs, timedOut}` on `dispatch.json` — enough to see in `loop retro` whether a slow or
|
|
312
|
+
flaky setup command is costing more dispatches than it saves.
|
|
313
|
+
|
|
151
314
|
## What the doctor checks
|
|
152
315
|
|
|
153
316
|
| Check | Source | Blocking |
|
|
@@ -157,6 +320,9 @@ Start from [`loop.config.example.yaml`](../loop.config.example.yaml) at the pack
|
|
|
157
320
|
| `routing.<role>` | tiers from `models.<role>` filtered by provider availability | yes — a role with no available provider blocks |
|
|
158
321
|
| `machine.slots` | `sampleMachine` + `adaptiveConcurrency`, free RAM reserve, WSL cap, running worktrees | no — 0 free slots is a warning, not a failure |
|
|
159
322
|
| `linear.queue` | `orca linear list-issues` per configured state, filtered and ordered locally | yes — an unreachable Linear blocks |
|
|
323
|
+
| `brief.skills` | existence + readability of each `brief.skills` path under `project.root` | yes when any are unreadable — dispatch would fail closed anyway |
|
|
324
|
+
| `plugins.modules` | each configured module exists and `import()`s without throwing | yes when any fails to load |
|
|
325
|
+
| `mcp.allowlist` | only runs when `mcp.enabled`; builds the default-deny bridge from `mcp.allowTools` and self-tests it (no live MCP server involved — ADR-0028) | warning on an empty allowlist, failed if the allow/deny wiring itself misbehaves |
|
|
160
326
|
|
|
161
327
|
|
|
162
328
|
## Dynamic model routing
|
|
@@ -181,11 +347,28 @@ its binary is on PATH, its auth is not known to be missing, no Orca usage window
|
|
|
181
347
|
|
|
182
348
|
Providers authenticate through their own CLI login (`claude login`, `codex login`, `grok login`); only providers
|
|
183
349
|
declared `auth: api-key` need an environment variable, and the loop never reads its value. Orca has no per-run model flag; the chosen model is rendered into `providers.<id>.tui` (for example
|
|
184
|
-
`codex -m {model}
|
|
350
|
+
`codex -m {model} -s workspace-write -a never`) and launched in the worker terminal. The explicit
|
|
351
|
+
approval policy keeps YOLO runs non-interactive while the workspace sandbox limits changes to the assigned worktree.
|
|
185
352
|
|
|
186
353
|
When a provider runs out of usage the loop records a cooldown in `<stateDir>/provider-cooldowns.json`:
|
|
187
354
|
`initialMin` doubling up to `maxMin`, never earlier than the reset instant Orca reported.
|
|
188
355
|
|
|
356
|
+
### Reasoning effort per role
|
|
357
|
+
|
|
358
|
+
`models.effort.<role>` (`low | medium | high | xhigh`; defaults: orchestrator/reviewer `high`, builder `medium`,
|
|
359
|
+
watcher `low`) is only applied for a provider that declares `providers.<id>.effortFlag` — a template such as
|
|
360
|
+
`-c model_reasoning_effort={effort}` (codex) or `--reasoning-effort {effort}` (grok); a provider without one
|
|
361
|
+
ignores it entirely, so leaving `effort` at its default is always safe. The flag (with `{effort}` substituted) is
|
|
362
|
+
appended to `tui` as literal text, and appended as its own argv elements (split on whitespace, since headless argv
|
|
363
|
+
is never shell-joined) to `headless`. `agentskit-review` has no reasoning-effort flag, so `models.effort.reviewer`
|
|
364
|
+
is not currently wired into the review CLI call — it is validated and recorded for symmetry and for a future
|
|
365
|
+
reviewer transport that supports it.
|
|
366
|
+
|
|
367
|
+
Whichever effort a dispatched builder actually used is recorded as `effort` on `dispatch.json` and the
|
|
368
|
+
`worker.dispatched` event; `loop retro`'s `dispatches.byProvider` groups by `provider/model@effort` (falling back
|
|
369
|
+
to plain `provider/model` for older events with no effort recorded) so a retro can tell a slow `gpt-5.6-luna@high`
|
|
370
|
+
run from a fast `@medium` one.
|
|
371
|
+
|
|
189
372
|
## Machine slots
|
|
190
373
|
|
|
191
374
|
`maxAgents = max(floor, min(adaptiveConcurrency(ceiling), ramBound, wslCap?))` where `ceiling` defaults to
|
|
@@ -208,7 +391,7 @@ the published dependency tree. `@agentskit/harness` stays dependency-light (`com
|
|
|
208
391
|
| Trigger | `.doc-bridge/index.json` exists under `project.root` |
|
|
209
392
|
| Knob | `contract.maxContextReferences` (default `6`; `0` disables) |
|
|
210
393
|
| Behaviour | Query is `"<issue id> <title>"`. Up to N deterministic references are appended to the orchestrator prompt. Missing or malformed index → **no refs** (loop continues). |
|
|
211
|
-
| Boundary | No `@agentskit/doc-bridge` import; the adapter only reads the local index contract ([ADR-0003](ADR-0003-doc-bridge-context-binding.md)). Index build/refresh stays with Doc Bridge (`pnpm docs:bridge:index` in repos that use it). |
|
|
394
|
+
| Boundary | No `@agentskit/doc-bridge` import; the adapter only reads the local index contract ([ADR-0003](ADR-0003-doc-bridge-context-binding.md)). Contract resolution rejects indexes older than `contract.docBridgeMaxAgeHours`; exact source freshness remains a Doc Bridge gate. Index build/refresh stays with Doc Bridge (`pnpm docs:bridge:index` in repos that use it). |
|
|
212
395
|
|
|
213
396
|
### Code Review (`agentskit-review`)
|
|
214
397
|
|
|
@@ -44,6 +44,7 @@ modules, not a provider dependency.
|
|
|
44
44
|
| `src/kernel/learning.ts` | Kernel | Retrospective parsing and human learning promotion | stdlib, `errors` | None |
|
|
45
45
|
| `src/kernel/status.ts` | Kernel | Deterministic status snapshot and digest | `errors`, `hash`, `block`, `types` (type-only) | None |
|
|
46
46
|
| `src/kernel/model-policy.ts` | Kernel | Role-to-model binding and validation | `errors`, `hash` | Provider is data, not an SDK |
|
|
47
|
+
| `src/kernel/pii.ts` | Kernel | Deterministic PII/secret pattern scan + redaction over a plain string | None | None |
|
|
47
48
|
| `src/execution/machine.ts` | Execution support | Machine sampling and adaptive concurrency | stdlib, `types`, `errors` | Host CPU/memory metrics |
|
|
48
49
|
| `src/execution/coordination.ts` | Execution support | Atomic issue/worktree claims and dispatch ledger | stdlib, `errors`, `hash` | Local state directory only |
|
|
49
50
|
| `src/kernel/resilience.ts` | Kernel | Failure classification and bounded retry/recovery policy | `errors` | Operation callback supplied by caller |
|
|
@@ -87,8 +88,13 @@ modules, not a provider dependency.
|
|
|
87
88
|
| `src/loop/doctor.ts` | Composition | Loop readiness report: Orca, providers, routing, slots, workers, queue | adapters, `config`, `cooldown`, `routing`, `slots` | Orca CLI via runner |
|
|
88
89
|
| `src/loop/contract.ts` | Composition | Task contract schema, orchestrator prompt (issue text as untrusted data), marked-JSON parsing, dispatchability assessment, candidate fallback with auth/quota classification, contract cache | `errors`, `hash`, `resilience`, `doc-bridge`, `config`, `routing` (type-only), `zod` | Headless coding-agent CLI via runner |
|
|
89
90
|
| `src/loop/brief.ts` | Composition | Worker prompt: frozen contract + repository rules + protected paths + done signal | `contract`, `config`, `linear-orca` (type-only) | None |
|
|
90
|
-
| `src/loop/
|
|
91
|
-
| `src/loop/
|
|
91
|
+
| `src/loop/resilience-state.ts` | Composition | Per-issue consecutive-failure counter + pause/resume, and per-stage (`tick`/`deliver`) crash-loop pause/resume | stdlib | `<stateDir>/issues/<id>/failures.json`, `<stateDir>/paused.json` |
|
|
92
|
+
| `src/loop/skills.ts` | Composition | `brief.skills` file loading: sha256 digest, truncation at `maxSkillChars`, fail-closed on a missing file | stdlib (`crypto`) | Local Markdown files under `project.root` |
|
|
93
|
+
| `src/loop/github-intake.ts` | Composition | GitHub label-based intake: discovers open PRs carrying `github.intakeLabel` and tracks them as `pr-<n>` (no Linear issue) | `github-cli` | GitHub via the injected runner; `<stateDir>/issues/pr-<n>/intake.json` |
|
|
94
|
+
| `src/loop/event-bus.ts` | Composition | Local pub/sub over the loop's event vocabulary plus orchestration lifecycle hooks (`beforeDispatch`, `beforeReview`, `beforeMerge`, …); loads local `plugins.modules` files | stdlib (`node:path`, `node:url`, dynamic `import()`) | Local `.mjs` files under `project.root`, never fetched over a network |
|
|
95
|
+
| `src/loop/progress.ts` | Composition | Best-effort read of `progress.json` from a worktree — the worker's optional, dynamic outcome-progress report | stdlib | The worktree filesystem only |
|
|
96
|
+
| `src/loop/tick.ts` | Composition | One keep-pushing tick: intake, admit (slots + dispatch ledger claim), contract, optional worktree setup command, skills pinning, dispatch into an Orca worktree, Linear transition, escalation, per-issue failure tracking, precheck | adapters, `coordination`, `errors`, `brief`, `config`, `contract`, `cooldown`, `doctor`, `resilience-state`, `routing`, `skills`, `slots` | Orca CLI + Linear via runner; `<stateDir>` files |
|
|
97
|
+
| `src/loop/deliver.ts` | Composition | Deliver stage per dispatched issue and per label-intake PR: PR detection, self-edit hold, conflict/CI/review fix rounds via terminal (or PR comment for intake), review at head, optimistic squash-merge (dispatched issues only — intake is review + comment, never merge), Linear Done, cleanup, stuck/abandoned escalation | adapters, `coordination`, `errors`, `config`, `cooldown`, `doctor`, `github-intake`, `routing`, `tick` | Orca, Linear, GitHub, agentskit-review via runner; `<stateDir>` files |
|
|
92
98
|
| `src/loop/install.ts` | Composition | Orca automation specs (`<prefix>-tick`, `<prefix>-deliver`) with read-only prechecks, idempotent create/edit by name, uninstall, status and the SessionStart hook line | `command`, `orca-cli`, `providers`, `errors`, `config`, `cooldown`, `doctor`, `routing` | Orca automations via runner |
|
|
93
99
|
| `src/loop/guided-install.ts` | Composition | Interactive install: doctor + environment preflight, dry-run rehearsal, confirmation, install, status; readline IO injected | `command`, `orca-cli`, `config`, `doctor`, `install`, `tick`, stdlib readline | Terminal prompts; Orca via runner |
|
|
94
100
|
| `src/loop/local-config.ts` | Composition | Per-machine overlay wizard: Linear team members via Orca, queue owner and machine tuning answers, YAML rendering and reload | `orca-cli`, `config`, `yaml` | Orca via runner; writes `loop.config.local.yaml` |
|
|
@@ -159,7 +165,7 @@ How these seams are wired into the keep-pushing loop (and what is still only a k
|
|
|
159
165
|
| Linear/GitHub/other tracker | `src/adapters/tracking.ts` callback | Caller-owned network mutation | Require idempotency key and explicit tracking authorization. |
|
|
160
166
|
| Orca CLI (loop) | `src/adapters/orca-cli.ts`, `src/adapters/linear-orca.ts` via `CommandRunner` | Read-only `--json` calls in the doctor; dispatch/mutation arrive in later loop phases | Argv only, never a shell string; every call bounded by a timeout; envelope `ok:false` fails closed. |
|
|
161
167
|
| Coding-agent CLIs | `src/adapters/providers.ts` | PATH lookup and optional probe command | Env keys are names only; usage comes from Orca, never from provider SDKs. |
|
|
162
|
-
| GitHub | `src/adapters/github-cli.ts` via `CommandRunner` | `gh pr view/list
|
|
168
|
+
| GitHub | `src/adapters/github-cli.ts` via `CommandRunner` | `gh pr view/list` (optionally `--label`), `gh api PUT …/merge` with `sha=<reviewed head>`, `gh pr comment`, `gh pr edit --remove-label` | Merge is refused by GitHub when the head moved; every call argv-based and bounded; label intake never calls the merge argv. |
|
|
163
169
|
| AgentsKit code review | `src/adapters/code-review.ts` via `CommandRunner` | `agentskit-review --pr … --result <file> [--post]` | Exit codes 0/1/2 plus the private result file decide clean/findings/incomplete; the floor is `--block`. |
|
|
164
170
|
| Process runtime | `src/execution/runtime.ts` | Starts child processes | Execution support; policy and evidence gates remain kernel decisions. |
|
|
165
171
|
| Docker runtime | `src/execution/runtime.ts` | Starts Docker containers | Optional sandbox selected by config, never a mandatory kernel dependency. |
|
package/loop.config.example.yaml
CHANGED
|
@@ -9,6 +9,11 @@ project:
|
|
|
9
9
|
baseBranch: main
|
|
10
10
|
root: . # relative to this file
|
|
11
11
|
stateDir: .codex/loop # ledger, cooldowns, contracts
|
|
12
|
+
setup:
|
|
13
|
+
# command: [pnpm, install, --frozen-lockfile] # argv (no shell), run once in a freshly created worktree
|
|
14
|
+
# before the worker terminal opens; unset = skip
|
|
15
|
+
timeoutSec: 600
|
|
16
|
+
required: true # failing/timing-out setup removes the worktree and counts as a dispatch failure
|
|
12
17
|
|
|
13
18
|
orca:
|
|
14
19
|
bin: orca # on Linux outside Orca terminals use orca-ide (bare `orca` is the GNOME screen reader)
|
|
@@ -24,6 +29,10 @@ linear:
|
|
|
24
29
|
person: my-linear-display-name # the ONLY queue this machine drains
|
|
25
30
|
people: # display name -> Linear user id (orca linear team members …)
|
|
26
31
|
my-linear-display-name: <linear-user-id>
|
|
32
|
+
rotation:
|
|
33
|
+
enabled: false # set true to advance through owners when the current dispatchable queue drains
|
|
34
|
+
owners: [my-linear-display-name] # ordered display names; the first entry is the initial owner
|
|
35
|
+
advanceWhenEmpty: true # never switch while an active lease remains
|
|
27
36
|
states: [Todo, Ready] # dispatchable states; anything already started is left alone
|
|
28
37
|
excludeLabels: [blocked, needs-info]
|
|
29
38
|
requireLabels: []
|
|
@@ -57,6 +66,12 @@ models:
|
|
|
57
66
|
maxMin: 240 # …up to 4 h
|
|
58
67
|
probeBeforeReenable: true
|
|
59
68
|
exhaustedPercent: 100
|
|
69
|
+
# Reasoning effort requested per role. Only applied for providers below that declare `effortFlag`; others ignore it.
|
|
70
|
+
effort:
|
|
71
|
+
orchestrator: high
|
|
72
|
+
reviewer: high # currently unused: agentskit-review has no reasoning-effort flag (see reviewProvider below)
|
|
73
|
+
builder: medium
|
|
74
|
+
watcher: low
|
|
60
75
|
providers:
|
|
61
76
|
claude:
|
|
62
77
|
bin: claude
|
|
@@ -64,12 +79,15 @@ models:
|
|
|
64
79
|
envKeys: [ANTHROPIC_API_KEY] # optional API-key fallback
|
|
65
80
|
tui: "claude --model {model} --permission-mode auto"
|
|
66
81
|
headless: [claude, -p, "{prompt}", --model, "{model}", --permission-mode, plan, --output-format, text]
|
|
82
|
+
# effortFlag: "--effort {effort}"
|
|
67
83
|
codex:
|
|
68
84
|
bin: codex
|
|
69
85
|
auth: subscription
|
|
70
86
|
envKeys: [OPENAI_API_KEY]
|
|
71
|
-
|
|
87
|
+
# Codex 0.154+ uses explicit approval/sandbox flags; --full-auto is not a valid CLI option.
|
|
88
|
+
tui: "codex -m {model} -s workspace-write -a never"
|
|
72
89
|
headless: [codex, exec, -m, "{model}", -s, read-only, --skip-git-repo-check, "{prompt}"]
|
|
90
|
+
# effortFlag: "-c model_reasoning_effort={effort}"
|
|
73
91
|
opencode:
|
|
74
92
|
bin: opencode
|
|
75
93
|
auth: none
|
|
@@ -81,6 +99,7 @@ models:
|
|
|
81
99
|
auth: subscription # `grok login` (OAuth) — no API key needed
|
|
82
100
|
tui: "grok -m {model}"
|
|
83
101
|
headless: [grok, -p, "{prompt}", -m, "{model}"]
|
|
102
|
+
# effortFlag: "--reasoning-effort {effort}"
|
|
84
103
|
|
|
85
104
|
machine:
|
|
86
105
|
floor: 1 # always allow one worker
|
|
@@ -110,10 +129,15 @@ delivery:
|
|
|
110
129
|
auto: true
|
|
111
130
|
method: squash
|
|
112
131
|
requireChecks: true
|
|
132
|
+
requireHumanApproval: false # true: also require a human GitHub review approval before merging, even
|
|
133
|
+
# after a clean loop review + green checks
|
|
113
134
|
maxFixRounds: 2
|
|
114
135
|
workerIdleTimeoutMin: 45
|
|
136
|
+
# maxDispatchMinutes: 240 # hard ceiling on one dispatch regardless of activity; unset = disabled
|
|
115
137
|
handoff: { enabled: true, maxHandoffs: 2, onlyWhenProviderUnavailable: true }
|
|
116
138
|
selfEditPaths: [loop.config.yaml, ".github/**"] # PRs touching these never auto-merge
|
|
139
|
+
# secretFilePatterns: [...] # default already covers .env, *.pem, *.key, id_rsa, credentials.json, etc.
|
|
140
|
+
# PRs touching a matching filename are held, never reviewed or merged
|
|
117
141
|
ignoreChecks: [] # advisory check names that never block
|
|
118
142
|
requiredChecks: [] # empty = every reported check must be green
|
|
119
143
|
cleanupWorktree: true # remove the Orca worktree after merge
|
|
@@ -154,6 +178,35 @@ contract:
|
|
|
154
178
|
maxBriefReferences: 4
|
|
155
179
|
contextProviders: [doc-bridge] # add rag when rag.enabled
|
|
156
180
|
|
|
181
|
+
github:
|
|
182
|
+
intakeLabel: loop:review # a PR with this label is reviewed even though the loop never dispatched it;
|
|
183
|
+
# null disables intake entirely
|
|
184
|
+
reviewOnly: true # fixed: intake PRs are always review + comment, never auto-merged
|
|
185
|
+
|
|
186
|
+
plugins:
|
|
187
|
+
modules: [] # e.g. [scripts/loop-plugins/notify.mjs] — local .mjs files (relative to
|
|
188
|
+
# project.root, never fetched over the network) loaded once per tick/deliver;
|
|
189
|
+
# each exports { id, apply(bus) } and can subscribe to loop events or block a
|
|
190
|
+
# beforeDispatch/beforeReview/beforeMerge hook — see docs/LOOP.md
|
|
191
|
+
|
|
192
|
+
security:
|
|
193
|
+
pii:
|
|
194
|
+
enabled: false # scan issue text / worker brief for PII-shaped patterns before embedding them
|
|
195
|
+
action: redact # redact | warn | block
|
|
196
|
+
|
|
197
|
+
resilience:
|
|
198
|
+
maxConsecutiveFailures: 3 # pause an issue after this many consecutive contract/dispatch failures
|
|
199
|
+
pausedLabel: loop:paused # applied to the issue in Linear; remove it (or `loop resume <id>`) to retry
|
|
200
|
+
stagePauseAfterRuns: 3 # pause `loop stage tick|deliver` itself after this many consecutive thrown runs
|
|
201
|
+
# maxUsageDeltaPercent: 40 # stop a dispatch if its provider's remaining usage drops by this many
|
|
202
|
+
# points while the issue is in flight; unset = disabled (cost circuit breaker)
|
|
203
|
+
|
|
204
|
+
brief:
|
|
205
|
+
skills: [] # e.g. [AGENTS.md, CLAUDE.md, docs/for-agents/INDEX.md] — paths relative to project.root,
|
|
206
|
+
# pinned verbatim into every worker brief (sha256-digested in dispatch.json); a
|
|
207
|
+
# missing file fails the dispatch instead of silently sending a worker without it
|
|
208
|
+
maxSkillChars: 6000 # per-file cap; longer files are truncated with a visible note
|
|
209
|
+
|
|
157
210
|
schedule:
|
|
158
211
|
tick: "*/5 * * * *"
|
|
159
212
|
deliver: "*/10 * * * *"
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@agentskit/harness",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.10.0",
|
|
4
4
|
"description": "Portable, evidence-backed development harness for coding agents.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -68,7 +68,7 @@
|
|
|
68
68
|
"test:phase-executor": "pnpm typecheck && vitest run --config vitest.config.ts test/phase-executor.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['phase-executor']}))\"",
|
|
69
69
|
"test:artifacts": "pnpm typecheck && vitest run --config vitest.config.ts test/artifacts.test.ts && node scripts/verify-artifact-cli.mjs",
|
|
70
70
|
"test:adapters": "pnpm typecheck && vitest run --config vitest.config.ts test/adapters.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['adapters']}))\"",
|
|
71
|
-
"test:loop": "pnpm typecheck && vitest run --config vitest.config.ts test/loop.test.ts test/loop-adapters.test.ts test/loop-tick.test.ts test/loop-deliver.test.ts test/loop-install.test.ts test/loop-guided-install.test.ts test/loop-retro.test.ts test/loop-debrief.test.ts test/loop-watch.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['loop-config','loop-providers','loop-routing','loop-doctor','loop-adapters','loop-tick','loop-deliver','loop-install','loop-debrief','loop-watch']}))\"",
|
|
71
|
+
"test:loop": "pnpm typecheck && vitest run --config vitest.config.ts test/loop.test.ts test/loop-adapters.test.ts test/loop-tick.test.ts test/loop-deliver.test.ts test/loop-install.test.ts test/loop-guided-install.test.ts test/loop-retro.test.ts test/loop-debrief.test.ts test/loop-watch.test.ts test/loop-agent-registry.test.ts test/loop-memory.test.ts test/loop-contract-failure.test.ts test/loop-resilience-state.test.ts test/loop-skills.test.ts test/loop-github-intake.test.ts test/loop-event-bus.test.ts test/loop-progress.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['loop-config','loop-providers','loop-routing','loop-doctor','loop-adapters','loop-tick','loop-deliver','loop-install','loop-debrief','loop-watch','loop-agent-registry','loop-memory','loop-contract-failure','loop-resilience-state','loop-skills','loop-github-intake','loop-event-bus','loop-progress']}))\"",
|
|
72
72
|
"test:review": "pnpm typecheck && vitest run --config vitest.config.ts test/review.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['review-delivery']}))\"",
|
|
73
73
|
"test:quality": "pnpm typecheck && vitest run --config vitest.config.ts test/quality.test.ts && node -e \"console.log(JSON.stringify({status:'passed',criteria:['quality-matrix']}))\"",
|
|
74
74
|
"test:eval-battery": "pnpm typecheck && vitest run --config vitest.config.ts test/eval-battery.test.ts test/issues-010-014.eval.test.ts test/optimization.test.ts && pnpm build >/dev/null && node scripts/verify-harness-eval-manifest.mjs && node -e \"console.log(JSON.stringify({status:'passed',criteria:['eval-battery','eval-manifest','eval-coverage']}))\"",
|
package/release/manifest.json
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"type": "agentskit-harness-release-manifest",
|
|
3
3
|
"schemaVersion": 1,
|
|
4
4
|
"package": "@agentskit/harness",
|
|
5
|
-
"version": "0.
|
|
5
|
+
"version": "0.10.0",
|
|
6
6
|
"channel": "latest",
|
|
7
7
|
"sourceRevision": "b8cfe596e5f30f2c373b89f00f271079b296feaf",
|
|
8
8
|
"requiredChecks": [
|
|
@@ -32,5 +32,5 @@
|
|
|
32
32
|
"pilot-benchmark"
|
|
33
33
|
],
|
|
34
34
|
"nextBaseline": "benchmarks/harness-0.4.0-baseline.json",
|
|
35
|
-
"digest": "
|
|
35
|
+
"digest": "231258f2636bbc19cd5b202f37c87146f7892cd5b79a41b97e12f4301c4573d4"
|
|
36
36
|
}
|