@cat-factory/executor-harness 1.88.0 → 1.92.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,7 +4,7 @@ The payload that runs **inside** a per-run Cloudflare Container (or a
4
4
  [self-hosted runner](../../docs/runner-pool-integration.md)) to perform real
5
5
  repo work with the [Pi coding agent](https://github.com/earendil-works/pi).
6
6
 
7
- It is a thin TypeScript wrapper a `node:http` server on `:8080` that the
7
+ It is a thin TypeScript wrapper (a `node:http` server on `:8080`) that the
8
8
  Worker drives over a small **job protocol**. Jobs run **asynchronously**: a `POST`
9
9
  accepts the job and returns immediately with a `jobId`; the driver then polls
10
10
  `GET /jobs/{id}` for live progress and the terminal result.
@@ -22,7 +22,7 @@ accepts the job and returns immediately with a `jobId`; the driver then polls
22
22
 
23
23
  | Method & path | Purpose |
24
24
  | ----------------- | ---------------------------------------------------------------------------------------------------------------------- |
25
- | `GET /health` | Liveness `{ "status": "ok" }`. |
25
+ | `GET /health` | Liveness: `{ "status": "ok" }`. |
26
26
  | `POST /run` | Start (or re-attach to) an **implementation** job (`coder` / `mocker` / `playwright`). Returns `202 { jobId, state }`. |
27
27
  | `POST /bootstrap` | Start a **repo-bootstrap** job (adapt a reference architecture → force-push a new repo). |
28
28
  | `POST /blueprint` | Start a **blueprint** job (decompose a repo → write the in-repo `blueprints/` map, commit on a branch). |
@@ -36,7 +36,7 @@ cat-factory sends are documented in
36
36
  [`docs/runner-pool-integration.md`](../../docs/runner-pool-integration.md).
37
37
 
38
38
  `GET /jobs/{id}` is also the harness's observability channel: `spans`, `followUps`
39
- and `callMetrics` are **drain-on-read** each poll returns what accumulated since
39
+ and `callMetrics` are **drain-on-read**; each poll returns what accumulated since
40
40
  the previous one and clears the buffer. That is deliberate. A job that dies before
41
41
  it can return a terminal result (an evicted container, an OOM-killed process) has
42
42
  still reported the tool spans it ran and the model calls it paid for. Each drained
@@ -44,8 +44,8 @@ still reported the tool spans it ran and the model calls it paid for. Each drain
44
44
  complete list, so the backend can take both channels without double-counting a call.
45
45
 
46
46
  Because the backend records a call as soon as it drains it (and ignores the terminal
47
- repeat), a drained call is FINAL. A call whose tokens are still open a CLI that reports
48
- only a cumulative total, costed at the end is withheld from the drain until it is
47
+ repeat), a drained call is FINAL. A call whose tokens are still open (a CLI that reports
48
+ only a cumulative total, costed at the end) is withheld from the drain until it is
49
49
  complete; see `createCallMetricPublisher` in `src/pi.ts`.
50
50
 
51
51
  ## What a job does
@@ -55,54 +55,54 @@ The implementation job (`POST /run`) is the canonical sequence:
55
55
  1. **clone** the target repo (shallow) with a short-lived GitHub installation token,
56
56
  2. write the composed system prompt (role + the block's best-practice fragments)
57
57
  to Pi's **global** context file `~/.pi/agent/AGENTS.md` (outside the checkout,
58
- so it never lands in a commit and never clobbers a repo's own `AGENTS.md` —
58
+ so it never lands in a commit and never clobbers a repo's own `AGENTS.md`:
59
59
  Pi reads and concatenates both), and point Pi at the Worker's LLM proxy via
60
- `~/.pi/agent/models.json` (provider `proxy`, `api: openai-completions`) at the
60
+ `~/.pi/agent/models.json` (provider `proxy`, `api: openai-completions`): at the
61
61
  phase-tagged completions path for the pass about to run (`.../phase/<phase>`) when the job
62
62
  body's `proxyPhasePath` says the backend serves it, which is how a repair round's model spend
63
63
  stays distinguishable from the first pass's in telemetry; without that flag the plain path is
64
64
  used and the calls are recorded as unattributed
65
65
  (see [token-burn instrumentation](../../../docs/initiatives/token-burn-instrumentation.md)),
66
- 3. **prepopulate dependencies**, when the job body carries `dependencyInstall` the
66
+ 3. **prepopulate dependencies**, when the job body carries `dependencyInstall`: the
67
67
  service's install command is run with `sh -c` in the checkout BEFORE the agent starts, so
68
68
  it reads real installed packages instead of inferring a library's capabilities from a
69
69
  manifest entry. Best-effort and never a gate: the outcome (success or the captured
70
- failure) is folded into the agent's prompt on EVERY pass, including the repair passes of
71
- steps 6 and 7, which start a fresh agent and the run continues either way. Whatever the
70
+ failure) is folded into the agent's prompt (on EVERY pass, including the repair passes of
71
+ steps 6 and 7, which start a fresh agent) and the run continues either way. Whatever the
72
72
  install materialises is excluded from git first, so no later `git add -A` can sweep a
73
73
  dependency tree into the pull request (see
74
74
  [dependency prepopulation](../../../docs/initiatives/agent-dependency-prepopulation.md)),
75
- 4. **resolve the repo's pull-request template**, when this dispatch opens a PR (`src/pr-template.ts`)
76
- `.github/PULL_REQUEST_TEMPLATE.md` and its root/`docs/`/multi-template-directory variants, or
75
+ 4. **resolve the repo's pull-request template**, when this dispatch opens a PR (`src/pr-template.ts`):
76
+ `.github/PULL_REQUEST_TEMPLATE.md` and its root/`docs/`/multi-template-directory variants, or
77
77
  GitLab's `.gitlab/merge_request_templates/`, read straight off the checkout (a symlinked template
78
- is followed only while it resolves INSIDE the checkout this is the one repo-chosen path the
78
+ is followed only while it resolves INSIDE the checkout: this is the one repo-chosen path the
79
79
  harness reads unprompted). Found, it is folded into the agent's prompt (on EVERY pass, as with
80
80
  the install above) asking it to write its briefing AS that template, filled in. This exists
81
81
  because neither host applies a template to an API-created pull request, so nothing else would:
82
82
  the template only reaches the web form a human opens. A directory of several templates with no
83
- `default` is left alone deliberately it exists so a human can choose per pull request,
83
+ `default` is left alone deliberately: it exists so a human can choose per pull request,
84
84
  5. **run Pi** non-interactively (`pi -p --mode json --model proxy/<model> --approve`),
85
- 6. **validate** the checkout, when the job body carries `validationChecks` the service's
85
+ 6. **validate** the checkout, when the job body carries `validationChecks`: the service's
86
86
  configured check commands (install/lint/test/build) run with `sh -c` in the checkout, and
87
87
  while they fail and the attempt budget remains the agent is re-run with the captured output
88
88
  as its instruction (see [pre-PR validation](../../../docs/initiatives/pre-pr-validation.md)),
89
- 7. **prove the reproduction**, when the job body carries `reproduction` the declared check is
89
+ 7. **prove the reproduction**, when the job body carries `reproduction`: the declared check is
90
90
  run against the pre-fix tree and the tree the PR will open from, in two freshly-created
91
91
  symmetric `git worktree` checkouts, and only red-then-green is reported as proof (see
92
- [bugfix reproduction proof](../../../docs/initiatives/bugfix-reproduction-proof.md)). Unlike
92
+ [bugfix reproduction proof](../../docs/adr/0033-bugfix-reproduction-proof.md)). Unlike
93
93
  step 6 this NEVER gates the PR: a failed verification is fed back to the agent while budget
94
94
  remains, then recorded as `inconclusive`. It runs BEFORE step 6 so validation stays the last
95
95
  thing to touch the tree,
96
- 8. **commit, push** a branch and **open a PR**, returning `{ prUrl, branch, summary }` but
96
+ 8. **commit, push** a branch and **open a PR**, returning `{ prUrl, branch, summary }`, but
97
97
  ONLY if step 6 ended green. A spent budget returns an error result with the validation report
98
98
  and opens no PR. Absent `validationChecks` / `reproduction`, steps 6 and 7 do not happen at
99
99
  all. The PR's description prefers the agent-authored reviewer briefing over the generic
100
100
  dispatch-time text the job body carries: a PR-opening agent is prompted to write one to the
101
101
  `.cat-pr-description.md` sentinel at the checkout root (one per sibling repo in a multi-repo
102
102
  run; an optional leading `# <title>` line, when it is the file's only `#` heading, sets the PR
103
- title), and `src/pr-description.ts` lifts it secret-scrubbed, size-capped with a visible
103
+ title), and `src/pr-description.ts` lifts it; secret-scrubbed, size-capped with a visible
104
104
  note, made inert for the host by `src/host-markdown.ts`, kept out of the commit like the
105
- effort/follow-ups sentinels onto `openPullRequest`. Absent or unusable ⇒ the fallback text,
105
+ effort/follow-ups sentinels; onto `openPullRequest`. Absent or unusable ⇒ the fallback text,
106
106
  unchanged. When the repo ships a template (step 4) that briefing IS the filled template: it
107
107
  crosses the same scrub/cap/inert boundary on the way out, but the leading-`#` title rule is
108
108
  switched OFF for it (`titleFromHeading: false`), because those headings are the repo's and
@@ -111,47 +111,60 @@ The implementation job (`POST /run`) is the canonical sequence:
111
111
  title/description in place (carrying the engine's managed report region across); the generic
112
112
  fallback never does, so a human's edit is safe.
113
113
 
114
- Bootstrap differs at the ends it may start from an empty dir, and **resets
114
+ Bootstrap differs at the ends: it may start from an empty dir, and **resets
115
115
  history to one commit and force-pushes** the default branch instead of opening a
116
116
  PR. Blueprint **commits onto a branch** (no history reset) and returns the tree.
117
117
 
118
118
  ### Skills and tool servers
119
119
 
120
- A job body may carry `skills[]` (procedural playbooks) and `mcpServers[]` (MCP tool servers) the
120
+ A job body may carry `skills[]` (procedural playbooks) and `mcpServers[]` (MCP tool servers): the
121
121
  harness MATERIALISES both and decides nothing about them; the backend has already resolved which
122
122
  apply and dropped what this harness cannot serve (see
123
123
  [`backend/docs/adr/0029-agent-kind-capabilities.md`](../../docs/adr/0029-agent-kind-capabilities.md)).
124
124
 
125
125
  - **Skills** install natively under `CLAUDE_CONFIG_DIR/skills/<name>/` for a leased-credential
126
126
  claude-code run (the CLI discovers and invokes them), and under
127
- `.cat-context/skill/<name>/` in the checkout for Pi, Codex, and an AMBIENT claude-code run
127
+ `.cat-context/skill/<name>/` in the checkout for Pi, Codex, and an AMBIENT claude-code run:
128
128
  whose prompt carries the instructions instead, because there is no isolated config home to
129
129
  install into and the runner refuses to write into the developer's own `~/.claude`.
130
130
  - **Tool servers** become a per-run `--mcp-config` file plus `--strict-mcp-config` for claude-code
131
131
  (so an ambient run never picks up the developer's personal servers), and `[mcp_servers.*]` blocks
132
- in the per-run `CODEX_HOME/config.toml` for Codex stdio only, and skipped entirely under
133
- ambient auth, which has no per-run home to write into. `--allowedTools` is passed ONLY when a
132
+ in the per-run `CODEX_HOME/config.toml` for Codex: stdio only, and skipped entirely under
133
+ ambient auth, which has no per-run home to write into. Both stdio-only skips are now BACKSTOPS
134
+ rather than decisions: the backend knows which transports each harness reaches and drops an
135
+ `http` server from a Codex dispatch with a stated reason, so the prompt names the gap instead of
136
+ advertising a tool this side then silently omitted. `--allowedTools` is passed ONLY when a
134
137
  server actually narrows its tools, and then carries the CLI's built-in tool names alongside the
135
- `mcp__*` patterns an allow-list is whole-session, not MCP-scoped, so a bare list of MCP
138
+ `mcp__*` patterns: an allow-list is whole-session, not MCP-scoped, so a bare list of MCP
136
139
  patterns would leave the agent unable to read, edit or build anything. Whether the CLI gates on
137
140
  that list at all is permission-mode dependent, so treat the narrowing as scoping rather than
138
- enforcement; the prompt states it either way.
141
+ enforcement; the prompt states it either way. An `allowedTools` entry that is not a single tool
142
+ name is DROPPED at the boundary, the comma above all: the list is joined into one argument with
143
+ commas, so `search_issues,get_issue` in one entry would become a pattern matching nothing.
144
+ - **An `mcp__*` call is exempt from the no-edit progress bound**, like a read or a subagent
145
+ dispatch: reaching a wired tool server is what the prompt tells the agent to do, so counting it
146
+ would abort an edits-expected run for following its own instructions. It is neutral rather than
147
+ edit-satisfying, and bounded by its own consecutive-call cap
148
+ (`JOB_MAX_CONSECUTIVE_MCP_CALLS`) for the same reason the web cap exists. Every exempt family
149
+ ALSO shares one backstop (`JOB_MAX_CONSECUTIVE_NON_ACTION_CALLS`), because each per-family cap
150
+ resets on any call outside its own family: a run alternating a web search with a tool-server
151
+ lookup trips neither, and having made no action call it never reaches the no-edit bound either.
139
152
  - **An `http` tool server must be `https`, or loopback.** Its headers carry a resolved credential,
140
153
  so the job boundary refuses a cleartext off-box URL (the backend refuses the same at
141
154
  registration). `secretKeys` names which `env`/`headers` entries are credentials, so exactly those
142
- values are registered for redaction scrubbing the whole map would turn ordinary config strings
155
+ values are registered for redaction: scrubbing the whole map would turn ordinary config strings
143
156
  into `***` in every later log line.
144
157
 
145
158
  Both config files carry this job's resolved credentials, so they are written to a per-job directory
146
- (mode `0600`) and never into the checkout or a HOME-global path see the next section.
159
+ (mode `0600`) and never into the checkout or a HOME-global path: see the next section.
147
160
 
148
161
  ## Per-job state: never a process- or HOME-global
149
162
 
150
163
  A job's staging state (the tester's secrets, private-registry auth, a repo-sourced Claude
151
164
  Skill) must be scoped to that job, not written into `process.env` or the home directory.
152
165
 
153
- In a container those two ARE per-job one job per process, and `HOME` belongs to that
154
- container so a global was a safe place to stage. The **local native transport** breaks both
166
+ In a container those two ARE per-job (one job per process, and `HOME` belongs to that
167
+ container) so a global was a safe place to stage. The **local native transport** breaks both
155
168
  assumptions: one long-lived host process serves every concurrent `ambientAuth` job, on the
156
169
  **developer's own home**. A global there is shared mutable state across siblings, and writing
157
170
  (or clearing) a dotfile destroys a file the developer owns.
@@ -162,16 +175,16 @@ under a per-job directory:
162
175
 
163
176
  | State | Container | Native (`ambientAuth`) |
164
177
  | -------------------- | -------------------------------------------- | ------------------------------------------------------------------------- |
165
- | Tester secrets | child env | child env (same path the old `process.env` set/restore is gone) |
178
+ | Tester secrets | child env | child env (same path: the old `process.env` set/restore is gone) |
166
179
  | Private-registry auth | `~/.npmrc`; cleared when a job has no entries | per-job `.npmrc` + `npm_config_userconfig`, seeded from the developer's; theirs is never written or removed |
167
- | Repo-sourced Claude Skill | installed into the isolated `CLAUDE_CONFIG_DIR` | not installed read from the checkout's `.cat-context/skill/`, like codex |
180
+ | Repo-sourced Claude Skill | installed into the isolated `CLAUDE_CONFIG_DIR` | not installed: read from the checkout's `.cat-context/skill/`, like codex |
168
181
 
169
182
  Two consequences worth knowing:
170
183
 
171
184
  - **The skill's PROMPT follows the same split.** A native install gets a short pointer; every
172
185
  checkout-reading case (Pi, codex, ambient claude-code) gets the instructions folded in plus a
173
186
  pointer to `.cat-context/skill/`. That decision is the backend's `renderSkillForHarness`, which
174
- keys off `ambientAuth` as well as the harness rendering an ambient run as an install would
187
+ keys off `ambientAuth` as well as the harness: rendering an ambient run as an install would
175
188
  point the agent at a skill that is nowhere on disk.
176
189
  - **`npm_config_userconfig` reaches less than `~/.npmrc` did.** npm and pnpm honour it; yarn does
177
190
  not. And it only reaches processes that are handed the job env, so anything the HARNESS itself
@@ -180,13 +193,13 @@ Two consequences worth knowing:
180
193
 
181
194
  When you add per-job state, put it in one of those two places. `~/.pi/*` and
182
195
  `~/.config/rpiv-web-tools` remain HOME-global, which is fine only because the Pi harness never
183
- runs natively (the native router sends `ambientAuth` jobs Claude/Codex only to the host
196
+ runs natively (the native router sends `ambientAuth` jobs (Claude/Codex only) to the host
184
197
  process and everything else to a container).
185
198
 
186
199
  ## No secrets in the image
187
200
 
188
201
  The image (built from the `Dockerfile`, base `node:26-trixie-slim`) contains
189
- only `git` + the Pi CLI + this compiled wrapper **no API keys, no GitHub
202
+ only `git` + the Pi CLI + this compiled wrapper: **no API keys, no GitHub
190
203
  credentials**. Per job, the Worker passes a short-lived GitHub token and a
191
204
  signed, model-locked LLM-proxy **session token** in the request body. Pi reaches
192
205
  models only through the Worker proxy, which injects the real provider key (qwen /
@@ -197,22 +210,22 @@ Kimi / DeepSeek) and meters spend. The provider key never enters the container.
197
210
  | File | Responsibility |
198
211
  | ------------------ | ------------------------------------------------------------------------------------------------------- |
199
212
  | `src/server.ts` | HTTP entry point; routes `/health`, `/run`, `/bootstrap`, `/blueprint`, `/jobs/{id}`. |
200
- | `src/runner.ts` | `JobRegistry` async job lifecycle, idempotent on `jobId`, progress tracking. |
213
+ | `src/runner.ts` | `JobRegistry`: async job lifecycle, idempotent on `jobId`, progress tracking. |
201
214
  | `src/job.ts` | Request types + validators for the job specs. |
202
215
  | `src/pi.ts` | Pi provider config, non-interactive run, JSON-line event + todo-progress parsing, global `AGENTS.md` guidance. |
203
216
  | `src/git.ts` | clone / branch / commit / push + GitHub PR creation; bootstrap history reset + force-push. |
204
217
  | `src/bootstrap.ts` | The `/bootstrap` handler (clone-or-empty → adapt → reinit + force-push). |
205
218
  | `src/blueprint.ts` | The `/blueprint` handler (decompose → render `blueprints/` → commit on branch). |
206
219
  | `src/embed.ts` | Bundled assets/templates written into the workspace. |
207
- | `src/package-registries.ts` | Private-registry (npm) auth: renders the job's allowlisted entries into an npmrc the user `~/.npmrc` in a container, a per-job file pointed at by `npm_config_userconfig` for a native job. |
208
- | `src/agent-runner.ts` | The subscription-harness runners (`runClaudeCode` / `runCodex`) talk direct to the vendor with a leased OAuth token, lift per-turn usage/telemetry off the CLI event stream. |
209
- | `src/claude-call-aggregator.ts` | Folds Claude Code's per-CONTENT-BLOCK `stream-json` envelopes back into the model calls they belong to (by `message.id`), reconstructs each call's request transcript, and routes subagent turns off the parent's chain. **Exported as the `./claude-call-aggregator` subpath and driven by the BACKEND too** (`runtimes/local`, for an inline step running on the developer's host `claude`), so it stays the ONE implementation the per-envelope over-count it fixes inflated a measured 1.47M tokens to 5.53M, and both drivers have to learn that only once. That second driver is why the transcript is retained only to `MAX_TRANSCRIPT_CHARS` (stating what it stopped retaining) and why assembling bodies at all is a `bodies` switch: in a container the reconstruction is one job's memory in a box sized for it, in the backend it is per concurrent inline step in the orchestrator process. Unlike the compile-only `./embed`, this subpath is a `dist` import, which is why the package emits declarations and why a consumer's typecheck depends on Turbo's `^build` edge having built this package first (see `tsconfig.json`'s `comment:buildOrder`). |
220
+ | `src/package-registries.ts` | Private-registry (npm) auth: renders the job's allowlisted entries into an npmrc; the user `~/.npmrc` in a container, a per-job file pointed at by `npm_config_userconfig` for a native job. |
221
+ | `src/agent-runner.ts` | The subscription-harness runners (`runClaudeCode` / `runCodex`): talk direct to the vendor with a leased OAuth token, lift per-turn usage/telemetry off the CLI event stream. |
222
+ | `src/claude-call-aggregator.ts` | Folds Claude Code's per-CONTENT-BLOCK `stream-json` envelopes back into the model calls they belong to (by `message.id`), reconstructs each call's request transcript, and routes subagent turns off the parent's chain. **Exported as the `./claude-call-aggregator` subpath and driven by the BACKEND too** (`runtimes/local`, for an inline step running on the developer's host `claude`), so it stays the ONE implementation: the per-envelope over-count it fixes inflated a measured 1.47M tokens to 5.53M, and both drivers have to learn that only once. That second driver is why the transcript is retained only to `MAX_TRANSCRIPT_CHARS` (stating what it stopped retaining) and why assembling bodies at all is a `bodies` switch: in a container the reconstruction is one job's memory in a box sized for it, in the backend it is per concurrent inline step in the orchestrator process. Unlike the compile-only `./embed`, this subpath is a `dist` import, which is why the package emits declarations, and why a consumer's typecheck depends on Turbo's `^build` edge having built this package first (see `tsconfig.json`'s `comment:buildOrder`). |
210
223
  | `src/transcript-retention.ts` | Lifts the CLI session transcripts (`projects/` / `sessions/`) out of the isolated, credential-bearing config home before it is deleted, and prunes them on a TTL (debugging artifact retention). |
211
224
  | `src/captured-command.ts` | The one way the harness runs a declared shell command on its own behalf: `sh -c` with a per-command watchdog, abort handling, conventional exit codes (124/127/130) and a scrub-then-bound output capture. Shared by both pre-PR verification phases so a fix to one cannot miss the other. |
212
- | `src/dependency-install.ts` | Dependency prepopulation: `prepopulateDependencies` is the ONE seam every checkout-having mode calls it runs the service's install command before the agent's first turn, excludes what the install materialised from git so no `git add -A` can sweep a dependency tree into the PR, and builds the prompt note describing the outcome. Best-effort every failure shape becomes a note, never a failed job. Generic keyed off the job body, never the agent kind. |
213
- | `src/validation-checks.ts` | Pre-PR validation: runs the job's check commands in the checkout (bounded, secret-scrubbed capture, per-command watchdog) and drives the retry-until-green loop that gates the PR. Generic keyed off the job body, never the agent kind. |
214
- | `src/reproduction-proof.ts` | Bugfix reproduction proof: runs the job's declared reproduction command against two symmetric fresh worktrees (the pre-fix tree and the final tree) and computes red-then-green from the exit codes, with a repair loop that never fails the run. Generic keyed off the job body, never the agent kind. |
215
- | `src/agent-capabilities.ts` | The agent CAPABILITIES a job body carries the run's `skills` (a `SKILL.md` payload + resources) and its `mcpServers` (tool servers) with their defensive parsing and the per-CLI config writers (`--mcp-config` JSON for claude-code, `[mcp_servers.*]` TOML for Codex). Backend-authored data the harness only MATERIALISES: adding a skill or a tool server is a backend registration, never a harness change. |
225
+ | `src/dependency-install.ts` | Dependency prepopulation: `prepopulateDependencies` is the ONE seam every checkout-having mode calls; it runs the service's install command before the agent's first turn, excludes what the install materialised from git so no `git add -A` can sweep a dependency tree into the PR, and builds the prompt note describing the outcome. Best-effort: every failure shape becomes a note, never a failed job. Generic: keyed off the job body, never the agent kind. |
226
+ | `src/validation-checks.ts` | Pre-PR validation: runs the job's check commands in the checkout (bounded, secret-scrubbed capture, per-command watchdog) and drives the retry-until-green loop that gates the PR. Generic: keyed off the job body, never the agent kind. |
227
+ | `src/reproduction-proof.ts` | Bugfix reproduction proof: runs the job's declared reproduction command against two symmetric fresh worktrees (the pre-fix tree and the final tree) and computes red-then-green from the exit codes, with a repair loop that never fails the run. Generic: keyed off the job body, never the agent kind. |
228
+ | `src/agent-capabilities.ts` | The agent CAPABILITIES a job body carries: the run's `skills` (a `SKILL.md` payload + resources) and its `mcpServers` (tool servers): with their defensive parsing and the per-CLI config writers (`--mcp-config` JSON for claude-code, `[mcp_servers.*]` TOML for Codex). Backend-authored data the harness only MATERIALISES: adding a skill or a tool server is a backend registration, never a harness change. |
216
229
  | `src/bootstrap-mode.ts` | The repo-bootstrap MODE: clone-a-reference-or-scaffold → run the agent → refuse to push an empty tree → reinit + force-push to the pre-created target repo. |
217
230
  | `src/agent-shared.ts` | The few helpers every agent MODE shares (effort-report folding, the capability fields forwarded to `runAgentInWorkspace`). |
218
231
  | `src/logger.ts` | Structured logging. |
@@ -227,15 +240,17 @@ runner):
227
240
  | `PORT` | `8080` | HTTP port the harness listens on. |
228
241
  | `JOB_MAX_DURATION_MS` | `3600000` (60m) | Hard ceiling on a job's wall-clock time; force-fails after. |
229
242
  | `JOB_INACTIVITY_MS` | `600000` (10m) | Kills a hung agent that produces no output for this long. |
230
- | `JOB_COLD_START_MS` | `120000` (2m) | First-output window (ADR 0026 D4). A job that has produced nothing this long records a cold-start diagnostic a likely onboarding/auth wedge WITHOUT being killed: logged, exposed on `GET /jobs/{id}`, and folded into the failure `detail` if the job goes on to fail. `0` disables it. |
231
- | `DEPENDENCY_INSTALL_TIMEOUT_MS` | a third of `JOB_MAX_DURATION_MS` (20m at its default) | Watchdog for the pre-agent dependency install; a timeout is reported as a failed install (exit 124), never a failed job. Derived from the job ceiling rather than fixed, and an explicit value is clamped by the same share the agent is what waits on this, so setup can never consume the run it is preparing for. |
243
+ | `JOB_MAX_CONSECUTIVE_MCP_CALLS` | `40` | Consecutive tool-server (`mcp__*`) calls with no other tool call between before the run counts as a lookup loop. The counter-bound the no-edit exemption above owes; a per-kind `tuning.guardLimits` entry can only RAISE it. |
244
+ | `JOB_MAX_CONSECUTIVE_NON_ACTION_CALLS` | `200` | Consecutive calls of ANY no-edit-exempt family (reads, searches, web, tool servers, subagent dispatches) with no action call between them. The backstop above the per-family caps, since each of those resets on a call outside its own family; sized as a backstop rather than a research judgement, and reset by any `bash`/edit. |
245
+ | `JOB_COLD_START_MS` | `120000` (2m) | First-output window (ADR 0026 D4). A job that has produced nothing this long records a cold-start diagnostic (a likely onboarding/auth wedge) WITHOUT being killed: logged, exposed on `GET /jobs/{id}`, and folded into the failure `detail` if the job goes on to fail. `0` disables it. |
246
+ | `DEPENDENCY_INSTALL_TIMEOUT_MS` | a third of `JOB_MAX_DURATION_MS` (20m at its default) | Watchdog for the pre-agent dependency install; a timeout is reported as a failed install (exit 124), never a failed job. Derived from the job ceiling rather than fixed, and an explicit value is clamped by the same share: the agent is what waits on this, so setup can never consume the run it is preparing for. |
232
247
  | `DEPENDENCY_INSTALL_HEARTBEAT_MS` | `30000` (30s) | How often the dependency install feeds the job inactivity watchdog. A cold install is activity-silent and `JOB_INACTIVITY_MS` is tighter than its own watchdog, so without this a healthy install aborts the run as "likely hung". |
233
248
  | `VALIDATION_COMMAND_TIMEOUT_MS` | `900000` (15m) | Per-command watchdog for a pre-PR validation check; a timeout counts as a failure (exit 124) so one hung command can't wedge the loop. |
234
249
  | `REPRODUCTION_COMMAND_TIMEOUT_MS` | `900000` (15m) | Per-command watchdog for a reproduction-proof setup or check command; a timeout counts as a failure (exit 124). |
235
250
  | `REPRODUCTION_HEARTBEAT_MS` | `30000` (30s) | How often the reproduction proof feeds the job inactivity watchdog while it runs commands the agent is not producing output for. |
236
251
  | `REPRODUCTION_TOTAL_BUDGET_MS` | `2700000` (45m) | Wall-clock ceiling on the WHOLE proof phase (every attempt, both trees, setup included). Attempts multiply two full tree runs each and the heartbeat above deliberately stops the inactivity watchdog from firing, so this is what bounds the phase. Checked at phase boundaries; exceeding it settles `inconclusive`, never a run failure. |
237
252
  | `HARNESS_TRANSCRIPT_TTL_MS` | `259200000` (3d) | How long lifted subscription-CLI session transcripts are kept before the retention sweep prunes them. |
238
- | `HARNESS_TRANSCRIPT_ROOT` | `<tmpdir>/cf-agent-transcripts` | Where retained session transcripts are moved to (one dir per run). Meaningful only on a reused (warm-pool) container; a per-run container is torn down with the job. The TTL sweep deletes only dirs it created (each carries a `.cf-retained` marker), so pointing this at a shared directory never touches unrelated content though a dedicated dir is still recommended. An override on a different filesystem than the config home falls back to copy-then-remove. |
253
+ | `HARNESS_TRANSCRIPT_ROOT` | `<tmpdir>/cf-agent-transcripts` | Where retained session transcripts are moved to (one dir per run). Meaningful only on a reused (warm-pool) container; a per-run container is torn down with the job. The TTL sweep deletes only dirs it created (each carries a `.cf-retained` marker), so pointing this at a shared directory never touches unrelated content, though a dedicated dir is still recommended. An override on a different filesystem than the config home falls back to copy-then-remove. |
239
254
 
240
255
  ## Build / test
241
256
 
@@ -283,11 +298,11 @@ pnpm --filter @cat-factory/executor-harness run image:publish
283
298
  The script ([`scripts/publish-image.sh`](./scripts/publish-image.sh)) builds the
284
299
  multi-arch image once and pushes it to the selected registries. Override defaults
285
300
  via env vars (`REGISTRIES`, `GHCR_OWNER`, `DOCKERHUB_ORG`, `TAG`, `PUSH_LATEST`,
286
- `PLATFORMS`, `EXTRA_CA`) see the header of the script. Example: GHCR only
301
+ `PLATFORMS`, `EXTRA_CA`): see the header of the script. Example: GHCR only;
287
302
  `REGISTRIES=ghcr pnpm --filter @cat-factory/executor-harness run image:publish`.
288
303
 
289
304
  A backend deployment references the image from `wrangler.toml`
290
- (`[[containers]] image = "ghcr.io/<owner>/cat-factory-executor:<version>"` see
305
+ (`[[containers]] image = "ghcr.io/<owner>/cat-factory-executor:<version>"`: see
291
306
  [`deploy/backend`](../../../deploy/backend)); a self-hosted runner pool pulls the
292
307
  same image (see [`docs/runner-pool-integration.md`](../../docs/runner-pool-integration.md)).
293
308
  The worker library's own test/dev `wrangler.toml` still references this
@@ -64,6 +64,19 @@ export declare function parseSkillSpecs(value: unknown): SkillSpec[] | undefined
64
64
  * `src/host-markdown.ts` uses.
65
65
  */
66
66
  export declare const MCP_SERVER_ID_PATTERN: RegExp;
67
+ /**
68
+ * A tool name an `allowedTools` entry may name. Kept byte-identical to kernel's
69
+ * `MCP_TOOL_NAME_PATTERN` for the same reason {@link MCP_SERVER_ID_PATTERN} is a copy, and pinned
70
+ * against it by `test/agent-capabilities.conformity.test.ts`.
71
+ *
72
+ * The comma is the reason the rule exists on THIS side of the boundary too:
73
+ * {@link claudeAllowedToolPatterns} builds the list that the runner joins into one
74
+ * `--allowedTools` argument with commas, so an entry carrying one splits into two patterns of which
75
+ * the second matches no tool the CLI has. Dropped rather than passed through, because the entries
76
+ * that survive are what narrows the session: a bad one would silently take the run's whole MCP
77
+ * surface with it.
78
+ */
79
+ export declare const MCP_TOOL_NAME_PATTERN: RegExp;
67
80
  /**
68
81
  * Whether an HTTP tool server's URL may be started. Mirrors kernel's `isAllowedMcpHttpUrl` (see
69
82
  * {@link MCP_SERVER_ID_PATTERN} for why it is a copy, and the conformity suite that pins it):
@@ -116,9 +129,13 @@ export declare const CLAUDE_BUILT_IN_TOOLS: readonly string[];
116
129
  export declare function claudeAllowedToolPatterns(servers: McpServerSpec[]): string[] | undefined;
117
130
  /**
118
131
  * The `[mcp_servers.<id>]` TOML block Codex reads from its `CODEX_HOME/config.toml`. Codex's MCP
119
- * client is stdio-only, so an `http` server is skipped here — the backend states such a server as
120
- * unavailable when it declares `harnesses: ['claude-code']`, and a deployment that wires an HTTP
121
- * server for Codex gets a no-op rather than a malformed config.
132
+ * client is stdio-only, so an `http` server is skipped here.
133
+ *
134
+ * The skip is now a BACKSTOP rather than the decision: the backend knows which transports each
135
+ * harness reaches (`MCP_HARNESS_TRANSPORTS`) and drops an `http` server from a Codex dispatch under
136
+ * its own `transport_unsupported` reason, so the prompt states the gap instead of advertising a tool
137
+ * this writer then silently omitted. It stays because a body that reached the container by any other
138
+ * route must still produce a valid config rather than a malformed one.
122
139
  */
123
140
  export declare function codexMcpConfigToml(servers: McpServerSpec[]): string;
124
141
  /**
@@ -125,6 +125,19 @@ function sanitizeServerId(value) {
125
125
  return undefined;
126
126
  return MCP_SERVER_ID_PATTERN.test(value) ? value : undefined;
127
127
  }
128
+ /**
129
+ * A tool name an `allowedTools` entry may name. Kept byte-identical to kernel's
130
+ * `MCP_TOOL_NAME_PATTERN` for the same reason {@link MCP_SERVER_ID_PATTERN} is a copy, and pinned
131
+ * against it by `test/agent-capabilities.conformity.test.ts`.
132
+ *
133
+ * The comma is the reason the rule exists on THIS side of the boundary too:
134
+ * {@link claudeAllowedToolPatterns} builds the list that the runner joins into one
135
+ * `--allowedTools` argument with commas, so an entry carrying one splits into two patterns of which
136
+ * the second matches no tool the CLI has. Dropped rather than passed through, because the entries
137
+ * that survive are what narrows the session: a bad one would silently take the run's whole MCP
138
+ * surface with it.
139
+ */
140
+ export const MCP_TOOL_NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/;
128
141
  /**
129
142
  * Whether an HTTP tool server's URL may be started. Mirrors kernel's `isAllowedMcpHttpUrl` (see
130
143
  * {@link MCP_SERVER_ID_PATTERN} for why it is a copy, and the conformity suite that pins it):
@@ -166,6 +179,17 @@ function parseStringArray(value) {
166
179
  const out = value.filter((v) => typeof v === 'string');
167
180
  return out.length ? out : undefined;
168
181
  }
182
+ /**
183
+ * The `allowedTools` list: string entries that are single tool NAMES (see
184
+ * {@link MCP_TOOL_NAME_PATTERN}). Undefined when nothing survives, which is the same answer as an
185
+ * absent field (every tool the server exposes) and the right one: the alternative is a list whose
186
+ * only surviving entries are the platform's own built-in tool names, i.e. a run narrowed to no MCP
187
+ * tools at all. The backend refuses these at registration; this is the boundary check.
188
+ */
189
+ function parseAllowedTools(value) {
190
+ const names = parseStringArray(value)?.filter((name) => MCP_TOOL_NAME_PATTERN.test(name));
191
+ return names?.length ? names : undefined;
192
+ }
169
193
  /** Validate one `mcpServers` entry, or undefined when malformed for its transport. */
170
194
  function parseMcpServerSpec(value) {
171
195
  if (typeof value !== 'object' || value === null)
@@ -174,7 +198,7 @@ function parseMcpServerSpec(value) {
174
198
  const id = sanitizeServerId(o.id);
175
199
  if (!id)
176
200
  return undefined;
177
- const allowedTools = parseStringArray(o.allowedTools);
201
+ const allowedTools = parseAllowedTools(o.allowedTools);
178
202
  const secretKeys = parseStringArray(o.secretKeys);
179
203
  if (o.transport === 'http') {
180
204
  // https anywhere, plain http only on loopback: the CLI would happily be pointed at a
@@ -313,9 +337,13 @@ function tomlString(value) {
313
337
  }
314
338
  /**
315
339
  * The `[mcp_servers.<id>]` TOML block Codex reads from its `CODEX_HOME/config.toml`. Codex's MCP
316
- * client is stdio-only, so an `http` server is skipped here — the backend states such a server as
317
- * unavailable when it declares `harnesses: ['claude-code']`, and a deployment that wires an HTTP
318
- * server for Codex gets a no-op rather than a malformed config.
340
+ * client is stdio-only, so an `http` server is skipped here.
341
+ *
342
+ * The skip is now a BACKSTOP rather than the decision: the backend knows which transports each
343
+ * harness reaches (`MCP_HARNESS_TRANSPORTS`) and drops an `http` server from a Codex dispatch under
344
+ * its own `transport_unsupported` reason, so the prompt states the gap instead of advertising a tool
345
+ * this writer then silently omitted. It stays because a body that reached the container by any other
346
+ * route must still produce a valid config rather than a malformed one.
319
347
  */
320
348
  export function codexMcpConfigToml(servers) {
321
349
  const blocks = [];
@@ -1,5 +1,5 @@
1
1
  import type { Logger } from './logger.js';
2
- import { type HarnessCallMetric, type PiRunOutcome, type TodoProgress } from './pi.js';
2
+ import { type HarnessCallMetric, type PiRunOutcome, type TodoProgress, type ToolSpan } from './pi.js';
3
3
  import { type McpServerSpec, type SkillSpec } from './agent-capabilities.js';
4
4
  import { type ProgressGuardLimits } from './progress-guard.js';
5
5
  import { type SliceReview } from './subagents.js';
@@ -72,6 +72,13 @@ export interface SubscriptionRunOptions {
72
72
  onActivity?: () => void;
73
73
  /** Called with the latest subtask counts each time the CLI updates its todo/plan list. */
74
74
  onProgress?: (progress: TodoProgress) => void;
75
+ /**
76
+ * Called once per completed tool call with a {@link ToolSpan}: the run's TRAJECTORY. The CLI's
77
+ * tool loop is internal to the CLI and never touches our proxy, so its own event stream is the
78
+ * only place these exist — without this hook a subscription-harness run's account of what it
79
+ * DID dies with the container.
80
+ */
81
+ onSpan?: (span: ToolSpan) => void;
75
82
  /**
76
83
  * Called with the FULL set of per-slice reviews each time one lands, so the backend can persist
77
84
  * a parallel review's completed work as it happens instead of only from the terminal result.
@@ -4,6 +4,7 @@ import { tmpdir } from 'node:os';
4
4
  import { dirname, join } from 'node:path';
5
5
  import { claudeAssistantContent, isObject, numberOf, redactBody } from './claude-stream.js';
6
6
  import { createClaudeRunTelemetry, subagentDispatchId } from './claude-call-aggregator.js';
7
+ import { ToolCallTracker, recordClaudeToolResults, } from './tool-trajectory.js';
7
8
  import { createCallMetricPublisher, publishCallMetric, } from './pi.js';
8
9
  import { claudeAllowedToolPatterns, codexMcpConfigToml, mcpServerSecretValues, writeClaudeMcpConfig, } from './agent-capabilities.js';
9
10
  import { ProgressGuard } from './progress-guard.js';
@@ -372,6 +373,27 @@ function createClaudeProgressGuard(opts) {
372
373
  reason: () => guardReason,
373
374
  };
374
375
  }
376
+ /**
377
+ * The run's TRAJECTORY, on the claude-code stream: each `tool_use` block paired with the
378
+ * `tool_result` that answers it on the following user turn, numbered and captured (scrubbed +
379
+ * capped). The CLI's stream is the only place this loop is visible at all — its tool calls never
380
+ * touch our proxy — so without this a subscription-harness run's account of what it DID dies with
381
+ * the container.
382
+ *
383
+ * Both halves are no-ops when the caller wants no spans, so a driver that only needs the run's
384
+ * output never pays to serialise a body nothing will read. Split out of {@link runClaudeCode} for
385
+ * the per-function line budget, like {@link createClaudeProgressGuard}.
386
+ */
387
+ function createClaudeToolTrajectory(opts, secrets) {
388
+ if (!opts.onSpan)
389
+ return { onToolUse: () => { }, onToolResults: () => { } };
390
+ const onSpan = opts.onSpan;
391
+ const tracker = new ToolCallTracker(secrets);
392
+ return {
393
+ onToolUse: (id, name, input) => tracker.started(id, name, input),
394
+ onToolResults: (content) => recordClaudeToolResults(tracker, content, (call) => onSpan({ ...call, bodies: 'stored' })),
395
+ };
396
+ }
375
397
  export async function runClaudeCode(opts) {
376
398
  const stats = { toolCalls: 0, assistantChars: 0 };
377
399
  let summary = '';
@@ -459,6 +481,7 @@ export async function runClaudeCode(opts) {
459
481
  // diagnostic. Disabled when the caller supplies no limits (only the external watchdog bounds it).
460
482
  const progressGuard = createClaudeProgressGuard(opts);
461
483
  const { rememberTool, feedGuard, guardAbort } = progressGuard;
484
+ const trajectory = createClaudeToolTrajectory(opts, secrets);
462
485
  const onEvent = (event, meta) => {
463
486
  const type = event.type;
464
487
  // A subagent's turns ride the parent's stdout tagged with the dispatch that spawned them;
@@ -481,6 +504,7 @@ export async function runClaudeCode(opts) {
481
504
  // `is_error` its `tool_result` carries on the next `user` turn.
482
505
  if (typeof block.id === 'string' && typeof block.name === 'string') {
483
506
  rememberTool(block.id, block.name);
507
+ trajectory.onToolUse(block.id, block.name, block.input);
484
508
  }
485
509
  if (block.name === 'TodoWrite') {
486
510
  const progress = todosToProgress(block.input?.todos);
@@ -506,6 +530,9 @@ export async function runClaudeCode(opts) {
506
530
  // would kill nothing and only convert a clean exit into a spurious failure.
507
531
  if (!meta?.final)
508
532
  feedGuard(content);
533
+ // The trajectory's other half — fed on the FINAL flush too, unlike the guard, since a
534
+ // CLI that has exited is exactly when the last calls' results matter.
535
+ trajectory.onToolResults(content);
509
536
  telemetry.onToolResult(dispatchId, content);
510
537
  }
511
538
  }
package/dist/job.d.ts CHANGED
@@ -493,6 +493,8 @@ export interface GuardLimitsSpec {
493
493
  maxToolCallsWithoutEdit?: number;
494
494
  maxConsecutiveErrors?: number;
495
495
  maxConsecutiveWebCalls?: number;
496
+ maxConsecutiveMcpCalls?: number;
497
+ maxConsecutiveNonActionCalls?: number;
496
498
  }
497
499
  /**
498
500
  * The record of standing the service's docker-compose dependencies up before a tester
package/dist/job.js CHANGED
@@ -36,12 +36,18 @@ function parseGuardLimits(value) {
36
36
  const noEdit = posInt(o.maxToolCallsWithoutEdit);
37
37
  const errors = posInt(o.maxConsecutiveErrors);
38
38
  const web = posInt(o.maxConsecutiveWebCalls);
39
+ const mcp = posInt(o.maxConsecutiveMcpCalls);
40
+ const nonAction = posInt(o.maxConsecutiveNonActionCalls);
39
41
  if (noEdit !== undefined)
40
42
  spec.maxToolCallsWithoutEdit = noEdit;
41
43
  if (errors !== undefined)
42
44
  spec.maxConsecutiveErrors = errors;
43
45
  if (web !== undefined)
44
46
  spec.maxConsecutiveWebCalls = web;
47
+ if (mcp !== undefined)
48
+ spec.maxConsecutiveMcpCalls = mcp;
49
+ if (nonAction !== undefined)
50
+ spec.maxConsecutiveNonActionCalls = nonAction;
45
51
  return Object.keys(spec).length > 0 ? spec : undefined;
46
52
  }
47
53
  /**
@@ -182,6 +182,9 @@ export async function runAgentInWorkspace(spec, opts = {}) {
182
182
  expectsEdits: spec.expectsEdits ?? true,
183
183
  onActivity: opts.onActivity,
184
184
  onProgress: opts.onProgress,
185
+ // The run's tool-call trajectory, the same hook the Pi path feeds — so a subscription run
186
+ // and a proxied one produce the same evidence rather than one of them producing none.
187
+ onSpan: opts.onSpan,
185
188
  // Per-slice review capture, so a parallel review's finished slices are persisted as they
186
189
  // land rather than only in the terminal output. Only the subscription runners fan work out
187
190
  // across subagents, so this is the only path that can produce it.
package/dist/pi.d.ts CHANGED
@@ -187,18 +187,39 @@ export interface TodoProgress {
187
187
  items?: TodoItem[];
188
188
  }
189
189
  /**
190
- * One tool invocation in Pi's loop, captured for the run's observability trace.
191
- * Metadata only (name + timing + ok) never the tool's args or result so the
192
- * harness buffer stays tiny. The backend drains these on its existing job poll and
193
- * emits them as child spans under the run trace.
190
+ * One tool invocation in an agent's loop, captured for the run's TRAJECTORY: the ordered
191
+ * account of what the agent did, drained by the backend on its existing job poll and
192
+ * both persisted and emitted as a child span under the run trace.
193
+ *
194
+ * It carries the call's arguments and result (scrubbed and capped at capture — see
195
+ * `tool-trajectory.ts`), because the question asked of a finished run is which command
196
+ * ran against what, not how long a tool named `bash` took. Whether those bodies are
197
+ * RETAINED is the backend's decision, taken against the deployment switch and the
198
+ * workspace's opt-out; the harness's job is to capture them bounded and scrubbed.
194
199
  */
195
200
  export interface ToolSpan {
196
201
  tool: string;
197
- /** Epoch ms the tool call started (approximated as the previous tool's end). */
202
+ /**
203
+ * The call's 0-based ordinal within this job. Two calls routinely land in the same
204
+ * millisecond, so this is the only thing that orders the trajectory — and it is what
205
+ * makes the backend's stored row id deterministic, so a replayed poll re-records
206
+ * instead of duplicating.
207
+ */
208
+ seq: number;
209
+ /** Epoch ms the tool call started (the previous call's end when no start was seen). */
198
210
  startedAt: number;
199
211
  /** Epoch ms the tool call ended (when its `tool_execution_end` event arrived). */
200
212
  endedAt: number;
201
213
  ok: boolean;
214
+ /** Whether the bodies below were captured at all — always `'stored'` from this harness. */
215
+ bodies: 'stored' | 'withheld';
216
+ /** The call's arguments, serialised, scrubbed and capped. `''` when it took none. */
217
+ args: string;
218
+ /** What the tool returned, scrubbed and capped. `''` when it returned nothing. */
219
+ result: string;
220
+ /** Characters the cap dropped from {@link args} / {@link result}; 0 when nothing was cut. */
221
+ argsDropped: number;
222
+ resultDropped: number;
202
223
  }
203
224
  /**
204
225
  * What the agent actually did this run, independent of any file changes. Used to