@cat-factory/executor-harness 1.88.0 → 1.90.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +64 -49
- package/dist/agent-capabilities.d.ts +20 -3
- package/dist/agent-capabilities.js +32 -4
- package/dist/job.d.ts +2 -0
- package/dist/job.js +6 -0
- package/dist/progress-guard.d.ts +29 -0
- package/dist/progress-guard.js +82 -8
- package/package.json +4 -4
- package/src/agent-capabilities.ts +34 -4
- package/src/job.ts +6 -0
- package/src/progress-guard.ts +129 -10
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@ The payload that runs **inside** a per-run Cloudflare Container (or a
|
|
|
4
4
|
[self-hosted runner](../../docs/runner-pool-integration.md)) to perform real
|
|
5
5
|
repo work with the [Pi coding agent](https://github.com/earendil-works/pi).
|
|
6
6
|
|
|
7
|
-
It is a thin TypeScript wrapper
|
|
7
|
+
It is a thin TypeScript wrapper (a `node:http` server on `:8080`) that the
|
|
8
8
|
Worker drives over a small **job protocol**. Jobs run **asynchronously**: a `POST`
|
|
9
9
|
accepts the job and returns immediately with a `jobId`; the driver then polls
|
|
10
10
|
`GET /jobs/{id}` for live progress and the terminal result.
|
|
@@ -22,7 +22,7 @@ accepts the job and returns immediately with a `jobId`; the driver then polls
|
|
|
22
22
|
|
|
23
23
|
| Method & path | Purpose |
|
|
24
24
|
| ----------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
|
25
|
-
| `GET /health` | Liveness
|
|
25
|
+
| `GET /health` | Liveness: `{ "status": "ok" }`. |
|
|
26
26
|
| `POST /run` | Start (or re-attach to) an **implementation** job (`coder` / `mocker` / `playwright`). Returns `202 { jobId, state }`. |
|
|
27
27
|
| `POST /bootstrap` | Start a **repo-bootstrap** job (adapt a reference architecture → force-push a new repo). |
|
|
28
28
|
| `POST /blueprint` | Start a **blueprint** job (decompose a repo → write the in-repo `blueprints/` map, commit on a branch). |
|
|
@@ -36,7 +36,7 @@ cat-factory sends are documented in
|
|
|
36
36
|
[`docs/runner-pool-integration.md`](../../docs/runner-pool-integration.md).
|
|
37
37
|
|
|
38
38
|
`GET /jobs/{id}` is also the harness's observability channel: `spans`, `followUps`
|
|
39
|
-
and `callMetrics` are **drain-on-read
|
|
39
|
+
and `callMetrics` are **drain-on-read**; each poll returns what accumulated since
|
|
40
40
|
the previous one and clears the buffer. That is deliberate. A job that dies before
|
|
41
41
|
it can return a terminal result (an evicted container, an OOM-killed process) has
|
|
42
42
|
still reported the tool spans it ran and the model calls it paid for. Each drained
|
|
@@ -44,8 +44,8 @@ still reported the tool spans it ran and the model calls it paid for. Each drain
|
|
|
44
44
|
complete list, so the backend can take both channels without double-counting a call.
|
|
45
45
|
|
|
46
46
|
Because the backend records a call as soon as it drains it (and ignores the terminal
|
|
47
|
-
repeat), a drained call is FINAL. A call whose tokens are still open
|
|
48
|
-
only a cumulative total, costed at the end
|
|
47
|
+
repeat), a drained call is FINAL. A call whose tokens are still open (a CLI that reports
|
|
48
|
+
only a cumulative total, costed at the end) is withheld from the drain until it is
|
|
49
49
|
complete; see `createCallMetricPublisher` in `src/pi.ts`.
|
|
50
50
|
|
|
51
51
|
## What a job does
|
|
@@ -55,54 +55,54 @@ The implementation job (`POST /run`) is the canonical sequence:
|
|
|
55
55
|
1. **clone** the target repo (shallow) with a short-lived GitHub installation token,
|
|
56
56
|
2. write the composed system prompt (role + the block's best-practice fragments)
|
|
57
57
|
to Pi's **global** context file `~/.pi/agent/AGENTS.md` (outside the checkout,
|
|
58
|
-
so it never lands in a commit and never clobbers a repo's own `AGENTS.md
|
|
58
|
+
so it never lands in a commit and never clobbers a repo's own `AGENTS.md`:
|
|
59
59
|
Pi reads and concatenates both), and point Pi at the Worker's LLM proxy via
|
|
60
|
-
`~/.pi/agent/models.json` (provider `proxy`, `api: openai-completions`)
|
|
60
|
+
`~/.pi/agent/models.json` (provider `proxy`, `api: openai-completions`): at the
|
|
61
61
|
phase-tagged completions path for the pass about to run (`.../phase/<phase>`) when the job
|
|
62
62
|
body's `proxyPhasePath` says the backend serves it, which is how a repair round's model spend
|
|
63
63
|
stays distinguishable from the first pass's in telemetry; without that flag the plain path is
|
|
64
64
|
used and the calls are recorded as unattributed
|
|
65
65
|
(see [token-burn instrumentation](../../../docs/initiatives/token-burn-instrumentation.md)),
|
|
66
|
-
3. **prepopulate dependencies**, when the job body carries `dependencyInstall
|
|
66
|
+
3. **prepopulate dependencies**, when the job body carries `dependencyInstall`: the
|
|
67
67
|
service's install command is run with `sh -c` in the checkout BEFORE the agent starts, so
|
|
68
68
|
it reads real installed packages instead of inferring a library's capabilities from a
|
|
69
69
|
manifest entry. Best-effort and never a gate: the outcome (success or the captured
|
|
70
|
-
failure) is folded into the agent's prompt
|
|
71
|
-
steps 6 and 7, which start a fresh agent
|
|
70
|
+
failure) is folded into the agent's prompt (on EVERY pass, including the repair passes of
|
|
71
|
+
steps 6 and 7, which start a fresh agent) and the run continues either way. Whatever the
|
|
72
72
|
install materialises is excluded from git first, so no later `git add -A` can sweep a
|
|
73
73
|
dependency tree into the pull request (see
|
|
74
74
|
[dependency prepopulation](../../../docs/initiatives/agent-dependency-prepopulation.md)),
|
|
75
|
-
4. **resolve the repo's pull-request template**, when this dispatch opens a PR (`src/pr-template.ts`)
|
|
76
|
-
|
|
75
|
+
4. **resolve the repo's pull-request template**, when this dispatch opens a PR (`src/pr-template.ts`):
|
|
76
|
+
`.github/PULL_REQUEST_TEMPLATE.md` and its root/`docs/`/multi-template-directory variants, or
|
|
77
77
|
GitLab's `.gitlab/merge_request_templates/`, read straight off the checkout (a symlinked template
|
|
78
|
-
is followed only while it resolves INSIDE the checkout
|
|
78
|
+
is followed only while it resolves INSIDE the checkout: this is the one repo-chosen path the
|
|
79
79
|
harness reads unprompted). Found, it is folded into the agent's prompt (on EVERY pass, as with
|
|
80
80
|
the install above) asking it to write its briefing AS that template, filled in. This exists
|
|
81
81
|
because neither host applies a template to an API-created pull request, so nothing else would:
|
|
82
82
|
the template only reaches the web form a human opens. A directory of several templates with no
|
|
83
|
-
`default` is left alone deliberately
|
|
83
|
+
`default` is left alone deliberately: it exists so a human can choose per pull request,
|
|
84
84
|
5. **run Pi** non-interactively (`pi -p --mode json --model proxy/<model> --approve`),
|
|
85
|
-
6. **validate** the checkout, when the job body carries `validationChecks
|
|
85
|
+
6. **validate** the checkout, when the job body carries `validationChecks`: the service's
|
|
86
86
|
configured check commands (install/lint/test/build) run with `sh -c` in the checkout, and
|
|
87
87
|
while they fail and the attempt budget remains the agent is re-run with the captured output
|
|
88
88
|
as its instruction (see [pre-PR validation](../../../docs/initiatives/pre-pr-validation.md)),
|
|
89
|
-
7. **prove the reproduction**, when the job body carries `reproduction
|
|
89
|
+
7. **prove the reproduction**, when the job body carries `reproduction`: the declared check is
|
|
90
90
|
run against the pre-fix tree and the tree the PR will open from, in two freshly-created
|
|
91
91
|
symmetric `git worktree` checkouts, and only red-then-green is reported as proof (see
|
|
92
|
-
[bugfix reproduction proof](
|
|
92
|
+
[bugfix reproduction proof](../../docs/adr/0033-bugfix-reproduction-proof.md)). Unlike
|
|
93
93
|
step 6 this NEVER gates the PR: a failed verification is fed back to the agent while budget
|
|
94
94
|
remains, then recorded as `inconclusive`. It runs BEFORE step 6 so validation stays the last
|
|
95
95
|
thing to touch the tree,
|
|
96
|
-
8. **commit, push** a branch and **open a PR**, returning `{ prUrl, branch, summary }
|
|
96
|
+
8. **commit, push** a branch and **open a PR**, returning `{ prUrl, branch, summary }`, but
|
|
97
97
|
ONLY if step 6 ended green. A spent budget returns an error result with the validation report
|
|
98
98
|
and opens no PR. Absent `validationChecks` / `reproduction`, steps 6 and 7 do not happen at
|
|
99
99
|
all. The PR's description prefers the agent-authored reviewer briefing over the generic
|
|
100
100
|
dispatch-time text the job body carries: a PR-opening agent is prompted to write one to the
|
|
101
101
|
`.cat-pr-description.md` sentinel at the checkout root (one per sibling repo in a multi-repo
|
|
102
102
|
run; an optional leading `# <title>` line, when it is the file's only `#` heading, sets the PR
|
|
103
|
-
title), and `src/pr-description.ts` lifts it
|
|
103
|
+
title), and `src/pr-description.ts` lifts it; secret-scrubbed, size-capped with a visible
|
|
104
104
|
note, made inert for the host by `src/host-markdown.ts`, kept out of the commit like the
|
|
105
|
-
effort/follow-ups sentinels
|
|
105
|
+
effort/follow-ups sentinels; onto `openPullRequest`. Absent or unusable ⇒ the fallback text,
|
|
106
106
|
unchanged. When the repo ships a template (step 4) that briefing IS the filled template: it
|
|
107
107
|
crosses the same scrub/cap/inert boundary on the way out, but the leading-`#` title rule is
|
|
108
108
|
switched OFF for it (`titleFromHeading: false`), because those headings are the repo's and
|
|
@@ -111,47 +111,60 @@ The implementation job (`POST /run`) is the canonical sequence:
|
|
|
111
111
|
title/description in place (carrying the engine's managed report region across); the generic
|
|
112
112
|
fallback never does, so a human's edit is safe.
|
|
113
113
|
|
|
114
|
-
Bootstrap differs at the ends
|
|
114
|
+
Bootstrap differs at the ends: it may start from an empty dir, and **resets
|
|
115
115
|
history to one commit and force-pushes** the default branch instead of opening a
|
|
116
116
|
PR. Blueprint **commits onto a branch** (no history reset) and returns the tree.
|
|
117
117
|
|
|
118
118
|
### Skills and tool servers
|
|
119
119
|
|
|
120
|
-
A job body may carry `skills[]` (procedural playbooks) and `mcpServers[]` (MCP tool servers)
|
|
120
|
+
A job body may carry `skills[]` (procedural playbooks) and `mcpServers[]` (MCP tool servers): the
|
|
121
121
|
harness MATERIALISES both and decides nothing about them; the backend has already resolved which
|
|
122
122
|
apply and dropped what this harness cannot serve (see
|
|
123
123
|
[`backend/docs/adr/0029-agent-kind-capabilities.md`](../../docs/adr/0029-agent-kind-capabilities.md)).
|
|
124
124
|
|
|
125
125
|
- **Skills** install natively under `CLAUDE_CONFIG_DIR/skills/<name>/` for a leased-credential
|
|
126
126
|
claude-code run (the CLI discovers and invokes them), and under
|
|
127
|
-
`.cat-context/skill/<name>/` in the checkout for Pi, Codex, and an AMBIENT claude-code run
|
|
127
|
+
`.cat-context/skill/<name>/` in the checkout for Pi, Codex, and an AMBIENT claude-code run:
|
|
128
128
|
whose prompt carries the instructions instead, because there is no isolated config home to
|
|
129
129
|
install into and the runner refuses to write into the developer's own `~/.claude`.
|
|
130
130
|
- **Tool servers** become a per-run `--mcp-config` file plus `--strict-mcp-config` for claude-code
|
|
131
131
|
(so an ambient run never picks up the developer's personal servers), and `[mcp_servers.*]` blocks
|
|
132
|
-
in the per-run `CODEX_HOME/config.toml` for Codex
|
|
133
|
-
ambient auth, which has no per-run home to write into.
|
|
132
|
+
in the per-run `CODEX_HOME/config.toml` for Codex: stdio only, and skipped entirely under
|
|
133
|
+
ambient auth, which has no per-run home to write into. Both stdio-only skips are now BACKSTOPS
|
|
134
|
+
rather than decisions: the backend knows which transports each harness reaches and drops an
|
|
135
|
+
`http` server from a Codex dispatch with a stated reason, so the prompt names the gap instead of
|
|
136
|
+
advertising a tool this side then silently omitted. `--allowedTools` is passed ONLY when a
|
|
134
137
|
server actually narrows its tools, and then carries the CLI's built-in tool names alongside the
|
|
135
|
-
`mcp__*` patterns
|
|
138
|
+
`mcp__*` patterns: an allow-list is whole-session, not MCP-scoped, so a bare list of MCP
|
|
136
139
|
patterns would leave the agent unable to read, edit or build anything. Whether the CLI gates on
|
|
137
140
|
that list at all is permission-mode dependent, so treat the narrowing as scoping rather than
|
|
138
|
-
enforcement; the prompt states it either way.
|
|
141
|
+
enforcement; the prompt states it either way. An `allowedTools` entry that is not a single tool
|
|
142
|
+
name is DROPPED at the boundary, the comma above all: the list is joined into one argument with
|
|
143
|
+
commas, so `search_issues,get_issue` in one entry would become a pattern matching nothing.
|
|
144
|
+
- **An `mcp__*` call is exempt from the no-edit progress bound**, like a read or a subagent
|
|
145
|
+
dispatch: reaching a wired tool server is what the prompt tells the agent to do, so counting it
|
|
146
|
+
would abort an edits-expected run for following its own instructions. It is neutral rather than
|
|
147
|
+
edit-satisfying, and bounded by its own consecutive-call cap
|
|
148
|
+
(`JOB_MAX_CONSECUTIVE_MCP_CALLS`) for the same reason the web cap exists. Every exempt family
|
|
149
|
+
ALSO shares one backstop (`JOB_MAX_CONSECUTIVE_NON_ACTION_CALLS`), because each per-family cap
|
|
150
|
+
resets on any call outside its own family: a run alternating a web search with a tool-server
|
|
151
|
+
lookup trips neither, and having made no action call it never reaches the no-edit bound either.
|
|
139
152
|
- **An `http` tool server must be `https`, or loopback.** Its headers carry a resolved credential,
|
|
140
153
|
so the job boundary refuses a cleartext off-box URL (the backend refuses the same at
|
|
141
154
|
registration). `secretKeys` names which `env`/`headers` entries are credentials, so exactly those
|
|
142
|
-
values are registered for redaction
|
|
155
|
+
values are registered for redaction: scrubbing the whole map would turn ordinary config strings
|
|
143
156
|
into `***` in every later log line.
|
|
144
157
|
|
|
145
158
|
Both config files carry this job's resolved credentials, so they are written to a per-job directory
|
|
146
|
-
(mode `0600`) and never into the checkout or a HOME-global path
|
|
159
|
+
(mode `0600`) and never into the checkout or a HOME-global path: see the next section.
|
|
147
160
|
|
|
148
161
|
## Per-job state: never a process- or HOME-global
|
|
149
162
|
|
|
150
163
|
A job's staging state (the tester's secrets, private-registry auth, a repo-sourced Claude
|
|
151
164
|
Skill) must be scoped to that job, not written into `process.env` or the home directory.
|
|
152
165
|
|
|
153
|
-
In a container those two ARE per-job
|
|
154
|
-
container
|
|
166
|
+
In a container those two ARE per-job (one job per process, and `HOME` belongs to that
|
|
167
|
+
container) so a global was a safe place to stage. The **local native transport** breaks both
|
|
155
168
|
assumptions: one long-lived host process serves every concurrent `ambientAuth` job, on the
|
|
156
169
|
**developer's own home**. A global there is shared mutable state across siblings, and writing
|
|
157
170
|
(or clearing) a dotfile destroys a file the developer owns.
|
|
@@ -162,16 +175,16 @@ under a per-job directory:
|
|
|
162
175
|
|
|
163
176
|
| State | Container | Native (`ambientAuth`) |
|
|
164
177
|
| -------------------- | -------------------------------------------- | ------------------------------------------------------------------------- |
|
|
165
|
-
| Tester secrets | child env | child env (same path
|
|
178
|
+
| Tester secrets | child env | child env (same path: the old `process.env` set/restore is gone) |
|
|
166
179
|
| Private-registry auth | `~/.npmrc`; cleared when a job has no entries | per-job `.npmrc` + `npm_config_userconfig`, seeded from the developer's; theirs is never written or removed |
|
|
167
|
-
| Repo-sourced Claude Skill | installed into the isolated `CLAUDE_CONFIG_DIR` | not installed
|
|
180
|
+
| Repo-sourced Claude Skill | installed into the isolated `CLAUDE_CONFIG_DIR` | not installed: read from the checkout's `.cat-context/skill/`, like codex |
|
|
168
181
|
|
|
169
182
|
Two consequences worth knowing:
|
|
170
183
|
|
|
171
184
|
- **The skill's PROMPT follows the same split.** A native install gets a short pointer; every
|
|
172
185
|
checkout-reading case (Pi, codex, ambient claude-code) gets the instructions folded in plus a
|
|
173
186
|
pointer to `.cat-context/skill/`. That decision is the backend's `renderSkillForHarness`, which
|
|
174
|
-
keys off `ambientAuth` as well as the harness
|
|
187
|
+
keys off `ambientAuth` as well as the harness: rendering an ambient run as an install would
|
|
175
188
|
point the agent at a skill that is nowhere on disk.
|
|
176
189
|
- **`npm_config_userconfig` reaches less than `~/.npmrc` did.** npm and pnpm honour it; yarn does
|
|
177
190
|
not. And it only reaches processes that are handed the job env, so anything the HARNESS itself
|
|
@@ -180,13 +193,13 @@ Two consequences worth knowing:
|
|
|
180
193
|
|
|
181
194
|
When you add per-job state, put it in one of those two places. `~/.pi/*` and
|
|
182
195
|
`~/.config/rpiv-web-tools` remain HOME-global, which is fine only because the Pi harness never
|
|
183
|
-
runs natively (the native router sends `ambientAuth` jobs
|
|
196
|
+
runs natively (the native router sends `ambientAuth` jobs (Claude/Codex only) to the host
|
|
184
197
|
process and everything else to a container).
|
|
185
198
|
|
|
186
199
|
## No secrets in the image
|
|
187
200
|
|
|
188
201
|
The image (built from the `Dockerfile`, base `node:26-trixie-slim`) contains
|
|
189
|
-
only `git` + the Pi CLI + this compiled wrapper
|
|
202
|
+
only `git` + the Pi CLI + this compiled wrapper: **no API keys, no GitHub
|
|
190
203
|
credentials**. Per job, the Worker passes a short-lived GitHub token and a
|
|
191
204
|
signed, model-locked LLM-proxy **session token** in the request body. Pi reaches
|
|
192
205
|
models only through the Worker proxy, which injects the real provider key (qwen /
|
|
@@ -197,22 +210,22 @@ Kimi / DeepSeek) and meters spend. The provider key never enters the container.
|
|
|
197
210
|
| File | Responsibility |
|
|
198
211
|
| ------------------ | ------------------------------------------------------------------------------------------------------- |
|
|
199
212
|
| `src/server.ts` | HTTP entry point; routes `/health`, `/run`, `/bootstrap`, `/blueprint`, `/jobs/{id}`. |
|
|
200
|
-
| `src/runner.ts` | `JobRegistry
|
|
213
|
+
| `src/runner.ts` | `JobRegistry`: async job lifecycle, idempotent on `jobId`, progress tracking. |
|
|
201
214
|
| `src/job.ts` | Request types + validators for the job specs. |
|
|
202
215
|
| `src/pi.ts` | Pi provider config, non-interactive run, JSON-line event + todo-progress parsing, global `AGENTS.md` guidance. |
|
|
203
216
|
| `src/git.ts` | clone / branch / commit / push + GitHub PR creation; bootstrap history reset + force-push. |
|
|
204
217
|
| `src/bootstrap.ts` | The `/bootstrap` handler (clone-or-empty → adapt → reinit + force-push). |
|
|
205
218
|
| `src/blueprint.ts` | The `/blueprint` handler (decompose → render `blueprints/` → commit on branch). |
|
|
206
219
|
| `src/embed.ts` | Bundled assets/templates written into the workspace. |
|
|
207
|
-
| `src/package-registries.ts` | Private-registry (npm) auth: renders the job's allowlisted entries into an npmrc
|
|
208
|
-
| `src/agent-runner.ts` | The subscription-harness runners (`runClaudeCode` / `runCodex`)
|
|
209
|
-
| `src/claude-call-aggregator.ts` | Folds Claude Code's per-CONTENT-BLOCK `stream-json` envelopes back into the model calls they belong to (by `message.id`), reconstructs each call's request transcript, and routes subagent turns off the parent's chain. **Exported as the `./claude-call-aggregator` subpath and driven by the BACKEND too** (`runtimes/local`, for an inline step running on the developer's host `claude`), so it stays the ONE implementation
|
|
220
|
+
| `src/package-registries.ts` | Private-registry (npm) auth: renders the job's allowlisted entries into an npmrc; the user `~/.npmrc` in a container, a per-job file pointed at by `npm_config_userconfig` for a native job. |
|
|
221
|
+
| `src/agent-runner.ts` | The subscription-harness runners (`runClaudeCode` / `runCodex`): talk direct to the vendor with a leased OAuth token, lift per-turn usage/telemetry off the CLI event stream. |
|
|
222
|
+
| `src/claude-call-aggregator.ts` | Folds Claude Code's per-CONTENT-BLOCK `stream-json` envelopes back into the model calls they belong to (by `message.id`), reconstructs each call's request transcript, and routes subagent turns off the parent's chain. **Exported as the `./claude-call-aggregator` subpath and driven by the BACKEND too** (`runtimes/local`, for an inline step running on the developer's host `claude`), so it stays the ONE implementation: the per-envelope over-count it fixes inflated a measured 1.47M tokens to 5.53M, and both drivers have to learn that only once. That second driver is why the transcript is retained only to `MAX_TRANSCRIPT_CHARS` (stating what it stopped retaining) and why assembling bodies at all is a `bodies` switch: in a container the reconstruction is one job's memory in a box sized for it, in the backend it is per concurrent inline step in the orchestrator process. Unlike the compile-only `./embed`, this subpath is a `dist` import, which is why the package emits declarations, and why a consumer's typecheck depends on Turbo's `^build` edge having built this package first (see `tsconfig.json`'s `comment:buildOrder`). |
|
|
210
223
|
| `src/transcript-retention.ts` | Lifts the CLI session transcripts (`projects/` / `sessions/`) out of the isolated, credential-bearing config home before it is deleted, and prunes them on a TTL (debugging artifact retention). |
|
|
211
224
|
| `src/captured-command.ts` | The one way the harness runs a declared shell command on its own behalf: `sh -c` with a per-command watchdog, abort handling, conventional exit codes (124/127/130) and a scrub-then-bound output capture. Shared by both pre-PR verification phases so a fix to one cannot miss the other. |
|
|
212
|
-
| `src/dependency-install.ts` | Dependency prepopulation: `prepopulateDependencies` is the ONE seam every checkout-having mode calls
|
|
213
|
-
| `src/validation-checks.ts` | Pre-PR validation: runs the job's check commands in the checkout (bounded, secret-scrubbed capture, per-command watchdog) and drives the retry-until-green loop that gates the PR. Generic
|
|
214
|
-
| `src/reproduction-proof.ts` | Bugfix reproduction proof: runs the job's declared reproduction command against two symmetric fresh worktrees (the pre-fix tree and the final tree) and computes red-then-green from the exit codes, with a repair loop that never fails the run. Generic
|
|
215
|
-
| `src/agent-capabilities.ts` | The agent CAPABILITIES a job body carries
|
|
225
|
+
| `src/dependency-install.ts` | Dependency prepopulation: `prepopulateDependencies` is the ONE seam every checkout-having mode calls; it runs the service's install command before the agent's first turn, excludes what the install materialised from git so no `git add -A` can sweep a dependency tree into the PR, and builds the prompt note describing the outcome. Best-effort: every failure shape becomes a note, never a failed job. Generic: keyed off the job body, never the agent kind. |
|
|
226
|
+
| `src/validation-checks.ts` | Pre-PR validation: runs the job's check commands in the checkout (bounded, secret-scrubbed capture, per-command watchdog) and drives the retry-until-green loop that gates the PR. Generic: keyed off the job body, never the agent kind. |
|
|
227
|
+
| `src/reproduction-proof.ts` | Bugfix reproduction proof: runs the job's declared reproduction command against two symmetric fresh worktrees (the pre-fix tree and the final tree) and computes red-then-green from the exit codes, with a repair loop that never fails the run. Generic: keyed off the job body, never the agent kind. |
|
|
228
|
+
| `src/agent-capabilities.ts` | The agent CAPABILITIES a job body carries: the run's `skills` (a `SKILL.md` payload + resources) and its `mcpServers` (tool servers): with their defensive parsing and the per-CLI config writers (`--mcp-config` JSON for claude-code, `[mcp_servers.*]` TOML for Codex). Backend-authored data the harness only MATERIALISES: adding a skill or a tool server is a backend registration, never a harness change. |
|
|
216
229
|
| `src/bootstrap-mode.ts` | The repo-bootstrap MODE: clone-a-reference-or-scaffold → run the agent → refuse to push an empty tree → reinit + force-push to the pre-created target repo. |
|
|
217
230
|
| `src/agent-shared.ts` | The few helpers every agent MODE shares (effort-report folding, the capability fields forwarded to `runAgentInWorkspace`). |
|
|
218
231
|
| `src/logger.ts` | Structured logging. |
|
|
@@ -227,15 +240,17 @@ runner):
|
|
|
227
240
|
| `PORT` | `8080` | HTTP port the harness listens on. |
|
|
228
241
|
| `JOB_MAX_DURATION_MS` | `3600000` (60m) | Hard ceiling on a job's wall-clock time; force-fails after. |
|
|
229
242
|
| `JOB_INACTIVITY_MS` | `600000` (10m) | Kills a hung agent that produces no output for this long. |
|
|
230
|
-
| `
|
|
231
|
-
| `
|
|
243
|
+
| `JOB_MAX_CONSECUTIVE_MCP_CALLS` | `40` | Consecutive tool-server (`mcp__*`) calls with no other tool call between before the run counts as a lookup loop. The counter-bound the no-edit exemption above owes; a per-kind `tuning.guardLimits` entry can only RAISE it. |
|
|
244
|
+
| `JOB_MAX_CONSECUTIVE_NON_ACTION_CALLS` | `200` | Consecutive calls of ANY no-edit-exempt family (reads, searches, web, tool servers, subagent dispatches) with no action call between them. The backstop above the per-family caps, since each of those resets on a call outside its own family; sized as a backstop rather than a research judgement, and reset by any `bash`/edit. |
|
|
245
|
+
| `JOB_COLD_START_MS` | `120000` (2m) | First-output window (ADR 0026 D4). A job that has produced nothing this long records a cold-start diagnostic (a likely onboarding/auth wedge) WITHOUT being killed: logged, exposed on `GET /jobs/{id}`, and folded into the failure `detail` if the job goes on to fail. `0` disables it. |
|
|
246
|
+
| `DEPENDENCY_INSTALL_TIMEOUT_MS` | a third of `JOB_MAX_DURATION_MS` (20m at its default) | Watchdog for the pre-agent dependency install; a timeout is reported as a failed install (exit 124), never a failed job. Derived from the job ceiling rather than fixed, and an explicit value is clamped by the same share: the agent is what waits on this, so setup can never consume the run it is preparing for. |
|
|
232
247
|
| `DEPENDENCY_INSTALL_HEARTBEAT_MS` | `30000` (30s) | How often the dependency install feeds the job inactivity watchdog. A cold install is activity-silent and `JOB_INACTIVITY_MS` is tighter than its own watchdog, so without this a healthy install aborts the run as "likely hung". |
|
|
233
248
|
| `VALIDATION_COMMAND_TIMEOUT_MS` | `900000` (15m) | Per-command watchdog for a pre-PR validation check; a timeout counts as a failure (exit 124) so one hung command can't wedge the loop. |
|
|
234
249
|
| `REPRODUCTION_COMMAND_TIMEOUT_MS` | `900000` (15m) | Per-command watchdog for a reproduction-proof setup or check command; a timeout counts as a failure (exit 124). |
|
|
235
250
|
| `REPRODUCTION_HEARTBEAT_MS` | `30000` (30s) | How often the reproduction proof feeds the job inactivity watchdog while it runs commands the agent is not producing output for. |
|
|
236
251
|
| `REPRODUCTION_TOTAL_BUDGET_MS` | `2700000` (45m) | Wall-clock ceiling on the WHOLE proof phase (every attempt, both trees, setup included). Attempts multiply two full tree runs each and the heartbeat above deliberately stops the inactivity watchdog from firing, so this is what bounds the phase. Checked at phase boundaries; exceeding it settles `inconclusive`, never a run failure. |
|
|
237
252
|
| `HARNESS_TRANSCRIPT_TTL_MS` | `259200000` (3d) | How long lifted subscription-CLI session transcripts are kept before the retention sweep prunes them. |
|
|
238
|
-
| `HARNESS_TRANSCRIPT_ROOT` | `<tmpdir>/cf-agent-transcripts` | Where retained session transcripts are moved to (one dir per run). Meaningful only on a reused (warm-pool) container; a per-run container is torn down with the job. The TTL sweep deletes only dirs it created (each carries a `.cf-retained` marker), so pointing this at a shared directory never touches unrelated content
|
|
253
|
+
| `HARNESS_TRANSCRIPT_ROOT` | `<tmpdir>/cf-agent-transcripts` | Where retained session transcripts are moved to (one dir per run). Meaningful only on a reused (warm-pool) container; a per-run container is torn down with the job. The TTL sweep deletes only dirs it created (each carries a `.cf-retained` marker), so pointing this at a shared directory never touches unrelated content, though a dedicated dir is still recommended. An override on a different filesystem than the config home falls back to copy-then-remove. |
|
|
239
254
|
|
|
240
255
|
## Build / test
|
|
241
256
|
|
|
@@ -283,11 +298,11 @@ pnpm --filter @cat-factory/executor-harness run image:publish
|
|
|
283
298
|
The script ([`scripts/publish-image.sh`](./scripts/publish-image.sh)) builds the
|
|
284
299
|
multi-arch image once and pushes it to the selected registries. Override defaults
|
|
285
300
|
via env vars (`REGISTRIES`, `GHCR_OWNER`, `DOCKERHUB_ORG`, `TAG`, `PUSH_LATEST`,
|
|
286
|
-
`PLATFORMS`, `EXTRA_CA`)
|
|
301
|
+
`PLATFORMS`, `EXTRA_CA`): see the header of the script. Example: GHCR only;
|
|
287
302
|
`REGISTRIES=ghcr pnpm --filter @cat-factory/executor-harness run image:publish`.
|
|
288
303
|
|
|
289
304
|
A backend deployment references the image from `wrangler.toml`
|
|
290
|
-
(`[[containers]] image = "ghcr.io/<owner>/cat-factory-executor:<version>"
|
|
305
|
+
(`[[containers]] image = "ghcr.io/<owner>/cat-factory-executor:<version>"`: see
|
|
291
306
|
[`deploy/backend`](../../../deploy/backend)); a self-hosted runner pool pulls the
|
|
292
307
|
same image (see [`docs/runner-pool-integration.md`](../../docs/runner-pool-integration.md)).
|
|
293
308
|
The worker library's own test/dev `wrangler.toml` still references this
|
|
@@ -64,6 +64,19 @@ export declare function parseSkillSpecs(value: unknown): SkillSpec[] | undefined
|
|
|
64
64
|
* `src/host-markdown.ts` uses.
|
|
65
65
|
*/
|
|
66
66
|
export declare const MCP_SERVER_ID_PATTERN: RegExp;
|
|
67
|
+
/**
|
|
68
|
+
* A tool name an `allowedTools` entry may name. Kept byte-identical to kernel's
|
|
69
|
+
* `MCP_TOOL_NAME_PATTERN` for the same reason {@link MCP_SERVER_ID_PATTERN} is a copy, and pinned
|
|
70
|
+
* against it by `test/agent-capabilities.conformity.test.ts`.
|
|
71
|
+
*
|
|
72
|
+
* The comma is the reason the rule exists on THIS side of the boundary too:
|
|
73
|
+
* {@link claudeAllowedToolPatterns} builds the list that the runner joins into one
|
|
74
|
+
* `--allowedTools` argument with commas, so an entry carrying one splits into two patterns of which
|
|
75
|
+
* the second matches no tool the CLI has. Dropped rather than passed through, because the entries
|
|
76
|
+
* that survive are what narrows the session: a bad one would silently take the run's whole MCP
|
|
77
|
+
* surface with it.
|
|
78
|
+
*/
|
|
79
|
+
export declare const MCP_TOOL_NAME_PATTERN: RegExp;
|
|
67
80
|
/**
|
|
68
81
|
* Whether an HTTP tool server's URL may be started. Mirrors kernel's `isAllowedMcpHttpUrl` (see
|
|
69
82
|
* {@link MCP_SERVER_ID_PATTERN} for why it is a copy, and the conformity suite that pins it):
|
|
@@ -116,9 +129,13 @@ export declare const CLAUDE_BUILT_IN_TOOLS: readonly string[];
|
|
|
116
129
|
export declare function claudeAllowedToolPatterns(servers: McpServerSpec[]): string[] | undefined;
|
|
117
130
|
/**
|
|
118
131
|
* The `[mcp_servers.<id>]` TOML block Codex reads from its `CODEX_HOME/config.toml`. Codex's MCP
|
|
119
|
-
* client is stdio-only, so an `http` server is skipped here
|
|
120
|
-
*
|
|
121
|
-
*
|
|
132
|
+
* client is stdio-only, so an `http` server is skipped here.
|
|
133
|
+
*
|
|
134
|
+
* The skip is now a BACKSTOP rather than the decision: the backend knows which transports each
|
|
135
|
+
* harness reaches (`MCP_HARNESS_TRANSPORTS`) and drops an `http` server from a Codex dispatch under
|
|
136
|
+
* its own `transport_unsupported` reason, so the prompt states the gap instead of advertising a tool
|
|
137
|
+
* this writer then silently omitted. It stays because a body that reached the container by any other
|
|
138
|
+
* route must still produce a valid config rather than a malformed one.
|
|
122
139
|
*/
|
|
123
140
|
export declare function codexMcpConfigToml(servers: McpServerSpec[]): string;
|
|
124
141
|
/**
|
|
@@ -125,6 +125,19 @@ function sanitizeServerId(value) {
|
|
|
125
125
|
return undefined;
|
|
126
126
|
return MCP_SERVER_ID_PATTERN.test(value) ? value : undefined;
|
|
127
127
|
}
|
|
128
|
+
/**
|
|
129
|
+
* A tool name an `allowedTools` entry may name. Kept byte-identical to kernel's
|
|
130
|
+
* `MCP_TOOL_NAME_PATTERN` for the same reason {@link MCP_SERVER_ID_PATTERN} is a copy, and pinned
|
|
131
|
+
* against it by `test/agent-capabilities.conformity.test.ts`.
|
|
132
|
+
*
|
|
133
|
+
* The comma is the reason the rule exists on THIS side of the boundary too:
|
|
134
|
+
* {@link claudeAllowedToolPatterns} builds the list that the runner joins into one
|
|
135
|
+
* `--allowedTools` argument with commas, so an entry carrying one splits into two patterns of which
|
|
136
|
+
* the second matches no tool the CLI has. Dropped rather than passed through, because the entries
|
|
137
|
+
* that survive are what narrows the session: a bad one would silently take the run's whole MCP
|
|
138
|
+
* surface with it.
|
|
139
|
+
*/
|
|
140
|
+
export const MCP_TOOL_NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/;
|
|
128
141
|
/**
|
|
129
142
|
* Whether an HTTP tool server's URL may be started. Mirrors kernel's `isAllowedMcpHttpUrl` (see
|
|
130
143
|
* {@link MCP_SERVER_ID_PATTERN} for why it is a copy, and the conformity suite that pins it):
|
|
@@ -166,6 +179,17 @@ function parseStringArray(value) {
|
|
|
166
179
|
const out = value.filter((v) => typeof v === 'string');
|
|
167
180
|
return out.length ? out : undefined;
|
|
168
181
|
}
|
|
182
|
+
/**
|
|
183
|
+
* The `allowedTools` list: string entries that are single tool NAMES (see
|
|
184
|
+
* {@link MCP_TOOL_NAME_PATTERN}). Undefined when nothing survives, which is the same answer as an
|
|
185
|
+
* absent field (every tool the server exposes) and the right one: the alternative is a list whose
|
|
186
|
+
* only surviving entries are the platform's own built-in tool names, i.e. a run narrowed to no MCP
|
|
187
|
+
* tools at all. The backend refuses these at registration; this is the boundary check.
|
|
188
|
+
*/
|
|
189
|
+
function parseAllowedTools(value) {
|
|
190
|
+
const names = parseStringArray(value)?.filter((name) => MCP_TOOL_NAME_PATTERN.test(name));
|
|
191
|
+
return names?.length ? names : undefined;
|
|
192
|
+
}
|
|
169
193
|
/** Validate one `mcpServers` entry, or undefined when malformed for its transport. */
|
|
170
194
|
function parseMcpServerSpec(value) {
|
|
171
195
|
if (typeof value !== 'object' || value === null)
|
|
@@ -174,7 +198,7 @@ function parseMcpServerSpec(value) {
|
|
|
174
198
|
const id = sanitizeServerId(o.id);
|
|
175
199
|
if (!id)
|
|
176
200
|
return undefined;
|
|
177
|
-
const allowedTools =
|
|
201
|
+
const allowedTools = parseAllowedTools(o.allowedTools);
|
|
178
202
|
const secretKeys = parseStringArray(o.secretKeys);
|
|
179
203
|
if (o.transport === 'http') {
|
|
180
204
|
// https anywhere, plain http only on loopback: the CLI would happily be pointed at a
|
|
@@ -313,9 +337,13 @@ function tomlString(value) {
|
|
|
313
337
|
}
|
|
314
338
|
/**
|
|
315
339
|
* The `[mcp_servers.<id>]` TOML block Codex reads from its `CODEX_HOME/config.toml`. Codex's MCP
|
|
316
|
-
* client is stdio-only, so an `http` server is skipped here
|
|
317
|
-
*
|
|
318
|
-
*
|
|
340
|
+
* client is stdio-only, so an `http` server is skipped here.
|
|
341
|
+
*
|
|
342
|
+
* The skip is now a BACKSTOP rather than the decision: the backend knows which transports each
|
|
343
|
+
* harness reaches (`MCP_HARNESS_TRANSPORTS`) and drops an `http` server from a Codex dispatch under
|
|
344
|
+
* its own `transport_unsupported` reason, so the prompt states the gap instead of advertising a tool
|
|
345
|
+
* this writer then silently omitted. It stays because a body that reached the container by any other
|
|
346
|
+
* route must still produce a valid config rather than a malformed one.
|
|
319
347
|
*/
|
|
320
348
|
export function codexMcpConfigToml(servers) {
|
|
321
349
|
const blocks = [];
|
package/dist/job.d.ts
CHANGED
|
@@ -493,6 +493,8 @@ export interface GuardLimitsSpec {
|
|
|
493
493
|
maxToolCallsWithoutEdit?: number;
|
|
494
494
|
maxConsecutiveErrors?: number;
|
|
495
495
|
maxConsecutiveWebCalls?: number;
|
|
496
|
+
maxConsecutiveMcpCalls?: number;
|
|
497
|
+
maxConsecutiveNonActionCalls?: number;
|
|
496
498
|
}
|
|
497
499
|
/**
|
|
498
500
|
* The record of standing the service's docker-compose dependencies up before a tester
|
package/dist/job.js
CHANGED
|
@@ -36,12 +36,18 @@ function parseGuardLimits(value) {
|
|
|
36
36
|
const noEdit = posInt(o.maxToolCallsWithoutEdit);
|
|
37
37
|
const errors = posInt(o.maxConsecutiveErrors);
|
|
38
38
|
const web = posInt(o.maxConsecutiveWebCalls);
|
|
39
|
+
const mcp = posInt(o.maxConsecutiveMcpCalls);
|
|
40
|
+
const nonAction = posInt(o.maxConsecutiveNonActionCalls);
|
|
39
41
|
if (noEdit !== undefined)
|
|
40
42
|
spec.maxToolCallsWithoutEdit = noEdit;
|
|
41
43
|
if (errors !== undefined)
|
|
42
44
|
spec.maxConsecutiveErrors = errors;
|
|
43
45
|
if (web !== undefined)
|
|
44
46
|
spec.maxConsecutiveWebCalls = web;
|
|
47
|
+
if (mcp !== undefined)
|
|
48
|
+
spec.maxConsecutiveMcpCalls = mcp;
|
|
49
|
+
if (nonAction !== undefined)
|
|
50
|
+
spec.maxConsecutiveNonActionCalls = nonAction;
|
|
45
51
|
return Object.keys(spec).length > 0 ? spec : undefined;
|
|
46
52
|
}
|
|
47
53
|
/**
|
package/dist/progress-guard.d.ts
CHANGED
|
@@ -35,11 +35,38 @@ export interface ProgressGuardLimits {
|
|
|
35
35
|
* without it.
|
|
36
36
|
*/
|
|
37
37
|
maxConsecutiveWebCalls?: number;
|
|
38
|
+
/**
|
|
39
|
+
* Abort after this many consecutive MCP tool-server calls (`mcp__*`) with no other
|
|
40
|
+
* tool call in between: the tool-server analogue of `maxConsecutiveWebCalls`, and
|
|
41
|
+
* present for the same reason. An `mcp__*` call is exempt from the no-edit bound (see
|
|
42
|
+
* `isMcpToolCall`), so without a streak of its own a run could query a tool server
|
|
43
|
+
* indefinitely without tripping any guard. Any non-MCP tool call resets the streak.
|
|
44
|
+
* Optional: defaults to {@link DEFAULT_PROGRESS_GUARD_LIMITS}.
|
|
45
|
+
*/
|
|
46
|
+
maxConsecutiveMcpCalls?: number;
|
|
47
|
+
/**
|
|
48
|
+
* Abort after this many consecutive calls that are EXEMPT from the no-edit bound
|
|
49
|
+
* (planning, read-only exploration, subagent dispatch, `mcp__*`) with no action call
|
|
50
|
+
* in between. The backstop that makes each individual exemption mean "not counted"
|
|
51
|
+
* rather than "unbounded": every per-family streak above resets on any call outside
|
|
52
|
+
* its own family, so a run alternating `web_search` with `mcp__issues__search` (or
|
|
53
|
+
* with `read`) trips none of them and, having never made an action call, never
|
|
54
|
+
* reaches `maxToolCallsWithoutEdit` either. Only the job's wall-clock ceiling
|
|
55
|
+
* bounded that.
|
|
56
|
+
*
|
|
57
|
+
* Deliberately far above every family cap, because it is not a research bound and
|
|
58
|
+
* must not become one: reading a hundred files before the first edit is legitimate
|
|
59
|
+
* work-up, and any `bash`/edit/action call resets the streak. Optional: defaults to
|
|
60
|
+
* {@link DEFAULT_PROGRESS_GUARD_LIMITS}.
|
|
61
|
+
*/
|
|
62
|
+
maxConsecutiveNonActionCalls?: number;
|
|
38
63
|
}
|
|
39
64
|
export declare const DEFAULT_PROGRESS_GUARD_LIMITS: {
|
|
40
65
|
maxToolCallsWithoutEdit: number;
|
|
41
66
|
maxConsecutiveErrors: number;
|
|
42
67
|
maxConsecutiveWebCalls: number;
|
|
68
|
+
maxConsecutiveMcpCalls: number;
|
|
69
|
+
maxConsecutiveNonActionCalls: number;
|
|
43
70
|
};
|
|
44
71
|
/** Read {@link ProgressGuardLimits} from the environment, falling back to the defaults. */
|
|
45
72
|
export declare function progressGuardLimitsFromEnv(env?: NodeJS.ProcessEnv): ProgressGuardLimits;
|
|
@@ -69,6 +96,8 @@ export declare class ProgressGuard {
|
|
|
69
96
|
private edits;
|
|
70
97
|
private consecutiveErrors;
|
|
71
98
|
private consecutiveWebCalls;
|
|
99
|
+
private consecutiveMcpCalls;
|
|
100
|
+
private consecutiveNonActionCalls;
|
|
72
101
|
constructor(limits: ProgressGuardLimits,
|
|
73
102
|
/** When false (assess-only runs like the merger), the no-edit bound is skipped. */
|
|
74
103
|
expectsEdits?: boolean);
|
package/dist/progress-guard.js
CHANGED
|
@@ -28,6 +28,16 @@ export const DEFAULT_PROGRESS_GUARD_LIMITS = {
|
|
|
28
28
|
// A genuine research burst is a handful of searches; an uninterrupted run of this
|
|
29
29
|
// many web calls (with no read/edit/bash between) is a search loop, not progress.
|
|
30
30
|
maxConsecutiveWebCalls: 25,
|
|
31
|
+
// Looser than the web cap: a tool server is usually the agent's route to the SYSTEM OF
|
|
32
|
+
// RECORD (the issue tracker, the advisory database, the design source), and reading a
|
|
33
|
+
// list and then each of its items is a normal opening move, not a rabbit-hole. A run
|
|
34
|
+
// that makes this many in a row with no read, edit or bash between is looping.
|
|
35
|
+
maxConsecutiveMcpCalls: 40,
|
|
36
|
+
// Well clear of every family cap above, and of any plausible read-up: a run that makes
|
|
37
|
+
// this many exempt calls with not one action call between them has stopped converging,
|
|
38
|
+
// whatever mix of reads, searches and lookups it is cycling through. Sized as a
|
|
39
|
+
// backstop rather than a judgement, because the families are where judgement belongs.
|
|
40
|
+
maxConsecutiveNonActionCalls: 200,
|
|
31
41
|
};
|
|
32
42
|
// Tool names that mutate files, so a call to one clears the no-edit suspicion. Kept
|
|
33
43
|
// broad on purpose: different models/extensions name the same capability differently
|
|
@@ -98,6 +108,28 @@ const EXPLORATION_TOOLS = new Set([
|
|
|
98
108
|
// call between) can be caught as a search loop — see `maxConsecutiveWebCalls`. Covers both
|
|
99
109
|
// Pi's `web_search`/`web_fetch` and Claude Code's `WebSearch`/`WebFetch`.
|
|
100
110
|
const WEB_TOOLS = new Set(['web_search', 'web_fetch', 'websearch', 'webfetch']);
|
|
111
|
+
// A call to a tool server (MCP). Every MCP client names these `mcp__<server>__<tool>`, and
|
|
112
|
+
// the prefix is the ONLY thing the harness can know about them: what a given server's tools
|
|
113
|
+
// do is a backend registration this image has never seen, so the guard classifies by shape.
|
|
114
|
+
//
|
|
115
|
+
// Matched, rather than enumerated in EXPLORATION_TOOLS, because the set is open: it is
|
|
116
|
+
// whatever tool servers the running kind was wired with. A prefix test is also why this is a
|
|
117
|
+
// function, `name.startsWith` on the already-lower-cased name, so `MCP__Issues__search`
|
|
118
|
+
// classifies the same as `mcp__issues__search`.
|
|
119
|
+
function isMcpToolCall(loweredName) {
|
|
120
|
+
return loweredName.startsWith('mcp__');
|
|
121
|
+
}
|
|
122
|
+
// Whether a call is EXEMPT from the no-edit bound: planning and bookkeeping, read-only
|
|
123
|
+
// exploration, a subagent dispatch, or a tool-server call. One predicate rather than the
|
|
124
|
+
// four tests inlined at the branch, because the combined non-action streak and the no-edit
|
|
125
|
+
// exemption must be the SAME set: a family exempted in one place and missed in the other is
|
|
126
|
+
// either an unbounded loop or a run killed for a call the bound says it may make.
|
|
127
|
+
function isNonActionToolCall(loweredName) {
|
|
128
|
+
return (PLANNING_TOOLS.has(loweredName) ||
|
|
129
|
+
EXPLORATION_TOOLS.has(loweredName) ||
|
|
130
|
+
SUBAGENT_DISPATCH_TOOLS.has(loweredName) ||
|
|
131
|
+
isMcpToolCall(loweredName));
|
|
132
|
+
}
|
|
101
133
|
/** Read {@link ProgressGuardLimits} from the environment, falling back to the defaults. */
|
|
102
134
|
export function progressGuardLimitsFromEnv(env = process.env) {
|
|
103
135
|
const num = (raw, fallback) => {
|
|
@@ -108,6 +140,8 @@ export function progressGuardLimitsFromEnv(env = process.env) {
|
|
|
108
140
|
maxToolCallsWithoutEdit: num(env.JOB_MAX_TOOLCALLS_WITHOUT_EDIT, DEFAULT_PROGRESS_GUARD_LIMITS.maxToolCallsWithoutEdit),
|
|
109
141
|
maxConsecutiveErrors: num(env.JOB_MAX_CONSECUTIVE_TOOL_ERRORS, DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveErrors),
|
|
110
142
|
maxConsecutiveWebCalls: num(env.JOB_MAX_CONSECUTIVE_WEB_CALLS, DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveWebCalls),
|
|
143
|
+
maxConsecutiveMcpCalls: num(env.JOB_MAX_CONSECUTIVE_MCP_CALLS, DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveMcpCalls),
|
|
144
|
+
maxConsecutiveNonActionCalls: num(env.JOB_MAX_CONSECUTIVE_NON_ACTION_CALLS, DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveNonActionCalls),
|
|
111
145
|
};
|
|
112
146
|
}
|
|
113
147
|
/**
|
|
@@ -127,9 +161,12 @@ export function mergeGuardLimits(base, overrides) {
|
|
|
127
161
|
return {
|
|
128
162
|
maxToolCallsWithoutEdit: loosen(base.maxToolCallsWithoutEdit, overrides.maxToolCallsWithoutEdit),
|
|
129
163
|
maxConsecutiveErrors: loosen(base.maxConsecutiveErrors, overrides.maxConsecutiveErrors),
|
|
130
|
-
//
|
|
131
|
-
//
|
|
164
|
+
// The streak knobs are optional on the interface (callers may omit them), so fall back
|
|
165
|
+
// to the default before loosening: it keeps `loosen`'s base a concrete number.
|
|
132
166
|
maxConsecutiveWebCalls: loosen(base.maxConsecutiveWebCalls ?? DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveWebCalls, overrides.maxConsecutiveWebCalls),
|
|
167
|
+
maxConsecutiveMcpCalls: loosen(base.maxConsecutiveMcpCalls ?? DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveMcpCalls, overrides.maxConsecutiveMcpCalls),
|
|
168
|
+
maxConsecutiveNonActionCalls: loosen(base.maxConsecutiveNonActionCalls ??
|
|
169
|
+
DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveNonActionCalls, overrides.maxConsecutiveNonActionCalls),
|
|
133
170
|
};
|
|
134
171
|
}
|
|
135
172
|
/**
|
|
@@ -146,6 +183,8 @@ export class ProgressGuard {
|
|
|
146
183
|
edits = 0;
|
|
147
184
|
consecutiveErrors = 0;
|
|
148
185
|
consecutiveWebCalls = 0;
|
|
186
|
+
consecutiveMcpCalls = 0;
|
|
187
|
+
consecutiveNonActionCalls = 0;
|
|
149
188
|
constructor(limits,
|
|
150
189
|
/** When false (assess-only runs like the merger), the no-edit bound is skipped. */
|
|
151
190
|
expectsEdits = true) {
|
|
@@ -189,14 +228,49 @@ export class ProgressGuard {
|
|
|
189
228
|
else {
|
|
190
229
|
this.consecutiveWebCalls = 0;
|
|
191
230
|
}
|
|
192
|
-
//
|
|
193
|
-
// no-edit bound
|
|
194
|
-
//
|
|
195
|
-
if (
|
|
196
|
-
|
|
197
|
-
|
|
231
|
+
// Tool-server (MCP) calls: bounded as their own streak for exactly the reason the web
|
|
232
|
+
// streak exists. They are exempt from the no-edit bound below, and an exemption with no
|
|
233
|
+
// counter-bound is a loop the guard cannot see. Any non-MCP call resets it.
|
|
234
|
+
if (isMcpToolCall(name)) {
|
|
235
|
+
this.consecutiveMcpCalls++;
|
|
236
|
+
const mcpCap = this.limits.maxConsecutiveMcpCalls ?? DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveMcpCalls;
|
|
237
|
+
if (this.consecutiveMcpCalls >= mcpCap) {
|
|
238
|
+
return (`no progress: ${this.consecutiveMcpCalls} consecutive tool-server (MCP) calls without ` +
|
|
239
|
+
`any other action. The agent is stuck querying its tools instead of doing the work. ` +
|
|
240
|
+
`Aborting.`);
|
|
241
|
+
}
|
|
242
|
+
}
|
|
243
|
+
else {
|
|
244
|
+
this.consecutiveMcpCalls = 0;
|
|
245
|
+
}
|
|
246
|
+
// Planning, read-only exploration, subagent-dispatch and tool-server calls don't count
|
|
247
|
+
// toward the no-edit bound (see `isNonActionToolCall`): only "action" calls without an
|
|
248
|
+
// edit do.
|
|
249
|
+
//
|
|
250
|
+
// An `mcp__*` call is exempt for the same reason a `read` is: the bound targets the
|
|
251
|
+
// credential rabbit-hole (endless `bash` probing with nothing implemented), and reaching
|
|
252
|
+
// a registered tool server is the platform TELLING the agent to look something up
|
|
253
|
+
// ("prefer them over guessing"). Counting them would abort an edits-expected kind for
|
|
254
|
+
// consulting the issue tracker the deployment wired for it, punishing the run for
|
|
255
|
+
// following its own prompt. They are neutral rather than edit-satisfying, exactly like a
|
|
256
|
+
// subagent dispatch: a read-only lookup must not clear the suspicion the bound holds.
|
|
257
|
+
//
|
|
258
|
+
// The exempt calls carry ONE streak of their own, and it is what keeps every exemption
|
|
259
|
+
// above from adding up to an unbounded run: each per-family cap resets on any call
|
|
260
|
+
// outside its family, so alternating two exempt families trips neither, and a run that
|
|
261
|
+
// never makes an action call never reaches the no-edit bound either.
|
|
262
|
+
if (isNonActionToolCall(name)) {
|
|
263
|
+
this.consecutiveNonActionCalls++;
|
|
264
|
+
const nonActionCap = this.limits.maxConsecutiveNonActionCalls ??
|
|
265
|
+
DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveNonActionCalls;
|
|
266
|
+
if (this.consecutiveNonActionCalls >= nonActionCap) {
|
|
267
|
+
return (`no progress: ${this.consecutiveNonActionCalls} consecutive read-only calls (searching, ` +
|
|
268
|
+
`reading, tool-server lookups, subagent dispatches) with no action call between them. ` +
|
|
269
|
+
`The agent is cycling through research instead of doing the work. Aborting.`);
|
|
270
|
+
}
|
|
198
271
|
return null;
|
|
199
272
|
}
|
|
273
|
+
this.consecutiveNonActionCalls = 0;
|
|
200
274
|
this.toolCalls++;
|
|
201
275
|
if (FILE_EDIT_TOOLS.has(name))
|
|
202
276
|
this.edits++;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cat-factory/executor-harness",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.90.0",
|
|
4
4
|
"description": "Container payload: a thin TypeScript wrapper that runs the Pi coding agent against a cloned repo and opens a PR. Runs in the Cloudflare Container (and, in local native mode, as a host process); carries no secrets.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -30,9 +30,9 @@
|
|
|
30
30
|
"hono": "^4.12.33",
|
|
31
31
|
"typescript": "7.0.2",
|
|
32
32
|
"vitest": "^4.1.10",
|
|
33
|
-
"@cat-factory/kernel": "0.
|
|
34
|
-
"@cat-factory/server": "0.
|
|
35
|
-
"@cat-factory/spend": "0.
|
|
33
|
+
"@cat-factory/kernel": "0.232.0",
|
|
34
|
+
"@cat-factory/server": "0.210.0",
|
|
35
|
+
"@cat-factory/spend": "0.14.7"
|
|
36
36
|
},
|
|
37
37
|
"scripts": {
|
|
38
38
|
"build": "tsc -p tsconfig.json",
|
|
@@ -178,6 +178,20 @@ function sanitizeServerId(value: unknown): string | undefined {
|
|
|
178
178
|
return MCP_SERVER_ID_PATTERN.test(value) ? value : undefined
|
|
179
179
|
}
|
|
180
180
|
|
|
181
|
+
/**
|
|
182
|
+
* A tool name an `allowedTools` entry may name. Kept byte-identical to kernel's
|
|
183
|
+
* `MCP_TOOL_NAME_PATTERN` for the same reason {@link MCP_SERVER_ID_PATTERN} is a copy, and pinned
|
|
184
|
+
* against it by `test/agent-capabilities.conformity.test.ts`.
|
|
185
|
+
*
|
|
186
|
+
* The comma is the reason the rule exists on THIS side of the boundary too:
|
|
187
|
+
* {@link claudeAllowedToolPatterns} builds the list that the runner joins into one
|
|
188
|
+
* `--allowedTools` argument with commas, so an entry carrying one splits into two patterns of which
|
|
189
|
+
* the second matches no tool the CLI has. Dropped rather than passed through, because the entries
|
|
190
|
+
* that survive are what narrows the session: a bad one would silently take the run's whole MCP
|
|
191
|
+
* surface with it.
|
|
192
|
+
*/
|
|
193
|
+
export const MCP_TOOL_NAME_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/
|
|
194
|
+
|
|
181
195
|
/**
|
|
182
196
|
* Whether an HTTP tool server's URL may be started. Mirrors kernel's `isAllowedMcpHttpUrl` (see
|
|
183
197
|
* {@link MCP_SERVER_ID_PATTERN} for why it is a copy, and the conformity suite that pins it):
|
|
@@ -219,13 +233,25 @@ function parseStringArray(value: unknown): string[] | undefined {
|
|
|
219
233
|
return out.length ? out : undefined
|
|
220
234
|
}
|
|
221
235
|
|
|
236
|
+
/**
|
|
237
|
+
* The `allowedTools` list: string entries that are single tool NAMES (see
|
|
238
|
+
* {@link MCP_TOOL_NAME_PATTERN}). Undefined when nothing survives, which is the same answer as an
|
|
239
|
+
* absent field (every tool the server exposes) and the right one: the alternative is a list whose
|
|
240
|
+
* only surviving entries are the platform's own built-in tool names, i.e. a run narrowed to no MCP
|
|
241
|
+
* tools at all. The backend refuses these at registration; this is the boundary check.
|
|
242
|
+
*/
|
|
243
|
+
function parseAllowedTools(value: unknown): string[] | undefined {
|
|
244
|
+
const names = parseStringArray(value)?.filter((name) => MCP_TOOL_NAME_PATTERN.test(name))
|
|
245
|
+
return names?.length ? names : undefined
|
|
246
|
+
}
|
|
247
|
+
|
|
222
248
|
/** Validate one `mcpServers` entry, or undefined when malformed for its transport. */
|
|
223
249
|
function parseMcpServerSpec(value: unknown): McpServerSpec | undefined {
|
|
224
250
|
if (typeof value !== 'object' || value === null) return undefined
|
|
225
251
|
const o = value as Record<string, unknown>
|
|
226
252
|
const id = sanitizeServerId(o.id)
|
|
227
253
|
if (!id) return undefined
|
|
228
|
-
const allowedTools =
|
|
254
|
+
const allowedTools = parseAllowedTools(o.allowedTools)
|
|
229
255
|
const secretKeys = parseStringArray(o.secretKeys)
|
|
230
256
|
if (o.transport === 'http') {
|
|
231
257
|
// https anywhere, plain http only on loopback: the CLI would happily be pointed at a
|
|
@@ -370,9 +396,13 @@ function tomlString(value: string): string {
|
|
|
370
396
|
|
|
371
397
|
/**
|
|
372
398
|
* The `[mcp_servers.<id>]` TOML block Codex reads from its `CODEX_HOME/config.toml`. Codex's MCP
|
|
373
|
-
* client is stdio-only, so an `http` server is skipped here
|
|
374
|
-
*
|
|
375
|
-
*
|
|
399
|
+
* client is stdio-only, so an `http` server is skipped here.
|
|
400
|
+
*
|
|
401
|
+
* The skip is now a BACKSTOP rather than the decision: the backend knows which transports each
|
|
402
|
+
* harness reaches (`MCP_HARNESS_TRANSPORTS`) and drops an `http` server from a Codex dispatch under
|
|
403
|
+
* its own `transport_unsupported` reason, so the prompt states the gap instead of advertising a tool
|
|
404
|
+
* this writer then silently omitted. It stays because a body that reached the container by any other
|
|
405
|
+
* route must still produce a valid config rather than a malformed one.
|
|
376
406
|
*/
|
|
377
407
|
export function codexMcpConfigToml(servers: McpServerSpec[]): string {
|
|
378
408
|
const blocks: string[] = []
|
package/src/job.ts
CHANGED
|
@@ -181,9 +181,13 @@ function parseGuardLimits(value: unknown): GuardLimitsSpec | undefined {
|
|
|
181
181
|
const noEdit = posInt(o.maxToolCallsWithoutEdit)
|
|
182
182
|
const errors = posInt(o.maxConsecutiveErrors)
|
|
183
183
|
const web = posInt(o.maxConsecutiveWebCalls)
|
|
184
|
+
const mcp = posInt(o.maxConsecutiveMcpCalls)
|
|
185
|
+
const nonAction = posInt(o.maxConsecutiveNonActionCalls)
|
|
184
186
|
if (noEdit !== undefined) spec.maxToolCallsWithoutEdit = noEdit
|
|
185
187
|
if (errors !== undefined) spec.maxConsecutiveErrors = errors
|
|
186
188
|
if (web !== undefined) spec.maxConsecutiveWebCalls = web
|
|
189
|
+
if (mcp !== undefined) spec.maxConsecutiveMcpCalls = mcp
|
|
190
|
+
if (nonAction !== undefined) spec.maxConsecutiveNonActionCalls = nonAction
|
|
187
191
|
return Object.keys(spec).length > 0 ? spec : undefined
|
|
188
192
|
}
|
|
189
193
|
|
|
@@ -874,6 +878,8 @@ export interface GuardLimitsSpec {
|
|
|
874
878
|
maxToolCallsWithoutEdit?: number
|
|
875
879
|
maxConsecutiveErrors?: number
|
|
876
880
|
maxConsecutiveWebCalls?: number
|
|
881
|
+
maxConsecutiveMcpCalls?: number
|
|
882
|
+
maxConsecutiveNonActionCalls?: number
|
|
877
883
|
}
|
|
878
884
|
|
|
879
885
|
/**
|
package/src/progress-guard.ts
CHANGED
|
@@ -50,6 +50,31 @@ export interface ProgressGuardLimits {
|
|
|
50
50
|
* without it.
|
|
51
51
|
*/
|
|
52
52
|
maxConsecutiveWebCalls?: number
|
|
53
|
+
/**
|
|
54
|
+
* Abort after this many consecutive MCP tool-server calls (`mcp__*`) with no other
|
|
55
|
+
* tool call in between: the tool-server analogue of `maxConsecutiveWebCalls`, and
|
|
56
|
+
* present for the same reason. An `mcp__*` call is exempt from the no-edit bound (see
|
|
57
|
+
* `isMcpToolCall`), so without a streak of its own a run could query a tool server
|
|
58
|
+
* indefinitely without tripping any guard. Any non-MCP tool call resets the streak.
|
|
59
|
+
* Optional: defaults to {@link DEFAULT_PROGRESS_GUARD_LIMITS}.
|
|
60
|
+
*/
|
|
61
|
+
maxConsecutiveMcpCalls?: number
|
|
62
|
+
/**
|
|
63
|
+
* Abort after this many consecutive calls that are EXEMPT from the no-edit bound
|
|
64
|
+
* (planning, read-only exploration, subagent dispatch, `mcp__*`) with no action call
|
|
65
|
+
* in between. The backstop that makes each individual exemption mean "not counted"
|
|
66
|
+
* rather than "unbounded": every per-family streak above resets on any call outside
|
|
67
|
+
* its own family, so a run alternating `web_search` with `mcp__issues__search` (or
|
|
68
|
+
* with `read`) trips none of them and, having never made an action call, never
|
|
69
|
+
* reaches `maxToolCallsWithoutEdit` either. Only the job's wall-clock ceiling
|
|
70
|
+
* bounded that.
|
|
71
|
+
*
|
|
72
|
+
* Deliberately far above every family cap, because it is not a research bound and
|
|
73
|
+
* must not become one: reading a hundred files before the first edit is legitimate
|
|
74
|
+
* work-up, and any `bash`/edit/action call resets the streak. Optional: defaults to
|
|
75
|
+
* {@link DEFAULT_PROGRESS_GUARD_LIMITS}.
|
|
76
|
+
*/
|
|
77
|
+
maxConsecutiveNonActionCalls?: number
|
|
53
78
|
}
|
|
54
79
|
|
|
55
80
|
// `satisfies` (not a type annotation) so each property keeps its concrete `number`
|
|
@@ -63,6 +88,16 @@ export const DEFAULT_PROGRESS_GUARD_LIMITS = {
|
|
|
63
88
|
// A genuine research burst is a handful of searches; an uninterrupted run of this
|
|
64
89
|
// many web calls (with no read/edit/bash between) is a search loop, not progress.
|
|
65
90
|
maxConsecutiveWebCalls: 25,
|
|
91
|
+
// Looser than the web cap: a tool server is usually the agent's route to the SYSTEM OF
|
|
92
|
+
// RECORD (the issue tracker, the advisory database, the design source), and reading a
|
|
93
|
+
// list and then each of its items is a normal opening move, not a rabbit-hole. A run
|
|
94
|
+
// that makes this many in a row with no read, edit or bash between is looping.
|
|
95
|
+
maxConsecutiveMcpCalls: 40,
|
|
96
|
+
// Well clear of every family cap above, and of any plausible read-up: a run that makes
|
|
97
|
+
// this many exempt calls with not one action call between them has stopped converging,
|
|
98
|
+
// whatever mix of reads, searches and lookups it is cycling through. Sized as a
|
|
99
|
+
// backstop rather than a judgement, because the families are where judgement belongs.
|
|
100
|
+
maxConsecutiveNonActionCalls: 200,
|
|
66
101
|
} satisfies ProgressGuardLimits
|
|
67
102
|
|
|
68
103
|
// Tool names that mutate files, so a call to one clears the no-edit suspicion. Kept
|
|
@@ -139,6 +174,32 @@ const EXPLORATION_TOOLS = new Set([
|
|
|
139
174
|
// Pi's `web_search`/`web_fetch` and Claude Code's `WebSearch`/`WebFetch`.
|
|
140
175
|
const WEB_TOOLS = new Set(['web_search', 'web_fetch', 'websearch', 'webfetch'])
|
|
141
176
|
|
|
177
|
+
// A call to a tool server (MCP). Every MCP client names these `mcp__<server>__<tool>`, and
|
|
178
|
+
// the prefix is the ONLY thing the harness can know about them: what a given server's tools
|
|
179
|
+
// do is a backend registration this image has never seen, so the guard classifies by shape.
|
|
180
|
+
//
|
|
181
|
+
// Matched, rather than enumerated in EXPLORATION_TOOLS, because the set is open: it is
|
|
182
|
+
// whatever tool servers the running kind was wired with. A prefix test is also why this is a
|
|
183
|
+
// function, `name.startsWith` on the already-lower-cased name, so `MCP__Issues__search`
|
|
184
|
+
// classifies the same as `mcp__issues__search`.
|
|
185
|
+
function isMcpToolCall(loweredName: string): boolean {
|
|
186
|
+
return loweredName.startsWith('mcp__')
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
// Whether a call is EXEMPT from the no-edit bound: planning and bookkeeping, read-only
|
|
190
|
+
// exploration, a subagent dispatch, or a tool-server call. One predicate rather than the
|
|
191
|
+
// four tests inlined at the branch, because the combined non-action streak and the no-edit
|
|
192
|
+
// exemption must be the SAME set: a family exempted in one place and missed in the other is
|
|
193
|
+
// either an unbounded loop or a run killed for a call the bound says it may make.
|
|
194
|
+
function isNonActionToolCall(loweredName: string): boolean {
|
|
195
|
+
return (
|
|
196
|
+
PLANNING_TOOLS.has(loweredName) ||
|
|
197
|
+
EXPLORATION_TOOLS.has(loweredName) ||
|
|
198
|
+
SUBAGENT_DISPATCH_TOOLS.has(loweredName) ||
|
|
199
|
+
isMcpToolCall(loweredName)
|
|
200
|
+
)
|
|
201
|
+
}
|
|
202
|
+
|
|
142
203
|
/** Read {@link ProgressGuardLimits} from the environment, falling back to the defaults. */
|
|
143
204
|
export function progressGuardLimitsFromEnv(
|
|
144
205
|
env: NodeJS.ProcessEnv = process.env,
|
|
@@ -160,6 +221,14 @@ export function progressGuardLimitsFromEnv(
|
|
|
160
221
|
env.JOB_MAX_CONSECUTIVE_WEB_CALLS,
|
|
161
222
|
DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveWebCalls,
|
|
162
223
|
),
|
|
224
|
+
maxConsecutiveMcpCalls: num(
|
|
225
|
+
env.JOB_MAX_CONSECUTIVE_MCP_CALLS,
|
|
226
|
+
DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveMcpCalls,
|
|
227
|
+
),
|
|
228
|
+
maxConsecutiveNonActionCalls: num(
|
|
229
|
+
env.JOB_MAX_CONSECUTIVE_NON_ACTION_CALLS,
|
|
230
|
+
DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveNonActionCalls,
|
|
231
|
+
),
|
|
163
232
|
}
|
|
164
233
|
}
|
|
165
234
|
|
|
@@ -186,12 +255,21 @@ export function mergeGuardLimits(
|
|
|
186
255
|
overrides.maxToolCallsWithoutEdit,
|
|
187
256
|
),
|
|
188
257
|
maxConsecutiveErrors: loosen(base.maxConsecutiveErrors, overrides.maxConsecutiveErrors),
|
|
189
|
-
//
|
|
190
|
-
//
|
|
258
|
+
// The streak knobs are optional on the interface (callers may omit them), so fall back
|
|
259
|
+
// to the default before loosening: it keeps `loosen`'s base a concrete number.
|
|
191
260
|
maxConsecutiveWebCalls: loosen(
|
|
192
261
|
base.maxConsecutiveWebCalls ?? DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveWebCalls,
|
|
193
262
|
overrides.maxConsecutiveWebCalls,
|
|
194
263
|
),
|
|
264
|
+
maxConsecutiveMcpCalls: loosen(
|
|
265
|
+
base.maxConsecutiveMcpCalls ?? DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveMcpCalls,
|
|
266
|
+
overrides.maxConsecutiveMcpCalls,
|
|
267
|
+
),
|
|
268
|
+
maxConsecutiveNonActionCalls: loosen(
|
|
269
|
+
base.maxConsecutiveNonActionCalls ??
|
|
270
|
+
DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveNonActionCalls,
|
|
271
|
+
overrides.maxConsecutiveNonActionCalls,
|
|
272
|
+
),
|
|
195
273
|
}
|
|
196
274
|
}
|
|
197
275
|
|
|
@@ -207,6 +285,8 @@ export class ProgressGuard {
|
|
|
207
285
|
private edits = 0
|
|
208
286
|
private consecutiveErrors = 0
|
|
209
287
|
private consecutiveWebCalls = 0
|
|
288
|
+
private consecutiveMcpCalls = 0
|
|
289
|
+
private consecutiveNonActionCalls = 0
|
|
210
290
|
|
|
211
291
|
constructor(
|
|
212
292
|
private readonly limits: ProgressGuardLimits,
|
|
@@ -257,16 +337,55 @@ export class ProgressGuard {
|
|
|
257
337
|
this.consecutiveWebCalls = 0
|
|
258
338
|
}
|
|
259
339
|
|
|
260
|
-
//
|
|
261
|
-
// no-edit bound
|
|
262
|
-
//
|
|
263
|
-
if (
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
340
|
+
// Tool-server (MCP) calls: bounded as their own streak for exactly the reason the web
|
|
341
|
+
// streak exists. They are exempt from the no-edit bound below, and an exemption with no
|
|
342
|
+
// counter-bound is a loop the guard cannot see. Any non-MCP call resets it.
|
|
343
|
+
if (isMcpToolCall(name)) {
|
|
344
|
+
this.consecutiveMcpCalls++
|
|
345
|
+
const mcpCap =
|
|
346
|
+
this.limits.maxConsecutiveMcpCalls ?? DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveMcpCalls
|
|
347
|
+
if (this.consecutiveMcpCalls >= mcpCap) {
|
|
348
|
+
return (
|
|
349
|
+
`no progress: ${this.consecutiveMcpCalls} consecutive tool-server (MCP) calls without ` +
|
|
350
|
+
`any other action. The agent is stuck querying its tools instead of doing the work. ` +
|
|
351
|
+
`Aborting.`
|
|
352
|
+
)
|
|
353
|
+
}
|
|
354
|
+
} else {
|
|
355
|
+
this.consecutiveMcpCalls = 0
|
|
356
|
+
}
|
|
357
|
+
|
|
358
|
+
// Planning, read-only exploration, subagent-dispatch and tool-server calls don't count
|
|
359
|
+
// toward the no-edit bound (see `isNonActionToolCall`): only "action" calls without an
|
|
360
|
+
// edit do.
|
|
361
|
+
//
|
|
362
|
+
// An `mcp__*` call is exempt for the same reason a `read` is: the bound targets the
|
|
363
|
+
// credential rabbit-hole (endless `bash` probing with nothing implemented), and reaching
|
|
364
|
+
// a registered tool server is the platform TELLING the agent to look something up
|
|
365
|
+
// ("prefer them over guessing"). Counting them would abort an edits-expected kind for
|
|
366
|
+
// consulting the issue tracker the deployment wired for it, punishing the run for
|
|
367
|
+
// following its own prompt. They are neutral rather than edit-satisfying, exactly like a
|
|
368
|
+
// subagent dispatch: a read-only lookup must not clear the suspicion the bound holds.
|
|
369
|
+
//
|
|
370
|
+
// The exempt calls carry ONE streak of their own, and it is what keeps every exemption
|
|
371
|
+
// above from adding up to an unbounded run: each per-family cap resets on any call
|
|
372
|
+
// outside its family, so alternating two exempt families trips neither, and a run that
|
|
373
|
+
// never makes an action call never reaches the no-edit bound either.
|
|
374
|
+
if (isNonActionToolCall(name)) {
|
|
375
|
+
this.consecutiveNonActionCalls++
|
|
376
|
+
const nonActionCap =
|
|
377
|
+
this.limits.maxConsecutiveNonActionCalls ??
|
|
378
|
+
DEFAULT_PROGRESS_GUARD_LIMITS.maxConsecutiveNonActionCalls
|
|
379
|
+
if (this.consecutiveNonActionCalls >= nonActionCap) {
|
|
380
|
+
return (
|
|
381
|
+
`no progress: ${this.consecutiveNonActionCalls} consecutive read-only calls (searching, ` +
|
|
382
|
+
`reading, tool-server lookups, subagent dispatches) with no action call between them. ` +
|
|
383
|
+
`The agent is cycling through research instead of doing the work. Aborting.`
|
|
384
|
+
)
|
|
385
|
+
}
|
|
268
386
|
return null
|
|
269
387
|
}
|
|
388
|
+
this.consecutiveNonActionCalls = 0
|
|
270
389
|
this.toolCalls++
|
|
271
390
|
if (FILE_EDIT_TOOLS.has(name)) this.edits++
|
|
272
391
|
|