@cat-factory/executor-harness 1.52.0 → 1.54.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +39 -0
- package/dist/agent-runner.js +31 -40
- package/dist/agent.js +57 -42
- package/dist/coding-agent.js +4 -0
- package/dist/frontend-infra.js +9 -2
- package/dist/package-registries.js +78 -13
- package/dist/pi-workspace.js +24 -8
- package/dist/pi.js +4 -3
- package/dist/progress.js +217 -0
- package/dist/subagents.js +52 -27
- package/package.json +2 -2
- package/src/agent-runner.ts +51 -41
- package/src/agent.ts +57 -40
- package/src/coding-agent.ts +7 -2
- package/src/frontend-infra.ts +10 -3
- package/src/package-registries.ts +95 -15
- package/src/pi-workspace.ts +27 -8
- package/src/pi.ts +4 -3
- package/src/progress.ts +232 -0
- package/src/runner.ts +11 -0
- package/src/subagents.ts +25 -34
package/README.md
CHANGED
|
@@ -65,6 +65,44 @@ Bootstrap differs at the ends — it may start from an empty dir, and **resets
|
|
|
65
65
|
history to one commit and force-pushes** the default branch instead of opening a
|
|
66
66
|
PR. Blueprint **commits onto a branch** (no history reset) and returns the tree.
|
|
67
67
|
|
|
68
|
+
## Per-job state: never a process- or HOME-global
|
|
69
|
+
|
|
70
|
+
A job's staging state (the tester's secrets, private-registry auth, a repo-sourced Claude
|
|
71
|
+
Skill) must be scoped to that job, not written into `process.env` or the home directory.
|
|
72
|
+
|
|
73
|
+
In a container those two ARE per-job — one job per process, and `HOME` belongs to that
|
|
74
|
+
container — so a global was a safe place to stage. The **local native transport** breaks both
|
|
75
|
+
assumptions: one long-lived host process serves every concurrent `ambientAuth` job, on the
|
|
76
|
+
**developer's own home**. A global there is shared mutable state across siblings, and writing
|
|
77
|
+
(or clearing) a dotfile destroys a file the developer owns.
|
|
78
|
+
|
|
79
|
+
So per-job values ride explicit **child env** (`RunOptions.agentEnv` →
|
|
80
|
+
`SubscriptionRunOptions.extraEnv`, merged over the inherited env at spawn) and per-job files go
|
|
81
|
+
under a per-job directory:
|
|
82
|
+
|
|
83
|
+
| State | Container | Native (`ambientAuth`) |
|
|
84
|
+
| -------------------- | -------------------------------------------- | ------------------------------------------------------------------------- |
|
|
85
|
+
| Tester secrets | child env | child env (same path — the old `process.env` set/restore is gone) |
|
|
86
|
+
| Private-registry auth | `~/.npmrc`; cleared when a job has no entries | per-job `.npmrc` + `npm_config_userconfig`, seeded from the developer's; theirs is never written or removed |
|
|
87
|
+
| Repo-sourced Claude Skill | installed into the isolated `CLAUDE_CONFIG_DIR` | not installed — read from the checkout's `.cat-context/skill/`, like codex |
|
|
88
|
+
|
|
89
|
+
Two consequences worth knowing:
|
|
90
|
+
|
|
91
|
+
- **The skill's PROMPT follows the same split.** A native install gets a short pointer; every
|
|
92
|
+
checkout-reading case (Pi, codex, ambient claude-code) gets the instructions folded in plus a
|
|
93
|
+
pointer to `.cat-context/skill/`. That decision is the backend's `renderSkillForHarness`, which
|
|
94
|
+
keys off `ambientAuth` as well as the harness — rendering an ambient run as an install would
|
|
95
|
+
point the agent at a skill that is nowhere on disk.
|
|
96
|
+
- **`npm_config_userconfig` reaches less than `~/.npmrc` did.** npm and pnpm honour it; yarn does
|
|
97
|
+
not. And it only reaches processes that are handed the job env, so anything the HARNESS itself
|
|
98
|
+
spawns (the frontend stand-up's install/build, a ralph validation command) is passed
|
|
99
|
+
`RunOptions.agentEnv` explicitly rather than relying on inheritance.
|
|
100
|
+
|
|
101
|
+
When you add per-job state, put it in one of those two places. `~/.pi/*` and
|
|
102
|
+
`~/.config/rpiv-web-tools` remain HOME-global, which is fine only because the Pi harness never
|
|
103
|
+
runs natively (the native router sends `ambientAuth` jobs — Claude/Codex only — to the host
|
|
104
|
+
process and everything else to a container).
|
|
105
|
+
|
|
68
106
|
## No secrets in the image
|
|
69
107
|
|
|
70
108
|
The image (built from the `Dockerfile`, base `node:26-trixie-slim`) contains
|
|
@@ -86,6 +124,7 @@ Kimi / DeepSeek) and meters spend. The provider key never enters the container.
|
|
|
86
124
|
| `src/bootstrap.ts` | The `/bootstrap` handler (clone-or-empty → adapt → reinit + force-push). |
|
|
87
125
|
| `src/blueprint.ts` | The `/blueprint` handler (decompose → render `blueprints/` → commit on branch). |
|
|
88
126
|
| `src/embed.ts` | Bundled assets/templates written into the workspace. |
|
|
127
|
+
| `src/package-registries.ts` | Private-registry (npm) auth: renders the job's allowlisted entries into an npmrc — the user `~/.npmrc` in a container, a per-job file pointed at by `npm_config_userconfig` for a native job. |
|
|
89
128
|
| `src/agent-runner.ts` | The subscription-harness runners (`runClaudeCode` / `runCodex`) — talk direct to the vendor with a leased OAuth token, lift per-turn usage/telemetry off the CLI event stream. |
|
|
90
129
|
| `src/transcript-retention.ts` | Lifts the CLI session transcripts (`projects/` / `sessions/`) out of the isolated, credential-bearing config home before it is deleted, and prunes them on a TTL (debugging artifact retention). |
|
|
91
130
|
| `src/logger.ts` | Structured logging. |
|
package/dist/agent-runner.js
CHANGED
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
import { spawn } from 'node:child_process';
|
|
2
2
|
import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';
|
|
3
|
-
import {
|
|
3
|
+
import { tmpdir } from 'node:os';
|
|
4
4
|
import { dirname, join } from 'node:path';
|
|
5
5
|
import { claudeAssistantContent, claudeCallUsage, isObject, numberOf, redactBody, } from './claude-stream.js';
|
|
6
6
|
import { createCallMetricPublisher, publishCallMetric, } from './pi.js';
|
|
7
7
|
import { killChildProcess, spawnDetached } from './process.js';
|
|
8
8
|
import { redact, secretsToRedact } from './redact.js';
|
|
9
|
-
import { createSliceTracker,
|
|
9
|
+
import { createSliceTracker, startSubagentWatcher } from './subagents.js';
|
|
10
|
+
import { createTaskPlanTracker, normalizeStatus, pickProgress, toProgress, todosToProgress, } from './progress.js';
|
|
10
11
|
import { assertOnboardingKeysCurrent, writeOnboardingPreseed } from './onboarding-preseed.js';
|
|
11
12
|
import { retainSessionTranscripts } from './transcript-retention.js';
|
|
12
13
|
/**
|
|
@@ -224,18 +225,24 @@ export async function runClaudeCode(opts) {
|
|
|
224
225
|
// may still rewrite below (a published call must be final — see the publisher).
|
|
225
226
|
const publisher = createCallMetricPublisher(calls, opts.onCallMetric);
|
|
226
227
|
// ADR 0026 D2.1 + ADR 0027 Defect B: surface live slice progress from TWO reconciled
|
|
227
|
-
// sources. The parent's
|
|
228
|
+
// sources. The parent's subagent dispatches + their terminal tool_results DO appear on this
|
|
228
229
|
// stream (only a subagent's intermediate turns don't), so `sliceTracker` derives per-slice
|
|
229
|
-
// progress for the parallel
|
|
230
|
-
//
|
|
231
|
-
// update, so neither masks the other — the pr-reviewer prompt writes its
|
|
232
|
-
//
|
|
230
|
+
// progress for the parallel shape; the parent's own plan (the sequential shape) is tracked
|
|
231
|
+
// by `planTracker` + `lastTodo`. `pickProgress` picks whichever is further along on each
|
|
232
|
+
// update, so neither masks the other — the pr-reviewer prompt writes its plan ONCE and never
|
|
233
|
+
// marks it done, which used to gate the slice signal off and pin progress at 0%.
|
|
234
|
+
//
|
|
235
|
+
// The plan arrives in one of two tool vocabularies depending on the bundled CLI build:
|
|
236
|
+
// `TodoWrite` (whole-list snapshots, tracked in `lastTodo`) or the incremental
|
|
237
|
+
// `TaskCreate`/`TaskUpdate` pair (tracked by `planTracker`, which needs the tool RESULTS too
|
|
238
|
+
// because the task id is minted there). Both are read — see ./progress.ts.
|
|
233
239
|
const sliceTracker = createSliceTracker();
|
|
240
|
+
const planTracker = createTaskPlanTracker();
|
|
234
241
|
let lastTodo;
|
|
235
242
|
const emitProgress = () => {
|
|
236
243
|
if (!opts.onProgress)
|
|
237
244
|
return;
|
|
238
|
-
const progress = pickProgress(lastTodo, sliceTracker.progress());
|
|
245
|
+
const progress = pickProgress(pickProgress(lastTodo, planTracker.progress()), sliceTracker.progress());
|
|
239
246
|
if (progress)
|
|
240
247
|
opts.onProgress(progress);
|
|
241
248
|
};
|
|
@@ -255,6 +262,7 @@ export async function runClaudeCode(opts) {
|
|
|
255
262
|
}
|
|
256
263
|
}
|
|
257
264
|
sliceTracker.onAssistant(content);
|
|
265
|
+
planTracker.onAssistant(content);
|
|
258
266
|
emitProgress();
|
|
259
267
|
// Record this call BEFORE appending its turn: the prompt is the history that
|
|
260
268
|
// produced this response. The append-only array keeps each call's prompt a strict
|
|
@@ -278,6 +286,7 @@ export async function runClaudeCode(opts) {
|
|
|
278
286
|
const content = event.message.content;
|
|
279
287
|
if (Array.isArray(content)) {
|
|
280
288
|
sliceTracker.onUser(content);
|
|
289
|
+
planTracker.onUser(content);
|
|
281
290
|
emitProgress();
|
|
282
291
|
messages.push({ role: 'tool', content });
|
|
283
292
|
}
|
|
@@ -314,14 +323,14 @@ export async function runClaudeCode(opts) {
|
|
|
314
323
|
await assertOnboardingKeysCurrent(configHome, process.env.CLAUDE_CLI_VERSION, opts.log);
|
|
315
324
|
}
|
|
316
325
|
// Repo-sourced Claude Skill (slice 2): install it as a native skill under the config dir's
|
|
317
|
-
// `skills/<name>/` so the CLI discovers and can invoke it.
|
|
318
|
-
//
|
|
319
|
-
//
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
await writeNativeSkill(
|
|
326
|
+
// `skills/<name>/` so the CLI discovers and can invoke it. ONLY into the isolated per-run config
|
|
327
|
+
// home — never the developer's own `~/.claude` (ambient/native mode), where it would persist in
|
|
328
|
+
// their personal setup after the run and two concurrent jobs carrying same-named skills from
|
|
329
|
+
// different repos would clobber each other. An ambient run reads the skill from the checkout
|
|
330
|
+
// instead (`.cat-context/skill/`, materialised by the caller). Best-effort: a write failure must
|
|
331
|
+
// not wedge the run — the prompt still names the skill.
|
|
332
|
+
if (opts.skill && configHome) {
|
|
333
|
+
await writeNativeSkill(join(configHome, 'skills'), opts.skill).catch(() => { });
|
|
325
334
|
}
|
|
326
335
|
const env = buildClaudeEnv(opts, configHome);
|
|
327
336
|
// ADR 0026 D3 (path corrected by ADR 0027 Defect A): while the run is live, tail the CLI's
|
|
@@ -393,9 +402,12 @@ export async function runClaudeCode(opts) {
|
|
|
393
402
|
* keep its cyclomatic complexity down; behaviour is a straight move of the original expression.
|
|
394
403
|
*/
|
|
395
404
|
function buildClaudeEnv(opts, configHome) {
|
|
405
|
+
// The job-scoped env rides along in BOTH modes; the credential/config vars below are what
|
|
406
|
+
// ambient mode drops (the developer's own logged-in `~/.claude` is used instead).
|
|
396
407
|
if (opts.ambientAuth)
|
|
397
|
-
return {};
|
|
408
|
+
return { ...opts.extraEnv };
|
|
398
409
|
return {
|
|
410
|
+
...opts.extraEnv,
|
|
399
411
|
CLAUDE_CONFIG_DIR: configHome,
|
|
400
412
|
...(opts.subscriptionBaseUrl
|
|
401
413
|
? {
|
|
@@ -442,25 +454,6 @@ async function assembleClaudeOutcome(args) {
|
|
|
442
454
|
...(mergedCalls.length ? { callMetrics: mergedCalls } : {}),
|
|
443
455
|
};
|
|
444
456
|
}
|
|
445
|
-
/** Map Claude Code's `TodoWrite` todos array onto subtask counts. */
|
|
446
|
-
function todosToProgress(todos) {
|
|
447
|
-
if (!Array.isArray(todos))
|
|
448
|
-
return undefined;
|
|
449
|
-
const items = todos.filter(isObject).map((t) => ({
|
|
450
|
-
label: typeof t.content === 'string' ? t.content : String(t.content ?? ''),
|
|
451
|
-
status: normalizeStatus(t.status),
|
|
452
|
-
}));
|
|
453
|
-
const completed = items.filter((i) => i.status === 'completed').length;
|
|
454
|
-
const inProgress = items.filter((i) => i.status === 'in_progress').length;
|
|
455
|
-
return { completed, inProgress, total: items.length, items };
|
|
456
|
-
}
|
|
457
|
-
function normalizeStatus(status) {
|
|
458
|
-
if (status === 'completed')
|
|
459
|
-
return 'completed';
|
|
460
|
-
if (status === 'in_progress')
|
|
461
|
-
return 'in_progress';
|
|
462
|
-
return 'pending';
|
|
463
|
-
}
|
|
464
457
|
function claudeUsage(raw) {
|
|
465
458
|
if (!isObject(raw))
|
|
466
459
|
return undefined;
|
|
@@ -584,7 +577,7 @@ export async function runCodex(opts) {
|
|
|
584
577
|
opts.model,
|
|
585
578
|
'-',
|
|
586
579
|
],
|
|
587
|
-
}, prompt, opts, codexHome ? { CODEX_HOME: codexHome } : {}, opts.subscriptionToken ? secretsToRedact(opts.subscriptionToken) : [], onEvent);
|
|
580
|
+
}, prompt, opts, { ...opts.extraEnv, ...(codexHome ? { CODEX_HOME: codexHome } : {}) }, opts.subscriptionToken ? secretsToRedact(opts.subscriptionToken) : [], onEvent);
|
|
588
581
|
// Fallback for a CLI/version that never emits per-turn `last_token_usage`: record a
|
|
589
582
|
// single call from the cumulative total + final text so the run is still observable.
|
|
590
583
|
if (calls.length === 0 && (usage || summary)) {
|
|
@@ -670,9 +663,7 @@ function codexPlanProgress(event) {
|
|
|
670
663
|
}));
|
|
671
664
|
if (items.length === 0)
|
|
672
665
|
return undefined;
|
|
673
|
-
|
|
674
|
-
const inProgress = items.filter((i) => i.status === 'in_progress').length;
|
|
675
|
-
return { completed, inProgress, total: items.length, items };
|
|
666
|
+
return toProgress(items);
|
|
676
667
|
}
|
|
677
668
|
/**
|
|
678
669
|
* Best-effort: pull token usage out of a Codex usage event. Codex `exec --json`
|
package/dist/agent.js
CHANGED
|
@@ -96,12 +96,12 @@ async function standUpInfra(dir, infra, signal, logger) {
|
|
|
96
96
|
* `package.json` / `outputDir` / `mocks/` all live under the service subtree, so installing,
|
|
97
97
|
* building, serving and seeding WireMock from the root would target the wrong directory.
|
|
98
98
|
*/
|
|
99
|
-
async function manageInfra(dir, workDir, infra,
|
|
99
|
+
async function manageInfra(dir, workDir, infra, opts, logger) {
|
|
100
100
|
if (infra.kind === 'frontend') {
|
|
101
101
|
// `onActivity` feeds the inactivity watchdog through the frontend build/serve stand-up,
|
|
102
102
|
// which (unlike docker-compose's 5-min-capped `up`) can run past the inactivity window.
|
|
103
103
|
// Runs in `workDir` so a monorepo frontend builds/serves from its own package subtree.
|
|
104
|
-
const fe = await standUpFrontend(workDir, infra,
|
|
104
|
+
const fe = await standUpFrontend(workDir, infra, opts, logger);
|
|
105
105
|
return {
|
|
106
106
|
...(fe.note ? { note: fe.note } : {}),
|
|
107
107
|
...(fe.serveUrl ? { serveUrl: fe.serveUrl } : {}),
|
|
@@ -109,7 +109,7 @@ async function manageInfra(dir, workDir, infra, signal, onActivity, logger) {
|
|
|
109
109
|
cleanup: () => tearDownFrontend(fe.processes, logger),
|
|
110
110
|
};
|
|
111
111
|
}
|
|
112
|
-
const standUp = await standUpInfra(dir, infra, signal, logger);
|
|
112
|
+
const standUp = await standUpInfra(dir, infra, opts.signal, logger);
|
|
113
113
|
return {
|
|
114
114
|
...(standUp.note ? { note: standUp.note } : {}),
|
|
115
115
|
...(standUp.record ? { record: standUp.record } : {}),
|
|
@@ -238,14 +238,39 @@ function mergeEffort(result, effortReport) {
|
|
|
238
238
|
}
|
|
239
239
|
/** Run one generic agent job end to end, dispatching on `mode`. */
|
|
240
240
|
export async function handleAgent(job, opts = {}) {
|
|
241
|
-
//
|
|
242
|
-
//
|
|
243
|
-
//
|
|
244
|
-
//
|
|
245
|
-
await
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
241
|
+
// An `ambientAuth` job runs in the SHARED native host process on the developer's own HOME
|
|
242
|
+
// (see `LocalProcessRunnerTransport`), so anything this job would otherwise write to a
|
|
243
|
+
// process- or HOME-global gets a per-job directory instead — it can't corrupt the
|
|
244
|
+
// developer's files, and concurrent jobs can't race on them.
|
|
245
|
+
const scopeDir = job.ambientAuth ? await mkdtemp(join(tmpdir(), 'cf-jobenv-')) : undefined;
|
|
246
|
+
try {
|
|
247
|
+
// Private-registry auth first, before any mode runs: every mode with a checkout may
|
|
248
|
+
// install dependencies (the agent's own shell and the frontend-infra stand-up both
|
|
249
|
+
// inherit this env, so they all read the written npmrc). In a container a job with no
|
|
250
|
+
// entries clears any stale ~/.npmrc from a prior job on a reused (warm-pool) container.
|
|
251
|
+
const registryEnv = await configurePackageRegistries(job.packageRegistries, scopeDir ? { isolatedDir: scopeDir } : {});
|
|
252
|
+
const scoped = withAgentEnv(opts, registryEnv);
|
|
253
|
+
if (job.mode === 'preview')
|
|
254
|
+
return await runPreviewMode(job, scoped);
|
|
255
|
+
return job.mode === 'coding'
|
|
256
|
+
? await runCodingMode(job, scoped)
|
|
257
|
+
: await runExploreMode(job, scoped);
|
|
258
|
+
}
|
|
259
|
+
finally {
|
|
260
|
+
if (scopeDir)
|
|
261
|
+
await rm(scopeDir, { recursive: true, force: true }).catch(() => { });
|
|
262
|
+
}
|
|
263
|
+
}
|
|
264
|
+
/**
|
|
265
|
+
* Layer extra child-process env onto a job's {@link RunOptions}. The agent CLI is spawned with
|
|
266
|
+
* `{...process.env, ...agentEnv}`, so this is how per-job values reach the agent (and the shell
|
|
267
|
+
* tools it spawns) WITHOUT mutating the harness's own `process.env` — which is shared by every
|
|
268
|
+
* concurrent job when the harness runs as a native host process. Empty `env` ⇒ `opts` unchanged.
|
|
269
|
+
*/
|
|
270
|
+
function withAgentEnv(opts, env) {
|
|
271
|
+
if (Object.keys(env).length === 0)
|
|
272
|
+
return opts;
|
|
273
|
+
return { ...opts, agentEnv: { ...opts.agentEnv, ...env } };
|
|
249
274
|
}
|
|
250
275
|
/**
|
|
251
276
|
* Decide a preview stand-up's outcome from its result (pure, so the success/failure boundary
|
|
@@ -300,7 +325,7 @@ async function runPreviewMode(job, opts) {
|
|
|
300
325
|
logger.info('agent(preview): building + serving', {
|
|
301
326
|
serviceDirectory: job.repo.serviceDirectory,
|
|
302
327
|
});
|
|
303
|
-
const fe = await standUpFrontend(workDir, infra, opts
|
|
328
|
+
const fe = await standUpFrontend(workDir, infra, opts, logger);
|
|
304
329
|
const infraSetupFields = fe.record
|
|
305
330
|
? { infraSetup: fe.record }
|
|
306
331
|
: {};
|
|
@@ -332,30 +357,23 @@ async function runPreviewMode(job, opts) {
|
|
|
332
357
|
}
|
|
333
358
|
}
|
|
334
359
|
/**
|
|
335
|
-
*
|
|
336
|
-
*
|
|
337
|
-
*
|
|
338
|
-
*
|
|
339
|
-
*
|
|
340
|
-
*
|
|
360
|
+
* Build the env carrying the tester's sensitive secrets, so the agent's shell tools (spawned as
|
|
361
|
+
* child processes that inherit it) can read `$KEY` — the out-of-band delivery channel. Each value
|
|
362
|
+
* is registered for redaction so it can't leak into captured output/logs. Reserved/toolchain env
|
|
363
|
+
* names were already dropped at parse. No secrets ⇒ an empty env.
|
|
364
|
+
*
|
|
365
|
+
* Returned as EXPLICIT child env rather than written onto `process.env`: a process-global
|
|
366
|
+
* set/restore is only safe when the process runs one job, which the native host-process transport
|
|
367
|
+
* breaks (it serves every concurrent ambient job from one process). There, two overlapping tester
|
|
368
|
+
* runs would read each other's secrets, and whichever finished first would delete the other's
|
|
369
|
+
* mid-run. Scoping them to the spawn env makes the delivery correct under concurrency and drops
|
|
370
|
+
* the restore step entirely.
|
|
341
371
|
*/
|
|
342
|
-
function
|
|
372
|
+
export function testSecretEnv(secrets) {
|
|
343
373
|
if (!secrets?.length)
|
|
344
|
-
return
|
|
374
|
+
return {};
|
|
345
375
|
registerKnownSecrets(secrets.map((s) => s.value));
|
|
346
|
-
|
|
347
|
-
for (const { key, value } of secrets) {
|
|
348
|
-
previous.set(key, process.env[key]);
|
|
349
|
-
process.env[key] = value;
|
|
350
|
-
}
|
|
351
|
-
return () => {
|
|
352
|
-
for (const [key, prior] of previous) {
|
|
353
|
-
if (prior === undefined)
|
|
354
|
-
delete process.env[key];
|
|
355
|
-
else
|
|
356
|
-
process.env[key] = prior;
|
|
357
|
-
}
|
|
358
|
-
};
|
|
376
|
+
return Object.fromEntries(secrets.map(({ key, value }) => [key, value]));
|
|
359
377
|
}
|
|
360
378
|
/**
|
|
361
379
|
* Read-only exploration: clone `branch`, run the agent making no edits, and return its
|
|
@@ -442,9 +460,7 @@ async function runExploreMode(job, opts) {
|
|
|
442
460
|
// The run-mode guidance itself lives in the backend-composed system/user prompt; the
|
|
443
461
|
// harness only manages the lifecycle + this dynamic stand-up note.
|
|
444
462
|
const infra = job.infra;
|
|
445
|
-
const managed = infra
|
|
446
|
-
? await manageInfra(dir, workDir, infra, opts.signal, opts.onActivity, logger)
|
|
447
|
-
: undefined;
|
|
463
|
+
const managed = infra ? await manageInfra(dir, workDir, infra, opts, logger) : undefined;
|
|
448
464
|
// Fold the stand-up outcome into the agent prompt: a stand-up problem (build/compose
|
|
449
465
|
// failure) is flagged as a concern; a frontend serve URL points the UI tester at the
|
|
450
466
|
// app it just built + served (the backend env resolution already reached the harness).
|
|
@@ -458,10 +474,10 @@ async function runExploreMode(job, opts) {
|
|
|
458
474
|
const infraSetupFields = managed?.record
|
|
459
475
|
? { infraSetup: managed.record }
|
|
460
476
|
: {};
|
|
461
|
-
//
|
|
462
|
-
// shell can read them as `$KEY
|
|
463
|
-
//
|
|
464
|
-
const
|
|
477
|
+
// Hand the tester's sensitive secrets to the agent's child process (out of band) so its
|
|
478
|
+
// shell can read them as `$KEY`. Scoped to this job's env, so a concurrent job in the same
|
|
479
|
+
// harness process never sees them. A no-op for non-tester runs (no `testSecrets`).
|
|
480
|
+
const agentOpts = withAgentEnv(opts, testSecretEnv(job.testSecrets));
|
|
465
481
|
try {
|
|
466
482
|
opts.onPhase?.('agent');
|
|
467
483
|
logger.info('agent(explore): running agent', { serviceDirectory });
|
|
@@ -484,11 +500,10 @@ async function runExploreMode(job, opts) {
|
|
|
484
500
|
webSearchProxy: job.webSearch,
|
|
485
501
|
contextFiles: job.contextFiles,
|
|
486
502
|
guardLimits: job.guardLimits,
|
|
487
|
-
},
|
|
503
|
+
}, agentOpts);
|
|
488
504
|
return mergeEffort(await finalizeExploreResult(job, { summary, stats, stderrTail, usage, callMetrics, runDiag }, { infra, infraSetupFields, logger, signal: opts.signal }), effortReport);
|
|
489
505
|
}
|
|
490
506
|
finally {
|
|
491
|
-
restoreSecrets();
|
|
492
507
|
if (managed)
|
|
493
508
|
await managed.cleanup();
|
|
494
509
|
}
|
package/dist/coding-agent.js
CHANGED
|
@@ -401,6 +401,10 @@ async function runRalphValidation(cwd, validation, logger, opts) {
|
|
|
401
401
|
cwd,
|
|
402
402
|
detached: spawnDetached,
|
|
403
403
|
stdio: ['ignore', 'pipe', 'pipe'],
|
|
404
|
+
// The job's own env (see `RunOptions.agentEnv`): a validation command typically installs
|
|
405
|
+
// before it tests, and this is spawned by the HARNESS rather than the agent, so it does not
|
|
406
|
+
// otherwise inherit the job's private-registry npmrc pointer on the native path.
|
|
407
|
+
env: { ...process.env, ...opts.agentEnv },
|
|
404
408
|
});
|
|
405
409
|
// Keep only the tail; guard against unbounded buffering on a chatty command.
|
|
406
410
|
const capture = (chunk) => {
|
package/dist/frontend-infra.js
CHANGED
|
@@ -56,7 +56,8 @@ function guardProcess(child, label, logger) {
|
|
|
56
56
|
* the agent as a prompt note (and captured on the record) rather than failing the job — the
|
|
57
57
|
* agent then reports the gap as a concern. Every path returns the processes to tear down.
|
|
58
58
|
*/
|
|
59
|
-
export async function standUpFrontend(dir, infra,
|
|
59
|
+
export async function standUpFrontend(dir, infra, run, logger = log) {
|
|
60
|
+
const { signal, onActivity } = run;
|
|
60
61
|
const startedAt = Date.now();
|
|
61
62
|
const processes = [];
|
|
62
63
|
// The frontend app's directory: the checkout root, or a monorepo subdirectory when the config
|
|
@@ -94,6 +95,11 @@ export async function standUpFrontend(dir, infra, signal, onActivity, logger = l
|
|
|
94
95
|
};
|
|
95
96
|
};
|
|
96
97
|
const buildEnv = (infra.envInjection ?? DEFAULTS.envInjection) === 'build' ? (infra.env ?? {}) : {};
|
|
98
|
+
// The job's own env (see `RunOptions.agentEnv`) — today the private-registry npmrc pointer.
|
|
99
|
+
// The stand-up is spawned by the HARNESS, not by the agent, so it does not inherit whatever the
|
|
100
|
+
// agent's CLI child was given: without this the install here would miss the job's registry auth
|
|
101
|
+
// on the native path, where the npmrc is per-job rather than the process's `~/.npmrc`.
|
|
102
|
+
const jobEnv = run.agentEnv ?? {};
|
|
97
103
|
try {
|
|
98
104
|
// 1) Install dependencies.
|
|
99
105
|
const install = installCommand(infra);
|
|
@@ -103,6 +109,7 @@ export async function standUpFrontend(dir, infra, signal, onActivity, logger = l
|
|
|
103
109
|
signal,
|
|
104
110
|
timeout: 8 * 60_000,
|
|
105
111
|
maxBuffer: 16 * 1024 * 1024,
|
|
112
|
+
env: { ...process.env, ...jobEnv },
|
|
106
113
|
});
|
|
107
114
|
pushOutput(installed.stdout, installed.stderr);
|
|
108
115
|
// 2) Build (build-time env injected here; runtime injection writes a shim after).
|
|
@@ -114,7 +121,7 @@ export async function standUpFrontend(dir, infra, signal, onActivity, logger = l
|
|
|
114
121
|
signal,
|
|
115
122
|
timeout: 12 * 60_000,
|
|
116
123
|
maxBuffer: 16 * 1024 * 1024,
|
|
117
|
-
env: { ...process.env, ...buildEnv },
|
|
124
|
+
env: { ...process.env, ...jobEnv, ...buildEnv },
|
|
118
125
|
});
|
|
119
126
|
pushOutput(built.stdout, built.stderr);
|
|
120
127
|
// Runtime injection: write a `window.env` shim into the build output the app can load
|
|
@@ -1,15 +1,29 @@
|
|
|
1
|
-
import { chmod, rm, writeFile } from 'node:fs/promises';
|
|
1
|
+
import { chmod, readFile, rm, writeFile } from 'node:fs/promises';
|
|
2
2
|
import { homedir } from 'node:os';
|
|
3
3
|
import { join } from 'node:path';
|
|
4
4
|
import { registerKnownSecrets } from './redact.js';
|
|
5
5
|
// Private package-registry auth for the checkout's installs (npm private orgs,
|
|
6
|
-
// GitHub Packages). The job's allowlisted entries are rendered into
|
|
7
|
-
//
|
|
8
|
-
//
|
|
9
|
-
// the
|
|
10
|
-
//
|
|
11
|
-
//
|
|
12
|
-
|
|
6
|
+
// GitHub Packages). The job's allowlisted entries are rendered into an npmrc — read by
|
|
7
|
+
// npm, pnpm and yarn v1 alike, and inherited by every child process (the agent's own
|
|
8
|
+
// shell installs and the frontend-infra stand-up's) — so the token never rides argv or
|
|
9
|
+
// the checkout.
|
|
10
|
+
//
|
|
11
|
+
// WHERE that npmrc lands depends on whether the harness process owns its HOME:
|
|
12
|
+
// - container (the default): the user `~/.npmrc`. HOME belongs to that one container, so
|
|
13
|
+
// writing it is safe and a job with NO entries CLEARS it — warm-pool containers are
|
|
14
|
+
// reused across jobs and must not leak a prior workspace's token.
|
|
15
|
+
// - shared native host process (`ambientAuth`, the local native transport): HOME is the
|
|
16
|
+
// DEVELOPER's. Writing there would overwrite their own npm config, clearing there would
|
|
17
|
+
// DELETE it, and concurrent jobs in the one process would race on the single file. Such a
|
|
18
|
+
// job gets its own npmrc under a per-job directory instead, pointed at by
|
|
19
|
+
// `npm_config_userconfig`; the developer's file is never written and never removed.
|
|
20
|
+
//
|
|
21
|
+
// Note the isolated path trades a little reach for that safety: `~/.npmrc` is read by npm, pnpm
|
|
22
|
+
// and yarn v1 alike, whereas `npm_config_userconfig` is honoured by npm and pnpm but NOT by yarn
|
|
23
|
+
// (v1 or Berry). A yarn-based checkout on the native path therefore sees only the developer's own
|
|
24
|
+
// registries, not the job's. Since the alternative is overwriting the file they actually use, the
|
|
25
|
+
// limitation stands — a yarn repo needing private-registry auth wants the container path.
|
|
26
|
+
/** Where the per-job npm auth lands in a container (the user npmrc, outside any checkout). */
|
|
13
27
|
export function npmrcPath() {
|
|
14
28
|
return join(homedir(), '.npmrc');
|
|
15
29
|
}
|
|
@@ -34,18 +48,69 @@ export function renderNpmrc(entries) {
|
|
|
34
48
|
return `${lines.join('\n')}\n`;
|
|
35
49
|
}
|
|
36
50
|
/**
|
|
37
|
-
* Write (or clear) the
|
|
38
|
-
*
|
|
51
|
+
* Write (or clear) the job's npmrc before the agent runs, and return the env the agent's child
|
|
52
|
+
* process needs to find it (empty for the container default, which npm picks up from HOME).
|
|
53
|
+
* Tokens are registered for output redaction so a token echoed in an npm error never reaches
|
|
39
54
|
* logs or stored output.
|
|
40
55
|
*/
|
|
41
|
-
export async function configurePackageRegistries(entries) {
|
|
56
|
+
export async function configurePackageRegistries(entries, scope = {}) {
|
|
57
|
+
const hasEntries = Boolean(entries?.length);
|
|
58
|
+
if (scope.isolatedDir) {
|
|
59
|
+
// A job with no entries needs no file at all: emitting no override leaves the developer's
|
|
60
|
+
// own `~/.npmrc` in effect (their private registries keep working) — and, crucially, leaves
|
|
61
|
+
// it ALONE. Clearing a stale file is a container concern; here nothing stale can exist,
|
|
62
|
+
// because the per-job dir is created and removed with the job.
|
|
63
|
+
if (!hasEntries)
|
|
64
|
+
return {};
|
|
65
|
+
const path = join(scope.isolatedDir, '.npmrc');
|
|
66
|
+
await writeIsolatedNpmrc(path, entries);
|
|
67
|
+
return { npm_config_userconfig: path };
|
|
68
|
+
}
|
|
42
69
|
const path = npmrcPath();
|
|
43
|
-
if (!
|
|
70
|
+
if (!hasEntries) {
|
|
44
71
|
await rm(path, { force: true });
|
|
45
|
-
return;
|
|
72
|
+
return {};
|
|
46
73
|
}
|
|
47
74
|
registerKnownSecrets(entries.map((entry) => entry.token));
|
|
48
75
|
await writeFile(path, renderNpmrc(entries), { mode: 0o600 });
|
|
49
76
|
// writeFile's mode only applies on create — tighten an existing file too.
|
|
50
77
|
await chmod(path, 0o600);
|
|
78
|
+
return {};
|
|
79
|
+
}
|
|
80
|
+
/**
|
|
81
|
+
* Write the per-job npmrc, seeded from the developer's own `~/.npmrc` when they have one so
|
|
82
|
+
* their unrelated settings (a corporate registry, a proxy) keep working for this run. The job's
|
|
83
|
+
* lines are APPENDED, and npm resolves the last occurrence of a key, so the job's entries win on
|
|
84
|
+
* any host they both configure. Copying their file into a 0600 temp adds no exposure: an ambient
|
|
85
|
+
* run already has the developer's full file access by definition.
|
|
86
|
+
*
|
|
87
|
+
* The seeded credentials are registered for redaction alongside the job's own. The job's tokens
|
|
88
|
+
* were always registered; the developer's were not, because before this path existed their file
|
|
89
|
+
* was overwritten and no credential of theirs was in play during the run. Now that theirs is in
|
|
90
|
+
* effect, an npm error echoing one must be scrubbed on exactly the same terms.
|
|
91
|
+
*/
|
|
92
|
+
async function writeIsolatedNpmrc(path, entries) {
|
|
93
|
+
registerKnownSecrets(entries.map((entry) => entry.token));
|
|
94
|
+
// Best-effort: no personal npmrc (or an unreadable one) just means the job's entries stand alone.
|
|
95
|
+
const inherited = await readFile(npmrcPath(), 'utf8').catch(() => '');
|
|
96
|
+
registerKnownSecrets(npmrcCredentials(inherited));
|
|
97
|
+
const prefix = inherited && !inherited.endsWith('\n') ? `${inherited}\n` : inherited;
|
|
98
|
+
await writeFile(path, `${prefix}${renderNpmrc(entries)}`, { mode: 0o600 });
|
|
99
|
+
await chmod(path, 0o600);
|
|
100
|
+
}
|
|
101
|
+
/**
|
|
102
|
+
* The credential VALUES in npmrc content: the three keys npm accepts a secret under, on any host
|
|
103
|
+
* line. Used to register a seeded (developer-owned) file's tokens for redaction. An `${ENV_VAR}`
|
|
104
|
+
* reference is not itself a secret — npm expands it at read time — so it is skipped rather than
|
|
105
|
+
* registered as a literal to scrub.
|
|
106
|
+
*/
|
|
107
|
+
export function npmrcCredentials(content) {
|
|
108
|
+
const found = [];
|
|
109
|
+
for (const line of content.split(/\r?\n/)) {
|
|
110
|
+
const match = /^\s*(?:.*:)?_(?:authToken|auth|password)\s*=\s*(.+?)\s*$/.exec(line);
|
|
111
|
+
const value = match?.[1]?.replace(/^["']|["']$/g, '');
|
|
112
|
+
if (value && !/^\$\{.*\}$/.test(value))
|
|
113
|
+
found.push(value);
|
|
114
|
+
}
|
|
115
|
+
return found;
|
|
51
116
|
}
|
package/dist/pi-workspace.js
CHANGED
|
@@ -118,11 +118,13 @@ export async function runAgentInWorkspace(spec, opts = {}) {
|
|
|
118
118
|
// harness paths; kept out of the agent's commits via a local git exclude entry.
|
|
119
119
|
const contextFiles = spec.contextFiles ?? [];
|
|
120
120
|
await materializeContextFiles(spec.dir, contextFiles);
|
|
121
|
-
// Repo-sourced skill (slice 2): claude-code installs it natively
|
|
122
|
-
//
|
|
123
|
-
//
|
|
124
|
-
//
|
|
125
|
-
|
|
121
|
+
// Repo-sourced skill (slice 2): claude-code installs it natively into its ISOLATED config dir,
|
|
122
|
+
// so it reads from there. Everything else reads the checkout, so materialise the skill's
|
|
123
|
+
// resources under `.cat-context/skill/` (its instructions are folded into the prompt by the
|
|
124
|
+
// backend) — Pi, codex, and AMBIENT claude-code, which has no isolated config dir to install
|
|
125
|
+
// into (the runner refuses to write a repo's skill into the developer's own `~/.claude`; see
|
|
126
|
+
// `runClaudeCode`). A resource-free skill is a no-op here.
|
|
127
|
+
if (spec.skill && !installsSkillNatively(spec)) {
|
|
126
128
|
await materializeSkillResources(spec.dir, spec.skill);
|
|
127
129
|
}
|
|
128
130
|
// Subscription harnesses (Claude Code / Codex) authenticate with the leased
|
|
@@ -144,6 +146,7 @@ export async function runAgentInWorkspace(spec, opts = {}) {
|
|
|
144
146
|
subscriptionBaseUrl: spec.subscriptionBaseUrl,
|
|
145
147
|
...(spec.ambientAuth ? { ambientAuth: true } : {}),
|
|
146
148
|
...(spec.skill ? { skill: spec.skill } : {}),
|
|
149
|
+
...(opts.agentEnv ? { extraEnv: opts.agentEnv } : {}),
|
|
147
150
|
signal: opts.signal,
|
|
148
151
|
onActivity: opts.onActivity,
|
|
149
152
|
onProgress: opts.onProgress,
|
|
@@ -169,9 +172,11 @@ export async function runAgentInWorkspace(spec, opts = {}) {
|
|
|
169
172
|
// container env, which `webSearchConfigFromEnv` autodetects.
|
|
170
173
|
// The proxy vars are handed to Pi's child via `extraEnv` (not the harness's own
|
|
171
174
|
// process.env), so detection runs against the same merged view the extension sees.
|
|
172
|
-
const extraEnv =
|
|
173
|
-
? webSearchProxyEnv(proxyBaseUrl, sessionToken)
|
|
174
|
-
|
|
175
|
+
const extraEnv = {
|
|
176
|
+
...(spec.webSearchProxy ? webSearchProxyEnv(proxyBaseUrl, sessionToken) : {}),
|
|
177
|
+
// Per-job env (tester secrets, a private-registry npmrc pointer) — see `RunOptions.agentEnv`.
|
|
178
|
+
...opts.agentEnv,
|
|
179
|
+
};
|
|
175
180
|
const webSearch = webSearchConfigFromEnv({ ...process.env, ...extraEnv });
|
|
176
181
|
if (webSearch)
|
|
177
182
|
await writeWebToolsConfig(webSearch);
|
|
@@ -201,6 +206,17 @@ export async function runAgentInWorkspace(spec, opts = {}) {
|
|
|
201
206
|
});
|
|
202
207
|
return withEffortReport(spec.dir, piOutcome);
|
|
203
208
|
}
|
|
209
|
+
/**
|
|
210
|
+
* Whether the claude-code runner will install this run's repo-sourced skill natively (into the
|
|
211
|
+
* CLI's config dir) rather than the caller materialising it into the checkout. True ONLY for a
|
|
212
|
+
* leased-credential claude-code run, which gets a throwaway per-run config home. An AMBIENT run
|
|
213
|
+
* uses the developer's own `~/.claude`, which the runner will not write a repo's skill into —
|
|
214
|
+
* it would outlive the run in their personal setup, and two concurrent jobs carrying same-named
|
|
215
|
+
* skills from different repos would overwrite each other's.
|
|
216
|
+
*/
|
|
217
|
+
export function installsSkillNatively(spec) {
|
|
218
|
+
return spec.harness === 'claude-code' && !spec.ambientAuth;
|
|
219
|
+
}
|
|
204
220
|
/**
|
|
205
221
|
* Lift the agent's effort self-assessment off its sentinel file in `dir` and fold it onto the
|
|
206
222
|
* run outcome. Shared by both harness paths so EVERY container agent's effort report is captured
|
package/dist/pi.js
CHANGED
|
@@ -210,9 +210,10 @@ export async function materializeContextFiles(cwd, files) {
|
|
|
210
210
|
export const SKILL_CONTEXT_SUBDIR = 'skill';
|
|
211
211
|
/**
|
|
212
212
|
* Materialise a repo-sourced skill's RESOURCE files under `.cat-context/skill/` in the checkout
|
|
213
|
-
* (repo-sourced Claude Skills, slice 2) — the
|
|
214
|
-
*
|
|
215
|
-
* the
|
|
213
|
+
* (repo-sourced Claude Skills, slice 2) — the path for every run that does NOT get a native
|
|
214
|
+
* install: Pi, codex, and ambient claude-code (no isolated `CLAUDE_CONFIG_DIR` to install into).
|
|
215
|
+
* Their agents read the checkout, and the skill's instructions are folded into their prompt by the
|
|
216
|
+
* backend (`renderSkillForHarness`, which keys off ambient auth as well as the harness). Resource sub-paths were sanitized at the job boundary (no traversal), so nested
|
|
216
217
|
* dirs are created as needed. Kept out of the agent's commits via the same `.cat-context/` git
|
|
217
218
|
* exclude entry. A skill with no resource bodies is a no-op.
|
|
218
219
|
*/
|