@cat-factory/executor-harness 1.58.0 → 1.62.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -2
- package/dist/agent.js +28 -14
- package/dist/captured-command.js +112 -0
- package/dist/coding-agent.js +90 -9
- package/dist/git.js +108 -319
- package/dist/host-markdown.js +142 -0
- package/dist/job.js +5 -46
- package/dist/pr-description.js +157 -0
- package/dist/reproduction-proof.js +614 -0
- package/dist/runner.js +3 -0
- package/dist/validation-checks.js +70 -82
- package/dist/vcs-api.js +402 -0
- package/package.json +4 -3
- package/src/agent.ts +28 -14
- package/src/captured-command.ts +144 -0
- package/src/coding-agent.ts +133 -6
- package/src/git.ts +134 -385
- package/src/host-markdown.ts +155 -0
- package/src/job.ts +32 -46
- package/src/pr-description.ts +171 -0
- package/src/reproduction-proof.ts +806 -0
- package/src/runner.ts +20 -0
- package/src/validation-checks.ts +71 -81
- package/src/vcs-api.ts +512 -0
package/README.md
CHANGED
|
@@ -63,9 +63,26 @@ The implementation job (`POST /run`) is the canonical sequence:
|
|
|
63
63
|
configured check commands (install/lint/test/build) run with `sh -c` in the checkout, and
|
|
64
64
|
while they fail and the attempt budget remains the agent is re-run with the captured output
|
|
65
65
|
as its instruction (see [pre-PR validation](../../../docs/initiatives/pre-pr-validation.md)),
|
|
66
|
-
5. **
|
|
66
|
+
5. **prove the reproduction**, when the job body carries `reproduction` — the declared check is
|
|
67
|
+
run against the pre-fix tree and the tree the PR will open from, in two freshly-created
|
|
68
|
+
symmetric `git worktree` checkouts, and only red-then-green is reported as proof (see
|
|
69
|
+
[bugfix reproduction proof](../../../docs/initiatives/bugfix-reproduction-proof.md)). Unlike
|
|
70
|
+
step 4 this NEVER gates the PR: a failed verification is fed back to the agent while budget
|
|
71
|
+
remains, then recorded as `inconclusive`. It runs BEFORE step 4 so validation stays the last
|
|
72
|
+
thing to touch the tree,
|
|
73
|
+
6. **commit, push** a branch and **open a PR**, returning `{ prUrl, branch, summary }` — but
|
|
67
74
|
ONLY if step 4 ended green. A spent budget returns an error result with the validation report
|
|
68
|
-
and opens no PR. Absent `validationChecks`,
|
|
75
|
+
and opens no PR. Absent `validationChecks` / `reproduction`, steps 4 and 5 do not happen at
|
|
76
|
+
all. The PR's description prefers the agent-authored reviewer briefing over the generic
|
|
77
|
+
dispatch-time text the job body carries: a PR-opening agent is prompted to write one to the
|
|
78
|
+
`.cat-pr-description.md` sentinel at the checkout root (one per sibling repo in a multi-repo
|
|
79
|
+
run; an optional leading `# <title>` line, when it is the file's only `#` heading, sets the PR
|
|
80
|
+
title), and `src/pr-description.ts` lifts it — secret-scrubbed, size-capped with a visible
|
|
81
|
+
note, made inert for the host by `src/host-markdown.ts`, kept out of the commit like the
|
|
82
|
+
effort/follow-ups sentinels — onto `openPullRequest`. Absent or unusable ⇒ the fallback text,
|
|
83
|
+
unchanged. On a RESUMED run the PR already exists, so an agent briefing additionally refreshes
|
|
84
|
+
its title/description in place (carrying the engine's managed report region across); the
|
|
85
|
+
generic fallback never does, so a human's edit is safe.
|
|
69
86
|
|
|
70
87
|
Bootstrap differs at the ends — it may start from an empty dir, and **resets
|
|
71
88
|
history to one commit and force-pushes** the default branch instead of opening a
|
|
@@ -133,7 +150,9 @@ Kimi / DeepSeek) and meters spend. The provider key never enters the container.
|
|
|
133
150
|
| `src/package-registries.ts` | Private-registry (npm) auth: renders the job's allowlisted entries into an npmrc — the user `~/.npmrc` in a container, a per-job file pointed at by `npm_config_userconfig` for a native job. |
|
|
134
151
|
| `src/agent-runner.ts` | The subscription-harness runners (`runClaudeCode` / `runCodex`) — talk direct to the vendor with a leased OAuth token, lift per-turn usage/telemetry off the CLI event stream. |
|
|
135
152
|
| `src/transcript-retention.ts` | Lifts the CLI session transcripts (`projects/` / `sessions/`) out of the isolated, credential-bearing config home before it is deleted, and prunes them on a TTL (debugging artifact retention). |
|
|
153
|
+
| `src/captured-command.ts` | The one way the harness runs a declared shell command on its own behalf: `sh -c` with a per-command watchdog, abort handling, conventional exit codes (124/127/130) and a scrub-then-bound output capture. Shared by both pre-PR verification phases so a fix to one cannot miss the other. |
|
|
136
154
|
| `src/validation-checks.ts` | Pre-PR validation: runs the job's check commands in the checkout (bounded, secret-scrubbed capture, per-command watchdog) and drives the retry-until-green loop that gates the PR. Generic — keyed off the job body, never the agent kind. |
|
|
155
|
+
| `src/reproduction-proof.ts` | Bugfix reproduction proof: runs the job's declared reproduction command against two symmetric fresh worktrees (the pre-fix tree and the final tree) and computes red-then-green from the exit codes, with a repair loop that never fails the run. Generic — keyed off the job body, never the agent kind. |
|
|
137
156
|
| `src/logger.ts` | Structured logging. |
|
|
138
157
|
|
|
139
158
|
## Runner lifecycle knobs
|
|
@@ -147,6 +166,9 @@ runner):
|
|
|
147
166
|
| `JOB_MAX_DURATION_MS` | `3600000` (60m) | Hard ceiling on a job's wall-clock time; force-fails after. |
|
|
148
167
|
| `JOB_INACTIVITY_MS` | `600000` (10m) | Kills a hung agent that produces no output for this long. |
|
|
149
168
|
| `VALIDATION_COMMAND_TIMEOUT_MS` | `900000` (15m) | Per-command watchdog for a pre-PR validation check; a timeout counts as a failure (exit 124) so one hung command can't wedge the loop. |
|
|
169
|
+
| `REPRODUCTION_COMMAND_TIMEOUT_MS` | `900000` (15m) | Per-command watchdog for a reproduction-proof setup or check command; a timeout counts as a failure (exit 124). |
|
|
170
|
+
| `REPRODUCTION_HEARTBEAT_MS` | `30000` (30s) | How often the reproduction proof feeds the job inactivity watchdog while it runs commands the agent is not producing output for. |
|
|
171
|
+
| `REPRODUCTION_TOTAL_BUDGET_MS` | `2700000` (45m) | Wall-clock ceiling on the WHOLE proof phase (every attempt, both trees, setup included). Attempts multiply two full tree runs each and the heartbeat above deliberately stops the inactivity watchdog from firing, so this is what bounds the phase. Checked at phase boundaries; exceeding it settles `inconclusive`, never a run failure. |
|
|
150
172
|
| `HARNESS_TRANSCRIPT_TTL_MS` | `259200000` (3d) | How long lifted subscription-CLI session transcripts are kept before the retention sweep prunes them. |
|
|
151
173
|
| `HARNESS_TRANSCRIPT_ROOT` | `<tmpdir>/cf-agent-transcripts` | Where retained session transcripts are moved to (one dir per run). Meaningful only on a reused (warm-pool) container; a per-run container is torn down with the job. The TTL sweep deletes only dirs it created (each carries a `.cf-retained` marker), so pointing this at a shared directory never touches unrelated content — though a dedicated dir is still recommended. An override on a different filesystem than the config home falls back to copy-then-remove. |
|
|
152
174
|
|
package/dist/agent.js
CHANGED
|
@@ -6,7 +6,9 @@ import { promisify } from 'node:util';
|
|
|
6
6
|
import { standUpFrontend, tearDownFrontend } from './frontend-infra.js';
|
|
7
7
|
import { configurePackageRegistries } from './package-registries.js';
|
|
8
8
|
import { captureRedactedOutput, redactSecrets, registerKnownSecrets } from './redact.js';
|
|
9
|
-
import { cloneRepo, commitAll, conflictDiff, fetchPullRequestHead, fetchReferenceBranches, hasAgentChanges, headCommit,
|
|
9
|
+
import { cloneRepo, commitAll, conflictDiff, fetchPullRequestHead, fetchReferenceBranches, hasAgentChanges, headCommit, mergeBranch, prepareExistingCheckout, pushBranch, reinitAndPush, unmergedPaths, } from './git.js';
|
|
10
|
+
import { inferVcsProvider, openPullRequest } from './vcs-api.js';
|
|
11
|
+
import { applyPrDescription } from './pr-description.js';
|
|
10
12
|
import { makeDirClaimer, noChangesReason, runCodingAgent, runMultiRepoCoding, } from './coding-agent.js';
|
|
11
13
|
import { validationFailureMessage } from './validation-checks.js';
|
|
12
14
|
import { acquireRepoCheckout, agentNeverActed, agentOutputTail, NEVER_ACTED_CAUSE, runAgentInWorkspace, unusableFinalAnswerCause, withWorkspace, } from './pi-workspace.js';
|
|
@@ -808,6 +810,10 @@ function buildSingleRepoCodingSpec(job, pushBranch) {
|
|
|
808
810
|
// Forwarded straight off the job body — the loop is generic machinery keyed on the data, not
|
|
809
811
|
// on the agent kind.
|
|
810
812
|
...(job.validationChecks ? { validationChecks: job.validationChecks } : {}),
|
|
813
|
+
// Bugfix reproduction proof: the declared command run against the pre-fix and final trees
|
|
814
|
+
// (see docs/initiatives/bugfix-reproduction-proof.md). Forwarded straight off the job body —
|
|
815
|
+
// like the checks above, the loop is generic machinery keyed on the data, not the agent kind.
|
|
816
|
+
...(job.reproduction ? { reproduction: job.reproduction } : {}),
|
|
811
817
|
};
|
|
812
818
|
}
|
|
813
819
|
/**
|
|
@@ -818,16 +824,20 @@ function buildSingleRepoCodingSpec(job, pushBranch) {
|
|
|
818
824
|
*/
|
|
819
825
|
async function runSingleRepoCoding(job, opts) {
|
|
820
826
|
const pushBranch = job.pushBranch ?? job.newBranch ?? job.branch;
|
|
821
|
-
const { summary, stats, stderrTail, pushed, usage, callMetrics, validation, validationReport, effortReport, } = await runCodingAgent(buildSingleRepoCodingSpec(job, pushBranch), opts);
|
|
827
|
+
const { summary, stats, stderrTail, pushed, usage, callMetrics, validation, validationReport, reproductionReport, effortReport, prDescription, } = await runCodingAgent(buildSingleRepoCodingSpec(job, pushBranch), opts);
|
|
822
828
|
// Ralph loop: the harness-computed validation verdict, forwarded onto the coding result as
|
|
823
829
|
// `ralphVerdict` so the backend's `toRunResult` lifts it onto `AgentRunResult.ralphVerdict`.
|
|
824
830
|
const ralphVerdict = validation ? { ralphVerdict: validation } : {};
|
|
825
831
|
// The agent's effort self-assessment, spread onto every result path below (mirrors ralphVerdict).
|
|
826
832
|
const effort = effortReport ? { effortReport } : {};
|
|
827
|
-
// The
|
|
828
|
-
// the captured proof the checkout was green when the PR opened; on the
|
|
829
|
-
// evidence behind the failure below.
|
|
830
|
-
|
|
833
|
+
// The two PRE-PR VERIFICATION reports, spread onto every result path below. The validation one:
|
|
834
|
+
// on the passing path it is the captured proof the checkout was green when the PR opened; on the
|
|
835
|
+
// exhausted path it is the evidence behind the failure below. The reproduction one is evidence
|
|
836
|
+
// on every path — it never gates the PR. Each is absent when its phase was not configured.
|
|
837
|
+
const verificationFields = {
|
|
838
|
+
...(validationReport ? { validationReport } : {}),
|
|
839
|
+
...(reproductionReport ? { reproductionReport } : {}),
|
|
840
|
+
};
|
|
831
841
|
// Pre-PR validation spent its attempt budget with the checkout still red. FAIL the job — do
|
|
832
842
|
// NOT open a pull request, and do not pretend the push succeeded as a deliverable. The work is
|
|
833
843
|
// still on the branch (a retry resumes on it); the report carries each failing command's exit
|
|
@@ -846,7 +856,7 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
846
856
|
failureCause: 'agent',
|
|
847
857
|
...(usage ? { usage } : {}),
|
|
848
858
|
...(callMetrics ? { callMetrics } : {}),
|
|
849
|
-
...
|
|
859
|
+
...verificationFields,
|
|
850
860
|
...effort,
|
|
851
861
|
};
|
|
852
862
|
}
|
|
@@ -861,7 +871,7 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
861
871
|
...(usage ? { usage } : {}),
|
|
862
872
|
...(callMetrics ? { callMetrics } : {}),
|
|
863
873
|
...ralphVerdict,
|
|
864
|
-
...
|
|
874
|
+
...verificationFields,
|
|
865
875
|
...effort,
|
|
866
876
|
};
|
|
867
877
|
}
|
|
@@ -874,7 +884,7 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
874
884
|
failureCause: 'no-changes',
|
|
875
885
|
...(usage ? { usage } : {}),
|
|
876
886
|
...(callMetrics ? { callMetrics } : {}),
|
|
877
|
-
...
|
|
887
|
+
...verificationFields,
|
|
878
888
|
...effort,
|
|
879
889
|
};
|
|
880
890
|
}
|
|
@@ -886,7 +896,11 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
886
896
|
ghToken: job.ghToken,
|
|
887
897
|
head: pushBranch,
|
|
888
898
|
base: job.repo.baseBranch,
|
|
889
|
-
|
|
899
|
+
// The agent-authored briefing (title/body) wins field-wise over the dispatch-time text.
|
|
900
|
+
pr: applyPrDescription(job.pr, prDescription),
|
|
901
|
+
// A resumed run's PR is already open, so refresh it rather than lose the briefing to the
|
|
902
|
+
// duplicate-PR 422 — only from a REAL briefing (see `refreshExisting` for why).
|
|
903
|
+
...(prDescription ? { refreshExisting: true } : {}),
|
|
890
904
|
apiBase: job.githubApiBase,
|
|
891
905
|
// The provider (set by the server from the configured backend) selects GitHub-PR vs
|
|
892
906
|
// GitLab-MR authoritatively; the clone URL supplies the GitLab REST base + project path.
|
|
@@ -908,7 +922,7 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
908
922
|
stats,
|
|
909
923
|
...(usage ? { usage } : {}),
|
|
910
924
|
...(callMetrics ? { callMetrics } : {}),
|
|
911
|
-
...
|
|
925
|
+
...verificationFields,
|
|
912
926
|
...effort,
|
|
913
927
|
};
|
|
914
928
|
}
|
|
@@ -921,7 +935,7 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
921
935
|
failureCause: 'no-changes',
|
|
922
936
|
...(usage ? { usage } : {}),
|
|
923
937
|
...(callMetrics ? { callMetrics } : {}),
|
|
924
|
-
...
|
|
938
|
+
...verificationFields,
|
|
925
939
|
...effort,
|
|
926
940
|
};
|
|
927
941
|
}
|
|
@@ -934,7 +948,7 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
934
948
|
...(usage ? { usage } : {}),
|
|
935
949
|
...(callMetrics ? { callMetrics } : {}),
|
|
936
950
|
...ralphVerdict,
|
|
937
|
-
...
|
|
951
|
+
...verificationFields,
|
|
938
952
|
...effort,
|
|
939
953
|
};
|
|
940
954
|
}
|
|
@@ -946,7 +960,7 @@ async function runSingleRepoCoding(job, opts) {
|
|
|
946
960
|
...(usage ? { usage } : {}),
|
|
947
961
|
...(callMetrics ? { callMetrics } : {}),
|
|
948
962
|
...ralphVerdict,
|
|
949
|
-
...
|
|
963
|
+
...verificationFields,
|
|
950
964
|
...effort,
|
|
951
965
|
};
|
|
952
966
|
}
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
import { spawn } from 'node:child_process';
|
|
2
|
+
import { killChildProcess, spawnDetached } from './process.js';
|
|
3
|
+
import { MAX_CAPTURED_OUTPUT_CHARS, redactSecrets } from './redact.js';
|
|
4
|
+
// The ONE way the harness runs a declared shell command on its own behalf (rather than through
|
|
5
|
+
// the agent) and keeps a bounded, secret-scrubbed record of what it printed.
|
|
6
|
+
//
|
|
7
|
+
// Both pre-PR verification phases need exactly this — the PRE-PR VALIDATION checks
|
|
8
|
+
// (`validation-checks.ts`) and the BUGFIX REPRODUCTION PROOF (`reproduction-proof.ts`) — and they
|
|
9
|
+
// need it to behave IDENTICALLY: same watchdog semantics, same abort handling, same conventional
|
|
10
|
+
// exit codes, same scrub-then-bound pipeline. They were two near-verbatim copies; a fix applied to
|
|
11
|
+
// one of them (a redaction ordering, an exit-code convention) silently missed the other, which is
|
|
12
|
+
// the whole reason this seam exists.
|
|
13
|
+
//
|
|
14
|
+
// Everything is PER-JOB by construction: the command, the cwd and the environment all arrive as
|
|
15
|
+
// arguments and nothing is read from or written to `process.env`/`HOME`. The local NATIVE
|
|
16
|
+
// transport serves every concurrent job from ONE host process, so a global would leak one job's
|
|
17
|
+
// state into a sibling's — and the container path would never catch it.
|
|
18
|
+
/**
|
|
19
|
+
* A little slack kept in the rolling capture buffer ON TOP of {@link MAX_CAPTURED_OUTPUT_CHARS},
|
|
20
|
+
* so scrubbing sees whole secrets.
|
|
21
|
+
*
|
|
22
|
+
* The buffer discards from the FRONT as output arrives, and `redactSecrets` only runs once the
|
|
23
|
+
* command settles. Without the margin a token straddling that rolling cut would already have lost
|
|
24
|
+
* its `KEY=` prefix by scrub time and would survive as an unrecognised partial. Capturing a bit
|
|
25
|
+
* more than we keep, scrubbing, and only THEN bounding to the real limit closes that window; 512
|
|
26
|
+
* chars comfortably exceeds any single credential assignment the rules match.
|
|
27
|
+
*/
|
|
28
|
+
const CAPTURE_MARGIN_CHARS = 512;
|
|
29
|
+
/**
|
|
30
|
+
* Run ONE command as `sh -c` in `cwd`, capturing a bounded, secret-scrubbed tail of its combined
|
|
31
|
+
* stdout+stderr. The exit code is the verdict — computed here by the harness, never self-reported
|
|
32
|
+
* by the model, which is the whole point of a programmatic phase. A watchdog kills the process
|
|
33
|
+
* tree on timeout and an aborted run resolves non-zero, so a phase is never what blocks a job
|
|
34
|
+
* from settling.
|
|
35
|
+
*
|
|
36
|
+
* The child inherits the JOB's environment (`RunOptions.agentEnv` layered over the process env),
|
|
37
|
+
* not a mutated global: the harness spawns this itself rather than through the agent, so without
|
|
38
|
+
* the explicit merge a native-mode job would run without the private-registry npmrc pointer (and,
|
|
39
|
+
* had this been staged in `process.env`, against a sibling job's state).
|
|
40
|
+
*
|
|
41
|
+
* `logLabel`/`logFields` shape only the two warnings this runner emits itself (the watchdog kill
|
|
42
|
+
* and a spawn failure); the caller keeps its own start/finish logging, which knows what the
|
|
43
|
+
* command MEANS.
|
|
44
|
+
*/
|
|
45
|
+
export async function runCapturedCommand(args) {
|
|
46
|
+
const { cwd, command, timeoutMs, reportTailChars, logLabel, logFields, logger, opts } = args;
|
|
47
|
+
const startedAt = Date.now();
|
|
48
|
+
return new Promise((resolve) => {
|
|
49
|
+
let out = '';
|
|
50
|
+
let settled = false;
|
|
51
|
+
let timedOut = false;
|
|
52
|
+
const child = spawn('sh', ['-c', command], {
|
|
53
|
+
cwd,
|
|
54
|
+
detached: spawnDetached,
|
|
55
|
+
stdio: ['ignore', 'pipe', 'pipe'],
|
|
56
|
+
env: { ...process.env, ...opts.agentEnv },
|
|
57
|
+
});
|
|
58
|
+
// Keep only the tail (plus the scrub margin); guard against unbounded buffering on a chatty
|
|
59
|
+
// command.
|
|
60
|
+
const capture = (chunk) => {
|
|
61
|
+
out = (out + chunk.toString('utf8')).slice(-(MAX_CAPTURED_OUTPUT_CHARS + CAPTURE_MARGIN_CHARS));
|
|
62
|
+
};
|
|
63
|
+
child.stdout?.on('data', capture);
|
|
64
|
+
child.stderr?.on('data', capture);
|
|
65
|
+
const finish = (exitCode) => {
|
|
66
|
+
if (settled)
|
|
67
|
+
return;
|
|
68
|
+
settled = true;
|
|
69
|
+
clearTimeout(timer);
|
|
70
|
+
opts.signal?.removeEventListener('abort', onAbort);
|
|
71
|
+
const trimmed = out.trim();
|
|
72
|
+
// Scrub BEFORE either bound: the pattern rules need a whole assignment to match, so the
|
|
73
|
+
// margin above is trimmed away only once the secrets are already gone.
|
|
74
|
+
const scrubbed = trimmed ? redactSecrets(trimmed).slice(-MAX_CAPTURED_OUTPUT_CHARS) : '';
|
|
75
|
+
resolve({
|
|
76
|
+
exitCode,
|
|
77
|
+
passed: exitCode === 0,
|
|
78
|
+
...(scrubbed ? { outputTail: boundTail(scrubbed, reportTailChars) } : {}),
|
|
79
|
+
durationMs: Date.now() - startedAt,
|
|
80
|
+
...(timedOut ? { timedOut: true } : {}),
|
|
81
|
+
...(scrubbed ? { fullTail: scrubbed } : {}),
|
|
82
|
+
});
|
|
83
|
+
};
|
|
84
|
+
const timer = setTimeout(() => {
|
|
85
|
+
logger.warn(`${logLabel}: command timed out`, { ...logFields, timeoutMs });
|
|
86
|
+
timedOut = true;
|
|
87
|
+
killChildProcess(child, undefined, logger);
|
|
88
|
+
finish(124); // conventional timeout exit code (a non-zero fail)
|
|
89
|
+
}, timeoutMs);
|
|
90
|
+
timer.unref?.();
|
|
91
|
+
const onAbort = () => {
|
|
92
|
+
killChildProcess(child, undefined, logger);
|
|
93
|
+
finish(130); // aborted (a non-zero fail)
|
|
94
|
+
};
|
|
95
|
+
opts.signal?.addEventListener('abort', onAbort, { once: true });
|
|
96
|
+
child.on('error', (err) => {
|
|
97
|
+
logger.warn(`${logLabel}: command failed to spawn`, {
|
|
98
|
+
...logFields,
|
|
99
|
+
error: err instanceof Error ? err.message : String(err),
|
|
100
|
+
});
|
|
101
|
+
finish(127); // spawn error / command not found (a non-zero fail)
|
|
102
|
+
});
|
|
103
|
+
child.on('close', (code) => finish(code ?? 1));
|
|
104
|
+
});
|
|
105
|
+
}
|
|
106
|
+
/** Bound an already-scrubbed output tail to what a REPORT carries, saying what it dropped. */
|
|
107
|
+
export function boundTail(scrubbed, maxChars) {
|
|
108
|
+
if (scrubbed.length <= maxChars)
|
|
109
|
+
return scrubbed;
|
|
110
|
+
const trimmed = scrubbed.length - maxChars;
|
|
111
|
+
return `…(${trimmed} earlier chars trimmed)\n${scrubbed.slice(-maxChars)}`;
|
|
112
|
+
}
|
package/dist/coding-agent.js
CHANGED
|
@@ -3,12 +3,15 @@ import { join } from 'node:path';
|
|
|
3
3
|
import { spawn } from 'node:child_process';
|
|
4
4
|
import { killChildProcess, spawnDetached } from './process.js';
|
|
5
5
|
import { MAX_CAPTURED_OUTPUT_CHARS, redactSecrets } from './redact.js';
|
|
6
|
-
import { branchAheadOfBase, branchHasCommitsSince, cloneExistingBranch, cloneRepo, commitTrackedEdits, createBranch, excludeFromGit, fetchReferenceBranches, headCommit, listUntrackedFiles,
|
|
6
|
+
import { branchAheadOfBase, changedFilesSinceBase, branchHasCommitsSince, cloneExistingBranch, cloneRepo, commitTrackedEdits, createBranch, excludeFromGit, fetchReferenceBranches, headCommit, listUntrackedFiles, prepareExistingCheckout, pushBranch, refreshFromBaseIfClean, remoteBranchExists, } from './git.js';
|
|
7
|
+
import { openPullRequest } from './vcs-api.js';
|
|
7
8
|
import { FOLLOW_UPS_FILENAME, FollowUpTailer } from './follow-ups.js';
|
|
8
9
|
import { EFFORT_REPORT_FILE } from './effort.js';
|
|
10
|
+
import { applyPrDescription, PR_DESCRIPTION_FILE, readPrDescription, } from './pr-description.js';
|
|
9
11
|
import { acquireRepoCheckout, agentNeverActed, agentOutputTail, runAgentInWorkspace, withWorkspace, } from './pi-workspace.js';
|
|
10
12
|
import { log } from './logger.js';
|
|
11
13
|
import { runValidationLoop, } from './validation-checks.js';
|
|
14
|
+
import { runReproductionLoop, } from './reproduction-proof.js';
|
|
12
15
|
/**
|
|
13
16
|
* How often the harness checkpoints the agent's work mid-run by pushing the branch.
|
|
14
17
|
* A per-run container can be evicted at any moment; pushing the agent's commits
|
|
@@ -114,6 +117,9 @@ export async function runCodingAgent(spec, opts = {}) {
|
|
|
114
117
|
// but that cannot un-stage a mid-run commit; the per-clone exclude is what prevents it. A bare
|
|
115
118
|
// filename pattern matches the file in any subdirectory, so it covers a monorepo `workDir` too.
|
|
116
119
|
await excludeFromGit(dir, EFFORT_REPORT_FILE, signal);
|
|
120
|
+
// Same treatment for the agent-authored PR-description sentinel: excluded locally so the
|
|
121
|
+
// agent's own `git add` can never stage the briefing into the PR it describes.
|
|
122
|
+
await excludeFromGit(dir, PR_DESCRIPTION_FILE, signal);
|
|
117
123
|
// Follow-up companion: tail the Coder's sentinel file and stream new items out on the
|
|
118
124
|
// job view. Locally exclude it from git first so the agent's own `git add` can never
|
|
119
125
|
// stage it and it never surfaces as an untracked leftover or in the PR. The sentinel
|
|
@@ -154,6 +160,55 @@ export async function runCodingAgent(spec, opts = {}) {
|
|
|
154
160
|
opts.onPhase?.('agent');
|
|
155
161
|
logger.info('coding-agent: running agent', { serviceDirectory });
|
|
156
162
|
let agentRun = await runAgentPass(spec.userPrompt);
|
|
163
|
+
const foldPass = (run) => {
|
|
164
|
+
agentRun = mergeAgentPasses(agentRun, run);
|
|
165
|
+
};
|
|
166
|
+
// The new files the agent left unadded, folded into either loop's repair prompt. Both
|
|
167
|
+
// loops judge state the push will NOT carry unless it is committed — the checks run
|
|
168
|
+
// against the working tree, the proof against committed trees — so an unadded file is
|
|
169
|
+
// exactly the thing to name. A throw degrades to "no warning" inside each loop.
|
|
170
|
+
const listUncommittedNewFiles = () => listUntrackedFiles(workDir, opts.signal);
|
|
171
|
+
// BUGFIX REPRODUCTION PROOF: run the run's declared reproduction command against the
|
|
172
|
+
// pre-fix tree and the tree the PR will open from, and record whether it was red then
|
|
173
|
+
// green. Runs BEFORE the validation loop below, deliberately: validation is the GATE
|
|
174
|
+
// ("only a green checkout opens a PR"), so it has to stay the last thing that touches the
|
|
175
|
+
// tree — otherwise a reproduction repair round could leave the checkout red behind it and
|
|
176
|
+
// the PR would open anyway. Keyed purely off the job body carrying a spec (no agent-kind
|
|
177
|
+
// switch); absent ⇒ a no-op and the flow below is byte-for-byte what it was.
|
|
178
|
+
const reproduction = spec.reproduction;
|
|
179
|
+
let reproductionReport;
|
|
180
|
+
if (reproduction && (await producedWork(dir, spec, baseSha, resumed, opts))) {
|
|
181
|
+
opts.onPhase?.('reproduction');
|
|
182
|
+
reproductionReport = await runReproductionLoop({
|
|
183
|
+
dir,
|
|
184
|
+
baseSha,
|
|
185
|
+
// Re-read per attempt: a repair pass commits, so the final tree moves under the loop.
|
|
186
|
+
// `producedWork` has already committed forgotten tracked edits, and each repair round
|
|
187
|
+
// re-commits before the next read.
|
|
188
|
+
resolveFinalSha: async () => {
|
|
189
|
+
await commitTrackedEdits(dir, spec.commitMessage, signal);
|
|
190
|
+
return headCommit(dir, signal);
|
|
191
|
+
},
|
|
192
|
+
...(serviceDirectory ? { serviceDirectory } : {}),
|
|
193
|
+
spec: reproduction,
|
|
194
|
+
logger,
|
|
195
|
+
opts,
|
|
196
|
+
runAgentPass,
|
|
197
|
+
onAgentPass: foldPass,
|
|
198
|
+
listUncommittedNewFiles,
|
|
199
|
+
// Only a RESUMED run can have a pre-fix tree that already carries work: a fresh run
|
|
200
|
+
// branched off base, so `baseSha` IS base. Wiring the probe unconditionally would buy
|
|
201
|
+
// an always-empty answer for the price of a fetch — and a fresh clone is shallow, so
|
|
202
|
+
// it could not resolve a merge base to answer with anyway. Lazy inside the loop: it
|
|
203
|
+
// only runs if a tree comes back green.
|
|
204
|
+
...(resumed
|
|
205
|
+
? {
|
|
206
|
+
listBaseTreeChanges: () => changedFilesSinceBase(dir, spec.repo.baseBranch, spec.ghToken, baseSha, opts.signal),
|
|
207
|
+
}
|
|
208
|
+
: {}),
|
|
209
|
+
});
|
|
210
|
+
opts.onPhase?.('agent');
|
|
211
|
+
}
|
|
157
212
|
// PRE-PR VALIDATION: run the service's configured checks against the checkout and, while
|
|
158
213
|
// they fail and budget remains, hand the captured output back to the agent and run it
|
|
159
214
|
// again. Sits BETWEEN the agent and the finalize/push/PR step so a red checkout never
|
|
@@ -169,17 +224,16 @@ export async function runCodingAgent(spec, opts = {}) {
|
|
|
169
224
|
logger,
|
|
170
225
|
opts,
|
|
171
226
|
runAgentPass,
|
|
172
|
-
onAgentPass:
|
|
173
|
-
agentRun = mergeAgentPasses(agentRun, run);
|
|
174
|
-
},
|
|
227
|
+
onAgentPass: foldPass,
|
|
175
228
|
// The checks run against the WORKING TREE, but only tracked edits are staged for the
|
|
176
229
|
// push — so a repair round can go green on a new file the PR would never contain.
|
|
177
230
|
// Name those files in the next repair prompt so the agent adds them.
|
|
178
|
-
listUncommittedNewFiles
|
|
231
|
+
listUncommittedNewFiles,
|
|
179
232
|
});
|
|
180
233
|
}
|
|
181
234
|
outcome = await finalizeCodingRun({
|
|
182
235
|
validationReport,
|
|
236
|
+
reproductionReport,
|
|
183
237
|
dir,
|
|
184
238
|
spec,
|
|
185
239
|
logger,
|
|
@@ -314,7 +368,7 @@ async function prepareCodingCheckout(dir, spec, logger, opts) {
|
|
|
314
368
|
* {@link runCodingAgent} so its body stays small; returns the built {@link CodingAgentOutcome}.
|
|
315
369
|
*/
|
|
316
370
|
async function finalizeCodingRun(args) {
|
|
317
|
-
const { validationReport, dir, spec, logger, opts, baseSha, resumed, workDir, checkpoint, followUpTick, followUpTailer, pushWorkOnce, inFlightPush, agentRun, } = args;
|
|
371
|
+
const { validationReport, reproductionReport, dir, spec, logger, opts, baseSha, resumed, workDir, checkpoint, followUpTick, followUpTailer, pushWorkOnce, inFlightPush, agentRun, } = args;
|
|
318
372
|
const { signal } = opts;
|
|
319
373
|
const { summary, stats, stderrTail, usage, callMetrics, effortReport } = agentRun;
|
|
320
374
|
let outcome;
|
|
@@ -327,6 +381,12 @@ async function finalizeCodingRun(args) {
|
|
|
327
381
|
// Safety net for forgotten edits: commit changes to TRACKED files only (never
|
|
328
382
|
// untracked scratch files/artifacts — the agent owns committing new files).
|
|
329
383
|
await commitTrackedEdits(dir, spec.commitMessage, signal);
|
|
384
|
+
// The agent-authored PR description, read AFTER the validation loop (a repair round may have
|
|
385
|
+
// changed what the briefing should say) and removed so it never lingers in the checkout. The
|
|
386
|
+
// prompt asks for it at the top level of the checkout; a monorepo agent working in a service
|
|
387
|
+
// subdirectory may drop it in its cwd instead, so probe the checkout root first, then the cwd.
|
|
388
|
+
const prDescription = (await readPrDescription(dir)) ??
|
|
389
|
+
(workDir !== dir ? await readPrDescription(workDir) : undefined);
|
|
330
390
|
// Stop periodic checkpoints and let any in-flight one settle BEFORE the final
|
|
331
391
|
// push, so the two never run a concurrent `git push` to the same branch (the
|
|
332
392
|
// final push below is then a fresh attempt whose failure is the real signal).
|
|
@@ -389,6 +449,7 @@ async function finalizeCodingRun(args) {
|
|
|
389
449
|
...(usage ? { usage } : {}),
|
|
390
450
|
...(callMetrics ? { callMetrics } : {}),
|
|
391
451
|
...(effortReport ? { effortReport } : {}),
|
|
452
|
+
...(prDescription ? { prDescription } : {}),
|
|
392
453
|
};
|
|
393
454
|
}
|
|
394
455
|
// Ralph loop: run the programmatic completion command against the pushed/committed
|
|
@@ -403,6 +464,10 @@ async function finalizeCodingRun(args) {
|
|
|
403
464
|
// backend to record on the step.
|
|
404
465
|
if (validationReport)
|
|
405
466
|
outcome.validationReport = validationReport;
|
|
467
|
+
// The reproduction proof: attached to EVERY outcome, including a no-op or an `inconclusive`
|
|
468
|
+
// verdict. It is evidence about the change, not a gate on it — see the loop's D6 note.
|
|
469
|
+
if (reproductionReport)
|
|
470
|
+
outcome.reproductionReport = reproductionReport;
|
|
406
471
|
return outcome;
|
|
407
472
|
}
|
|
408
473
|
/**
|
|
@@ -645,7 +710,7 @@ export async function runMultiRepoCoding(job, opts = {}) {
|
|
|
645
710
|
multiRepo: true,
|
|
646
711
|
}, opts);
|
|
647
712
|
// Commit forgotten tracked edits, then push + open a PR for each repo the run actually changed.
|
|
648
|
-
const { primaryPushed, primaryPrUrl, peerPullRequests } = await pushMultiRepoLegs(legs, job, logger, opts);
|
|
713
|
+
const { primaryPushed, primaryPrUrl, peerPullRequests } = await pushMultiRepoLegs(legs, job, logger, opts, root);
|
|
649
714
|
const anyWork = primaryPushed || peerPullRequests.length > 0;
|
|
650
715
|
if (!anyWork) {
|
|
651
716
|
// Nothing changed in ANY repo. For the implementer this is a failure (as in the
|
|
@@ -746,6 +811,9 @@ async function prepareMultiRepoCheckouts(root, legs, job, logger, opts) {
|
|
|
746
811
|
await createBranch(dir, leg.workBranch, signal);
|
|
747
812
|
}
|
|
748
813
|
leg.dir = dir;
|
|
814
|
+
// Exclude the agent-authored PR-description sentinel locally (as the single-repo path does)
|
|
815
|
+
// so the agent's own `git add` can never stage the briefing into the PR it describes.
|
|
816
|
+
await excludeFromGit(dir, PR_DESCRIPTION_FILE, signal);
|
|
749
817
|
// The branch tip before the agent runs. Captured BEFORE the resume base refresh below so
|
|
750
818
|
// that refresh's merge commit counts as advancement and is pushed (as in the single-repo
|
|
751
819
|
// path). A fresh leg produced work iff its branch advances past this; a resumed leg already
|
|
@@ -791,7 +859,9 @@ async function prepareMultiRepoCheckouts(root, legs, job, logger, opts) {
|
|
|
791
859
|
* no PR; a read-only reference leg is never committed or pushed). Extracted so the multi-repo body
|
|
792
860
|
* stays small; returns the primary's push/PR state plus the peer PRs.
|
|
793
861
|
*/
|
|
794
|
-
async function pushMultiRepoLegs(legs, job, logger, opts
|
|
862
|
+
async function pushMultiRepoLegs(legs, job, logger, opts,
|
|
863
|
+
/** The workspace root the agent ran in — the fallback probe for the primary's briefing. */
|
|
864
|
+
root) {
|
|
795
865
|
const { signal } = opts;
|
|
796
866
|
opts.onPhase?.('push');
|
|
797
867
|
let primaryPushed = false;
|
|
@@ -802,6 +872,14 @@ async function pushMultiRepoLegs(legs, job, logger, opts) {
|
|
|
802
872
|
// guarantee (the spec carries no branch/PR, and the clone phase gave it no work branch).
|
|
803
873
|
if (leg.readOnly)
|
|
804
874
|
continue;
|
|
875
|
+
// Lift (and remove) the agent-authored PR description for THIS repo's PR before anything
|
|
876
|
+
// else touches the checkout — each sibling checkout carries its own briefing for its own PR.
|
|
877
|
+
// The agent's cwd here is the WORKSPACE ROOT rather than any one checkout, so an agent that
|
|
878
|
+
// read the prompt loosely may well have written a single briefing there instead. Fall back
|
|
879
|
+
// to it for the PRIMARY leg only: at the root there is nothing to say which repo it
|
|
880
|
+
// describes, and the primary is the one the run is actually about.
|
|
881
|
+
const agentPrDescription = (await readPrDescription(leg.dir)) ??
|
|
882
|
+
(leg.primary ? await readPrDescription(root) : undefined);
|
|
805
883
|
await commitTrackedEdits(leg.dir, job.commitMessage ?? leg.pr?.title ?? 'Agent changes', signal);
|
|
806
884
|
const advanced = await branchHasCommitsSince(leg.dir, leg.baseSha, signal);
|
|
807
885
|
let hasWork = advanced || leg.resumed;
|
|
@@ -831,7 +909,10 @@ async function pushMultiRepoLegs(legs, job, logger, opts) {
|
|
|
831
909
|
ghToken: leg.ghToken,
|
|
832
910
|
head: leg.workBranch,
|
|
833
911
|
base: leg.repo.baseBranch,
|
|
834
|
-
pr: leg.pr,
|
|
912
|
+
pr: applyPrDescription(leg.pr, agentPrDescription),
|
|
913
|
+
// See the single-repo call site: refresh a resumed leg's already-open PR, but only
|
|
914
|
+
// when the text is the agent's own briefing rather than the dispatch-time fallback.
|
|
915
|
+
...(agentPrDescription ? { refreshExisting: true } : {}),
|
|
835
916
|
apiBase: job.githubApiBase,
|
|
836
917
|
cloneUrl: leg.repo.cloneUrl,
|
|
837
918
|
...(leg.repo.provider ? { provider: leg.repo.provider } : {}),
|