nomarmy 0.1.0-alpha.21 → 0.1.0-alpha.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/nomarmy.mjs +155 -16
- package/lib/acceptance-fill.mjs +180 -0
- package/lib/acceptance-impact.mjs +215 -0
- package/lib/acceptance-result.mjs +21 -0
- package/lib/acceptance.mjs +166 -0
- package/lib/admission.mjs +33 -3
- package/lib/army.mjs +2 -2
- package/lib/codex-link.mjs +6 -1
- package/lib/connect.mjs +6 -2
- package/lib/continue-from.mjs +29 -12
- package/lib/coordinator-instructions.mjs +4 -0
- package/lib/diff-checks.mjs +9 -3
- package/lib/doctor.mjs +3 -1
- package/lib/execute.mjs +187 -32
- package/lib/git-record.mjs +9 -2
- package/lib/health.mjs +25 -3
- package/lib/install-freshness.mjs +2 -2
- package/lib/jev-checks.mjs +13 -0
- package/lib/job-format.mjs +3 -1
- package/lib/judge.mjs +26 -5
- package/lib/line-diff.mjs +46 -0
- package/lib/mutation.mjs +27 -13
- package/lib/openclaw-run.mjs +21 -7
- package/lib/process.mjs +30 -4
- package/lib/repo-query.mjs +4 -4
- package/lib/sandbox-images.mjs +31 -4
- package/lib/schema.mjs +28 -0
- package/lib/server-context.mjs +3 -2
- package/lib/share.mjs +53 -2
- package/lib/stats.mjs +11 -0
- package/lib/trust-checks.mjs +269 -0
- package/lib/trust-files.mjs +123 -0
- package/lib/trust-judgment.mjs +157 -0
- package/lib/trust-learning.mjs +153 -0
- package/lib/trust-map.mjs +149 -0
- package/lib/trust-reach.mjs +220 -0
- package/lib/trust.mjs +227 -0
- package/lib/validators.mjs +7 -0
- package/lib/verification-flow.mjs +25 -18
- package/lib/verification-network.mjs +2 -2
- package/lib/verify.mjs +46 -3
- package/lib/worktree-pointer.mjs +246 -0
- package/lib/worktree-write.mjs +128 -0
- package/mcp/server.mjs +28 -11
- package/package.json +1 -1
- package/playbooks/feature.md +5 -5
- package/scripts/trust-measure.mjs +139 -0
|
@@ -16,10 +16,14 @@ Before dispatching:
|
|
|
16
16
|
- To run tests without changing anything, use mode: verify; it costs no model usage.
|
|
17
17
|
- Mark an implement job stakes: high when a mistake would be costly (security, access control, personal or tenant data, data loss, money, irreversible changes), however small it is. It then needs a verification profile, keeps the revert check, and needs an independent review before you accept it: a scout on another vendor with reviews: <job id>.
|
|
18
18
|
- Brief outcomes, not edits: a task, explicit acceptance criteria, and the tests that prove it. Put facts you've already resolved in evidence.
|
|
19
|
+
- At feature planning, write acceptance/<feature>.yml from durable outcome promises, not job hygiene. Assign stable criterion IDs in each job brief before parallel dispatch and pass them as criteria; jobs must not pick the next free ID. After integration, preview nomarmy acceptance fill <run-id> --dry-run, prune unrelated proposed mappings, fill for real, and commit the contract with the feature. A CONTRACT BROKEN: issue means the change is wrong or the contract must intentionally change in the same pull request.
|
|
19
20
|
- Prefer local_worker_start for anything longer than a few minutes. Right after, if you can run a background command, run \`nomarmy jobs --wait <job_id>\` in the background so you're told the moment it finishes and can tell the operator (for several jobs, \`nomarmy jobs --wait <id> <id> ...\`; for a whole run, \`nomarmy jobs --events --until-done --run <run-id>\`); otherwise poll local_worker_status with wait_seconds. To read a finished job, use local_worker_status with report: true (the report, a scout's cited findings, the outcome, issues and commit); full: true is the whole record and rarely needed. Review scouts (reviews set, or a review-phase role) default to 20 minutes, other jobs to 10; pass timeout_seconds for more. Never use unscoped \`--until-done\` when other sessions may have jobs. Never run the plain \`nomarmy jobs --events\` stream as a background command: it only reports when it exits, so you'd never hear; it's for a monitor that reads each line. For a whole feature, use /feature (run_start keeps a run's jobs, spend and hours bounded).
|
|
20
21
|
|
|
21
22
|
Trust boundary:
|
|
22
23
|
- A worker's four-line report is a claim; nomArmy's verified git record and independent verification are the evidence. A job isn't complete if its report is missing or malformed, its STATUS is partial or blocked, STATUS done lacks VERIFICATION pass, or its changes aren't committed by nomArmy.
|
|
24
|
+
- If a repository has no accepted trust map, suggest nomarmy trust map once; dispatch its scout brief and use nomarmy trust review for operator ratification.
|
|
25
|
+
- A job with trust.level: review or human needs an independent review on another vendor before acceptance, like stakes: high.
|
|
26
|
+
- A job with trust.level: human is a hard stop before integration: show every trust reason to the operator and wait for explicit approval. Never decide this gate alone. After the operator decides, record nomarmy trust ack <job> --accept|--reject --reason ...; rejection never permits integration.
|
|
23
27
|
- Read the diff of anything material before integrating it. nomArmy commits on the worker's branch and never merges into yours: integration, conflicts and pushes are yours.
|
|
24
28
|
- A job on the wrong track can be stopped with local_worker_stop; it keeps its worktree. To finish a job that came back partial, blocked, failing verification or stopped, dispatch the correction with continue_from: <that job id> (a cheaper model is fine). The new job starts with its unfinished work in place and verifies the whole. Don't fix and commit a worker's files yourself: that lands them unverified. If you must, commit them on a branch, run mode: verify on it before building on it, and say so in your report.
|
|
25
29
|
- Failed and incomplete worktrees are kept for review; clean up with local_worker_cleanup or local_worker_sweep once you've decided.
|
package/lib/diff-checks.mjs
CHANGED
|
@@ -41,10 +41,16 @@ export function isCiConfigPath(file) {
|
|
|
41
41
|
|| normalized === "bitbucket-pipelines.yml";
|
|
42
42
|
}
|
|
43
43
|
|
|
44
|
+
export function isAcceptanceContractPath(file) {
|
|
45
|
+
const normalized = String(file ?? "").replace(/\\/g, "/").replace(/^\.\//, "");
|
|
46
|
+
return /^acceptance\/[^/]+\.ya?ml$/.test(normalized);
|
|
47
|
+
}
|
|
48
|
+
|
|
44
49
|
export function planRegressionProductionFiles(files) {
|
|
45
50
|
// Local repository tests cannot exercise CI services, so reverting CI
|
|
46
|
-
// configuration cannot provide useful regression evidence.
|
|
47
|
-
|
|
51
|
+
// configuration cannot provide useful regression evidence. Acceptance
|
|
52
|
+
// contracts are checked directly against their proofs instead.
|
|
53
|
+
return (files ?? []).filter((file) => !isDocumentationPath(file) && !isCiConfigPath(file) && !isAcceptanceContractPath(file));
|
|
48
54
|
}
|
|
49
55
|
|
|
50
56
|
export function isTestPath(file) {
|
|
@@ -266,7 +272,7 @@ function meaningfulTokens(name) {
|
|
|
266
272
|
}
|
|
267
273
|
const TEST_NAME_PATTERN = /^test[_A-Za-z]/i;
|
|
268
274
|
const MIN_CLAIM_OVERLAP = 2; // fewer shared, meaningful words is not a specific-enough claim to check
|
|
269
|
-
const escRegex = (s) => String(s).replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
275
|
+
export const escRegex = (s) => String(s).replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
|
270
276
|
|
|
271
277
|
/**
|
|
272
278
|
* @param {{ cwd: string, productionFiles: string[], testFiles: string[], gitDiffFn: (file: string) => Promise<string>, outlineFn: Function, readFileFn: (cwd: string, file: string) => string }} input
|
package/lib/doctor.mjs
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { prepareHostGit } from "./worktree-pointer.mjs";
|
|
1
2
|
// lib/doctor.mjs
|
|
2
3
|
// Pure logic for the `nomarmy doctor` command.
|
|
3
4
|
// Side effects (probing PATH, spawning podman, hitting the worker endpoint)
|
|
@@ -486,7 +487,8 @@ async function probeGitLongPaths(gitPath) {
|
|
|
486
487
|
let out = "";
|
|
487
488
|
let child;
|
|
488
489
|
try {
|
|
489
|
-
|
|
490
|
+
const prep = prepareHostGit(process.cwd());
|
|
491
|
+
child = spawn(gitPath, [...(prep?.argsPrefix ?? []), "config", "--get", "core.longpaths"], { env: prep ? prep.env() : process.env, stdio: ["ignore", "pipe", "ignore"] });
|
|
490
492
|
} catch { return resolve(null); }
|
|
491
493
|
const timer = setTimeout(() => { child.kill(); resolve(null); }, 3000);
|
|
492
494
|
child.stdout.on("data", (d) => { out += d.toString(); });
|
package/lib/execute.mjs
CHANGED
|
@@ -1,3 +1,7 @@
|
|
|
1
|
+
import { recordReviewEvidence } from "./trust-learning.mjs";
|
|
2
|
+
import { checkJobContracts, changedContractWeakening, touchesAcceptance } from "./acceptance-impact.mjs";
|
|
3
|
+
import { jobAcceptanceProposals } from "./acceptance-fill.mjs";
|
|
4
|
+
import { readWorktreeFile } from "./worktree-write.mjs";
|
|
1
5
|
import { writeStatus, shouldRetryTransientAbort, shouldAttemptScoutRecovery, readStopRequest } from "./openclaw-run.mjs";
|
|
2
6
|
import fs from "node:fs";
|
|
3
7
|
import path from "node:path";
|
|
@@ -5,16 +9,22 @@ import { fileURLToPath } from "node:url";
|
|
|
5
9
|
import { vmRestartIssue } from "./podman-health.mjs";
|
|
6
10
|
import { measureReads } from "./job-budgets.mjs";
|
|
7
11
|
import { coordinatorCommitMessage, worktreePointerState } from "./git-record.mjs";
|
|
12
|
+
import { assertWorktreePointer, assertRegisteredWorktree, registerJobWorktree, registerJobsRoot, sealWorktree } from "./worktree-pointer.mjs";
|
|
8
13
|
import { parseScoutReport, verifyCitations, resolveScoutOutcome, renderScoutReport, isScoutReportUnusable, scoutReportRecoveryPrompt } from "./scout.mjs";
|
|
9
14
|
import { parseDecomposeReport, buildDecomposeFindings, resolveDecomposeOutcome, checkDecompositionOverlap, renderDecomposeReport } from "./decompose.mjs";
|
|
10
15
|
import { deriveTimeBudget } from "./budget.mjs";
|
|
11
|
-
import { continuationProblem, continuationBase, snapshotRetainedWork, applyRetainedWork, continuationNote } from "./continue-from.mjs";
|
|
16
|
+
import { continuationStakes, continuationProblem, continuationBase, snapshotRetainedWork, applyRetainedWork, continuationNote } from "./continue-from.mjs";
|
|
12
17
|
import { checkScoutCitations, checkReportClaims } from "./jev-checks.mjs";
|
|
13
18
|
import { runJudge } from "./judge.mjs";
|
|
14
19
|
import { pickMutants, runMutants, describeSurvivors } from "./mutation.mjs";
|
|
15
20
|
import { estimateDisplacement, readOpenClawTranscript } from "./transcript.mjs";
|
|
16
21
|
import { outlineFile, findReferences } from "./repo-query.mjs";
|
|
17
22
|
import { loadConfig } from "./config.mjs";
|
|
23
|
+
import { loadTrustMap } from "./trust-map.mjs";
|
|
24
|
+
import { cachedTrustReach } from "./trust-reach.mjs";
|
|
25
|
+
import { snapshotTrustFiles } from "./trust-files.mjs";
|
|
26
|
+
import { evaluateDiffTrust, trustValidatorSettings } from "./trust-judgment.mjs";
|
|
27
|
+
import { evaluateTrust, codeownersPaths, readTrustWorktreeFile, UNCHECKABLE_TRUST_FILE } from "./trust.mjs";
|
|
18
28
|
import { linkNodePackages, nodeModulesState, repairHostInstalls } from "./sandbox-images.mjs";
|
|
19
29
|
import { detectTestSabotage, addedLinesOf, loadDependencyNames } from "./sabotage.mjs";
|
|
20
30
|
import { describeRecoveryChanges, reportRecoveryPrompt } from "./worker-prompt.mjs";
|
|
@@ -33,6 +43,7 @@ export const JOB_PHASES = Object.freeze(["starting", "worktree", "worker", "veri
|
|
|
33
43
|
export const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
|
|
34
44
|
|
|
35
45
|
export function createExecutor(deps) {
|
|
46
|
+
registerJobsRoot(deps.jobsRoot);
|
|
36
47
|
const { VERSION, projectDir, jobsRoot, run, git, gitRaw,
|
|
37
48
|
collectGitRecord, createCoordinatorCommit, ensureJobsRoot, slug, assertRepo,
|
|
38
49
|
resolveBase, workerModelThinkingSupported, budgetState, execution, buildMetrics,
|
|
@@ -41,8 +52,24 @@ export function createExecutor(deps) {
|
|
|
41
52
|
runRegressionCheck, repoPolicy } = deps;
|
|
42
53
|
// Optional Jev checks (lib/validators.mjs): the server wires the settings; without them (tests), none run.
|
|
43
54
|
const jevSettingsFor = (check) => { try { const s = deps.jevSettings?.(); return s?.checks?.includes(check) ? s : null; } catch { return null; } };
|
|
55
|
+
// Seal a worktree nomArmy just created. A mocked worktree add (tests) may
|
|
56
|
+
// not plant a .git file; production git always does, and that is what we record.
|
|
57
|
+
function recordCreatedWorktree(worktree) {
|
|
58
|
+
const dotGit = path.join(worktree, ".git");
|
|
59
|
+
let st;
|
|
60
|
+
try { st = fs.lstatSync(dotGit); }
|
|
61
|
+
catch { return worktreePointerState(worktree); }
|
|
62
|
+
if (st.isSymbolicLink() || !st.isFile()) throw new Error(`worktree Git pointer integrity failure: ${st.isSymbolicLink() ? "symlink" : "not a regular file"} at creation`);
|
|
63
|
+
const sealed = sealWorktree(worktree, projectDir);
|
|
64
|
+
writeStatus(path.dirname(worktree), { worktree, projectDir, worktreePointerBefore: sealed });
|
|
65
|
+
return sealed;
|
|
66
|
+
}
|
|
67
|
+
function requireWorktreePointer(worktree, sealed) {
|
|
68
|
+
if (!sealed?.bytes) throw new Error("worktree Git pointer integrity failure: no recorded pointer bytes");
|
|
69
|
+
assertWorktreePointer({ worktree, expectedBytes: sealed.bytes, repoRoot: projectDir });
|
|
70
|
+
}
|
|
44
71
|
|
|
45
|
-
async function executeJob({ task, acceptance, verification, mode = "implement", baseRef, timeoutSeconds = 600, profile = "coder", reasoning = "high", pool = null, subscriptionWorker = null, onBehalfOf = null, model = null, reportSize = null, workerId, evidence = null, verifyRegression = false, commitSubject = null, refactor = false, continueFrom = null, stakes = null, reviews = null, jobId: presetJobId = null }) {
|
|
72
|
+
async function executeJob({ task, acceptance, criteria, verification, mode = "implement", baseRef, timeoutSeconds = 600, profile = "coder", reasoning = "high", pool = null, subscriptionWorker = null, onBehalfOf = null, model = null, reportSize = null, workerId, evidence = null, verifyRegression = false, commitSubject = null, refactor = false, continueFrom = null, stakes = null, trustAdmission = null, reviews = null, jobId: presetJobId = null }) {
|
|
46
73
|
await assertRepo();
|
|
47
74
|
// A continuation starts from the retained job's own base commit.
|
|
48
75
|
let continuation = null;
|
|
@@ -51,6 +78,7 @@ export function createExecutor(deps) {
|
|
|
51
78
|
if (checked.problem) throw new Error(checked.problem);
|
|
52
79
|
continuation = { jobId: continueFrom, record: checked.record };
|
|
53
80
|
baseRef = continuationBase(checked.record);
|
|
81
|
+
stakes = continuationStakes(checked.record.stakes, stakes);
|
|
54
82
|
}
|
|
55
83
|
ensureJobsRoot();
|
|
56
84
|
// Fire-and-forget: sweeps whatever this or any other nomArmy install left
|
|
@@ -61,14 +89,14 @@ export function createExecutor(deps) {
|
|
|
61
89
|
fs.mkdirSync(runtimeDir, { recursive: true });
|
|
62
90
|
const progress = (phase, extra = {}) => writeStatus(jobDir, {
|
|
63
91
|
jobId, workerId: workerId || jobId, mode, phase, state: phase === "finished" ? "finished" : "running",
|
|
64
|
-
serverPid: process.pid, baseSha: base.sha, timeoutSeconds, ...extra
|
|
92
|
+
projectDir, ...(stakes ? { stakes } : {}), serverPid: process.pid, baseSha: base.sha, timeoutSeconds, ...extra
|
|
65
93
|
});
|
|
66
94
|
progress("starting", { startedAt: new Date().toISOString(), agent: mode === "verify" ? null : pool ?? subscriptionWorker ?? "local", model: model ?? null });
|
|
67
95
|
const common = { task, acceptance, base, jobId, jobDir, runtimeDir, timeoutSeconds, profile, reasoning, pool, subscriptionWorker, onBehalfOf, model, reportSize, workerId, progress, jobStartedMs };
|
|
68
96
|
if (mode === "verify") return executeVerify({ ...common, verification });
|
|
69
97
|
if (mode === "scout") return executeScout({ ...common, reviews });
|
|
70
98
|
if (mode === "decompose") return executeDecompose(common);
|
|
71
|
-
return executeImplement({ ...common, verification, evidence, verifyRegression, commitSubject, refactor, continuation, stakes });
|
|
99
|
+
return executeImplement({ ...common, criteria, verification, evidence, verifyRegression, commitSubject, refactor, continuation, stakes, trustAdmission });
|
|
72
100
|
}
|
|
73
101
|
|
|
74
102
|
async function executeVerify({ task, verification: profile, base, jobId, jobDir, workerId, progress, jobStartedMs }) {
|
|
@@ -77,6 +105,7 @@ export function createExecutor(deps) {
|
|
|
77
105
|
try {
|
|
78
106
|
progress("worktree");
|
|
79
107
|
await run("git", ["worktree", "add", "--detach", worktree, base.sha], { cwd: projectDir });
|
|
108
|
+
recordCreatedWorktree(worktree);
|
|
80
109
|
record = await collectGitRecord({ cwd: worktree, baseSha: base.sha, baseRef: base.ref, branch: null, jobId });
|
|
81
110
|
progress("verification");
|
|
82
111
|
verification = await runIndependentVerification({ profile, cwd: worktree, jobId, baseSha: base.sha, branch: null, mode: "verify", record, logFile: path.join(jobDir, "verification.log") });
|
|
@@ -84,7 +113,7 @@ export function createExecutor(deps) {
|
|
|
84
113
|
error = err.message;
|
|
85
114
|
verification = normalizeVerification({ status: "not_run", reason: error }, profile);
|
|
86
115
|
} finally {
|
|
87
|
-
try { await run("git", ["worktree", "remove", "--force", worktree], { cwd: projectDir }); }
|
|
116
|
+
try { assertRegisteredWorktree(worktree); await run("git", ["worktree", "remove", "--force", worktree], { cwd: projectDir }); }
|
|
88
117
|
catch (err) { error = [error, err.message].filter(Boolean).join("\n"); }
|
|
89
118
|
}
|
|
90
119
|
const retained = fs.existsSync(worktree);
|
|
@@ -103,21 +132,33 @@ export function createExecutor(deps) {
|
|
|
103
132
|
return { ok: outcome === OUTCOMES.VERIFIED, manifest, jobDir, report: "" };
|
|
104
133
|
}
|
|
105
134
|
|
|
106
|
-
async function executeImplement({ task, acceptance, verification, base, jobId, jobDir, runtimeDir, timeoutSeconds, profile, reasoning, pool = null, subscriptionWorker = null, onBehalfOf = null, model = null, reportSize = null, workerId, evidence, verifyRegression = false, commitSubject = null, refactor = false, continuation = null, stakes = null, progress, jobStartedMs }) {
|
|
135
|
+
async function executeImplement({ task, acceptance, criteria, verification, base, jobId, jobDir, runtimeDir, timeoutSeconds, profile, reasoning, pool = null, subscriptionWorker = null, onBehalfOf = null, model = null, reportSize = null, workerId, evidence, verifyRegression = false, commitSubject = null, refactor = false, continuation = null, stakes = null, trustAdmission = null, progress, jobStartedMs }) {
|
|
107
136
|
const mode = "implement";
|
|
108
137
|
let branch = `agent/${jobId}`, worktree = path.join(jobDir, "worktree");
|
|
109
138
|
try {
|
|
110
139
|
progress("worktree");
|
|
111
140
|
await run("git", ["worktree", "add", "-b", branch, worktree, base.sha], { cwd: projectDir });
|
|
112
141
|
const cwd = worktree;
|
|
142
|
+
const beforePointer = recordCreatedWorktree(worktree), startedAt = new Date().toISOString();
|
|
143
|
+
// Snapshot the operator's floor before dispatch. Never read the worker's
|
|
144
|
+
// config to decide which trust rules or owners govern this job.
|
|
145
|
+
const trustConfig = loadConfig(projectDir)?.config?.trust;
|
|
146
|
+
const trustedOwners = trustConfig?.codeowners ? codeownersPaths(projectDir) : [];
|
|
147
|
+
const trustedFiles = snapshotTrustFiles(cwd);
|
|
148
|
+
const trustedMap = loadTrustMap(projectDir);
|
|
149
|
+
const trustedReach = trustedMap.length ? cachedTrustReach({ baseDir: cwd, baseCommit: base.sha,
|
|
150
|
+
stateDir: path.join(jobsRoot, ".."), entries: trustedMap }) : null;
|
|
113
151
|
// continue_from: lay the retained job's unfinished work into this
|
|
114
152
|
// worktree as uncommitted changes, so this job's diff (and so its
|
|
115
153
|
// verification and revert check) covers that work too.
|
|
116
154
|
let continuedFrom = null;
|
|
117
155
|
if (continuation) {
|
|
156
|
+
const retainedBytes = continuation.record.worktreePointerBefore?.bytes;
|
|
157
|
+
if (!retainedBytes) throw new Error("worktree Git pointer integrity failure: continuation has no recorded pointer bytes");
|
|
158
|
+
registerJobWorktree(continuation.record, { repoRoot: projectDir });
|
|
118
159
|
const git = async (args, opts = {}) => (await run("git", args, { trim: false, ...opts })).stdout;
|
|
119
|
-
const snapshot = await snapshotRetainedWork({ worktree: continuation.record.worktree, baseSha: base.sha, jobId: continuation.jobId, git });
|
|
120
|
-
await applyRetainedWork({ worktree, baseSha: base.sha, commit: snapshot.commit, git });
|
|
160
|
+
const snapshot = await snapshotRetainedWork({ worktree: continuation.record.worktree, baseSha: base.sha, jobId: continuation.jobId, git, repoRoot: projectDir, expectedPointer: retainedBytes });
|
|
161
|
+
await applyRetainedWork({ worktree, baseSha: base.sha, commit: snapshot.commit, git, repoRoot: projectDir, expectedPointer: beforePointer.bytes });
|
|
121
162
|
continuedFrom = { jobId: continuation.jobId, snapshot: snapshot.commit, files: snapshot.files };
|
|
122
163
|
evidence = [continuationNote({ continueFrom: continuation.jobId, record: continuation.record, files: snapshot.files }), evidence].filter(Boolean).join("\n\n");
|
|
123
164
|
fs.appendFileSync(path.join(jobDir, "coordinator.log"), `${new Date().toISOString()} continuing job ${continuation.jobId}: ${snapshot.files.length} file(s) of its unfinished work carried into this worktree (snapshot ${snapshot.commit})\n`);
|
|
@@ -127,7 +168,6 @@ export function createExecutor(deps) {
|
|
|
127
168
|
try { linkNodePackages(worktree, nodeConfig); } catch { /* verification reports what's missing */ }
|
|
128
169
|
let nodeModulesBefore = {};
|
|
129
170
|
try { nodeModulesBefore = nodeModulesState(worktree, nodeConfig); } catch { /* no Node packages */ }
|
|
130
|
-
const beforePointer = worktreePointerState(worktree), startedAt = new Date().toISOString();
|
|
131
171
|
|
|
132
172
|
// The caller's timeout is split up front into a work phase and a
|
|
133
173
|
// reserved report phase (see deriveTimeBudget) rather than letting the
|
|
@@ -257,16 +297,16 @@ export function createExecutor(deps) {
|
|
|
257
297
|
}
|
|
258
298
|
}
|
|
259
299
|
|
|
300
|
+
requireWorktreePointer(worktree, beforePointer);
|
|
260
301
|
const afterPointer = worktreePointerState(worktree);
|
|
261
|
-
if (!afterPointer.exists || afterPointer.kind !== "file") throw new Error(`worktree Git pointer integrity failure after worker: ${JSON.stringify(afterPointer)}`);
|
|
262
302
|
const preCommit = await collectGitRecord({ cwd, baseSha: base.sha, branch, baseRef: base.ref, jobId });
|
|
263
303
|
const repositoryChanged = preCommit.repoStatusFiles.length > 0;
|
|
264
304
|
|
|
265
305
|
// A worker whose tools ran outside the sandbox can leave a host-built
|
|
266
306
|
// node_modules behind; verification must not run against it.
|
|
267
307
|
let hostInstalls = [];
|
|
268
|
-
try { hostInstalls = repairHostInstalls(cwd, nodeConfig, nodeModulesBefore); } catch {
|
|
269
|
-
if (hostInstalls.length) fs.appendFileSync(path.join(jobDir, "coordinator.log"), `${new Date().toISOString()}
|
|
308
|
+
try { hostInstalls = repairHostInstalls(cwd, nodeConfig, nodeModulesBefore); } catch (error) { hostInstalls = [`repair refused: ${error.message}`]; }
|
|
309
|
+
if (hostInstalls.length) fs.appendFileSync(path.join(jobDir, "coordinator.log"), `${new Date().toISOString()} dependency repair findings: ${hostInstalls.join(", ")}; safe paths repaired and unsafe paths retained for review\n`);
|
|
270
310
|
|
|
271
311
|
progress("verification");
|
|
272
312
|
let independentVerification = normalizeVerification({ status: "not_run", basis: "not-applicable", reason: "no verification runner registered" }, verification ?? null);
|
|
@@ -282,11 +322,35 @@ export function createExecutor(deps) {
|
|
|
282
322
|
independentVerification = await runIndependentVerification({ profile: verification ?? null, cwd, jobId, baseSha: base.sha, branch, mode, record: preCommit, logFile: path.join(jobDir, "verification.log") });
|
|
283
323
|
}
|
|
284
324
|
|
|
325
|
+
let changedContracts = { weakened: [], issues: [], files: [] };
|
|
326
|
+
if (touchesAcceptance(preCommit.nameStatus ?? []) && workerStopReason !== "stopped") {
|
|
327
|
+
changedContracts = await changedContractWeakening({
|
|
328
|
+
worktree: cwd, baseSha: base.sha, nameStatus: preCommit.nameStatus, gitRaw,
|
|
329
|
+
});
|
|
330
|
+
const contractFiles = changedContracts.files;
|
|
331
|
+
const contractCheck = contractFiles.length ? await runIndependentVerification({
|
|
332
|
+
profile: "acceptance-contracts", acceptanceFiles: contractFiles,
|
|
333
|
+
cwd, jobId: `${jobId}-acceptance`, baseSha: base.sha, branch, mode,
|
|
334
|
+
record: preCommit, logFile: path.join(jobDir, "acceptance-check.log"),
|
|
335
|
+
}) : { status: changedContracts.issues.length ? "fail" : "pass", basis: "not-applicable",
|
|
336
|
+
reason: "no changed contracts in the new contract set" };
|
|
337
|
+
// Missing infrastructure is not permission to accept unchecked contracts.
|
|
338
|
+
// Keep both verdicts and the dedicated log in the job record.
|
|
339
|
+
independentVerification = {
|
|
340
|
+
...independentVerification,
|
|
341
|
+
status: contractCheck.status !== "pass" ? "fail"
|
|
342
|
+
: !verification && !planRegressionProductionFiles(preCommit.testChanges.production_files_changed).length
|
|
343
|
+
&& independentVerification.status === "not_run" ? "pass" : independentVerification.status,
|
|
344
|
+
...(contractCheck.status !== "pass" ? { reason: "changed acceptance contracts did not pass verification" } : {}),
|
|
345
|
+
contractCheck: { ...contractCheck, files: contractFiles, weakened: changedContracts.weakened },
|
|
346
|
+
};
|
|
347
|
+
}
|
|
348
|
+
|
|
285
349
|
// verify_regression: on by default whenever there's a verification
|
|
286
350
|
// profile (resolveVerifyRegression). It doubles verification wall-clock,
|
|
287
351
|
// so it runs only when there's something to re-check: a passing
|
|
288
352
|
// first-pass verification on a diff that touched production code.
|
|
289
|
-
// Documentation
|
|
353
|
+
// Documentation, CI configuration and acceptance contracts are not code,
|
|
290
354
|
// so neither is reverted or used as a reason to run this check.
|
|
291
355
|
const codeFilesChanged = planRegressionProductionFiles(preCommit.testChanges.production_files_changed);
|
|
292
356
|
let regressionCheck = null, regressionCheckFatal = false, regressionCheckElapsedMs = null;
|
|
@@ -322,13 +386,13 @@ export function createExecutor(deps) {
|
|
|
322
386
|
for (const file of codeFilesChanged.filter((f) => !deleted.has(f))) {
|
|
323
387
|
const full = path.join(cwd, file);
|
|
324
388
|
let lines;
|
|
325
|
-
if (untracked.has(file)) { try { lines =
|
|
389
|
+
if (untracked.has(file)) { try { lines = readWorktreeFile(cwd, full).content.toString("utf8").split("\n").map((_, i) => i + 1); } catch { continue; } }
|
|
326
390
|
else lines = parseAddedLineNumbers(await gitRaw(["diff", "-U0", base.sha, "--", file], cwd));
|
|
327
391
|
if (lines.length) files.push({ path: file, full, lines });
|
|
328
392
|
}
|
|
329
|
-
const mutants = pickMutants(files, mutationConfig.mutants);
|
|
393
|
+
const mutants = pickMutants(files, mutationConfig.mutants, cwd);
|
|
330
394
|
let n = 0;
|
|
331
|
-
mutation = await runMutants({ mutants, deadlineMs: mutationStartedMs + mutationConfig.max_seconds * 1000,
|
|
395
|
+
mutation = await runMutants({ mutants, root: cwd, deadlineMs: mutationStartedMs + mutationConfig.max_seconds * 1000,
|
|
332
396
|
verify: () => runIndependentVerification({ profile: verification, cwd, jobId: `${jobId}-mutant-${++n}`, baseSha: base.sha, branch, mode, record: preCommit }) });
|
|
333
397
|
mutation.planned = mutants.length;
|
|
334
398
|
} catch (error) {
|
|
@@ -338,6 +402,21 @@ export function createExecutor(deps) {
|
|
|
338
402
|
fs.appendFileSync(path.join(jobDir, "coordinator.log"), `${new Date().toISOString()} mutation testing: ${mutation.killed ?? 0} killed, ${mutation.survived?.length ?? 0} survived, ${mutation.inconclusive ?? 0} inconclusive of ${mutation.tried ?? 0} tried (${Math.round(mutationElapsedMs / 1000)}s)\n`);
|
|
339
403
|
}
|
|
340
404
|
|
|
405
|
+
// Operator contracts are authoritative even if the worker edits acceptance/.
|
|
406
|
+
// Reuse the registered sandbox runner, including its image, policy and budgets.
|
|
407
|
+
let contractCheck;
|
|
408
|
+
try {
|
|
409
|
+
contractCheck = await checkJobContracts({
|
|
410
|
+
projectDir, worktree: cwd, jobDir, jobId: `${jobId}-contract-impact`,
|
|
411
|
+
verify: runIndependentVerification,
|
|
412
|
+
changedFiles: [...new Set([...(preCommit.changedFiles ?? []),
|
|
413
|
+
...(preCommit.nameStatus ?? []).flatMap(entry => [entry.path, entry.oldPath].filter(Boolean))])],
|
|
414
|
+
});
|
|
415
|
+
} catch (error) {
|
|
416
|
+
contractCheck = { record: { affected: [], broken: [] }, criteria: [],
|
|
417
|
+
issues: [`CONTRACT CHECK ERROR: ${error.message}`] };
|
|
418
|
+
}
|
|
419
|
+
|
|
341
420
|
// resolveOutcome's own contract only ever sees pass/fail/not_run for
|
|
342
421
|
// regressionCheck -- a restore_failed status is substituted to not_run
|
|
343
422
|
// here so resolveOutcome never needs a fourth value; the hard override
|
|
@@ -425,7 +504,7 @@ export function createExecutor(deps) {
|
|
|
425
504
|
for (const c of (preCommit.nameStatus ?? []).slice(0, 300)) {
|
|
426
505
|
let addedLines = [];
|
|
427
506
|
if (c.status === "A") {
|
|
428
|
-
try { const text =
|
|
507
|
+
try { const text = readTrustWorktreeFile(cwd, c.path).bytes.toString("utf8"); if (text.length < 2_000_000) addedLines = text.split("\n"); } catch { /* unreadable: status alone still counts */ }
|
|
429
508
|
} else if (c.status !== "D") {
|
|
430
509
|
try { addedLines = addedLinesOf(await gitRaw(["diff", "-U0", base.sha, "--", c.path], cwd)); } catch { /* skip this file */ }
|
|
431
510
|
}
|
|
@@ -458,7 +537,9 @@ export function createExecutor(deps) {
|
|
|
458
537
|
} catch { /* best-effort; never blocks a commit on the scan's OWN failure -- the absence of a signal is not evidence of safety, but a hard block on a scanner crash would be a self-inflicted denial of service */ }
|
|
459
538
|
}
|
|
460
539
|
const afterHostInstalls = hostInstalls.length
|
|
461
|
-
? { ...afterMislabeledTests, reviewRequired: true, reasons: [...afterMislabeledTests.reasons,
|
|
540
|
+
? { ...afterMislabeledTests, reviewRequired: true, reasons: [...afterMislabeledTests.reasons, hostInstalls.some(entry => entry.includes("refused"))
|
|
541
|
+
? `UNSAFE DEPENDENCY PATHS: ${hostInstalls.join(", ")}. Unsafe paths were retained without following symlinks; inspect them before accepting the job.`
|
|
542
|
+
: `TOOLS OUTSIDE THE SANDBOX: the worker left a real ${hostInstalls.join(", ")}, so packages were installed where the sandbox (no network) couldn't have: its tool calls ran on this machine. nomArmy removed them and verified against the sandbox's own dependencies.`] }
|
|
462
543
|
: afterMislabeledTests;
|
|
463
544
|
const afterSecrets = possibleSecrets
|
|
464
545
|
? { ...afterHostInstalls, reviewRequired: true, commitAllowed: false,
|
|
@@ -472,12 +553,13 @@ export function createExecutor(deps) {
|
|
|
472
553
|
const jevImplement = jevSettingsFor("report-claims");
|
|
473
554
|
const judge = (() => { try { return deps.judgeSettings?.() ?? null; } catch { return null; } })();
|
|
474
555
|
// The whole change, new files included, for whichever validators run.
|
|
475
|
-
const jobDiff = async () => {
|
|
476
|
-
let diff = await gitRaw(["diff", base.sha, "--"], cwd);
|
|
556
|
+
const jobDiff = async (fullContext = false) => {
|
|
557
|
+
let diff = await gitRaw(["diff", ...(fullContext ? ["--unified=1000000"] : []), base.sha, "--"], cwd);
|
|
477
558
|
for (const entry of (preCommit.nameStatus ?? []).filter((e) => e.untracked)) {
|
|
478
559
|
let text = "";
|
|
479
|
-
try { text =
|
|
480
|
-
|
|
560
|
+
try { text = readTrustWorktreeFile(cwd, entry.path).bytes.toString("utf8"); }
|
|
561
|
+
catch (error) { if (trustConfig) throw error; continue; }
|
|
562
|
+
diff += `\ndiff --git a/${entry.path} b/${entry.path}\nnew file\n--- /dev/null\n+++ b/${entry.path}\n@@ -0,0 +1,${text.split("\n").length} @@\n${text.split("\n").map((l) => `+${l}`).join("\n")}\n`;
|
|
481
563
|
}
|
|
482
564
|
return diff;
|
|
483
565
|
};
|
|
@@ -496,11 +578,15 @@ export function createExecutor(deps) {
|
|
|
496
578
|
const modifiedTests = preCommit.testChanges?.existing_tests_modified ?? [];
|
|
497
579
|
let testDiff = "";
|
|
498
580
|
if (modifiedTests.length) { try { testDiff = await gitRaw(["diff", base.sha, "--", ...modifiedTests], cwd); } catch { testDiff = ""; } }
|
|
499
|
-
judged = await runJudge({ settings: judge, task, acceptance: acceptance ?? [], report: reportValidation, diff: diffText, testDiff, stateRoot: path.join(jobsRoot, "..") });
|
|
581
|
+
judged = await (deps.runJudge ?? runJudge)({ settings: judge, task, acceptance: acceptance ?? [], contracts: contractCheck.criteria, report: reportValidation, diff: diffText, testDiff, stateRoot: path.join(jobsRoot, "..") });
|
|
500
582
|
}
|
|
501
|
-
const
|
|
502
|
-
|
|
583
|
+
const contractIssues = [...contractCheck.issues, ...changedContracts.issues];
|
|
584
|
+
const afterContracts = contractIssues.length
|
|
585
|
+
? { ...afterSecrets, reviewRequired: true, reasons: [...afterSecrets.reasons, ...contractIssues] }
|
|
503
586
|
: afterSecrets;
|
|
587
|
+
const afterJevOnly = jevClaims?.flag
|
|
588
|
+
? { ...afterContracts, reviewRequired: true, reasons: [...afterContracts.reasons, `REPORT MAY NOT MATCH THE DIFF (Jev, ${jevClaims.flag.probability.toFixed(2)}): the report says "${String(reportValidation.note ?? "").slice(0, 200)}", and the diff may show otherwise. Read the diff before accepting.`] }
|
|
589
|
+
: afterContracts;
|
|
504
590
|
const afterJev = judged?.flags?.length
|
|
505
591
|
? { ...afterJevOnly, reviewRequired: true, reasons: [...afterJevOnly.reasons, `JUDGE (${judge.agent}/${judge.model}): ${judged.flags.join("; ")}. Read the diff before accepting.`] }
|
|
506
592
|
: afterJevOnly;
|
|
@@ -525,18 +611,62 @@ export function createExecutor(deps) {
|
|
|
525
611
|
: mutation?.status === "survivors"
|
|
526
612
|
? { ...afterConfig, reviewRequired: true, reasons: [...afterConfig.reasons, describeSurvivors(mutation, verification)] }
|
|
527
613
|
: afterConfig;
|
|
528
|
-
|
|
614
|
+
let trust = null;
|
|
615
|
+
{
|
|
616
|
+
const changedFiles = [...new Set([...(preCommit.changedFiles ?? []),
|
|
617
|
+
...(preCommit.nameStatus ?? []).flatMap((entry) => [entry.path, entry.oldPath].filter(Boolean))])];
|
|
618
|
+
// ls-tree distinguishes absent base paths from a failed blob read. Never
|
|
619
|
+
// suppress base cat-file failures or filter the trusted base bytes.
|
|
620
|
+
const baseModes = new Map((await gitRaw(["ls-tree", "-r", "-z", base.sha], projectDir)).split("\0").filter(Boolean).map((entry) => {
|
|
621
|
+
const tab = entry.indexOf("\t");
|
|
622
|
+
return [entry.slice(tab + 1), entry.slice(0, entry.indexOf(" "))];
|
|
623
|
+
}));
|
|
624
|
+
const fileChanges = [];
|
|
625
|
+
for (const file of changedFiles) {
|
|
626
|
+
const mode = baseModes.get(file);
|
|
627
|
+
const baseProblem = mode && !["100644", "100755", "120000"].includes(mode) ? UNCHECKABLE_TRUST_FILE : null;
|
|
628
|
+
const before = mode && !baseProblem
|
|
629
|
+
? (await run("git", ["cat-file", "blob", `${base.sha}:${file}`], { cwd: projectDir, encoding: null, trim: false })).stdout
|
|
630
|
+
: Buffer.alloc(0);
|
|
631
|
+
const { bytes: after, problem, regular } = readTrustWorktreeFile(cwd, file);
|
|
632
|
+
fileChanges.push({ file, before, after, problem: baseProblem ?? problem });
|
|
633
|
+
if (regular && !problem) {
|
|
634
|
+
try {
|
|
635
|
+
// Feed the safely read bytes, never reopen a worker path. Worktree
|
|
636
|
+
// attributes must apply here exactly as they will during git add.
|
|
637
|
+
const hashed = await run("git", ["hash-object", "-w", "--stdin", `--path=${file}`], { cwd, input: after });
|
|
638
|
+
const sha = hashed.stdout.trim();
|
|
639
|
+
if (!/^(?:[a-f0-9]{40}|[a-f0-9]{64})$/.test(sha)) throw new Error("invalid blob id");
|
|
640
|
+
const committed = (await run("git", ["cat-file", "blob", sha], { cwd: projectDir, encoding: null, trim: false })).stdout;
|
|
641
|
+
fileChanges.push({ file, before, after: committed });
|
|
642
|
+
} catch {
|
|
643
|
+
fileChanges.push({ file, before, after, problem: "could not be converted to committed bytes" });
|
|
644
|
+
}
|
|
645
|
+
}
|
|
646
|
+
}
|
|
647
|
+
// Unlike optional validators, failure to read this deterministic floor
|
|
648
|
+
// must not silently mark sensitive work normal.
|
|
649
|
+
const floor = evaluateTrust({ rules: trustConfig?.sensitive ?? [], changedFiles,
|
|
650
|
+
fileChanges, codeowners: trustedOwners });
|
|
651
|
+
trust = await evaluateDiffTrust({ floor, fileChanges, repository: trustedFiles, reach: trustedReach, tenantColumns: trustConfig?.tenant_columns, judgment: trustConfig?.judgment, previous: continuation?.record?.trust,
|
|
652
|
+
...trustValidatorSettings(deps), stateRoot: path.join(jobsRoot, ".."),
|
|
653
|
+
askJev: deps.askTrustJev, askJudge: deps.askTrustJudge });
|
|
654
|
+
}
|
|
655
|
+
if (["review", "human"].includes(trust?.level) || continuation?.record?.stakes === "high") stakes = "high";
|
|
656
|
+
const afterStakes = stakes === "high" || ["review", "human"].includes(trust?.level) ? { ...afterMutation, reviewRequired: true, reasons: [...afterMutation.reasons, HIGH_STAKES_NOTE] } : afterMutation;
|
|
529
657
|
const finalOutcome = applyRefactorContract(applyVerificationPolicy(afterStakes, independentVerification.status, repoPolicy()),
|
|
530
658
|
{ refactor, verificationStatus: independentVerification.status, testChanges: preCommit.testChanges });
|
|
531
659
|
|
|
660
|
+
const proposalDiff = criteria?.length && regressionCheck?.status === "pass" && finalOutcome.commitAllowed ? await jobDiff(true) : "";
|
|
532
661
|
progress("commit");
|
|
533
662
|
const commit = await createCoordinatorCommit({ cwd, jobId, outcome: finalOutcome,
|
|
534
663
|
message: coordinatorCommitMessage({ task: continuation?.record?.objective ?? task, subject: commitSubject, note: reportValidation?.note ?? null, jobId, workerId, recovered: Boolean(finalOutcome.recovered), provider: (result ?? attempted)?.provider ?? null, model: (result ?? attempted)?.model ?? null, continuedFrom: continuedFrom?.jobId ?? null }) });
|
|
664
|
+
const acceptanceProposals = await jobAcceptanceProposals({ criteria, commit, regressionCheck, diff: () => proposalDiff });
|
|
535
665
|
progress("record");
|
|
536
666
|
const record = await collectGitRecord({ cwd, baseSha: base.sha, branch, baseRef: base.ref, jobId }), worker = workerMetadata(result ?? attempted);
|
|
537
667
|
|
|
538
668
|
let coordinatorStatus = COORDINATOR_STATUS_BY_OUTCOME[finalOutcome.outcome] ?? "incomplete";
|
|
539
|
-
const issues = [...finalOutcome.reasons, ...(preCommit.issues ?? [])];
|
|
669
|
+
const issues = [...finalOutcome.reasons, ...(trustAdmission?.notes ?? []), ...(preCommit.issues ?? [])];
|
|
540
670
|
if (workerStopReason === "stopped") {
|
|
541
671
|
const request = readStopRequest(jobDir);
|
|
542
672
|
issues.push(`stopped on request${request?.reason ? `: ${request.reason}` : ""}; the worktree is kept, so continue_from can pick the work up (on another model too)`);
|
|
@@ -574,11 +704,16 @@ export function createExecutor(deps) {
|
|
|
574
704
|
: `report-recovery follow-up call did not produce a usable report either (${cause})`);
|
|
575
705
|
}
|
|
576
706
|
|
|
707
|
+
if (trust?.level === "human") issues.unshift(`HUMAN REVIEW REQUIRED (trust): ${trust.reasons.map((entry) => entry.reason).join("; ")}`);
|
|
708
|
+
|
|
577
709
|
const metrics = buildMetrics({ result: result ?? attempted, record, reportValidation, outcome: finalOutcome, workerElapsedMs, totalElapsedMs: Date.now() - jobStartedMs, regressionCheckElapsedMs, transientAbortRetried });
|
|
578
710
|
const manifest = { version: VERSION, jobId, workerId: workerId || jobId, mode, projectDir, worktree, branch, startedAt, finishedAt,
|
|
579
|
-
objective: task, acceptance: acceptance ?? [], verificationProfile: verification ?? null, ...(continuedFrom ? { continuedFrom } : {}), ...(stakes ? { stakes } : {}),
|
|
711
|
+
objective: task, acceptance: acceptance ?? [], ...(criteria ? { criteria } : {}), ...(acceptanceProposals !== undefined ? { acceptanceProposals } : {}), verificationProfile: verification ?? null, ...(continuedFrom ? { continuedFrom } : {}), ...(stakes ? { stakes } : {}),
|
|
712
|
+
...(trust ? { trust } : {}),
|
|
713
|
+
...(trustAdmission ? { trustAdmission } : {}),
|
|
580
714
|
...(mutation ? { mutation: { ...mutation, elapsedSeconds: Math.round((mutationElapsedMs ?? 0) / 1000) } } : {}),
|
|
581
715
|
...(jevClaims || judged ? { validators: { ...(jevClaims ? { jev: { check: "report-claims", verdict: jevClaims.verdict, flagged: Boolean(jevClaims.flag), error: jevClaims.error, truncated: Boolean(jevClaims.truncated), inputTokens: jevClaims.usage } } : {}), ...(judged ? { judge: { agent: judge?.agent ?? null, provider: judge?.provider ?? null, model: judge?.model ?? null, answer: judged.answer, flags: judged.flags, error: judged.error } } : {}) } } : {}),
|
|
716
|
+
...(contractCheck.record.affected.length || contractCheck.record.bounded ? { contract: contractCheck.record } : {}),
|
|
582
717
|
outcome: finalOutcome.outcome, recovered: finalOutcome.recovered, recoveryAttempted: finalOutcome.recoveryAttempted,
|
|
583
718
|
reportRecoveryAttempted, reportRecovered,
|
|
584
719
|
reviewRequired: finalOutcome.reviewRequired || record.testChanges.reviewRequired,
|
|
@@ -608,8 +743,8 @@ export function createExecutor(deps) {
|
|
|
608
743
|
progress("finished", { coordinatorStatus, outcome: finalOutcome.outcome });
|
|
609
744
|
return { ok: coordinatorStatus === "complete", report: report || "(worker returned no final report)", manifest, jobDir };
|
|
610
745
|
} catch (error) {
|
|
611
|
-
const failure = { version: VERSION, jobId, workerId: workerId || jobId, mode, branch, worktree, outcome: OUTCOMES.WORKER_FAILED,
|
|
612
|
-
coordinatorStatus: "failed", error: error.stack || error.message, retained: Boolean(worktree), worktreeRetained: Boolean(worktree), execution };
|
|
746
|
+
const failure = { version: VERSION, jobId, workerId: workerId || jobId, mode, branch, worktree, ...(stakes ? { stakes } : {}), outcome: OUTCOMES.WORKER_FAILED,
|
|
747
|
+
coordinatorStatus: "failed", ...(criteria ? { criteria } : {}), error: error.stack || error.message, retained: Boolean(worktree), worktreeRetained: Boolean(worktree), execution };
|
|
613
748
|
fs.writeFileSync(path.join(jobDir, "failure.json"), JSON.stringify(failure, null, 2));
|
|
614
749
|
progress("finished", { coordinatorStatus: "failed", outcome: OUTCOMES.WORKER_FAILED });
|
|
615
750
|
return { ok: false, report: `LOCAL WORKER FAILED:\n${error.stack || error.message}`, manifest: failure, jobDir };
|
|
@@ -627,6 +762,7 @@ export function createExecutor(deps) {
|
|
|
627
762
|
try {
|
|
628
763
|
progress("worktree");
|
|
629
764
|
await run("git", ["worktree", "add", "--detach", worktree, base.sha], { cwd: projectDir });
|
|
765
|
+
const sealedPointer = recordCreatedWorktree(worktree);
|
|
630
766
|
const startedAt = new Date().toISOString();
|
|
631
767
|
|
|
632
768
|
// Place the deterministic evidence CLI where the sandbox can run it. It
|
|
@@ -712,6 +848,7 @@ export function createExecutor(deps) {
|
|
|
712
848
|
}
|
|
713
849
|
}
|
|
714
850
|
|
|
851
|
+
requireWorktreePointer(worktree, sealedPointer);
|
|
715
852
|
const record = await collectGitRecord({ cwd: worktree, baseSha: base.sha, branch: null, baseRef: base.ref, jobId });
|
|
716
853
|
const dirty = record.repoStatusFiles.length > 0;
|
|
717
854
|
const readFile = async p => { try { return await gitRaw(["show", `${base.sha}:${p}`], projectDir); } catch { return null; } };
|
|
@@ -729,7 +866,10 @@ export function createExecutor(deps) {
|
|
|
729
866
|
|
|
730
867
|
progress("record");
|
|
731
868
|
if (outcome.retainWorktree) worktreeRetained = true;
|
|
732
|
-
else
|
|
869
|
+
else {
|
|
870
|
+
try { assertRegisteredWorktree(worktree); await run("git", ["worktree", "remove", "--force", worktree], { cwd: projectDir }); }
|
|
871
|
+
catch (error) { worktreeRetained = fs.existsSync(worktree); outcome.reasons.push(error.message); outcome.reviewRequired = true; }
|
|
872
|
+
}
|
|
733
873
|
|
|
734
874
|
const worker = workerMetadata(result ?? attempted);
|
|
735
875
|
const issues = [...outcome.reasons];
|
|
@@ -802,6 +942,16 @@ export function createExecutor(deps) {
|
|
|
802
942
|
// reasoningApplied is what the field previously conflated it with.
|
|
803
943
|
requestedProfile: profile, requestedReasoning: reasoning, reasoningApplied: resolveReasoningApplied({ result, profile, reasoning, workerModelThinkingSupported }), execution };
|
|
804
944
|
fs.writeFileSync(path.join(jobDir, "metadata.json"), JSON.stringify(manifest, null, 2));
|
|
945
|
+
if (reviews) {
|
|
946
|
+
try {
|
|
947
|
+
const job = JSON.parse(fs.readFileSync(path.join(jobsRoot, reviews, "metadata.json"), "utf8"));
|
|
948
|
+
recordReviewEvidence({ stateDir: path.dirname(jobsRoot), operatorDir: projectDir, review: manifest, job });
|
|
949
|
+
} catch (error) {
|
|
950
|
+
// The finished review remains available; trust review retries collection.
|
|
951
|
+
manifest.issues.push(`Trust learning evidence not recorded: ${error.message}`);
|
|
952
|
+
fs.writeFileSync(path.join(jobDir, "metadata.json"), JSON.stringify(manifest, null, 2));
|
|
953
|
+
}
|
|
954
|
+
}
|
|
805
955
|
if (result) fs.writeFileSync(path.join(jobDir, "result.json"), JSON.stringify(result, null, 2));
|
|
806
956
|
progress("finished", { coordinatorStatus: outcome.coordinatorStatus, outcome: outcome.outcome });
|
|
807
957
|
return { ok: outcome.coordinatorStatus === "complete", report: rendered, manifest, jobDir };
|
|
@@ -832,6 +982,7 @@ export function createExecutor(deps) {
|
|
|
832
982
|
try {
|
|
833
983
|
progress("worktree");
|
|
834
984
|
await run("git", ["worktree", "add", "--detach", worktree, base.sha], { cwd: projectDir });
|
|
985
|
+
const sealedPointer = recordCreatedWorktree(worktree);
|
|
835
986
|
const startedAt = new Date().toISOString();
|
|
836
987
|
|
|
837
988
|
const evidenceTool = ".openclaw/nomarmy-evidence.mjs";
|
|
@@ -859,6 +1010,7 @@ export function createExecutor(deps) {
|
|
|
859
1010
|
progress("verification");
|
|
860
1011
|
const used = result?.budgetsUsed ?? budgetState.budgets; // see executeScout: the job's own budget, not the local one
|
|
861
1012
|
const report = parseDecomposeReport(reportText, used.decompose);
|
|
1013
|
+
requireWorktreePointer(worktree, sealedPointer);
|
|
862
1014
|
const record = await collectGitRecord({ cwd: worktree, baseSha: base.sha, branch: null, baseRef: base.ref, jobId });
|
|
863
1015
|
const dirty = record.repoStatusFiles.length > 0;
|
|
864
1016
|
const readFile = async p => { try { return await gitRaw(["show", `${base.sha}:${p}`], projectDir); } catch { return null; } };
|
|
@@ -868,7 +1020,10 @@ export function createExecutor(deps) {
|
|
|
868
1020
|
|
|
869
1021
|
progress("record");
|
|
870
1022
|
if (outcome.retainWorktree) worktreeRetained = true;
|
|
871
|
-
else
|
|
1023
|
+
else {
|
|
1024
|
+
try { assertRegisteredWorktree(worktree); await run("git", ["worktree", "remove", "--force", worktree], { cwd: projectDir }); }
|
|
1025
|
+
catch (error) { worktreeRetained = fs.existsSync(worktree); outcome.reasons.push(error.message); outcome.reviewRequired = true; }
|
|
1026
|
+
}
|
|
872
1027
|
|
|
873
1028
|
const worker = workerMetadata(result ?? attempted);
|
|
874
1029
|
const issues = [...outcome.reasons];
|
package/lib/git-record.mjs
CHANGED
|
@@ -94,8 +94,15 @@ export function wrapText(text, width) {
|
|
|
94
94
|
|
|
95
95
|
export function worktreePointerState(worktree) {
|
|
96
96
|
if (!worktree) return { applicable: false, exists: null, kind: null };
|
|
97
|
-
const dotGit = path.join(worktree, ".git");
|
|
98
|
-
|
|
97
|
+
const dotGit = path.join(worktree, ".git");
|
|
98
|
+
let stat;
|
|
99
|
+
try { stat = fs.lstatSync(dotGit); }
|
|
100
|
+
catch (error) {
|
|
101
|
+
if (error.code === "ENOENT") return { applicable: true, exists: false, kind: "missing" };
|
|
102
|
+
return { applicable: true, exists: false, kind: "unreadable" };
|
|
103
|
+
}
|
|
104
|
+
const kind = stat.isSymbolicLink() ? "symlink" : stat.isFile() ? "file" : stat.isDirectory() ? "directory" : "other";
|
|
105
|
+
return { applicable: true, exists: true, kind };
|
|
99
106
|
}
|
|
100
107
|
|
|
101
108
|
export function createGitRecord({ run, git, gitRaw }) {
|
package/lib/health.mjs
CHANGED
|
@@ -27,6 +27,7 @@ import { executionMode } from "./execution.mjs";
|
|
|
27
27
|
import { modelRejection } from "./openclaw-errors.mjs";
|
|
28
28
|
import { providerConfigured, readOpenclawConfig } from "./openclaw-config.mjs";
|
|
29
29
|
import { readUsageSnapshots, usageStatus, usageDisplayText, usageReadingIsStale, refreshStaleOverLimitReadings } from "./usage-limits.mjs";
|
|
30
|
+
import { PORTABLE_SERVER } from "./connect.mjs";
|
|
30
31
|
import { freshnessIssues, readInstallVersions } from "./install-freshness.mjs";
|
|
31
32
|
|
|
32
33
|
import { PINNED_OPENCLAW_VERSION } from "./openclaw-install.mjs";
|
|
@@ -280,12 +281,33 @@ export function leftoverIssues({ retainedWorktrees = 0, jobsBytes = 0, staleRunn
|
|
|
280
281
|
return issues;
|
|
281
282
|
}
|
|
282
283
|
|
|
284
|
+
/** Check repository registrations against the same launch command connect writes. */
|
|
285
|
+
export function projectRegistrationIssues(projectDir) {
|
|
286
|
+
if (!projectDir) return [];
|
|
287
|
+
const issues = [];
|
|
288
|
+
// Codex connect supports user scope only, so it has no project equivalent.
|
|
289
|
+
for (const [target, relative] of [["claude", ".mcp.json"], ["cursor", ".cursor/mcp.json"]]) {
|
|
290
|
+
const file = path.join(projectDir, relative);
|
|
291
|
+
let entry;
|
|
292
|
+
try { entry = JSON.parse(fs.readFileSync(file, "utf8"))?.mcpServers?.["nomarmy-local-worker"]; }
|
|
293
|
+
catch (error) { if (error.code === "ENOENT") continue; entry = {}; }
|
|
294
|
+
if (entry === undefined) continue;
|
|
295
|
+
if (entry?.command === PORTABLE_SERVER.command && Array.isArray(entry.args)
|
|
296
|
+
&& entry.args.length === PORTABLE_SERVER.args.length && entry.args.every((arg, i) => arg === PORTABLE_SERVER.args[i])) continue;
|
|
297
|
+
issues.push({ id: `project-mcp:${target}`, severity: "warn",
|
|
298
|
+
title: `Unexpected project MCP registration in ${relative}`,
|
|
299
|
+
detail: `${file} is executable repository configuration; its command and args do not match what nomarmy connect writes today.`,
|
|
300
|
+
fix: `nomarmy connect ${target} --scope project (rewrite it), or review ${file} by hand`, short: "project MCP changed" });
|
|
301
|
+
}
|
|
302
|
+
return issues;
|
|
303
|
+
}
|
|
304
|
+
|
|
283
305
|
/**
|
|
284
306
|
* Run every check. `env` supplies what each needs, with real defaults;
|
|
285
307
|
* tests pass their own.
|
|
286
308
|
*/
|
|
287
|
-
export async function runHealthChecks({ now = Date.now(), mode = "local", openclawCmd = process.env.NOMARMY_OPENCLAW_CMD || "openclaw", run = runBounded, armySummary = null, agentsError = null, jobsRoot = null, pidAlive = () => true, agents = null, openclawConfig = null, vendors = {}, modelsInUse = null, autoPruned = null, usageSnapshots = null, stateRoot = null, install = null } = {}) {
|
|
288
|
-
const issues =
|
|
309
|
+
export async function runHealthChecks({ now = Date.now(), mode = "local", openclawCmd = process.env.NOMARMY_OPENCLAW_CMD || "openclaw", run = runBounded, armySummary = null, agentsError = null, jobsRoot = null, pidAlive = () => true, agents = null, openclawConfig = null, vendors = {}, modelsInUse = null, autoPruned = null, usageSnapshots = null, stateRoot = null, install = null, projectDir = null } = {}) {
|
|
310
|
+
const issues = projectRegistrationIssues(projectDir);
|
|
289
311
|
if (autoPruned?.freedBytes) issues.push({ id: `auto-prune:${new Date(now).toISOString()}`, severity: "info",
|
|
290
312
|
title: `Freed ${(autoPruned.freedBytes / 1024 ** 3).toFixed(2)} GB: ${[autoPruned.pruned ? `runtime data of ${autoPruned.pruned} finished job${autoPruned.pruned === 1 ? "" : "s"} older than ${autoPruned.olderThanHours}h` : null, autoPruned.scratchCleared ? `OpenClaw scratch files of ${autoPruned.scratchCleared} more` : null].filter(Boolean).join(", ")}`,
|
|
291
313
|
detail: "Automatic; each job's record and report are kept. NOMARMY_AUTO_PRUNE_HOURS sets the age (0 turns it off).", fix: null, short: null });
|
|
@@ -413,7 +435,7 @@ export async function checkAndRecordHealth({ projectDir, stateRoot, configDir, n
|
|
|
413
435
|
if (ageMs !== null) { try { autoPruned = { ...pruneJobRuntime({ stateRoot, olderThanMs: ageMs, now }), olderThanHours: ageMs / 3600000 }; } catch { /* best-effort */ } }
|
|
414
436
|
const mode = executionMode(env).mode;
|
|
415
437
|
const install = await readRegisteredInstall({ projectDir });
|
|
416
|
-
const result = await runHealthChecks({ now, mode, armySummary, agentsError, jobsRoot: path.join(stateRoot, "jobs"), pidAlive,
|
|
438
|
+
const result = await runHealthChecks({ now, mode, projectDir, armySummary, agentsError, jobsRoot: path.join(stateRoot, "jobs"), pidAlive,
|
|
417
439
|
agents, openclawConfig: readOpenclawConfig(), vendors: SUBSCRIPTION_VENDORS, modelsInUse, autoPruned, usageSnapshots: readUsageSnapshots(stateRoot), stateRoot, install });
|
|
418
440
|
try {
|
|
419
441
|
const { loadJobRecords, agentLookup } = await import("./stats.mjs");
|