@gethmy/harness 1.6.0 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/cli.js +376 -219
- package/dist/index.js +293 -146
- package/package.json +2 -2
- package/src/cli.ts +34 -2
- package/src/exec-types.ts +33 -4
- package/src/git-pr.ts +53 -5
- package/src/oracle-collector.ts +38 -8
- package/src/oracle.ts +464 -53
- package/src/repair-sandbox.test.ts +166 -1
- package/src/repair-sandbox.ts +203 -8
- package/src/run-containment.ts +191 -37
- package/src/stage-cli.ts +32 -8
- package/src/verification.ts +200 -21
- package/src/worktree.ts +19 -2
package/src/run-containment.ts
CHANGED
|
@@ -78,44 +78,85 @@
|
|
|
78
78
|
* was found by a review round rather than by reading the code, and a residual
|
|
79
79
|
* nobody wrote down is indistinguishable from one nobody saw.
|
|
80
80
|
*
|
|
81
|
-
* - **
|
|
82
|
-
* the four verification steps AND the `build_green` gate
|
|
83
|
-
* re-runs the same two scripts: `runBuild`, `runTests`,
|
|
84
|
-
*
|
|
85
|
-
* `verification.sandboxImage` is set — worktree the only mount, no
|
|
86
|
-
* no host fallback.
|
|
87
|
-
*
|
|
88
|
-
*
|
|
89
|
-
*
|
|
90
|
-
*
|
|
91
|
-
*
|
|
92
|
-
*
|
|
93
|
-
*
|
|
94
|
-
*
|
|
95
|
-
*
|
|
96
|
-
*
|
|
97
|
-
*
|
|
98
|
-
*
|
|
99
|
-
*
|
|
100
|
-
*
|
|
101
|
-
*
|
|
102
|
-
*
|
|
103
|
-
*
|
|
104
|
-
*
|
|
105
|
-
*
|
|
106
|
-
*
|
|
107
|
-
*
|
|
108
|
-
*
|
|
81
|
+
* - **Egress from the two `dev` servers, plus the standalone harness CLI.**
|
|
82
|
+
* #1036 closed the four verification steps AND the `build_green` gate
|
|
83
|
+
* collector, which re-runs the same two scripts: `runBuild`, `runTests`,
|
|
84
|
+
* `runFormatFix` and `runLint` run inside `repair-sandbox.ts`'s container
|
|
85
|
+
* when `verification.sandboxImage` is set — worktree the only mount, no
|
|
86
|
+
* network, no host fallback. #1037 took the same key and moved the two dev
|
|
87
|
+
* servers in behind it: `runDeepReview`'s and the review worker's, both
|
|
88
|
+
* through one `devServerLaunch`, so they can no longer be fixed one at a
|
|
89
|
+
* time the way #988 fixed one and shipped the other.
|
|
90
|
+
*
|
|
91
|
+
* **What is left of this bullet is egress, and only egress.** A dev server
|
|
92
|
+
* exists to be connected to, so `--network=none` is not available to it: the
|
|
93
|
+
* container publishes `127.0.0.1:<port>` on a default bridge, which gives it
|
|
94
|
+
* a route out. That is a strictly weaker property than the four verification
|
|
95
|
+
* steps get, and it is named here rather than left to be inferred from a
|
|
96
|
+
* missing flag. Two shapes that would have kept both halves were measured on
|
|
97
|
+
* the daemon's own platform and neither works — `--internal` drops the
|
|
98
|
+
* published port along with the egress, and a bridge with
|
|
99
|
+
* `enable_ip_masquerade=false` keeps the egress and additionally reaches the
|
|
100
|
+
* operator's own host services. `devServerSandboxArgs` carries the numbers.
|
|
101
|
+
*
|
|
102
|
+
* What the container still takes is the part that made the command worth
|
|
103
|
+
* attacking: the run's `dev` script no longer executes in the daemon
|
|
104
|
+
* process, no longer reads `~/.ssh` or `~/.harmony-mcp/config.json`, and no
|
|
105
|
+
* longer holds a capability. What egress can still carry out is the
|
|
106
|
+
* worktree, which is the run's own output.
|
|
107
|
+
*
|
|
108
|
+
* The containment is **opt-in**, so on a host with no `sandboxImage` all six
|
|
109
|
+
* commands still run in the daemon process, with `containedEnv()` as the
|
|
110
|
+
* only bound. That is a deliberate asymmetry with this module's own
|
|
111
|
+
* containment, which defaults on: the SDK sandbox is inside the CLI, whereas
|
|
112
|
+
* a container needs a reachable Docker and an image carrying the repo's
|
|
113
|
+
* toolchain.
|
|
114
|
+
*
|
|
115
|
+
* One correction to the severity this was first written with. The dev
|
|
116
|
+
* servers were called the rare path because `verification.deepReview`
|
|
117
|
+
* defaults to false. That is true of `runDeepReview` and **false of the
|
|
118
|
+
* review worker's**, which has no such gate and no toolchain check:
|
|
119
|
+
* `review.enabled` defaults to true, so that server starts on every card
|
|
120
|
+
* that reaches Review. Of the two, the one on the default path was the one
|
|
121
|
+
* described as conditional.
|
|
122
|
+
*
|
|
123
|
+
* `packages/harmony-harness/src/cli.ts` used to build the same collector and
|
|
124
|
+
* pass no image, because the motor held no operator config of its own. #1021
|
|
125
|
+
* gave it `--sandbox-image`, threaded from the daemon's same
|
|
126
|
+
* `verification.sandboxImage`, so a daemon-driven stage now contains the
|
|
127
|
+
* `build_green` gate and the HELD TEST alike. A person running the CLI by
|
|
128
|
+
* hand against their own checkout still passes no image and still runs on
|
|
129
|
+
* their host — a different threat model, and named here rather than left to
|
|
130
|
+
* be rediscovered.
|
|
131
|
+
*
|
|
132
|
+
* The count is in this bullet on purpose, and it has now been wrong three
|
|
133
|
+
* times. The first version named the four verification steps and stopped, and
|
|
134
|
+
* both dev servers then shipped with the daemon's whole environment while the
|
|
109
135
|
* changelog told users these commands run "without your credentials". The
|
|
110
136
|
* second version — #1036's own — said "the two dev servers" while the
|
|
111
137
|
* `build_green` gate ran the worktree's script on the host for any operator
|
|
112
|
-
* who had set an image precisely to stop that.
|
|
113
|
-
*
|
|
114
|
-
* `
|
|
115
|
-
*
|
|
116
|
-
*
|
|
117
|
-
*
|
|
118
|
-
*
|
|
138
|
+
* who had set an image precisely to stop that. The third omission was not a
|
|
139
|
+
* missing NAME but a wrong SPELLING, which is worse: `review-worker.ts`'s dev
|
|
140
|
+
* server carried `env: containedEnv()` on a `spawnInGroup` call, and
|
|
141
|
+
* `spawnInGroup` merges `process.env` back over whatever `env` it is handed —
|
|
142
|
+
* so that strip removed nothing for as long as it shipped, and the source
|
|
143
|
+
* scan reported the property as held because the scan read the spelling.
|
|
144
|
+
* Every one of the three was caught by a reader, not by a gate.
|
|
145
|
+
*
|
|
146
|
+
* A fourth was caught by the gate, which is the point of having one. #1037
|
|
147
|
+
* consolidated the two dev servers behind `devServerLaunch` — the right move,
|
|
148
|
+
* and it instantly took `review-worker.ts` OUT of the scan, because the file
|
|
149
|
+
* stopped containing the word `spawnRunArgs` and that list was how the walk
|
|
150
|
+
* chose which files to open. The property assertion would have gone on
|
|
151
|
+
* passing over the very file whose inert strip had just been fixed. The scan
|
|
152
|
+
* now names both dev-server files explicitly, so a file leaving the walk
|
|
153
|
+
* fails instead of quietly shrinking the count.
|
|
154
|
+
*
|
|
155
|
+
* `spawn-run-containment.test.ts` is therefore spawn-function aware, covers
|
|
156
|
+
* the held-test spawn, and checks its own coverage; and
|
|
157
|
+
* `verification-sandbox.test.ts` asserts that a configured image takes the
|
|
158
|
+
* four out of this process entirely — because the residual is a completeness
|
|
159
|
+
* claim over CALL SITES and a hand-list has already lost it twice.
|
|
119
160
|
* - **The MCP surface.** The sandbox governs commands. MCP stdio servers are
|
|
120
161
|
* hosted by the CLI and are not sandboxed, so `mcp__harmony__*` is bounded by
|
|
121
162
|
* the tool allow-list and the daemon-owned denials (#525/#576), not by this.
|
|
@@ -598,6 +639,53 @@ export function containedEnv(
|
|
|
598
639
|
return out;
|
|
599
640
|
}
|
|
600
641
|
|
|
642
|
+
/**
|
|
643
|
+
* The model credentials {@link containedEnv} deliberately KEEPS, named so a
|
|
644
|
+
* spawn that is not a model call can drop them.
|
|
645
|
+
*
|
|
646
|
+
* They are in {@link KEEP_ENV_KEYS} because an implement run IS the model call
|
|
647
|
+
* — stripping them stops the run existing rather than containing it. Nothing
|
|
648
|
+
* generalizes from that: it is a statement about one spawn.
|
|
649
|
+
*/
|
|
650
|
+
const MODEL_CREDENTIAL_KEYS: readonly string[] = [
|
|
651
|
+
"ANTHROPIC_API_KEY",
|
|
652
|
+
"ANTHROPIC_AUTH_TOKEN",
|
|
653
|
+
"CLAUDE_CODE_OAUTH_TOKEN",
|
|
654
|
+
];
|
|
655
|
+
|
|
656
|
+
/**
|
|
657
|
+
* Environment keys the HELD-TEST run must not carry (#1021): everything
|
|
658
|
+
* {@link secretEnvKeysToStrip} names, plus the model credentials.
|
|
659
|
+
*
|
|
660
|
+
* A held test is arbitrary code an untrusted `author` subagent wrote, and
|
|
661
|
+
* before this the oracle spawn passed no `env` at all — so it inherited the
|
|
662
|
+
* daemon's entire environment, `HARMONY_API_KEY` and `ANTHROPIC_API_KEY`
|
|
663
|
+
* included. The first class is `secretEnvKeysToStrip`'s. The second is kept by
|
|
664
|
+
* `containedEnv` on purpose, which is right for the spawn that one was written
|
|
665
|
+
* for and wrong here: a test file has no model call to make, so a live billing
|
|
666
|
+
* credential in its environment is spend and reach it has no use for.
|
|
667
|
+
*
|
|
668
|
+
* **KEY NAMES, not an environment, and that is the whole point.** The oracle
|
|
669
|
+
* spawns through `spawnInGroup`, which MERGES `process.env` back on top of any
|
|
670
|
+
* `env` it is handed — so passing a narrowed object there strips nothing. The
|
|
671
|
+
* removal has to be a `stripEnvKeys` DELETE. `sdk-agent-runner.ts` records the
|
|
672
|
+
* same trap for the same reason; `review-worker.ts` shipped the inert spelling.
|
|
673
|
+
*
|
|
674
|
+
* This is a REDUCTION, not the boundary — the same thing #988 said about
|
|
675
|
+
* `containedEnv()` and for the same reason: the command still runs as the
|
|
676
|
+
* operator and can read `~/.ssh` and `~/.harmony-mcp/config.json` off disk. The
|
|
677
|
+
* boundary is the container, which `oracle.ts` uses when
|
|
678
|
+
* `verification.sandboxImage` is configured. This is what the DEFAULT path gets,
|
|
679
|
+
* and the default path is the one most operators are on.
|
|
680
|
+
*/
|
|
681
|
+
export function heldTestEnvKeysToStrip(
|
|
682
|
+
parentEnv: Record<string, string | undefined> = process.env,
|
|
683
|
+
): string[] {
|
|
684
|
+
return [
|
|
685
|
+
...new Set([...secretEnvKeysToStrip(parentEnv), ...MODEL_CREDENTIAL_KEYS]),
|
|
686
|
+
];
|
|
687
|
+
}
|
|
688
|
+
|
|
601
689
|
/**
|
|
602
690
|
* Git metadata the contained run may not write — the escape the worktree grant
|
|
603
691
|
* would otherwise hand it.
|
|
@@ -1048,8 +1136,36 @@ export const IMPLEMENT_ALLOWED_DOMAINS: readonly string[] = [
|
|
|
1048
1136
|
* lockstep with its own release, and a contained run reaching for `@latest`
|
|
1049
1137
|
* would fetch a version this daemon was never tested against — over a network
|
|
1050
1138
|
* the containment then has to allow the registry for.
|
|
1139
|
+
*
|
|
1140
|
+
* ## `run` — how the agent inside the run learns its own session (#1035)
|
|
1141
|
+
*
|
|
1142
|
+
* A daemon-run agent cannot call `harmony_start_agent_session`: the daemon owns
|
|
1143
|
+
* the lifecycle and denies the tool (`STAGE_DAEMON_OWNED_TOOLS`). So the MCP
|
|
1144
|
+
* process it talks to used to know nothing about the run it serves, and
|
|
1145
|
+
* `harmony_add_comment` had to leave the attribution to the API — which could
|
|
1146
|
+
* only ask "does this ACCOUNT have a session on this card?", true of every
|
|
1147
|
+
* concurrent run of the same account. That is the borrowed attribution measured
|
|
1148
|
+
* on card #1029.
|
|
1149
|
+
*
|
|
1150
|
+
* Passing `run` closes it at the source: the run's `card_agent_context` id
|
|
1151
|
+
* reaches the MCP server as its own environment, so the comment names the
|
|
1152
|
+
* session instead of the server inferring one.
|
|
1153
|
+
*
|
|
1154
|
+
* **The `env` field, not the spawn environment, and that is the point.** The
|
|
1155
|
+
* CLI hosts a stdio MCP server with `{...its own env, ...this env}` — measured
|
|
1156
|
+
* 2026-09-05 against `claude 2.1.261` with a probe server that dumped its
|
|
1157
|
+
* environment: both halves arrive, and `PATH`/`HOME` survive. Putting the id
|
|
1158
|
+
* here rather than on the run's own spawn keeps it out of the model's shell, so
|
|
1159
|
+
* the agent cannot read it, echo it, or pass a different one.
|
|
1160
|
+
*
|
|
1161
|
+
* An OLDER `@gethmy/mcp` — one published before #1035 — simply ignores two
|
|
1162
|
+
* unknown environment keys, so this is safe to ship ahead of the pin bump. A
|
|
1163
|
+
* CLI FLAG would not be: `cli.js serve` parses with commander, which exits 1 on
|
|
1164
|
+
* an unknown option, and the run would lose `mcp__harmony__*` entirely.
|
|
1051
1165
|
*/
|
|
1052
|
-
export function harmonyMcpServer(
|
|
1166
|
+
export function harmonyMcpServer(
|
|
1167
|
+
run?: HarmonyMcpRunBinding,
|
|
1168
|
+
): Record<string, McpServerConfig> {
|
|
1053
1169
|
const require = createRequire(import.meta.url);
|
|
1054
1170
|
// The package's `exports` map publishes "." → dist/index.js and no bin
|
|
1055
1171
|
// subpath, so the CLI is named as its sibling rather than resolved directly.
|
|
@@ -1061,10 +1177,37 @@ export function harmonyMcpServer(): Record<string, McpServerConfig> {
|
|
|
1061
1177
|
// and no chance of a different Node than the one that was tested.
|
|
1062
1178
|
command: process.execPath,
|
|
1063
1179
|
args: [cli, "serve"],
|
|
1180
|
+
// Both keys or neither: the card id is what lets the MCP server tell
|
|
1181
|
+
// "this comment is on the card I am running" from "this comment is on a
|
|
1182
|
+
// neighbour", and without it the session id would be claimed on every
|
|
1183
|
+
// card the run comments on.
|
|
1184
|
+
...(run
|
|
1185
|
+
? {
|
|
1186
|
+
env: {
|
|
1187
|
+
HARMONY_AGENT_CARD_ID: run.cardId,
|
|
1188
|
+
HARMONY_AGENT_SESSION_ID: run.agentSessionId,
|
|
1189
|
+
},
|
|
1190
|
+
}
|
|
1191
|
+
: {}),
|
|
1064
1192
|
},
|
|
1065
1193
|
};
|
|
1066
1194
|
}
|
|
1067
1195
|
|
|
1196
|
+
/**
|
|
1197
|
+
* The run whose `card_agent_context` session the MCP server may claim (#1035).
|
|
1198
|
+
*
|
|
1199
|
+
* Both fields are required together — see {@link harmonyMcpServer}. A caller
|
|
1200
|
+
* that has no live session (a verification or review spawn that never opened
|
|
1201
|
+
* one) passes nothing, and the MCP server then posts sessionless rather than
|
|
1202
|
+
* borrowing.
|
|
1203
|
+
*/
|
|
1204
|
+
export interface HarmonyMcpRunBinding {
|
|
1205
|
+
/** `cards.id` — the card this run holds its session on. */
|
|
1206
|
+
cardId: string;
|
|
1207
|
+
/** `card_agent_context.id` — the run's own session row. */
|
|
1208
|
+
agentSessionId: string;
|
|
1209
|
+
}
|
|
1210
|
+
|
|
1068
1211
|
/** The fields {@link implementRunContainment} contributes to an `SdkRunnerConfig`. */
|
|
1069
1212
|
export interface RunContainment {
|
|
1070
1213
|
sandbox: SandboxSettings;
|
|
@@ -1107,6 +1250,12 @@ export function implementRunContainment(args: {
|
|
|
1107
1250
|
* it says instead of being auto-approved because the run is sandboxed.
|
|
1108
1251
|
*/
|
|
1109
1252
|
readOnly?: boolean;
|
|
1253
|
+
/**
|
|
1254
|
+
* The board session this spawn IS, so its agent can name it on a comment
|
|
1255
|
+
* instead of the API inferring one (#1035). Omit when the spawn holds no
|
|
1256
|
+
* session — the agent then comments sessionless, which is the honest answer.
|
|
1257
|
+
*/
|
|
1258
|
+
run?: HarmonyMcpRunBinding;
|
|
1110
1259
|
}): RunContainment {
|
|
1111
1260
|
// Inside the builder rather than at the call sites, so a future spawn cannot
|
|
1112
1261
|
// adopt the containment and skip the check that makes it hold. Throws — see
|
|
@@ -1229,7 +1378,7 @@ export function implementRunContainment(args: {
|
|
|
1229
1378
|
// Handed back explicitly, because the line above took it away — see
|
|
1230
1379
|
// `harmonyMcpServer`. Without this the run loses `mcp__harmony__*` and can
|
|
1231
1380
|
// no longer report its own progress.
|
|
1232
|
-
mcpServers: harmonyMcpServer(),
|
|
1381
|
+
mcpServers: harmonyMcpServer(args.run),
|
|
1233
1382
|
// Paired with the two above: the run sees only the server declared to it,
|
|
1234
1383
|
// and no MCP config the operator happens to have registered for themselves.
|
|
1235
1384
|
strictMcpConfig: true,
|
|
@@ -1288,6 +1437,11 @@ export function implementRunContainmentCliArgs(args: {
|
|
|
1288
1437
|
* it says instead of being auto-approved because the run is sandboxed.
|
|
1289
1438
|
*/
|
|
1290
1439
|
readOnly?: boolean;
|
|
1440
|
+
/**
|
|
1441
|
+
* The board session this spawn IS (#1035) — travels inside `--mcp-config`,
|
|
1442
|
+
* the same way the SDK path travels it inside `mcpServers`.
|
|
1443
|
+
*/
|
|
1444
|
+
run?: HarmonyMcpRunBinding;
|
|
1291
1445
|
}): string[] {
|
|
1292
1446
|
const containment = implementRunContainment(args);
|
|
1293
1447
|
return [
|
package/src/stage-cli.ts
CHANGED
|
@@ -26,14 +26,19 @@ import { buildRoleLaunch, envKeysDroppedByLaunch } from "./runner.js";
|
|
|
26
26
|
import type { SdkRunnerConfig } from "./sdk-agent-runner.js";
|
|
27
27
|
|
|
28
28
|
export const STAGE_RUN_USAGE =
|
|
29
|
-
"usage: harmony-harness stage run --card <id> --stage <id> --workspace <id> --repo <path> --session <id> [--metrics <json-path>]";
|
|
29
|
+
"usage: harmony-harness stage run --card <id> --stage <id> --workspace <id> --repo <path> --session <id> [--metrics <json-path>] [--sandbox-image <image>]";
|
|
30
30
|
|
|
31
31
|
/** The five identifiers one stage run needs. Every one is required.
|
|
32
|
-
* `metricsPath`
|
|
33
|
-
*
|
|
34
|
-
* `
|
|
35
|
-
*
|
|
36
|
-
*
|
|
32
|
+
* `metricsPath` and `sandboxImage` are the optional inputs, and both carry the
|
|
33
|
+
* DRIVER's operator config into a motor that holds none of its own:
|
|
34
|
+
* - `metricsPath` is a JSON file holding the metric allowlist for `custom`
|
|
35
|
+
* gates (same shape as the daemon operator's `agent.playbooks.metrics`).
|
|
36
|
+
* Without it a `custom` gate reports "metric not declared" and blocks,
|
|
37
|
+
* exactly as before.
|
|
38
|
+
* - `sandboxImage` is the daemon's `verification.sandboxImage` (#1021): the
|
|
39
|
+
* container the held test and the gate's build/lint commands run in. Without
|
|
40
|
+
* it both run in the motor's own process tree, which is what a person
|
|
41
|
+
* running this CLI against their own checkout gets. */
|
|
37
42
|
export interface StageRunArgs {
|
|
38
43
|
cardId: string;
|
|
39
44
|
stageId: string;
|
|
@@ -41,6 +46,7 @@ export interface StageRunArgs {
|
|
|
41
46
|
repoPath: string;
|
|
42
47
|
sessionId: string;
|
|
43
48
|
metricsPath: string | null;
|
|
49
|
+
sandboxImage: string | null;
|
|
44
50
|
}
|
|
45
51
|
|
|
46
52
|
export type StageRunArgsResult =
|
|
@@ -78,14 +84,16 @@ export function parseStageRunArgs(argv: string[]): StageRunArgsResult {
|
|
|
78
84
|
};
|
|
79
85
|
}
|
|
80
86
|
|
|
81
|
-
const fields: Array<
|
|
87
|
+
const fields: Array<
|
|
88
|
+
[Exclude<keyof StageRunArgs, "metricsPath" | "sandboxImage">, string]
|
|
89
|
+
> = [
|
|
82
90
|
["cardId", "card"],
|
|
83
91
|
["stageId", "stage"],
|
|
84
92
|
["workspaceId", "workspace"],
|
|
85
93
|
["repoPath", "repo"],
|
|
86
94
|
["sessionId", "session"],
|
|
87
95
|
];
|
|
88
|
-
const args = { metricsPath: null } as StageRunArgs;
|
|
96
|
+
const args = { metricsPath: null, sandboxImage: null } as StageRunArgs;
|
|
89
97
|
for (const [field, flag] of fields) {
|
|
90
98
|
const value = readFlag(argv, flag);
|
|
91
99
|
if (value === null) {
|
|
@@ -105,6 +113,22 @@ export function parseStageRunArgs(argv: string[]): StageRunArgsResult {
|
|
|
105
113
|
}
|
|
106
114
|
args.metricsPath = metricsPath;
|
|
107
115
|
}
|
|
116
|
+
|
|
117
|
+
// Optional: --sandbox-image <image> (#1021). Same absent/valueless split as
|
|
118
|
+
// --metrics, and the refusal matters MORE here: a driver that passed the flag
|
|
119
|
+
// asked for the held test to run without its credentials in scope, and
|
|
120
|
+
// silently running it on the host would hand back exactly the exposure the
|
|
121
|
+
// flag exists to remove. There is no host fallback anywhere on this path.
|
|
122
|
+
if (argv.includes("--sandbox-image")) {
|
|
123
|
+
const sandboxImage = readFlag(argv, "sandbox-image");
|
|
124
|
+
if (sandboxImage === null) {
|
|
125
|
+
return {
|
|
126
|
+
ok: false,
|
|
127
|
+
message: "missing value for --sandbox-image <image>",
|
|
128
|
+
};
|
|
129
|
+
}
|
|
130
|
+
args.sandboxImage = sandboxImage;
|
|
131
|
+
}
|
|
108
132
|
return { ok: true, args };
|
|
109
133
|
}
|
|
110
134
|
|
package/src/verification.ts
CHANGED
|
@@ -14,7 +14,14 @@ import {
|
|
|
14
14
|
supportsDevServer,
|
|
15
15
|
testCommand,
|
|
16
16
|
} from "./project-type.js";
|
|
17
|
-
import {
|
|
17
|
+
import {
|
|
18
|
+
devServerContainerName,
|
|
19
|
+
devServerSandboxArgs,
|
|
20
|
+
removeSandboxContainer,
|
|
21
|
+
runInSandbox,
|
|
22
|
+
SANDBOX_DEV_SERVER_BIND,
|
|
23
|
+
sandboxAvailable,
|
|
24
|
+
} from "./repair-sandbox.js";
|
|
18
25
|
import { findDeletedTestFiles } from "./revert-guard.js";
|
|
19
26
|
import {
|
|
20
27
|
containedEnv,
|
|
@@ -70,7 +77,9 @@ export async function runVerification(
|
|
|
70
77
|
|
|
71
78
|
// Resolved once (#1036). Undefined unless the operator configured an image,
|
|
72
79
|
// in which case the four script-resolved steps run in a container instead of
|
|
73
|
-
// in this process.
|
|
80
|
+
// in this process. `runDeepReview` resolves the same value for its dev server
|
|
81
|
+
// (#1037) rather than taking it from here, because it is also called directly
|
|
82
|
+
// by the review path — one key, six commands.
|
|
74
83
|
const sandbox = verificationSandbox(config);
|
|
75
84
|
|
|
76
85
|
if (config.verification.revertGuard) {
|
|
@@ -220,22 +229,59 @@ export async function runVerification(
|
|
|
220
229
|
* takes the same sandbox, wired at the worker's registry call site. One config
|
|
221
230
|
* value, both paths.
|
|
222
231
|
*
|
|
223
|
-
* ##
|
|
232
|
+
* ## The two `dev` servers: contained too, on a weaker network (#1037)
|
|
233
|
+
*
|
|
234
|
+
* `runDeepReview`'s and the review worker's take the SAME `sandboxImage` and go
|
|
235
|
+
* through one `devServerLaunch`, so they can no longer be fixed one at a time —
|
|
236
|
+
* which is how #988 came to fix one of them and ship the other.
|
|
237
|
+
*
|
|
238
|
+
* **That is not a hypothetical, and #1037's card got it wrong.** The card says
|
|
239
|
+
* both dev servers already carried a working `containedEnv()`. The review
|
|
240
|
+
* worker's did not: it passed `env: containedEnv()` to `spawnInGroup`, which
|
|
241
|
+
* merges `process.env` back on top, so the strip removed NOTHING for as long as
|
|
242
|
+
* it shipped — and `spawn-run-containment.test.ts` reported the property as held
|
|
243
|
+
* because it read the spelling. #1021 fixed the call and made the scan
|
|
244
|
+
* spawn-function aware. So on that server the credential half closed one commit
|
|
245
|
+
* before this one, not in #988.
|
|
246
|
+
*
|
|
247
|
+
* Their container differs from the four steps' in exactly one flag, and it
|
|
248
|
+
* cannot be otherwise: a dev server exists to be connected to, so it publishes
|
|
249
|
+
* `127.0.0.1:<port>` instead of taking `--network=none`, and therefore has
|
|
250
|
+
* egress. Every other bound is identical — worktree the only mount, `HOME=/tmp`,
|
|
251
|
+
* no capabilities. `devServerSandboxArgs` records the two alternative network
|
|
252
|
+
* shapes that were measured and rejected.
|
|
253
|
+
*
|
|
254
|
+
* The cost lands very differently from the four steps', and this was measured
|
|
255
|
+
* rather than carried over. A verification step pays container start-up per
|
|
256
|
+
* COMMAND; a dev server pays it once and then lives for the whole review.
|
|
257
|
+
* Measured on macOS / Docker Desktop 29.7.2 with a warm `oven/bun:1`, as
|
|
258
|
+
* time-to-first-HTTP-response — what `probeDevServer` actually waits for:
|
|
224
259
|
*
|
|
225
|
-
*
|
|
226
|
-
*
|
|
227
|
-
*
|
|
228
|
-
*
|
|
229
|
-
*
|
|
230
|
-
*
|
|
231
|
-
*
|
|
260
|
+
* host 0.11 / 0.11 / 0.25 s
|
|
261
|
+
* contained 0.24 / 0.26 / 0.26 s -> ~0.15 s added, once per run
|
|
262
|
+
*
|
|
263
|
+
* That is noise beside the 10–20 s a real dev server takes to boot, which is
|
|
264
|
+
* why this one is not a cost argument the way #1036's was. (Taking the
|
|
265
|
+
* measurement also failed to reproduce #1036's own ~5.1 s figure on this host:
|
|
266
|
+
* its one-shot shape measures 0.50 s contained against 0.37 s on the host, with
|
|
267
|
+
* or without the bind mount. Left as a note rather than an edit, because the
|
|
268
|
+
* conditions that produced 5.1 s cannot be reconstructed from here — but the
|
|
269
|
+
* number is load-bearing for "opt-in, not default" and is worth re-taking.)
|
|
270
|
+
*
|
|
271
|
+
* `harmony-harness`'s own CLI no longer passes no image: #1021 gave it
|
|
272
|
+
* `--sandbox-image`, threaded from this same config by the daemon's motor
|
|
273
|
+
* driver, so a daemon-driven stage contains the `build_green` gate and the held
|
|
274
|
+
* test alike. A person running the CLI by hand still passes none and still runs
|
|
275
|
+
* on their own host, which is a different threat model. The dev servers are not
|
|
276
|
+
* on that path at all — neither gate starts one.
|
|
232
277
|
*
|
|
233
278
|
* That distinction is stated rather than left implied because the first version
|
|
234
279
|
* of both lists named the four verification steps only, and the two dev servers
|
|
235
280
|
* then shipped with the daemon's full environment, unlisted and unnoticed
|
|
236
281
|
* through every gate. `spawn-run-containment.test.ts` asserts the environment
|
|
237
|
-
* property over call sites
|
|
238
|
-
*
|
|
282
|
+
* property over call sites AND that both dev-server files are still in its
|
|
283
|
+
* scan; `verification-sandbox.test.ts` asserts that a configured image takes
|
|
284
|
+
* the four out of the daemon entirely.
|
|
239
285
|
*/
|
|
240
286
|
/**
|
|
241
287
|
* Resolve the sandbox for the verification steps out of the daemon's config
|
|
@@ -476,25 +522,44 @@ export async function runDeepReview(
|
|
|
476
522
|
|
|
477
523
|
const port = config.verification.devServerBasePort + workerId;
|
|
478
524
|
let devServer: ChildProcess | null = null;
|
|
525
|
+
// Resolved before the `try`, so the `finally` can tear the container down
|
|
526
|
+
// even if the spawn itself throws — `--rm` does not collect a server that
|
|
527
|
+
// never exits, and this one never does.
|
|
528
|
+
const launch = devServerLaunch({
|
|
529
|
+
worktreePath,
|
|
530
|
+
port,
|
|
531
|
+
sandbox: verificationSandbox(config),
|
|
532
|
+
});
|
|
479
533
|
|
|
480
534
|
try {
|
|
535
|
+
// Clear a predecessor on this port BEFORE starting, and await it. The
|
|
536
|
+
// teardown below cannot be awaited everywhere it is reached, and a
|
|
537
|
+
// container still shutting down still holds the published port — so the
|
|
538
|
+
// next start would fail with "port is already allocated" through no fault
|
|
539
|
+
// of the branch. Also collects a container orphaned by a killed daemon,
|
|
540
|
+
// which no in-process handle survives to remove. No-op when nothing is
|
|
541
|
+
// there; `removeSandboxContainer` swallows "No such container".
|
|
542
|
+
if (launch.containerName) {
|
|
543
|
+
await removeSandboxContainer(launch.containerName);
|
|
544
|
+
}
|
|
545
|
+
|
|
481
546
|
// Start dev server in background
|
|
482
|
-
|
|
483
|
-
devServer = spawn(cmd, args, {
|
|
547
|
+
devServer = spawn(launch.cmd, launch.args, {
|
|
484
548
|
cwd: worktreePath,
|
|
485
549
|
stdio: ["ignore", "pipe", "pipe"],
|
|
486
550
|
// Contained (#988), and it was the one spawn in this file that was not.
|
|
487
|
-
// `
|
|
488
|
-
//
|
|
551
|
+
// `devServerLaunch` resolves the worktree's OWN `dev` script, which the
|
|
552
|
+
// contained implement run may rewrite, so this is the same
|
|
489
553
|
// attacker-choosable command as `runBuild`/`runTests` — except it ran
|
|
490
554
|
// with the daemon's full environment, `HARMONY_API_KEY` and
|
|
491
555
|
// `GITHUB_PERSONAL_ACCESS_TOKEN` included, while `changelog.ts` told
|
|
492
556
|
// users the verification commands run "without your credentials".
|
|
493
557
|
//
|
|
494
|
-
//
|
|
495
|
-
//
|
|
496
|
-
//
|
|
497
|
-
//
|
|
558
|
+
// #988 removed the environment; #1037 removes the rest of the daemon
|
|
559
|
+
// process, when `verification.sandboxImage` is set. The env strip stays
|
|
560
|
+
// either way: it is what covers the host path, which is still the
|
|
561
|
+
// default, and it is what `docker` itself is spawned with on the
|
|
562
|
+
// contained one.
|
|
498
563
|
env: containedEnv(),
|
|
499
564
|
});
|
|
500
565
|
|
|
@@ -584,6 +649,14 @@ export async function runDeepReview(
|
|
|
584
649
|
if (devServer && !devServer.killed) {
|
|
585
650
|
devServer.kill("SIGTERM");
|
|
586
651
|
}
|
|
652
|
+
// Killing the `docker run` CLI is not the same as stopping the container it
|
|
653
|
+
// launched — the trap `runInSandbox` already documents for its timeout
|
|
654
|
+
// path, except here it is every path, because a dev server only ever ends
|
|
655
|
+
// by being killed. Without this a deep review leaves a server running with
|
|
656
|
+
// the operator's worktree bind-mounted, once per run.
|
|
657
|
+
if (launch.containerName) {
|
|
658
|
+
await removeSandboxContainer(launch.containerName);
|
|
659
|
+
}
|
|
587
660
|
}
|
|
588
661
|
}
|
|
589
662
|
|
|
@@ -836,6 +909,104 @@ export class DevServerReadinessError extends Error {
|
|
|
836
909
|
}
|
|
837
910
|
}
|
|
838
911
|
|
|
912
|
+
/** How a dev server is started, and what has to be torn down afterwards. */
|
|
913
|
+
export interface DevServerLaunch {
|
|
914
|
+
/** Command for `spawn` / `spawnInGroup`. */
|
|
915
|
+
cmd: string;
|
|
916
|
+
args: string[];
|
|
917
|
+
/**
|
|
918
|
+
* The container to force-remove once the server is no longer wanted. Absent
|
|
919
|
+
* on the host path, where killing the process is the whole teardown.
|
|
920
|
+
*/
|
|
921
|
+
containerName?: string;
|
|
922
|
+
}
|
|
923
|
+
|
|
924
|
+
/**
|
|
925
|
+
* Resolve how to start the worktree's `dev` server — in a container when one is
|
|
926
|
+
* configured, in the daemon process when one is not (#1037).
|
|
927
|
+
*
|
|
928
|
+
* ## Why both dev servers come through here
|
|
929
|
+
*
|
|
930
|
+
* There are two: `runDeepReview`'s, below, and the review worker's
|
|
931
|
+
* (`review-worker.ts`, in the daemon package). They are the same command with
|
|
932
|
+
* the same exposure, and they have already drifted apart once — #988 found the
|
|
933
|
+
* first, shipped the fix, and the second went out uncontained in the same
|
|
934
|
+
* release because nothing tied them together. A shared resolver is what makes
|
|
935
|
+
* "both dev servers" a fact about the code instead of a claim in a comment.
|
|
936
|
+
*
|
|
937
|
+
* `spawn-run-containment.test.ts` scans for this function by name alongside
|
|
938
|
+
* `spawnRunArgs`, so moving the resolution in here does not take the review
|
|
939
|
+
* worker's spawn out of that scan.
|
|
940
|
+
*
|
|
941
|
+
* ## What the container does and does not buy
|
|
942
|
+
*
|
|
943
|
+
* The `dev` script is resolved out of the worktree's OWN `package.json`, which
|
|
944
|
+
* a contained implement run is allowed to rewrite — the worktree is inside
|
|
945
|
+
* `sandbox.filesystem.allowWrite` by design. So this is the same
|
|
946
|
+
* attacker-choosable command as `runBuild` and `runTests`, and until now it ran
|
|
947
|
+
* in the daemon process as the operator. Contained, it loses the operator's
|
|
948
|
+
* home directory, every capability, and the daemon's uid on anything outside
|
|
949
|
+
* the worktree. It does NOT lose egress: see `devServerSandboxArgs`, which
|
|
950
|
+
* records the two shapes that were measured and rejected.
|
|
951
|
+
*
|
|
952
|
+
* ## Opt-in, behind the SAME key as #1036
|
|
953
|
+
*
|
|
954
|
+
* `verification.sandboxImage`, and deliberately not a second key. The image
|
|
955
|
+
* requirement is identical — it has to carry the repo's own toolchain — and an
|
|
956
|
+
* operator who set one key but not the other would get a half-contained daemon,
|
|
957
|
+
* which is the failure this module keeps having to write down. The cost of
|
|
958
|
+
* being wrong is bounded and already designed for: a contained dev server that
|
|
959
|
+
* cannot start fails exactly as an unstartable host one does, and the review
|
|
960
|
+
* worker already treats that as infrastructure rather than as the branch's
|
|
961
|
+
* fault — it keeps the card in Review and labels it for a human.
|
|
962
|
+
*
|
|
963
|
+
* `--host` is appended only on the contained path, because only there does the
|
|
964
|
+
* bind address matter. See `SANDBOX_DEV_SERVER_BIND`.
|
|
965
|
+
*/
|
|
966
|
+
export function devServerLaunch(args: {
|
|
967
|
+
worktreePath: string;
|
|
968
|
+
port: number;
|
|
969
|
+
sandbox?: VerificationSandbox;
|
|
970
|
+
}): DevServerLaunch {
|
|
971
|
+
const [cmd, runArgs] = spawnRunArgs(
|
|
972
|
+
"dev",
|
|
973
|
+
"--port",
|
|
974
|
+
String(args.port),
|
|
975
|
+
...(args.sandbox ? ["--host", SANDBOX_DEV_SERVER_BIND] : []),
|
|
976
|
+
);
|
|
977
|
+
if (!args.sandbox) return { cmd, args: runArgs };
|
|
978
|
+
const containerName = devServerContainerName(args.port);
|
|
979
|
+
return {
|
|
980
|
+
cmd: "docker",
|
|
981
|
+
args: devServerSandboxArgs({
|
|
982
|
+
image: args.sandbox.image,
|
|
983
|
+
worktree: args.worktreePath,
|
|
984
|
+
command: { cmd, args: runArgs },
|
|
985
|
+
port: args.port,
|
|
986
|
+
name: containerName,
|
|
987
|
+
}),
|
|
988
|
+
containerName,
|
|
989
|
+
};
|
|
990
|
+
}
|
|
991
|
+
|
|
992
|
+
/**
|
|
993
|
+
* The word "ready", and NOT the tail of "already" (#1037).
|
|
994
|
+
*
|
|
995
|
+
* A substring test read `already` as a readiness signal, which stopped being
|
|
996
|
+
* academic the moment the server moved into a container: `docker run` refuses a
|
|
997
|
+
* duplicate with `Conflict. The container name "…" is **already** in use`, so a
|
|
998
|
+
* failed start announced itself as a successful one. Reproduced end to end — the
|
|
999
|
+
* probe that followed was answered by the PREVIOUS run's container, still up on
|
|
1000
|
+
* that port, and the review would have graded the previous card's page believing
|
|
1001
|
+
* it was looking at this one. A wrong answer, not a slow one.
|
|
1002
|
+
*
|
|
1003
|
+
* `\b` is what fixes it: `already` has a word character before `ready`, so it no
|
|
1004
|
+
* longer matches, while `ready in 320 ms` and `Ready in 2.3s` both still do —
|
|
1005
|
+
* the second only since this became case-insensitive, which is a small widening
|
|
1006
|
+
* in the direction the check was always reaching for.
|
|
1007
|
+
*/
|
|
1008
|
+
const DEV_SERVER_READY = /\bready\b/i;
|
|
1009
|
+
|
|
839
1010
|
/**
|
|
840
1011
|
* Wait for a dev server to signal readiness on stdout/stderr.
|
|
841
1012
|
*
|
|
@@ -843,6 +1014,14 @@ export class DevServerReadinessError extends Error {
|
|
|
843
1014
|
* need the server to be live for correctness (e.g. the review worker)
|
|
844
1015
|
* must not proceed without a confirmed signal. If the server dies before
|
|
845
1016
|
* becoming ready, we reject with the exit details.
|
|
1017
|
+
*
|
|
1018
|
+
* **A match here is not proof the server is up**, and the caller's
|
|
1019
|
+
* `probeDevServer` is not redundant: this reads text the child chose to print.
|
|
1020
|
+
* The container path adds a second writer to that stream — docker itself — and
|
|
1021
|
+
* the two are indistinguishable once they are bytes. That is why the contained
|
|
1022
|
+
* path both suppresses docker's progress output (`--quiet`) and removes the
|
|
1023
|
+
* previous container before starting, so a stale server cannot be the thing
|
|
1024
|
+
* that answers the probe.
|
|
846
1025
|
*/
|
|
847
1026
|
export function waitForDevServer(
|
|
848
1027
|
proc: ChildProcess,
|
|
@@ -882,7 +1061,7 @@ export function waitForDevServer(
|
|
|
882
1061
|
const onData = (data: Buffer) => {
|
|
883
1062
|
const text = data.toString();
|
|
884
1063
|
if (
|
|
885
|
-
|
|
1064
|
+
DEV_SERVER_READY.test(text) ||
|
|
886
1065
|
text.includes("localhost") ||
|
|
887
1066
|
text.includes("Local:")
|
|
888
1067
|
) {
|