@codedology/code-forge 0.9.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +58 -0
- package/README.md +29 -8
- package/docs/reference/blocks.json +385 -1
- package/docs/reference/config.md +36 -0
- package/package.json +1 -1
- package/schema/code-forge.schema.json +89 -0
- package/skill/references/adapters/claude-code.md +1 -1
- package/skill/references/adapters/solo.md +2 -2
- package/skill/references/adapters/subprocess.md +12 -2
- package/skill/references/code.md +8 -3
- package/skill/references/ledger.md +2 -2
- package/skill/references/proof.md +77 -2
- package/skill/references/review.md +11 -4
- package/skill/references/security.md +4 -4
- package/skill/scripts/forge +1 -1
- package/skill/templates/coder-brief.md +2 -0
- package/src/autopilot/binnacle.mjs +9 -3
- package/src/cli/block.mjs +164 -24
- package/src/cli/jev.mjs +22 -3
- package/src/cli/keys.mjs +156 -21
- package/src/cli/proof.mjs +35 -7
- package/src/cli/spawn.mjs +44 -3
- package/src/config/validate.mjs +320 -5
- package/src/doctor/index.mjs +2 -1
- package/src/doctor/local.mjs +27 -6
- package/src/doctor/probes.mjs +38 -0
- package/src/engines/builders/codex.mjs +44 -4
- package/src/engines/sandbox-dirs.mjs +257 -0
- package/src/keys/store.mjs +128 -21
- package/src/proof/junit.mjs +530 -0
- package/src/proof/red-green.mjs +57 -9
- package/src/proof/red-parse.mjs +2 -0
- package/src/proof/runners.mjs +472 -0
- package/src/proof/xcresult.mjs +285 -0
- package/src/review/engine.mjs +82 -30
- package/src/review/fixloop.mjs +45 -13
- package/src/review/gate-check.mjs +3 -2
- package/src/review/lenses/A.md +1 -0
- package/src/review/lenses/B.md +1 -0
- package/src/review/lenses/full.md +1 -0
- package/src/review/lenses/quick.md +1 -0
- package/src/review/lenses/recheck.md +1 -0
- package/src/review/reopen.mjs +4 -6
- package/src/review/route.mjs +232 -0
- package/src/review/schema-fallback.mjs +77 -22
- package/src/review/session-retry.mjs +44 -9
- package/src/review/test-quality.mjs +201 -0
- package/src/review/verb-forms.mjs +39 -0
- package/src/session/spawn.mjs +39 -2
- package/src/state/block.mjs +13 -3
- package/src/state/config-snapshot.mjs +39 -0
- package/src/state/run.mjs +4 -0
- package/src/util/forbidden.mjs +2 -2
- package/src/worker/loop.mjs +79 -14
- package/src/worker/ticket.mjs +9 -1
package/docs/reference/config.md
CHANGED
|
@@ -108,6 +108,25 @@ Every other key defaults as shown, or is entirely absent from a minimal file. `a
|
|
|
108
108
|
| `review.min_tokens_out` | integer | | `120` | min 1; The stub guard: a review answer with fewer output tokens is unavailable (too_short). A clean pass (passed: true, no findings, a non-blank summary) needs only min(this, 40), whatever the diff size (B53). |
|
|
109
109
|
| `review.batch_small_files` | boolean | | `true` | |
|
|
110
110
|
| `review.session_timeout_s` | integer | | `300` | min 1 |
|
|
111
|
+
| `review.routes` | array<object> | | — | B90: size-based review routing. min_lines and model are required. A rule's level decides which sessions it changes: an L2 rule (level: L2, the default) changes only the reviewers' model, and in consensus mode only slot A (slot B stays review.second_levels.L2); an L3 rule changes only the judge's model. The count is the file's lines at the reviewed state (the bytes in the review packet): newline bytes, plus one for a last line without one; a deleted file is never routed. Rules compete only within their level: among the rules of one level, the largest min_lines at or under the count wins; the same level and min_lines twice is refused. Left out, an optional key gets: provider the top-level provider (the rule replaces the level entry, so the level's own provider is not inherited; validation checks the rule with that same provider); effort none; fallback none; session_timeout_s nothing (the rule does not count in the timeout); second review.second_levels for the rule's level. The timeout is one value for every session of the ticket: the largest session_timeout_s among the matched rules that set one (a matched rule without one is left out of that max), else review.session_timeout_s. The second-level try uses the rule's second, except for an L2 rule in consensus mode. The worker writes one signed review.route row per routed level before the first session, never twice for one ticket; its second_model is null when the rule's second is not used, which includes an L2 rule in consensus mode, because slot B is not the rule's. run reload accepts a change. |
|
|
112
|
+
| `review.routes[].min_lines` | integer | yes | — | min 0; B90: the rule applies to a file with at least this many lines (0: every file that is not deleted). |
|
|
113
|
+
| `review.routes[].level` | enum: `L2`, `L3` | | `"L2"` | B90: the level the rule replaces, L2 (the reviewers) or L3 (the judge). |
|
|
114
|
+
| `review.routes[].model` | string | yes | — | minLength 1 |
|
|
115
|
+
| `review.routes[].effort` | string | | — | |
|
|
116
|
+
| `review.routes[].provider` | enum: `anthropic`, `openai`, `xai` | | — | |
|
|
117
|
+
| `review.routes[].fallback` | array<object> | | — | |
|
|
118
|
+
| `review.routes[].fallback[].provider` | enum: `anthropic`, `openai`, `xai` | yes | — | |
|
|
119
|
+
| `review.routes[].fallback[].model` | string | yes | — | minLength 1 |
|
|
120
|
+
| `review.routes[].fallback[].effort` | string | | — | |
|
|
121
|
+
| `review.routes[].session_timeout_s` | integer | | — | min 1; B90: a matched rule's timeout; every session of the ticket runs at the largest one among the matched rules that set one. Unset: the rule does not count in that max (review.session_timeout_s applies only when no matched rule sets one). |
|
|
122
|
+
| `review.routes[].second` | object | | — | |
|
|
123
|
+
| `review.routes[].second.model` | string | yes | — | minLength 1 |
|
|
124
|
+
| `review.routes[].second.effort` | string | | — | |
|
|
125
|
+
| `review.routes[].second.provider` | enum: `anthropic`, `openai`, `xai` | | — | |
|
|
126
|
+
| `review.routes[].second.fallback` | array<object> | | — | |
|
|
127
|
+
| `review.routes[].second.fallback[].provider` | enum: `anthropic`, `openai`, `xai` | yes | — | |
|
|
128
|
+
| `review.routes[].second.fallback[].model` | string | yes | — | minLength 1 |
|
|
129
|
+
| `review.routes[].second.fallback[].effort` | string | | — | |
|
|
111
130
|
| `review.max_rounds_per_file` | integer | | `4` | min 2; max 6 |
|
|
112
131
|
| `review.recheck_scope` | enum: `fix_hunks`, `file` | | `"fix_hunks"` | |
|
|
113
132
|
| `review.late_findings` | enum: `sweep`, `block` | | `"sweep"` | |
|
|
@@ -153,6 +172,18 @@ Every other key defaults as shown, or is entirely absent from a minimal file. `a
|
|
|
153
172
|
| `proof.export` | object | | — | |
|
|
154
173
|
| `proof.export.link_dirs` | array<string> | | `["vendor","node_modules"]` | |
|
|
155
174
|
| `proof.export.copy_untracked` | array<string> | | `[".env",".env.testing"]` | |
|
|
175
|
+
| `proof.runners` | array<object> | | — | minItems 1; B87: how proof red-green runs a test file that is not a node:test file. proof red-green uses the first entry whose match covers the --test file (list order). With no match, a .mjs/.cjs/.js file uses the built-in node:test command and any other file is refused. Fixed for the run: run start copies the list into the run snapshot, run reload refuses to change it, and with it set no block may own .code-forge.yml. An empty list is refused: leave the key out for no runner. |
|
|
176
|
+
| `proof.runners[].match` | array<string> | yes | — | minItems 1; Globs of the test files this runner runs, relative to the repository (the same glob rules as owned entries). |
|
|
177
|
+
| `proof.runners[].adapter` | enum: `junit`, `swift`, `xcodebuild` | yes | — | How the result is read. junit: the runner writes a JUnit XML report to {report} (PHPUnit, Pest, pytest and other JUnit runners). swift: swift test with --xunit-output {report}. xcodebuild: xcodebuild test with -resultBundlePath {result_bundle}; the result is read with xcrun xcresulttool, and a precheck needs full Xcode and the destination's simulator first. |
|
|
178
|
+
| `proof.runners[].argv` | array<string> | yes | — | minItems 1; The command, an argv array (never a shell string). These items are replaced when an item is exactly the token. junit and swift: {test_file} (the test path), {test_id} (<file>::<case>, or <file> with no case), {test_filter} (the case, regex-escaped), {report} (a fresh report path outside the work dir); {report} and one test token are required. xcodebuild: {workspace}, {project}, {scheme}, {destination} (from xcode), {only_testing} (one item, -only-testing:<test_target>/<Suite>[/<case>]), {result_bundle} (a fresh result bundle path outside the work dir); {scheme}, {destination}, {only_testing} and {result_bundle} are required. A token cannot be the command (argv[0]) or part of a longer item. |
|
|
179
|
+
| `proof.runners[].xcode` | object | | — | |
|
|
180
|
+
| `proof.runners[].xcode.workspace` | string | | — | minLength 1; The .xcworkspace for {workspace}. At most one of workspace and project; neither = a Swift package in the work dir. |
|
|
181
|
+
| `proof.runners[].xcode.project` | string | | — | minLength 1; The .xcodeproj for {project}. At most one of workspace and project. |
|
|
182
|
+
| `proof.runners[].xcode.scheme` | string | yes | — | minLength 1; The scheme for {scheme}. |
|
|
183
|
+
| `proof.runners[].xcode.destination` | string | yes | — | minLength 1; The destination for {destination}, e.g. platform=iOS Simulator,name=iPhone 17. A simulator destination must name a device xcrun simctl lists as available (by name or id, and OS when given); platform=macOS needs none. code-forge never boots, creates or deletes a simulator. |
|
|
184
|
+
| `proof.runners[].xcode.test_target` | string | yes | — | minLength 1; The test target (one name, no /) that {only_testing} starts with; every test in the result must sit under <test_target>/<Suite>. |
|
|
185
|
+
| `proof.runners[].env` | object | | — | |
|
|
186
|
+
| `proof.runners[].timeout_s` | integer | | `600` | min 60; max 7200; Seconds each step (RED, GREEN) may run before it is stopped. |
|
|
156
187
|
| `budget` | object | | — | |
|
|
157
188
|
| `budget.block_lines` | integer | | `600` | min 1 |
|
|
158
189
|
| `budget.block_tokens_soft` | integer | | — | min 1 |
|
|
@@ -175,6 +206,7 @@ Every other key defaults as shown, or is entirely absent from a minimal file. `a
|
|
|
175
206
|
| `calibration` | object | | — | |
|
|
176
207
|
| `calibration.shadow_rate` | number | | `0` | min 0; max 0.5 |
|
|
177
208
|
| `keys` | object | | — | |
|
|
209
|
+
| `keys.cache_hours` | integer | | `8` | min 1; max 24; B84: how many whole hours a key read from 1Password stays cached in the per-user key store (keys set --op, keys test, jev ask, doctor, the worker). keys set --cache-hours wins over it. A cached key is never trusted for longer than the reader's value. |
|
|
178
210
|
| `keys.<key>` | string | | — | user-defined key |
|
|
179
211
|
| `tmp` | object | | — | |
|
|
180
212
|
| `tmp.root` | string \| null | | `null` | |
|
|
@@ -194,4 +226,8 @@ Every other key defaults as shown, or is entirely absent from a minimal file. `a
|
|
|
194
226
|
| `hooks.on_block_end.timeout_s` | integer | | `60` | min 1; max 600; Seconds before the hook's process group is killed (the hook.ran row then says timed_out). |
|
|
195
227
|
| `autopilot` | object | | — | |
|
|
196
228
|
| `autopilot.min_confidence` | number | | `0.7` | min 0; max 1; B46: the lowest confidence at which the autopilot delegate's answer is acted on; below it the question goes to the owner. |
|
|
229
|
+
| `coder` | object | | — | |
|
|
230
|
+
| `coder.sandbox` | object | | — | |
|
|
231
|
+
| `coder.sandbox.profile` | enum: `workspace`, `toolchain`, `full` | | `"workspace"` | workspace (the default; no key gives today's argv): `-s workspace-write`, the project tree is the only writable root. toolchain: the same plus one `--add-dir <abs>` per add_dirs entry (a build cache, a package store). full: `-s danger-full-access`, the coder writes anywhere the user can; like the Claude coder, the forbidden list, the transcript grep and the gates are the only limits, and doctor prints a WARN row. Every profile keeps the $TMPDIR and /tmp exclusions in the argv; Codex ignores them under danger-full-access. |
|
|
232
|
+
| `coder.sandbox.add_dirs` | array<string> | | `[]` | Extra writable directories for profile toolchain; under workspace or full they are unused and validate warns (rule sandbox-dirs-unused), and a refused entry is still refused. Each entry is absolute or starts with ~/ (expanded against the process HOME) and becomes an absolute realpath: symlinks are followed, even a dangling one, so a symlink is judged by its target. Refused (rule sandbox-dir-refused, by validate and again by the builder; the message names the key path, never the value): /, the home directory itself, and any directory that is, holds or sits inside ~/.code-forge, ~/.codex or $CODEX_HOME, ~/.ssh, ~/.gnupg, ~/.aws, ~/.config, ~/Library/Keychains, the run temp root (so $TMPDIR or /tmp when it holds it), or the session CODEX_HOME; for example ~/Library is refused because it holds ~/Library/Keychains. |
|
|
197
233
|
| `telemetry` | const `"off"` | | `"off"` | |
|
package/package.json
CHANGED
|
@@ -139,6 +139,37 @@
|
|
|
139
139
|
"min_tokens_out": { "type": "integer", "minimum": 1, "default": 120, "description": "The stub guard: a review answer with fewer output tokens is unavailable (too_short). A clean pass (passed: true, no findings, a non-blank summary) needs only min(this, 40), whatever the diff size (B53)." },
|
|
140
140
|
"batch_small_files": { "type": "boolean", "default": true },
|
|
141
141
|
"session_timeout_s": { "type": "integer", "minimum": 1, "default": 300 },
|
|
142
|
+
"routes": {
|
|
143
|
+
"type": "array",
|
|
144
|
+
"description": "B90: size-based review routing. min_lines and model are required. A rule's level decides which sessions it changes: an L2 rule (level: L2, the default) changes only the reviewers' model, and in consensus mode only slot A (slot B stays review.second_levels.L2); an L3 rule changes only the judge's model. The count is the file's lines at the reviewed state (the bytes in the review packet): newline bytes, plus one for a last line without one; a deleted file is never routed. Rules compete only within their level: among the rules of one level, the largest min_lines at or under the count wins; the same level and min_lines twice is refused. Left out, an optional key gets: provider the top-level provider (the rule replaces the level entry, so the level's own provider is not inherited; validation checks the rule with that same provider); effort none; fallback none; session_timeout_s nothing (the rule does not count in the timeout); second review.second_levels for the rule's level. The timeout is one value for every session of the ticket: the largest session_timeout_s among the matched rules that set one (a matched rule without one is left out of that max), else review.session_timeout_s. The second-level try uses the rule's second, except for an L2 rule in consensus mode. The worker writes one signed review.route row per routed level before the first session, never twice for one ticket; its second_model is null when the rule's second is not used, which includes an L2 rule in consensus mode, because slot B is not the rule's. run reload accepts a change.",
|
|
145
|
+
"items": {
|
|
146
|
+
"type": "object",
|
|
147
|
+
"additionalProperties": false,
|
|
148
|
+
"required": ["min_lines", "model"],
|
|
149
|
+
"properties": {
|
|
150
|
+
"min_lines": { "type": "integer", "minimum": 0, "description": "B90: the rule applies to a file with at least this many lines (0: every file that is not deleted)." },
|
|
151
|
+
"level": { "type": "string", "enum": ["L2", "L3"], "default": "L2", "description": "B90: the level the rule replaces, L2 (the reviewers) or L3 (the judge)." },
|
|
152
|
+
"model": { "type": "string", "minLength": 1 },
|
|
153
|
+
"effort": { "type": "string" },
|
|
154
|
+
"provider": { "$ref": "#/$defs/provider" },
|
|
155
|
+
"fallback": {
|
|
156
|
+
"type": "array",
|
|
157
|
+
"items": {
|
|
158
|
+
"type": "object",
|
|
159
|
+
"additionalProperties": false,
|
|
160
|
+
"required": ["provider", "model"],
|
|
161
|
+
"properties": {
|
|
162
|
+
"provider": { "$ref": "#/$defs/provider" },
|
|
163
|
+
"model": { "type": "string", "minLength": 1 },
|
|
164
|
+
"effort": { "type": "string" }
|
|
165
|
+
}
|
|
166
|
+
}
|
|
167
|
+
},
|
|
168
|
+
"session_timeout_s": { "type": "integer", "minimum": 1, "description": "B90: a matched rule's timeout; every session of the ticket runs at the largest one among the matched rules that set one. Unset: the rule does not count in that max (review.session_timeout_s applies only when no matched rule sets one)." },
|
|
169
|
+
"second": { "$ref": "#/$defs/level" }
|
|
170
|
+
}
|
|
171
|
+
}
|
|
172
|
+
},
|
|
142
173
|
"max_rounds_per_file": { "type": "integer", "minimum": 2, "maximum": 6, "default": 4 },
|
|
143
174
|
"recheck_scope": { "type": "string", "enum": ["fix_hunks", "file"], "default": "fix_hunks" },
|
|
144
175
|
"late_findings": { "type": "string", "enum": ["sweep", "block"], "default": "sweep" },
|
|
@@ -258,6 +289,45 @@
|
|
|
258
289
|
"default": [".env", ".env.testing"]
|
|
259
290
|
}
|
|
260
291
|
}
|
|
292
|
+
},
|
|
293
|
+
"runners": {
|
|
294
|
+
"type": "array",
|
|
295
|
+
"minItems": 1,
|
|
296
|
+
"description": "B87: how proof red-green runs a test file that is not a node:test file. proof red-green uses the first entry whose match covers the --test file (list order). With no match, a .mjs/.cjs/.js file uses the built-in node:test command and any other file is refused. Fixed for the run: run start copies the list into the run snapshot, run reload refuses to change it, and with it set no block may own .code-forge.yml. An empty list is refused: leave the key out for no runner.",
|
|
297
|
+
"items": {
|
|
298
|
+
"type": "object",
|
|
299
|
+
"additionalProperties": false,
|
|
300
|
+
"required": ["match", "adapter", "argv"],
|
|
301
|
+
"if": { "patternProperties": { "^adapter$": { "const": "xcodebuild" } } },
|
|
302
|
+
"then": { "not": { "propertyNames": { "not": { "const": "xcode" } } } },
|
|
303
|
+
"else": { "propertyNames": { "not": { "const": "xcode" } } },
|
|
304
|
+
"properties": {
|
|
305
|
+
"match": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 }, "description": "Globs of the test files this runner runs, relative to the repository (the same glob rules as owned entries)." },
|
|
306
|
+
"adapter": { "type": "string", "enum": ["junit", "swift", "xcodebuild"], "description": "How the result is read. junit: the runner writes a JUnit XML report to {report} (PHPUnit, Pest, pytest and other JUnit runners). swift: swift test with --xunit-output {report}. xcodebuild: xcodebuild test with -resultBundlePath {result_bundle}; the result is read with xcrun xcresulttool, and a precheck needs full Xcode and the destination's simulator first." },
|
|
307
|
+
"argv": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 }, "description": "The command, an argv array (never a shell string). These items are replaced when an item is exactly the token. junit and swift: {test_file} (the test path), {test_id} (<file>::<case>, or <file> with no case), {test_filter} (the case, regex-escaped), {report} (a fresh report path outside the work dir); {report} and one test token are required. xcodebuild: {workspace}, {project}, {scheme}, {destination} (from xcode), {only_testing} (one item, -only-testing:<test_target>/<Suite>[/<case>]), {result_bundle} (a fresh result bundle path outside the work dir); {scheme}, {destination}, {only_testing} and {result_bundle} are required. A token cannot be the command (argv[0]) or part of a longer item." },
|
|
308
|
+
"xcode": {
|
|
309
|
+
"type": "object",
|
|
310
|
+
"additionalProperties": false,
|
|
311
|
+
"required": ["scheme", "destination", "test_target"],
|
|
312
|
+
"dependentSchemas": { "workspace": { "propertyNames": { "not": { "const": "project" } } } },
|
|
313
|
+
"description": "B102: the xcodebuild adapter's values; required for it, refused on any other adapter. Each value is a non-empty string that does not start with -. At most one of workspace and project.",
|
|
314
|
+
"properties": {
|
|
315
|
+
"workspace": { "type": "string", "minLength": 1, "pattern": "^[^-]", "description": "The .xcworkspace for {workspace}. At most one of workspace and project; neither = a Swift package in the work dir." },
|
|
316
|
+
"project": { "type": "string", "minLength": 1, "pattern": "^[^-]", "description": "The .xcodeproj for {project}. At most one of workspace and project." },
|
|
317
|
+
"scheme": { "type": "string", "minLength": 1, "pattern": "^[^-]", "description": "The scheme for {scheme}." },
|
|
318
|
+
"destination": { "type": "string", "minLength": 1, "pattern": "^[^-]", "description": "The destination for {destination}, e.g. platform=iOS Simulator,name=iPhone 17. A simulator destination must name a device xcrun simctl lists as available (by name or id, and OS when given); platform=macOS needs none. code-forge never boots, creates or deletes a simulator." },
|
|
319
|
+
"test_target": { "type": "string", "minLength": 1, "pattern": "^[^-/][^/]*$", "description": "The test target (one name, no /) that {only_testing} starts with; every test in the result must sit under <test_target>/<Suite>." }
|
|
320
|
+
}
|
|
321
|
+
},
|
|
322
|
+
"env": {
|
|
323
|
+
"type": "object",
|
|
324
|
+
"additionalProperties": false,
|
|
325
|
+
"patternProperties": { "^[A-Za-z_][A-Za-z0-9_]*$": { "type": "string" } },
|
|
326
|
+
"description": "Variables added to the runner's environment, NAME: value. code-forge never prints or records an env value; only the names are shown (env_keys). Child output is masked only for values of 8 or more characters. Not a secret store."
|
|
327
|
+
},
|
|
328
|
+
"timeout_s": { "type": "integer", "minimum": 60, "maximum": 7200, "default": 600, "description": "Seconds each step (RED, GREEN) may run before it is stopped." }
|
|
329
|
+
}
|
|
330
|
+
}
|
|
261
331
|
}
|
|
262
332
|
}
|
|
263
333
|
},
|
|
@@ -326,6 +396,9 @@
|
|
|
326
396
|
},
|
|
327
397
|
"keys": {
|
|
328
398
|
"type": "object",
|
|
399
|
+
"properties": {
|
|
400
|
+
"cache_hours": { "type": "integer", "minimum": 1, "maximum": 24, "default": 8, "description": "B84: how many whole hours a key read from 1Password stays cached in the per-user key store (keys set --op, keys test, jev ask, doctor, the worker). keys set --cache-hours wins over it. A cached key is never trusted for longer than the reader's value." }
|
|
401
|
+
},
|
|
329
402
|
"additionalProperties": { "type": "string" }
|
|
330
403
|
},
|
|
331
404
|
"tmp": {
|
|
@@ -384,6 +457,22 @@
|
|
|
384
457
|
"min_confidence": { "type": "number", "minimum": 0, "maximum": 1, "default": 0.7, "description": "B46: the lowest confidence at which the autopilot delegate's answer is acted on; below it the question goes to the owner." }
|
|
385
458
|
}
|
|
386
459
|
},
|
|
460
|
+
"coder": {
|
|
461
|
+
"type": "object",
|
|
462
|
+
"additionalProperties": false,
|
|
463
|
+
"description": "B93: what the coder session may write beyond the project tree. Read by the Codex (openai) coder only; it has no effect on the Claude (anthropic) and Grok (xai) coders, whose argv does not change. No key means profile workspace and today's coder argv. Authority: fixed at run start; `run reload` refuses a change; `forge spawn --role coder --run <id>` refuses a live value that differs from the run-start snapshot (sandbox-changed). Without --run, spawn reads the live config.",
|
|
464
|
+
"properties": {
|
|
465
|
+
"sandbox": {
|
|
466
|
+
"type": "object",
|
|
467
|
+
"additionalProperties": false,
|
|
468
|
+
"description": "The Codex coder sandbox profile and its extra writable directories.",
|
|
469
|
+
"properties": {
|
|
470
|
+
"profile": { "type": "string", "enum": ["workspace", "toolchain", "full"], "default": "workspace", "description": "workspace (the default; no key gives today's argv): `-s workspace-write`, the project tree is the only writable root. toolchain: the same plus one `--add-dir <abs>` per add_dirs entry (a build cache, a package store). full: `-s danger-full-access`, the coder writes anywhere the user can; like the Claude coder, the forbidden list, the transcript grep and the gates are the only limits, and doctor prints a WARN row. Every profile keeps the $TMPDIR and /tmp exclusions in the argv; Codex ignores them under danger-full-access." },
|
|
471
|
+
"add_dirs": { "type": "array", "items": { "type": "string", "minLength": 1, "pattern": "^(/|~/)" }, "uniqueItems": true, "default": [], "description": "Extra writable directories for profile toolchain; under workspace or full they are unused and validate warns (rule sandbox-dirs-unused), and a refused entry is still refused. Each entry is absolute or starts with ~/ (expanded against the process HOME) and becomes an absolute realpath: symlinks are followed, even a dangling one, so a symlink is judged by its target. Refused (rule sandbox-dir-refused, by validate and again by the builder; the message names the key path, never the value): /, the home directory itself, and any directory that is, holds or sits inside ~/.code-forge, ~/.codex or $CODEX_HOME, ~/.ssh, ~/.gnupg, ~/.aws, ~/.config, ~/Library/Keychains, the run temp root (so $TMPDIR or /tmp when it holds it), or the session CODEX_HOME; for example ~/Library is refused because it holds ~/Library/Keychains." }
|
|
472
|
+
}
|
|
473
|
+
}
|
|
474
|
+
}
|
|
475
|
+
},
|
|
387
476
|
"telemetry": { "type": "string", "const": "off", "default": "off" }
|
|
388
477
|
},
|
|
389
478
|
"$defs": {
|
|
@@ -12,7 +12,7 @@ The harness's `Agent` tool runs the coder as a subagent; everything else — the
|
|
|
12
12
|
|
|
13
13
|
The tool return is the completion signal; the report must begin with `ACK <sha8>` and end with `===BLOCK <id> COMPLETE===` (+ test summary + `git diff --stat`) or `===BLOCK <id> FAILED: <reason>===`. A report without the sentinel is a failed block, never a partial success.
|
|
14
14
|
|
|
15
|
-
**Between completion and close come the review rounds.** The subagent ran `forge review-file <path> --block <id> --run <r>` per file and `--wait`ed on each ticket inside its turn, so most rounds happen before the sentinel; when the tool returns, first drain every `pending` ticket with `forge review-file --wait <ticket>` yourself — a pending ticket is a review in flight, not an open finding, and waiting on it costs no round. Then
|
|
15
|
+
**Between completion and close come the review rounds.** The subagent ran `forge review-file <path> --block <id> --run <r>` per file and `--wait`ed on each ticket inside its turn, so most rounds happen before the sentinel; when the tool returns, first drain every `pending` ticket with `forge review-file --wait <ticket>` yourself — a pending ticket is a review in flight, not an open finding, and waiting on it costs no round. Then the open `fix_now` findings, the failing tests and the files the coder never submitted go into one fix round. Fix rounds are one per block: wait for every `review-file` ticket of the block and its tests, then send ONE fix message with every failing test and every open finding; a finding that arrives later waits for the next round (`code.md` §4). Dispatch that message as one new `Agent` call at the level the ladder says (`review.md` §5 — the coder is gone with its tool return, so the fix brief carries every failing test, every open finding and the fix history). Its brief says: in that round the coder fixes, runs the block's tests, re-runs `review-file` on every changed file, and prints its `TESTS` line. One block round is one round for each file it touched. Rounds are bounded: four rounds per file, then the L3 rung once, then `stopped: review_cap` for the human. Only when every changed file has a signed `review.approved` row do you run the re-measurement of `code.md` §5 and `forge block close <id> --run <r> --report <file>` (the subagent's final report, saved to a file) — the transcript grep reads `.code-forge/runs/<run>/<block>.log`; pass `--transcript <file>` when the harness saved the subagent's transcript elsewhere. Closing with a file still unreviewed is a human decision: `forge block waive <id> <finding> --run <r> --file <path> --reason "<why>"` per finding, which asks for a yes at a terminal and has no `--yes`, or `forge block close … --no-require-reviews`, which also asks and is refused under an active autopilot grant, all audited as their own rows. `forge block reopen <id> --file <path> --run <r> --reason "<why>"` does not close anything: it sends a fixed stopped file back to review, and it is also human-only and asks the same way. Print the exact command: the orchestrator never answers this prompt; the human types the command in their own terminal. You and the subagent never run it. The prompt checks only that stdin and stdout are terminals; a pseudo-terminal passes it, so it does not prove a human (`security.md` §3).
|
|
16
16
|
|
|
17
17
|
## §2.1 Plan approval
|
|
18
18
|
|
|
@@ -21,8 +21,8 @@ Solo adds spawn, watch, timers, scratchpads, locks and `kv` on top of the CLI; i
|
|
|
21
21
|
4. **Launch-verification gate** — the one thing you babysit: poll `get_process_output` / `search_output` until the coder shows the ACK and tool activity. Dead-idle after the pointer (no prompt on screen) ⇒ one Enter (`send_input bytes=[13]`) and re-check. A **permission or model-picker prompt** is never answered with Enter — it would accept an unreviewed default: `close_process`, respawn with the correct `extra_args`, and record a `spawn-flag defect` line in the block record (the flags came from `forge resolve`; a prompt means they were not pinned).
|
|
22
22
|
5. **Wake channels, event-driven, never fixed polls:** record your own pid once (`whoami` → `kv_set("forge:orchestrator_pid")`); coders that can call Solo carry the push contract (post-sentinel `kv_set("block:<id>", …)` + `timer_set(1s, body, delivery_process_id=<your pid>)`); for every group arm `timer_fire_when_idle_any([coders], max_wait≈20–25 min, delivery_process_id=<your pid>)` right after the launch gate (idle-any ignores already-idle processes, so arm while they are provably running); keep one long dead-man fallback per 30–40 min with an L0 triage first. On wake: sentinel ⇒ gate; no sentinel and active again ⇒ re-arm; no sentinel and idle ⇒ parked, intervene now.
|
|
23
23
|
6. Completion = `search_output(pattern="===BLOCK <id> COMPLETE===")`, last occurrence (the `kv` key is the truncation backup). Completion is not closure.
|
|
24
|
-
7. **Review rounds, with the coder still alive.** The coder ran `forge review-file <path> --block <id> --run <r>` per file and `--wait`ed on each ticket; read the ledger (`forge ledger tail --slug <slug>`) for what is still open — a `fix_now` finding, a `pending` ticket, a file never submitted.
|
|
25
|
-
8. **Close** only when every changed file has a signed `review.approved` row: the re-measurement of `code.md` §5, then `forge block close <id> --run <r> --report <file>` (the coder's final report) — the transcript grep reads the Solo output. Closing with a file still unreviewed is a human decision, audited: `forge block waive <id> <finding> --run <r> --file <path> --reason "<why>"` per finding, or `forge block close … --no-require-reviews`, which
|
|
24
|
+
7. **Review rounds, with the coder still alive.** The coder ran `forge review-file <path> --block <id> --run <r>` per file and `--wait`ed on each ticket; read the ledger (`forge ledger tail --slug <slug>`) for what is still open — a `fix_now` finding, a `pending` ticket, a file never submitted. After the coder's first COMPLETE, wait until every `review-file` ticket of the block is answered and the block's tests ran; then send ONE fix message with every failing test (name and first error line) and every open finding (by file and id). Never one finding or one file at a time. A finding that arrives after that message waits for the next round. In that round the coder fixes, runs the block's tests, re-runs `review-file` on every changed file, and prints its `TESTS` line. The message goes to the **warm coder** by `send_input`, ending in `<<<EOM>>>`; when it is over the input limit (§1), write it to a file and send its path. Round ≥ 2 re-checks only the fix hunks (`review.md` §5), and one block round is one round for each file it touched. **One ladder** (`decisions.md` §5): a stall or exhausted rounds *below L2* climbs one level — close this coder and respawn a fresh one at the higher level with the open findings and the fix history in its brief; **the respawn keeps the per-file round count**, the cap does not reset. L2 is the highest running level: a trigger at L2, or the four-round cap on a file, goes to the **L3 rung once** — a fresh L3 patch session, never this coder and never a coder at L3 — and if findings stay open after the `patch_check` round the file is `stopped: review_cap` for the human. No other path exists.
|
|
25
|
+
8. **Close** only when every changed file has a signed `review.approved` row: the re-measurement of `code.md` §5, then `forge block close <id> --run <r> --report <file>` (the coder's final report) — the transcript grep reads the Solo output. Closing with a file still unreviewed is a human decision, audited: `forge block waive <id> <finding> --run <r> --file <path> --reason "<why>"` per finding, which asks for a yes at a terminal and has no `--yes`, or `forge block close … --no-require-reviews`, which also asks. `forge block reopen <id> --file <path> --run <r> --reason "<why>"` does not close anything: it sends a fixed stopped file back to review, and it is also human-only and asks the same way. Under an active autopilot grant `block waive` and `block reopen` still prompt; only `block close --no-require-reviews` is refused. Autopilot's own waive path, `forge autopilot waive` (`by: autopilot`), needs no terminal. Print the exact command: the orchestrator never answers this prompt; the human types the command in their own terminal. You and the coder never run it. The prompt checks only that stdin and stdout are terminals; a pseudo-terminal passes it, so it does not prove a human (`security.md` §3).
|
|
26
26
|
9. **Reap after the close**: `close_process` the coder and confirm `lock_status` shows no lease before the next group. Locks: a coder that can call Solo holds `lock_acquire` on its file area while editing; coders that cannot never overlap another block.
|
|
27
27
|
|
|
28
28
|
## §4 Latency overlaps (standing)
|
|
@@ -8,8 +8,18 @@ Never selected automatically (`degraded.md` §1). Coders are detached CLI proces
|
|
|
8
8
|
|
|
9
9
|
## §2 Completion and evidence
|
|
10
10
|
|
|
11
|
-
The coder's stdout is captured; completion is the **last** `===BLOCK <id> COMPLETE===` or `===BLOCK <id> FAILED: <reason>===` line in it (a quoted sentinel inside a sentence never matches). Process exit without a sentinel is a failed block. Then the re-measurement of `code.md` §5 and `forge block close <id> --run <r> --transcript <captured stdout> --report <captured stdout>` — the transcript grep runs on the capture. A coder that hangs is killed at `--timeout` together with its process group; the pid entry is cleared.
|
|
11
|
+
The coder's stdout is captured; completion is the **last** `===BLOCK <id> COMPLETE===` or `===BLOCK <id> FAILED: <reason>===` line in it (a quoted sentinel inside a sentence never matches). Process exit without a sentinel is a failed block. Then the re-measurement of `code.md` §5 and `forge block close <id> --run <r> --transcript <captured stdout> --report <captured stdout>` — the transcript grep runs on the capture. A coder that hangs is killed at `--timeout` together with its process group; the pid entry is cleared. Fix rounds are one per block: wait for every `review-file` ticket of the block and its tests, then send ONE fix message with every failing test and every open finding; a finding that arrives later waits for the next round (`code.md` §4). Here a fix round is a new spawn, and the brief of that spawn is the one fix message.
|
|
12
12
|
|
|
13
13
|
## §3 What the coder's sandbox looks like per provider
|
|
14
14
|
|
|
15
|
-
Each provider's coder runs in its non-interactive auto mode with writes limited to the workspace where the CLI supports it
|
|
15
|
+
Each provider's coder runs in its non-interactive auto mode with writes limited to the workspace where the CLI supports it, except under `coder.sandbox.profile: full`; every closed-book role runs read-only with an empty cwd and the packet on stdin. `forge doctor` probes, per provider, that the flags exist, that a one-token call works per role builder, that the closed-book session sees no project document, and that the deny rules are honoured — read its rows before the first real spawn. Where a provider cannot load a rules file, the forbidden list is prose-only in the brief and the transcript grep is the detector (`security.md` §2).
|
|
16
|
+
|
|
17
|
+
**The Codex coder's sandbox profile (`coder.sandbox`, B93).** Only the Codex (openai) coder reads it; it has no effect on the Claude and Grok coders, whose argv does not change. `doctor` prints no `coder sandbox` row without the key, one INFO row with it, a WARN row for `full`, and a no-effect INFO row only when a Claude or Grok coder is configured.
|
|
18
|
+
|
|
19
|
+
- `workspace` (the default, and what no `coder` key means): `-s workspace-write`, today's argv. The project tree (`-C <cwd>`) is the only writable root; `$TMPDIR` and `/tmp` stay excluded, so the session `CODEX_HOME` under the run temp root cannot be written.
|
|
20
|
+
- `toolchain`: the same, plus one `--add-dir <abs>` per `coder.sandbox.add_dirs` entry right after `-s workspace-write`, so a build cache or a package store is writable too.
|
|
21
|
+
- `full`: `-s danger-full-access`, which turns the Codex sandbox off: the coder writes anywhere you can, with network as the CLI allows. Like the Claude coder, the deny rules, the transcript grep and the gates are the only limits; `doctor` prints a WARN row. The argv keeps the `$TMPDIR` and `/tmp` exclusion flags, which Codex ignores there.
|
|
22
|
+
|
|
23
|
+
**`add_dirs` entries.** Each entry is absolute or starts with `~/`, which expands against HOME, and becomes an absolute realpath: symlinks are judged by their target, a dangling symlink too. `validate` (rule `sandbox-dir-refused`) and the builder refuse `/`, the home directory itself, and an entry that is, holds or sits inside any refused dir: `~/.code-forge`, `~/.codex` (and `$CODEX_HOME`), `~/.ssh`, `~/.gnupg`, `~/.aws`, `~/.config`, `~/Library/Keychains`, the run temp root or the session `CODEX_HOME`. So `~/Library` is refused (it holds `~/Library/Keychains`), and so is `$TMPDIR` or `/tmp` when it holds the run temp root. The message names the key path (`coder.sandbox.add_dirs[<i>]`), never the value. Under `workspace` or `full` the entries are unused: `validate` warns `sandbox-dirs-unused`, and a refused entry is still refused.
|
|
24
|
+
|
|
25
|
+
The key is fixed for the run: `run reload` refuses a change, and `forge spawn --role coder --run <r>` exits 1 `sandbox-changed` when the live value differs from the run-start snapshot. Without `--run`, spawn reads the live config, so give a run's coder `--run <r>`. A run recorded before config snapshots has no run-start value; spawn then uses the live config. A Codex coder session row carries `sandbox: <profile>` only when the key is set.
|
package/skill/references/code.md
CHANGED
|
@@ -28,6 +28,7 @@ The brief file carries, in this order:
|
|
|
28
28
|
4. The project's own rules digest (`CLAUDE.md`/`AGENTS.md`, ≤ 60 lines) — left out with a `context` key, because the pasted pack carries the rules and lessons — the forbidden list verbatim (`security.md` §2), and the sentinel contract: the `TESTS` line of item 6, then `===BLOCK <id> COMPLETE===` followed by the test summary, `git diff --stat` and one `reviewed <path> ticket <ticket-id>` line per changed file, or `===BLOCK <id> FAILED: <reason>===`.
|
|
29
29
|
5. **FAILED unless** (`templates/coder-brief.md`): a final report — first attempt or fix round — that does not list the `review-file` ticket id of every changed file counts as FAILED whatever its sentinel says. Pass the report to the gate with `forge block close <id> --run <r> --report <file>`; it refuses `coder report FAILED: no review-file ticket id for <files>`.
|
|
30
30
|
6. **The green test run.** The coder runs the block's test command and prints one line that is exactly `TESTS <command> exit=<n>` just before `===BLOCK <id> COMPLETE===` — on the first attempt and again, freshly run, in every fix round. `<command>` is the command it ran (not empty); `<n>` is its exit code as a decimal 0–255 with no leading zeros. The line is matched after trimming; a line quoted inside prose, a non-numeric, signed or out-of-range `<n>`, or a leading zero does not count.
|
|
31
|
+
7. **Tests prove behaviour (B92).** The brief says: proof is behaviour, not text: never read a source file in a test to assert on its text. A test calls the code and checks what it does; reading a fixture is fine. A test file that reads a source file and asserts on its text gets a `## test-quality` section in its review packet (`review.md` §3). The section is a hint, not a finding: the reviewer checks those lines and, when the test really reads source text to assert on it, raises a `warning` (unless the file read is a fixture).
|
|
31
32
|
|
|
32
33
|
The brief never mentions `block waive`, `block reopen`, keys, or `~/.code-forge/runs/`. The coder process gets no Jev key (the System 1 model's key, `keys.jev`) and no signer key.
|
|
33
34
|
|
|
@@ -45,6 +46,8 @@ Waived or external reviews (`--no-require-reviews`, `review.external`) do not li
|
|
|
45
46
|
|
|
46
47
|
Each `review-file` ticket is served by the worker `forge run start` launched: tools first (the project's gates on that file — red ⇒ `tools_red`, no model review), then S1 `risk`, then the planned depth (`review.md` §2). You watch the run, not the coder's output: `forge status` (B64a) is the progress check — read-only (no ledger row, never the run lock), every run of this repository that is not ended, newest first, with a blank line between runs, at most 10 lines per run: `run <id> · <status> · <plan|brief> · spent $<x>[ of $<budget>]` (money with 2 decimals; `$<x>+?` when some session has no known price, `$?` when the ledger cannot be read), then per open or in-attempt block `<id> <open|attempt-<n>> <level> · <model or -> · gate <pass|fail|none>[ (<gate>)] · findings <n>` (the gate field is `gate pass`, `gate fail (<gate>)` with the failing gate's name, `gate fail` when no gate was named, or `gate none` when no gate ran; findings = the count of open review findings the close gate would refuse today), `… +<k> more blocks` as the last block line when they do not fit, and `closed <a> · stopped <b>` only when either is non-zero — printed LAST, but it keeps its place in the cap ahead of block lines (block lines are dropped first). No such run: `no active run`, exit 0. `--run <id>` shows one run (ended too); an unknown run exits 1, a usage error exits 2. `--json` prints one line, `{"runs":[…]}`, with every open block (no cap, so `more` is always 0), `unknown_usd_sessions`, and per block `gate: {result, name}` (`name` null for `pass`, for `none` and for an unnamed fail). Use it instead of reading logs or polling the coder's transcript: it never prints transcript or diff text. `forge ledger tail --slug <slug>` shows the rows themselves (`review.plan`, `review.round`, `review.done`). Idle is a wake, never proof; the sentinel is the only completion signal (`adapters/*.md`).
|
|
47
48
|
|
|
49
|
+
**One fix round per block (B94).** After the coder's first COMPLETE, wait until every `review-file` ticket of the block is answered and the block's tests ran; then send ONE fix message with every failing test (name and first error line) and every open finding (by file and id). Never one finding or one file at a time. A finding that arrives after that message waits for the next round. In that round the coder fixes, runs the block's tests, re-runs `review-file` on every changed file, and prints its `TESTS` line. The per-file cap still counts per file: one block round is one round for each file it touched (`review.md` §5).
|
|
50
|
+
|
|
48
51
|
## §5 The block gate is a re-measurement — `forge block close <id> --run <r>`
|
|
49
52
|
|
|
50
53
|
The coder's report is a claim sheet. The gate reproduces each number in the measurement export (`proof.md` §3), and so do you when you read the report:
|
|
@@ -70,11 +73,13 @@ Without the flag the tree stays. On a block without a worktree the flag is a usa
|
|
|
70
73
|
|
|
71
74
|
**When a file stops** (`review_cap`, `l3_patch_exhausted`, `split_required`, …), the block cannot close. The stop's `exits` name the ways out. Take them in this order; `review.md` §5 gives each one's rules and refusals:
|
|
72
75
|
|
|
73
|
-
1. **Fix, then `block reopen`.** When the fix belongs in the stopped file, have it fixed. Then
|
|
76
|
+
1. **Fix, then `block reopen`.** When the fix belongs in the stopped file, have it fixed. Then print the exact command, `forge block reopen <id> --file <path> --run <r> --reason <text>` with the human's reason, and the human runs it in their own terminal, exactly like `block waive`. Never run it yourself, and never for a coder (it is on the coder forbidden list). After it, send the coder's `review-file` again: the next round is a full round 1 when the content differs from what the stop reviewed; on the stop's own content the stop repeats. A `reopen-same-content` refusal means the content is still what the stop reviewed: offer the waive instead.
|
|
74
77
|
2. **`review-file --answers`.** Use it when the fix belongs in another owned file (a missing test, say). The coder fixes it there and runs `forge review-file <that file> --block <id> --run <r> --answers <ids> --of <source>`. A fresh reviewer judges the fix. It needs no human step.
|
|
75
|
-
3. **`block waive`.**
|
|
78
|
+
3. **`block waive`.** The human waives each open finding: print the exact `forge block waive <id> <finding-id> --run <r> --file <path> --reason <text>` command per finding, and the human runs it in their own terminal. A file whose findings are all waived or answered closes with a signed `gate.review_waived_all` row.
|
|
76
79
|
4. **Re-decompose.** The human decides, and the architect plans new blocks.
|
|
77
80
|
|
|
81
|
+
**`block waive` and `block reopen` are run by the human, never by you (B80).** Both ask for a yes at a terminal. You print the exact command; the human runs it in their own terminal. The rule: a coder or the orchestrator never runs it and never answers its prompt. Separately, as a fact of the verb: when stdin or stdout is not a terminal (a pipe, or a process started with no terminal) it exits 2 with `block <verb> needs the human at a terminal to confirm; nothing was recorded (there is no --yes)` and records nothing. There is no `--yes`, and an autopilot grant does not change this (the delegate waives through `forge autopilot waive`). The prompt checks only that stdin and stdout are terminals; a pseudo-terminal passes it, so it does not prove a human (`security.md` §3).
|
|
82
|
+
|
|
78
83
|
`block attempt` never resets review state: it only raises the attempt counter, and a stopped file stays stopped. Use `review-file --fresh` only after a result that never counted (`stale`, `unavailable`, or `refused` with `other-run`, `context-unreadable` or `worktree-*`); it never re-runs a counted round. A missing state file needs no exit: the worker first rebuilds it from its ledger anchor (`review.state_restored`). Only a state whose anchor has no `state_body` is `lost` (`review-state-lost`); that one needs the human's `block reopen`, on its current content. A `review-state-tampered` file has none of these exits: only the owner's `block close --no-require-reviews` gets past it.
|
|
79
84
|
|
|
80
85
|
`--no-require-reviews` is the human's way out, written as its own audit row. The human runs it themselves at a terminal: it asks for a yes, has no `--yes`, exits 2 unless stdin and stdout are both terminals, lifts `no_changes` and is refused while an autopilot grant is active — so never run it for them; tell them the command when they say so in chat. The owner's other way out is `review.external` in the config before `run start` (`review.md` §1): the gate then skips the review rows and the report's ticket ids and writes `gate.reviews_waived {by: config}`; every other check, `no_changes` included, still applies. A finding counts only under the gate's own configuration; a tautology (an expected value computed by the code under test) is a finding. Reproduce disclosures: a coder that reports its own rule break did the hard part.
|
|
@@ -97,7 +102,7 @@ Both `autonomy` keys default to false. Whatever they say, a block's branch is ha
|
|
|
97
102
|
2. The block's test command green, case count read from the runner and written to the block record.
|
|
98
103
|
3. Type check and lint green under the project's gates.
|
|
99
104
|
4. Every changed or new file reviewed by a fresh L2 session and converged under the convergence rule (≤ `review.max_rounds_per_file`, or a human decision on record).
|
|
100
|
-
5. Red→green rows for every new test; proof rows per tier (`proof.md`).
|
|
105
|
+
5. Red→green rows for every new test; proof rows per tier (`proof.md`). A test that is not a `node:test` file (PHPUnit, Pest, pytest, Swift) is proven through the `proof.runners` entry the owner set before the run (`proof.md` §2): `forge proof red-green <id> --run <r> --test <file>::<case>`, with the case as the runner itself names it. The `::<case>` is needed only when the runner's argv has `{test_filter}`. Without `{test_filter}`, a case only labels the proof: `{test_id}` becomes `<file>::<case>`, and `{test_file}` still runs the whole file. A coder never edits `proof.runners`.
|
|
101
106
|
6. Forecast vs actual (cases and lines) recorded; misses attributed (architect-added scope vs coder under-forecast) — a finished green block is accepted and recorded either way.
|
|
102
107
|
7. One commit for the block; the draft PR (when the project uses PRs) carries the evidence.
|
|
103
108
|
8. Every artifact that proves something is committed, not left in a scratch dir; no claim shipped that the evidence does not carry.
|
|
@@ -4,11 +4,11 @@
|
|
|
4
4
|
|
|
5
5
|
Every row has the run, the block, the event and a timestamp; the ones the block gate relies on carry `mac` (`security.md` §3). Useful fields: `level`, `provider`, `model`, `effort`, `fallback_step` (every spawn), `role ∈ {coder, reviewer, judge, s2, author, facts}`, `trigger` (escalations), `source ∈ {jev, rules, s2, s2-fallback, shadow}` (decisions), `depth_unconstrained` / `depth_chosen` / `degrade_step` and `context_mode ∈ {whole, hunks, minimal, recheck}` (reviews), `round` and `kind ∈ {full, recheck, patch_check, sweep}` (rounds), `isolation ∈ {export, lock}` (proof), `l3_patch`, `l3_fallback`, `overspend`. **Tokens are facts, dollars are estimates**: `tokens_source: reported` when the CLI's JSON carries usage, else `estimated`. File paths in rows are relative to the repository root.
|
|
6
6
|
|
|
7
|
-
Events you will read most: `run.start`, `run.mode {mode: brief, plan: none}` (a brief-mode run, `plan.md` §6), `review.external {required, reviewer}` (the owner's outside reviewer, `review.md`), `run.stop {reason}`, `run.end`, `dispatch` (`test_files` when `block open --tests` named them), `gate.red` and `gate.done` (with `scope: related` + `test_count` or `scope: full` + `reason: threshold | no_tests` when a gate has the `{test_files}` item, `no_tests` also without `--run/--block`; with `--run/--block` they name `run` and `block`, and for a block with a worktree also `worktree: <path>` (the gates ran there); a gate with env names only `env_keys` in its result, `proof.md` §3), `review.plan`, `review.round`, `review.done`, `review.approved`, `review.unavailable {reason}`, `review.late_finding`, `review.waived {by: human}`, `review.state {block, file, seq, state_sha, l3_rung_used, state_body}` (signed; the fix-loop state anchor, `review.md` §5 step 8; `state_body` carries `reviewed_content_sha` and `reviewed_content_blob`, the git blob id of the reviewed text in the reviewed tree's object store, or null when there was no reviewed text or no content root, or the write failed; when the signed body is over 64 KB, `state_omitted: "too_large"` replaces `state_body`, and a lost state file then refuses as `review-state-lost`), `review.state_restored {block, file, seq, from_seq}` (signed; a lost state file was rebuilt from the anchor at `from_seq` and saved as `seq`), `review.reopened {block, file, from: stopped | lost, stop_reason, at_seq, from_hash, to_hash, reason, by: human}` (signed; B72: the human's `block reopen` of a stopped or lost file — `at_seq` the file's latest `review.state` anchor seq, `from_hash` the content the stop reviewed (null for `lost`), `to_hash` the current content, `stop_reason` null for `lost`; at most one row per `at_seq`, the verb refuses a second one for other content as `reopen-already-recorded`; the worker starts a fresh round-1 state for it once, saved at `at_seq + 1` — for `from: stopped` only when the current content still differs from the stop's content (the state's `stop_content_hash`) and from `from_hash`, else the stop repeats; round 1 is a full review (`reviewed_content` null) and `l3_rung_used` is kept, so the file gets up to `review.max_rounds_per_file` rounds at L2 again; the worker ignores a row that does not verify, of another run, not `by: human` or at another seq), `review.answered {block, file, content_hash, finding, by_file, by_hash, ticket}` (signed; B71: an answers ticket's reviewer found that `by_file` at `by_hash` resolves the open finding `finding` of `file` at `content_hash` — exactly one row per (finding, `by_hash`, ticket): a retried ticket writes no duplicate, only the rows a failed write left out; the close counts it only while both hashes are current, and ignores a row without a valid run-key MAC or from another run; an answer may clear a critical and never approves `by_file`, `review.md` §5 step 5), `gate.reviews_waived {by: human | config}` (`by: config` from `review.external`), `gate.review_waived_all {block, file, content_hash, findings, answered?, by: human | autopilot | null}` (signed; a file with no approval closed as `reviewed_by_waiver`: a signed `review.round` at its current hash, and every finding of the latest hash-bound refusal and every open `review.cap` id cleared by an accepted waiver or a still-current signed `review.answered` row (B71) — a file cleared only by answers is `reviewed_by_waiver` too. `findings` are the sorted WAIVED ids and `answered` (B71, only when there is one) the sorted ids `review.answered` rows cleared; `by: human` only when every clearing waiver is by a human, else `autopilot`, and `by: null` only when every id was answered (`findings: []`) — a reader must handle null. One row per such file just before `block.close`, none on a refused close, `review.md` §5 step 5), `coder.tests {block, command, exit}` (signed; under `context.require_coder_tests` the close read the coder's `TESTS <command> exit=0` line from `--report`, written with the close rows just before `block.close` and only when the block closes, `code.md` §3), `decision {question: lane, source: rules}` (a brief-mode `block open` without `--level`), `worktree.created {block, path, branch, base}` (`block open` under `proof.isolation: worktree` made the block's tree, written before `dispatch`; a re-open that reuses it writes none), `worktree.removed {block, path, branch}` and `worktree.remove_failed {block, path, reason}` (after `block close --remove-worktree`; the branch is always kept, and on a failure the tree stays, `proof.md` §3), `context.pack {block, path, sha256, tokens, lines, cut}` (signed; `forge context pack` wrote a new pack for the block — the same pack again, same sha256 as the block's last row, writes none; `code.md` §3), `hook.ran {block, hook: on_block_end, argv0, exit, signal, timed_out, duration_ms}` (signed; one row per block end with `hooks.on_block_end` set — also when the hook fails, times out or is not found; none when the key is absent; `argv0` is the basename only. Exited: `exit` its code, `signal` null. Killed by a signal: `exit` null, `signal` its name. Timed out: `exit` null, `signal` `SIGKILL`, `timed_out` true. Not found (argv[0] ENOENT) or any other spawn error (a missing cwd, EACCES): `exit` null, `signal` null, `timed_out` false; `code.md` §3), `outside_review {run, pr, rounds, findings, link (or null), blocks}` (signed; `forge ledger outside-review`, §4 — the latest row per PR wins in every reader), `review.budget`, `forecast.cases_exceeded`, `proof`, `outcome`, `block.rebase`, `block.orphan`, `ledger.tamper`, `worker.replaced`, `facts.built`, `tmp.swept`.
|
|
7
|
+
Events you will read most: `run.start`, `run.mode {mode: brief, plan: none}` (a brief-mode run, `plan.md` §6), `review.external {required, reviewer}` (the owner's outside reviewer, `review.md`), `run.stop {reason}`, `run.end`, `dispatch` (`test_files` when `block open --tests` named them), `gate.red` and `gate.done` (with `scope: related` + `test_count` or `scope: full` + `reason: threshold | no_tests` when a gate has the `{test_files}` item, `no_tests` also without `--run/--block`; with `--run/--block` they name `run` and `block`, and for a block with a worktree also `worktree: <path>` (the gates ran there); a gate with env names only `env_keys` in its result, `proof.md` §3), `review.plan`, `review.round`, `review.done`, `review.approved`, `review.unavailable {reason}`, `review.late_finding`, `review.waived {by: human}` (the human's `block waive`, run in their own terminal after a yes, B80; no row on a "no", a cancel, no terminal, `--yes` (a usage error), or a pre-check refusal (a malformed or control-character finding id or `--file`, an unknown block, a file the block does not own, an unknown finding). The prompt checks only that stdin and stdout are terminals; a pseudo-terminal passes it, so it does not prove a human (`security.md` §3).), `review.state {block, file, seq, state_sha, l3_rung_used, state_body}` (signed; the fix-loop state anchor, `review.md` §5 step 8; `state_body` carries `reviewed_content_sha` and `reviewed_content_blob`, the git blob id of the reviewed text in the reviewed tree's object store, or null when there was no reviewed text or no content root, or the write failed; when the signed body is over 64 KB, `state_omitted: "too_large"` replaces `state_body`, and a lost state file then refuses as `review-state-lost`), `review.state_restored {block, file, seq, from_seq}` (signed; a lost state file was rebuilt from the anchor at `from_seq` and saved as `seq`), `review.reopened {block, file, from: stopped | lost, stop_reason, at_seq, from_hash, to_hash, reason, by: human}` (signed; B72: the human's `block reopen` of a stopped or lost file, run in their own terminal after a yes (B80; no row on a "no", a cancel, no terminal, `--yes` (a usage error), or a pre-check refusal (a malformed or control-character `--file`, an unknown or not-open block, a file the block does not own)) — `at_seq` the file's latest `review.state` anchor seq, `from_hash` the content the stop reviewed (null for `lost`), `to_hash` the current content, `stop_reason` null for `lost`; at most one row per `at_seq`, the verb refuses a second one for other content as `reopen-already-recorded`; the worker starts a fresh round-1 state for it once, saved at `at_seq + 1` — for `from: stopped` only when the current content still differs from the stop's content (the state's `stop_content_hash`) and from `from_hash`, else the stop repeats; round 1 is a full review (`reviewed_content` null) and `l3_rung_used` is kept, so the file gets up to `review.max_rounds_per_file` rounds at L2 again; the worker ignores a row that does not verify, of another run, not `by: human` or at another seq), `review.answered {block, file, content_hash, finding, by_file, by_hash, ticket}` (signed; B71: an answers ticket's reviewer found that `by_file` at `by_hash` resolves the open finding `finding` of `file` at `content_hash` — exactly one row per (finding, `by_hash`, ticket): a retried ticket writes no duplicate, only the rows a failed write left out; the close counts it only while both hashes are current, and ignores a row without a valid run-key MAC or from another run; an answer may clear a critical and never approves `by_file`, `review.md` §5 step 5), `gate.reviews_waived {by: human | config}` (`by: config` from `review.external`), `gate.review_waived_all {block, file, content_hash, findings, answered?, by: human | autopilot | null}` (signed; a file with no approval closed as `reviewed_by_waiver`: a signed `review.round` at its current hash, and every finding of the latest hash-bound refusal and every open `review.cap` id cleared by an accepted waiver or a still-current signed `review.answered` row (B71) — a file cleared only by answers is `reviewed_by_waiver` too. `findings` are the sorted WAIVED ids and `answered` (B71, only when there is one) the sorted ids `review.answered` rows cleared; `by: human` only when every clearing waiver is by a human, else `autopilot`, and `by: null` only when every id was answered (`findings: []`) — a reader must handle null. One row per such file just before `block.close`, none on a refused close, `review.md` §5 step 5), `coder.tests {block, command, exit}` (signed; under `context.require_coder_tests` the close read the coder's `TESTS <command> exit=0` line from `--report`, written with the close rows just before `block.close` and only when the block closes, `code.md` §3), `decision {question: lane, source: rules}` (a brief-mode `block open` without `--level`), `worktree.created {block, path, branch, base}` (`block open` under `proof.isolation: worktree` made the block's tree, written before `dispatch`; a re-open that reuses it writes none), `worktree.removed {block, path, branch}` and `worktree.remove_failed {block, path, reason}` (after `block close --remove-worktree`; the branch is always kept, and on a failure the tree stays, `proof.md` §3), `context.pack {block, path, sha256, tokens, lines, cut}` (signed; `forge context pack` wrote a new pack for the block — the same pack again, same sha256 as the block's last row, writes none; `code.md` §3), `hook.ran {block, hook: on_block_end, argv0, exit, signal, timed_out, duration_ms}` (signed; one row per block end with `hooks.on_block_end` set — also when the hook fails, times out or is not found; none when the key is absent; `argv0` is the basename only. Exited: `exit` its code, `signal` null. Killed by a signal: `exit` null, `signal` its name. Timed out: `exit` null, `signal` `SIGKILL`, `timed_out` true. Not found (argv[0] ENOENT) or any other spawn error (a missing cwd, EACCES): `exit` null, `signal` null, `timed_out` false; `code.md` §3), `outside_review {run, pr, rounds, findings, link (or null), blocks}` (signed; `forge ledger outside-review`, §4 — the latest row per PR wins in every reader), `review.budget`, `forecast.cases_exceeded`, `proof`, `outcome`, `block.rebase`, `block.orphan`, `ledger.tamper`, `worker.replaced`, `facts.built`, `tmp.swept`.
|
|
8
8
|
|
|
9
9
|
## §2 `forge report --slug <slug> [--json] [--export <dir>]`
|
|
10
10
|
|
|
11
|
-
Terminal output in this version. Sections: cost per block — completed and still open, with `completed` and the count of unknown-price sessions (per level and lane, split review / coder / S1+S2 / facts / proof time) · lane distribution · escalations per block with `trigger` · S1 calls per block · S2 rate and overrule rate per question · the review budget table with `missed_after_degrade` per degrade step · review depth and context-mode distribution, findings by category and severity · fix rounds per file, rounds that stalled, files stopped at `review_cap` · `review.unavailable` by reason · proof time per block · forecast vs actual lines and cases with the measured lines-per-case · blocks stopped at L3 · `overspend` and `l3_fallback` counts · runs stopped for `no_engine` · `outside_review` (14th, last): per PR sorted by number, `{pr, rounds, findings, link, blocks, code_forge_usd, unknown_usd_sessions}` — `code_forge_usd` sums the `cost_per_block` totals of the PR's blocks in its run, rounded as `cost_per_block` rounds; unknown-price sessions count 0 and are counted; a PR with no blocks shows 0 and 0 unknown; `[]` with no row; in the terminal the lines print under `== outside_review ==`: `outside review: none`, or one line per PR (`PR #12 · 3 outside rounds · 7 findings · code-forge $3.75 (+1 unknown) · B1, B2`). `--export <dir>` writes one JSON file per section — the durable dataset a future UI reads. After the sections the terminal output prints the spend per run (open blocks included) and the total. `report` checks no row's signature in any section (an unsigned row counts like a signed one); shadow rows (`source: shadow`) are excluded from every section.
|
|
11
|
+
Terminal output in this version. Sections: cost per block — completed and still open, with `completed` and the count of unknown-price sessions (per level and lane, split review / coder / S1+S2 / facts / proof time) · lane distribution · escalations per block with `trigger` · S1 calls per block · S2 rate and overrule rate per question · the review budget table with `missed_after_degrade` per degrade step · review depth and context-mode distribution, findings by category and severity · fix rounds per file (one block round is one round for each file it touched, `review.md` §5), rounds that stalled, files stopped at `review_cap` · `review.unavailable` by reason · proof time per block · forecast vs actual lines and cases with the measured lines-per-case · blocks stopped at L3 · `overspend` and `l3_fallback` counts · runs stopped for `no_engine` · `outside_review` (14th, last): per PR sorted by number, `{pr, rounds, findings, link, blocks, code_forge_usd, unknown_usd_sessions}` — `code_forge_usd` sums the `cost_per_block` totals of the PR's blocks in its run, rounded as `cost_per_block` rounds; unknown-price sessions count 0 and are counted; a PR with no blocks shows 0 and 0 unknown; `[]` with no row; in the terminal the lines print under `== outside_review ==`: `outside review: none`, or one line per PR (`PR #12 · 3 outside rounds · 7 findings · code-forge $3.75 (+1 unknown) · B1, B2`). `--export <dir>` writes one JSON file per section — the durable dataset a future UI reads. After the sections the terminal output prints the spend per run (open blocks included) and the total. `report` checks no row's signature in any section (an unsigned row counts like a signed one); shadow rows (`source: shadow`) are excluded from every section.
|
|
12
12
|
|
|
13
13
|
## §3 Live views
|
|
14
14
|
|
|
@@ -9,7 +9,7 @@ Tier per file = `high` when `risk ≥ 2`, or a `proof.tiers.high.paths` match, o
|
|
|
9
9
|
| light | red→green recommended for every new or changed test; **optional at the gate** | the coder or orchestrator runs the runner per test (§2) |
|
|
10
10
|
| high | red→green **required at the gate** for every changed high-tier file, plus the deeper **review** depth of `review.md` §2 (two lenses and a judge) | as above; `block close` refuses `unproven <file>` |
|
|
11
11
|
|
|
12
|
-
**`forge block close` enforces the high tier:** tier inputs are `proof.tiers.high.paths` (the union of the config at the block's base and the current one), the recorded `risk`, and `security_sensitive`; a row naming the file whose MAC fails makes it high. Each changed high-tier file needs a signed `proof` row of this run and block with `step: red-green`, `proven: true`, `red_kind: assertion` whose `covers` list names the file (`revert` covers the sources it put back; `assertion-deletion` covers only its test), else `unproven <file>`; only the human's `forge block waive <id> proof --file <path>` clears it.
|
|
12
|
+
**`forge block close` enforces the high tier:** tier inputs are `proof.tiers.high.paths` (the union of the config at the block's base and the current one), the recorded `risk`, and `security_sensitive`; a row naming the file whose MAC fails makes it high. Each changed high-tier file needs a signed `proof` row of this run and block with `step: red-green`, `proven: true`, `red_kind: assertion` whose `covers` list names the file (`revert` covers the sources it put back; `assertion-deletion` covers only its test), else `unproven <file>`; only the human's `forge block waive <id> proof --file <path>` clears it (it asks for a yes at a terminal: the orchestrator prints the exact command and the human runs it in their own terminal; there is no `--yes`, `review.md` §5).
|
|
13
13
|
|
|
14
14
|
**Mutation testing is not part of this product.** The question whether tool-made mutants on changed lines (Infection, Stryker) should be a user-facing proof tier was decided: **cut**. There is no `proof mutate`, no nightly mutation run, no `min_msi` floor, and no mutation-tool key in `.code-forge.yml`; a consumer that runs a mutation tool does so outside code-forge and never beside a coder's test run in the same tree. What replaces it as proof is the rule itself: one exact test per acceptance clause plus one negative or boundary where the clause has a failure mode — assertions that say **which** element and **how many**, never merely whether something exists — and the isolated per-file review with the convergence rule, which on this package's own build found the defects green suites had missed.
|
|
15
15
|
|
|
@@ -21,6 +21,75 @@ Tier per file = `high` when `risk ≥ 2`, or a `proof.tiers.high.paths` match, o
|
|
|
21
21
|
|
|
22
22
|
The runner works in the export (§3) or under the lock with a journal: it refuses when a journal already exists (an earlier run was interrupted) → journals every source file the test covers (bytes, sha256, mode — or `existed: false`) → writes the **base** version of each (`git show <base>:<path>`, never `stash` or `checkout --`); **a file the block added has no base version, so it is removed for the red run** → runs the filtered tests → **parses** the output: an assertion failure ⇒ `RED`; a fatal, compile or import error ⇒ `RED_INVALID`, which proves nothing (a removed module that the test imports is the usual cause — such a test needs an assertion that fails against the base, not an import that crashes) → restores every journaled file byte for byte, **verifies each sha256**, deletes the journal (in a `finally`, so an ordinary error restores too) → when the red step was `RED`, runs again ⇒ `GREEN`. A characterization test (`proof: characterization`) is proven by assertion deletion. The ledger `proof` row records `mechanism` and `red_kind`; one `RED` + `GREEN` pair per proven test. SIGTERM mid-run leaves the journal; `forge proof restore <block> --run <r>` restores the exact hashes and refuses when no journal exists.
|
|
23
23
|
|
|
24
|
+
**How a test report is read (B86).** A `node:test` file is still classified from its console text. A runner that writes a JUnit XML report (PHPUnit, Pest, pytest, `swift test --xunit-output`) is classified from that report instead, because console text changes with the environment and is cut at 10 MiB. The reader fails closed: a report that is missing, empty, over 8 MiB, not UTF-8 or malformed, that has a `<!DOCTYPE` or `<!ENTITY` (no entity is ever expanded) or a character that is not an XML character, or that has an element other than `testsuites`, `testsuite`, `testcase`, `failure`, `error`, `skipped`, `system-out`, `system-err`, `properties`, `property` is unreadable. Each test's id is `classname.name` (or `name`). A `<failure>` is an assertion only when its `type` names one, or when it has no `type` and its `message` names one. The names: `AssertionError`, `ERR_ASSERTION`, `ExpectationFailedException`, `AssertionFailedError`, `Failed asserting that`, `XCTAssert…`, `XCTFail`, `Expectation failed` (Swift Testing), and pytest's `assert …`. Any other `<failure>` and every `<error>` is an error. One exception: `swift test --xunit-output` writes every failed XCTest as `<failure message="failure"></failure>` (Swift 6.3; older and Linux builds write `message="failed"`), with no type and no cause. Such a bare failure (no `type`, message exactly `failure` or `failed`, an empty body; a body of only whitespace, as text or CDATA, counts as empty) counts as an assertion only when the console text has XCTest's assertion line for that same test (`<file>:<line>: error: -[Module.Suite test] : XCTAssert… failed` on macOS, `Suite.test : XCTAssert… failed` on Linux, or XCTFail's `: failed - <message>`) and shows no Swift crash (`Fatal error`, `Precondition failed`, `Assertion failed`, `Unexpectedly found nil`, `signal code`, `Illegal instruction`, `Segmentation fault`); a thrown error (`: failed: caught error`), a missing line or a crash leaves it an error. This table applies only to a runner that writes a report; a `node:test` file is still classified from its console text, as above. The RED step, first row that applies:
|
|
25
|
+
|
|
26
|
+
| when | verdict | `red_kind` |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| exit 0, no signal | `NOT_RED` | `none` |
|
|
29
|
+
| a timeout or a signal | `RED_INVALID` | `fatal` |
|
|
30
|
+
| the console text shows an import, compile or fatal error | `RED_INVALID` | `import` / `compile` / `fatal` |
|
|
31
|
+
| no report, or a report is unreadable | `RED_INVALID` | `error` (no readable test report) |
|
|
32
|
+
| no test ran (zero tests, or only skipped ones) | `RED_INVALID` | `error` (the filter ran no test) |
|
|
33
|
+
| any test errored, or failed without an assertion | `RED_INVALID` | `error` |
|
|
34
|
+
| at least one test failed on an assertion | `RED` | `assertion` |
|
|
35
|
+
| the run failed, but the report has no failure | `RED_INVALID` | `error` |
|
|
36
|
+
|
|
37
|
+
GREEN needs exit 0, a readable report, at least one test that ran, no failed or errored test, and every test that failed at RED present and passed; anything else is `NOT_GREEN`. The result keeps names only: `failed` is the count, and `failed_tests` lists at most 20 test ids of at most 200 characters each, never a failure message.
|
|
38
|
+
|
|
39
|
+
**Runners (B87).** A test file that is not a `node:test` file is proven through a `proof.runners` entry. Without one, `proof red-green` refuses it (exit 1): `only node:test files (.mjs, .cjs, .js) have a filtered test command in this version — add a proof.runners entry for it (see proof.md §2)`.
|
|
40
|
+
|
|
41
|
+
```yaml
|
|
42
|
+
proof:
|
|
43
|
+
runners:
|
|
44
|
+
- match: ["tests/**/*Test.php"]
|
|
45
|
+
adapter: junit
|
|
46
|
+
argv: [php, artisan, test, "{test_file}", --filter, "{test_filter}", --log-junit, "{report}"]
|
|
47
|
+
env: {DB_CONNECTION: sqlite, DB_DATABASE: ":memory:"}
|
|
48
|
+
- match: ["tests/**/test_*.py"]
|
|
49
|
+
adapter: junit
|
|
50
|
+
argv: [pytest, "{test_id}", --junitxml, "{report}"]
|
|
51
|
+
timeout_s: 900
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
- **Keys.** `match`: a non-empty list of globs over repo-relative paths. `adapter`: `junit` (any runner that writes a JUnit XML report), `swift` (`swift test`: the `junit` rules plus the Swift rules below) or `xcodebuild` (`xcodebuild test`, read from its result bundle; it needs an `xcode` object, Xcode below). `argv`: the command as an argv array, never a shell string. `env` (optional): variables added to the runner's environment, which never inherits a `GIT_*` variable. A value is never recorded: the signed row and the JSON output both hold only the names (`env_keys`). When the runner's own output echoes a value of 8 or more characters, code-forge masks it in every line it prints, as for gate env (below). `timeout_s` (optional): 60 to 7200 seconds per step, default 600.
|
|
55
|
+
- **Tokens.** An argv item that is exactly one of these is replaced: `{test_file}` the repo-relative test path; `{test_id}` `<file>::<case>`, or `<file>` with no case (a pytest node id); `{test_filter}` the case, regex-escaped (for `swift`, anchored as `(?:^|[./])<case>(?![^/])`, and with no case the file's base name, regex-escaped and anchored as `\.<name>/`); `{report}` the path where the runner must write its JUnit report. The `xcodebuild` adapter has its own tokens instead (Xcode, below). A token is a whole item, never part of one (`--log-junit={report}` is refused; give `--log-junit`, `"{report}"`), and never `argv[0]`. `validate` refuses: an empty `proof.runners: []` (leave the key out for no runner); an entry key it does not know (a typo such as `timeout`); an unknown adapter; a missing or empty `match`; a missing or empty `argv`, an `argv` given as one shell string, or an argv item that is not a string; a NUL character in an argv item or an env value; a token inside a longer item, a token as `argv[0]`, and a token-shaped item the adapter does not list (with an unknown adapter, one no adapter lists); a `junit` or `swift` argv without `{report}`, or without a test token (`{test_file}`, `{test_id}` or `{test_filter}`: a proof must name its test); with an unknown adapter, an argv without a report item of any adapter (`{report}` or `{result_bundle}`) or without a test token of any adapter (`{test_file}`, `{test_id}`, `{test_filter}` or `{only_testing}`), so a misspelt adapter still shows every argv error; the `xcodebuild` checks below; a `timeout_s` out of range.
|
|
56
|
+
- **Selection.** The runner list is read from the run's config snapshot, never from the live `.code-forge.yml`: `run start` freezes it, `run reload` refuses to change it, and while it is set no block may own `.code-forge.yml`. A coder cannot change the command that proves its own work. The first entry whose `match` covers the `--test` file runs it. No entry matches and the file is `.mjs`, `.cjs` or `.js`: the built-in `node:test` command, as before. A run started without a snapshot uses no runner.
|
|
57
|
+
- **Fail closed.** A `junit` runner whose argv has `{test_filter}` needs a case: `--test <file>` with no `::case` is a usage error (exit 2, no row): give `--test <file>::<case>`. A `swift` runner accepts a missing case: it then filters by the file's suite (Swift, below). An empty case (`<file>::`), and a test path or a case that starts with `-`, are refused (exit 2, no row). A snapshot list that is not a list, is empty, or has an entry that fails the checks above refuses the proof (`runner-invalid`, exit 1, no row): the entry is never run, never skipped, and the file never falls through to another entry or to `node:test`. A matched entry that has no runner hook is a usage error instead (exit 2, no row, no precheck).
|
|
58
|
+
- **The run.** Each step (RED, GREEN) gets a fresh report dir under the run's temp root, outside the work dir; the runner runs in the work dir (the export, or the tree under the lock) and the dir is deleted after its report is read. RED is classified from the report (the table above), GREEN by the GREEN rule above; a GREEN step that throws is `NOT_GREEN`, and the row is still written. A changed file that a runner's `match` covers is a test, never a source: it is not reverted. The journal, restore and signal rules are the same as for `node:test`.
|
|
59
|
+
- **Swift (B88).** The `swift` adapter runs `swift test` through SwiftPM:
|
|
60
|
+
|
|
61
|
+
```yaml
|
|
62
|
+
proof:
|
|
63
|
+
runners:
|
|
64
|
+
- match: ["Tests/**/*.swift"]
|
|
65
|
+
adapter: swift
|
|
66
|
+
argv: [swift, test, --parallel, --filter, "{test_filter}", --xunit-output, "{report}"]
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`swift test --filter` searches its regex in each test's `Module.Suite/test` id, unanchored. `{test_filter}` is the case, regex-escaped and anchored: it starts the id or follows a `.` or `/`, and it ends the id or is followed by a `/` (`--test Tests/KitTests/ClampSwiftTests.swift::ClampSwiftTests/capsHigh()` gives `(?:^|[./])ClampSwiftTests/capsHigh\(\)(?![^/])`), so it never selects a near-name suite (`MoreClampSwiftTests/capsHigh()`) or a longer test name (`testCapsHigh` never selects `testCapsHighToo`); give the case as `Suite/test`, because a bare test name (`::capsHigh()`) selects that test in every suite. The end is not `$`: SwiftPM 6.3.3 matches a Swift Testing test against an id with more after the test name, so `$` selects no Swift Testing test. A `swift` runner accepts a missing case: `{test_filter}` is then the file's base name, regex-escaped and anchored with a `.` before it and a `/` after it (`ClampXCTests.swift` gives `\.ClampXCTests/`), so it selects that suite only and never a near-name suite such as `BarClampXCTests` or `ClampXCTestsHelpers`; name each XCTest class or Swift Testing suite after its file. A run that selects no test (a suite not named after its file) exits 0 and writes both reports with zero tests (SwiftPM warns "No matching test cases were run"): RED is `NOT_RED` (exit 1, no proof; like every verdict, it still writes its `red-green` row, with `green: null` and `proven: false`); a GREEN run that selects no test is `NOT_GREEN`. Keep `--parallel`: without it SwiftPM 6.3 writes no XCTest report at `{report}`, and a step that wrote the Swift Testing report but no XCTest report is `RED_INVALID error` (the reason names `--parallel`) and never `GREEN`. The adapter reads every `.xml` file in the step's report dir: the XCTest report at `{report}` and the Swift Testing report SwiftPM writes next to it as `<report stem>-swift-testing.xml` (`report-swift-testing.xml`); a failure in either counts. A missing or unreadable report proves nothing: at RED it is `RED_INVALID error`, with no `failed` count and no `failed_tests`; at GREEN it is `NOT_GREEN`, also when every test that failed at RED passed in the other report (GREEN needs every report file RED had). A test that crashes, such as one that hits `fatalError`, leaves the Swift Testing report cut off: `RED_INVALID error`. A failed run that wrote no report at all and whose console shows a `swiftc` error (`<file>.swift:<line>:<col>: error:`, `error: emit-module command failed`, `error: compile command failed`) is `RED_INVALID compile`; an XCTest assertion line (`<file>.swift:<line>: error: -[…]`, no column), a crash (`<file>.swift:<line>: Fatal error: …`) and SwiftPM's closing `error: fatalError` are not compile errors, and no report without a compile line stays `RED_INVALID error`. **A test of a new API needs the human waiver:** a test that calls a function, type or method the block added does not compile at base, so `revert` gives `RED_INVALID compile`, never `RED`. For a high-tier file the only way past the gate is the human's `forge block waive <id> proof --file <path>` (D3), run by the human in their own terminal; prove the behaviour with a test of an API that already exists at base where you can. For the block gate, a SwiftPM `gates.test` is `test: {argv: [swift, test, --parallel]}`.
|
|
70
|
+
- **PHP (B89).** PHPUnit and Pest are proven through the `junit` adapter with the Laravel runner above, `[php, artisan, test, "{test_file}", --filter, "{test_filter}", --log-junit, "{report}"]`: `php artisan test` passes the file, `--filter` and `--log-junit` through to PHPUnit and to Pest (verified live, §4), so a Laravel app needs no `vendor/bin/phpunit` or `vendor/bin/pest` runner. Give the case as the test's own name: the PHPUnit method (`--test tests/Unit/ClampTest.php::test_caps_high`) or the Pest description (`--test tests/Unit/ClampTest.php::caps high`). `{test_filter}` is the case, regex-escaped. PHPUnit and Pest read `--filter` as a regex and search it in the test name unanchored, so a longer name that contains the case runs too, and must then pass at GREEN. Because the case is escaped, a Pest description with `( ) [ ] . + ? /` is safe: it selects that test and not a near name (checked on Pest 4.7.8 with `clamp (high) [edge] a.b + c? x/y`). A description with `#` and digits or with `@` selects no test (PHPUnit reads those as a data-set name). **A PHP run that selects no test is `RED_INVALID error`, where a Swift one is `NOT_RED`,** because of the exit code: PHPUnit and Pest print "No tests found.", exit 1 and write an empty report (`<testsuites/>`; seen live on Pest 4.7.8 through artisan and on PHPUnit 12.5.33 run directly in the Pest app, in both console forms). The RED classifier (`classifyReport`) checks exit 0 first: exit 0 is `NOT_RED`, which is what `swift test` gives, since it exits 0 when no test matches. A non-zero exit goes on: the console holds no import, compile or fatal line, the report is readable, and no test in it ran, so the classifier stops at its "the filter ran no test" row: `RED_INVALID error`. Neither is a proof. The report's ids are `Tests.Unit.ClampTest.test_caps_high` (PHPUnit) and `Tests.Unit.ClampTest.caps high` (Pest); in a filtered run the root suite is named after the test class (in a full-config run after the `phpunit.xml` path), and the reader does not depend on that name. A failed assertion is `<failure type="PHPUnit\Framework\ExpectationFailedException">`: `RED assertion`. **A test of a class the block added is `RED_INVALID`:** at base the class is gone, the test errors (`<error type="Error">`, `Class "App\Support\Greeting" not found`, runner exit 2), and the kind is `import` in both of Laravel's console forms: its plain text, and the one-line JSON summary Laravel 13 prints under an AI agent (`CLAUDECODE` or another agent variable set), where the class name is JSON-escaped (`Class \"App\\Support\\Greeting\" not found`). It proves nothing; for a high-tier file the way past the gate is the human waiver, as for a new Swift API.
|
|
71
|
+
- **Xcode (B102).** An app that only builds with Xcode (an `.xcodeproj` or `.xcworkspace`, iOS targets) is proven through the `xcodebuild` adapter, which runs `xcodebuild test` and reads its result bundle:
|
|
72
|
+
|
|
73
|
+
```yaml
|
|
74
|
+
proof:
|
|
75
|
+
runners:
|
|
76
|
+
- match: ["AppTests/**/*.swift"]
|
|
77
|
+
adapter: xcodebuild
|
|
78
|
+
xcode:
|
|
79
|
+
workspace: App.xcworkspace # or project: App.xcodeproj; neither = a Swift package in the work dir
|
|
80
|
+
scheme: App
|
|
81
|
+
destination: "platform=iOS Simulator,name=iPhone 17"
|
|
82
|
+
test_target: AppTests
|
|
83
|
+
argv: [xcodebuild, test, -workspace, "{workspace}", -scheme, "{scheme}", -destination, "{destination}", "{only_testing}", -resultBundlePath, "{result_bundle}"]
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
The `xcode` object needs `scheme`, `destination` and `test_target` (one target name, no `/`), and at most one of `workspace` and `project`; each value is a non-empty string that does not start with `-`. Only an `xcodebuild` entry may have an `xcode` object, and an `xcodebuild` entry must have one. Its tokens are `{workspace}`, `{project}`, `{scheme}` and `{destination}` (the values from the snapshot's `xcode` object), `{only_testing}` (one item, `-only-testing:<test_target>/<Suite>[/<test>]`, the Suite from the case or the file's base name) and `{result_bundle}` (a fresh `<step dir>/r.xcresult` path under the run's temp root, outside the work dir, that does not exist yet). `validate` refuses an `xcodebuild` argv without `{scheme}`, `{destination}`, `{only_testing}` or `{result_bundle}`, a `{workspace}` or `{project}` token whose key is not set, a `workspace` or `project` key with no matching token, and `{report}` or a test token (those belong to `junit` and `swift`). The schema says the same where JSON Schema can: no value starting with `-`, no `/` in `test_target`, not both `workspace` and `project`, `xcode` required for `xcodebuild` and refused for any other adapter. The case is `<Suite>` or `<Suite>/<test>`: `--test Tests/KitTests/ClampXCTests.swift::ClampXCTests/testCapsHigh`, or with no case the file's suite (`-only-testing:KitTests/ClampXCTests`). What Xcode 26.6 was seen to do with `-only-testing`: a near-name suite (`ClampXCTestsExtra`, `MoreClampSwiftTests`) was not selected; **a Swift Testing case needs its `()`:** `::ClampSwiftTests/capsHigh()`; without them nothing is selected and `xcodebuild` exits 0; an XCTest case works with or without `()`. **Exit 0 with no test run is `RED_INVALID error`** (the filter ran no test). This differs from `junit` and `swift`, where any exit 0 is `NOT_RED`: `xcodebuild test` exits 0 when its filter selects nothing, so an empty run says nothing about the test and is never read as "the test passed". A clean exit with no bundle or an unreadable one is `RED_INVALID error` too. At GREEN, an empty run, a missing or unreadable bundle, or a test outside the suite is `NOT_GREEN`. With a case, the scope is still the whole `<test_target>/<Suite>/`: another test of the same suite in the result is not refused, and when it failed with an assertion too it is one more RED failure that GREEN must see pass. A nested Swift Testing suite cannot be named as a case: `Outer/Inner/test()` is refused (`usage`, nothing run); give the outer `<Suite>` to run all of it.
|
|
87
|
+
|
|
88
|
+
**The precheck** runs before the export is built and before any file is reverted, and only reads: `xcode-select -p` must point at a full Xcode, else exit 1 `xcode-missing` (Command Line Tools only, or no developer directory). A simulator destination (`platform=iOS Simulator`, or the watchOS, tvOS or visionOS simulator) must name, by `name` or `id` (and by `OS` when given; `OS=latest` matches any), a device that `xcrun simctl list devices available -j` lists, else exit 1 `destination-unavailable`, with the names that do exist (at most 10). A simulator destination with neither `name` nor `id` is `destination-unavailable` too, before `simctl` runs: the message says it names no simulator. `platform=macOS` needs no simulator check. Either refusal writes no row and no journal and leaves the tree unchanged. code-forge never boots, creates or deletes a simulator. Both commands get the runner's env (no `GIT_*`) and a 30 s timeout.
|
|
89
|
+
|
|
90
|
+
**The result** is read with `xcrun xcresulttool get test-results tests --path <bundle>` (argv only, 60 s timeout, output capped at 8 MiB). A test's id is `<test_target>/<nodeIdentifier>`: `KitTests/ClampXCTests/testCapsHigh()` (XCTest) or `KitTests/ClampSwiftTests/capsHigh()` (Swift Testing). A failure message is an assertion only when it is `XCTAssert… failed` (not `… failed: threw error`), XCTFail's `failed` or `failed - <message>`, or Swift Testing's `Expectation failed: …`. A thrown error (`failed: caught error: …`, `Caught error: …`), `Issue recorded: …` or a crash is an error, so the test proves nothing (`RED_INVALID error`). No bundle, a bundle that is not a directory, a failed `xcresulttool`, JSON that does not parse, two tests with the same id, and a node type, place or result the reader does not know: no readable result, `RED_INVALID error` at RED and `NOT_GREEN` at GREEN. Today that makes these unreadable: a parameterized Swift Testing test (`Arguments` nodes), repetitions (`Repetition`, `Test Case Run`), a run over several destinations or test plan configurations (`Device`, `Test Plan Configuration`), a `Runtime Warning`, and the result `Expected Failure`. Prove such a test through a plain case, one destination and no repetitions. **Only the named suite counts:** a test outside `<test_target>/<Suite>/` is `RED_INVALID error` at RED and `NOT_GREEN` at GREEN (the filter ran other tests). This holds for a run that exits 0 too: a passing run with any test outside the suite, or with only such tests, is `RED_INVALID error`, never `NOT_RED`, since the test under proof did not run cleanly in scope. **A build failure is `RED_INVALID compile`:** `** BUILD FAILED **` or `** TEST BUILD FAILED **` in the output, `xcodebuild test`'s `Testing cancelled because the build failed.` (Xcode 26.6 then ends with `** TEST FAILED **`), or no test results and a `swiftc` error. A test of a new function is such a case, and needs the human waiver, as for Swift.
|
|
91
|
+
- **The row.** The signed `proof` row and the JSON output add `runner: <adapter>`, `env_keys` (the entry's env variable names, `[]` without an `env`) and `failed_tests` (names only) when a runner was used; an invalid RED (a missing or unreadable report) has no `failed` count and an empty `failed_tests`. `failed_tests` is a bounded list of test names (at most 20, each cut at 200 characters, control characters removed), never a failure message. The full list of failed ids that GREEN checks (`failed_ids`) stays in memory and is never stored. A `node:test` row is unchanged.
|
|
92
|
+
|
|
24
93
|
## §3 The measurement export (R13) — `proof.isolation: export`, the default
|
|
25
94
|
|
|
26
95
|
Parallel blocks share one working tree, so nothing is measured there. `forge proof export <block> --run <r>` builds `.code-forge/export/<block>/`: (a) `git archive <base_sha>` extracted; (b) every path the base's `.gitattributes` marks `export-ignore` restored from `git show` — the hazard is real: repositories mark `/.github` and `CHANGELOG.md` that way; (c) every path in `proof.export.copy_untracked` that exists in the main tree added (`.env`, `.env.testing` by default; never anything under `.code-forge/`); (d) the block's owned files at their **current** content — new files included, and an owned file that existed at the base but is deleted in the tree is **removed from the export** too (listed under `removed` in the export's result), so a deletion is measured as a deletion; (e) the dependency dirs in `proof.export.link_dirs` symlinked read-only — except a **Composer vendor dir** (one holding `composer/autoload_real.php`), which is **copied** (listed under `copied` in the export's result): PHP resolves `__DIR__` through symlinks, so a linked `vendor` makes Composer's autoloader load `App\…` and `Tests\…` from the main tree, where another block's half-written file would leak into the measurement. Gates, red→green and the test-count delta run there; `--remove` deletes it at block close. A gate command that fails inside the export with a missing-file error is reported as `export.incomplete: <path>` with the sentence *add it to `proof.export.copy_untracked` or set `proof.isolation: lock`* — never as a red test.
|
|
@@ -65,11 +134,17 @@ Test-count delta (blocks without a worktree) = the count on the block's filtered
|
|
|
65
134
|
|
|
66
135
|
The export is proven on the package's own example repository (a Node library with a `node:test` suite, `/.github` and `CHANGELOG.md` restored from `export-ignore`).
|
|
67
136
|
|
|
68
|
-
**PHP (Pest and PHPUnit, Laravel) under export: VERIFIED** on 2026-10-11 with PHP 8.4.25, Composer 2.10.2 and Laravel Framework 13.35.0 — a fresh `laravel/laravel` skeleton once on PHPUnit 12.5.38 (as shipped) and once on Pest 4.7.8 (`pestphp/pest` + `pestphp/pest-plugin-laravel` added). Re-run it with `CODE_FORGE_LIVE_PHP=1 node --import ./test/helpers/isolate.mjs --test test/proof/laravel-live.test.mjs` (needs `php`, `composer` and the network; skipped without the flag, so `npm test` never needs them; POSIX only — it was run on macOS, not on Windows). Each check builds the skeleton, commits it
|
|
137
|
+
**PHP (Pest and PHPUnit, Laravel) under export: VERIFIED** on 2026-10-11 with PHP 8.4.25, Composer 2.10.2 and Laravel Framework 13.35.0 — a fresh `laravel/laravel` skeleton once on PHPUnit 12.5.38 (as shipped) and once on Pest 4.7.8 (`pestphp/pest` + `pestphp/pest-plugin-laravel` added). Re-run it with `CODE_FORGE_LIVE_PHP=1 node --import ./test/helpers/isolate.mjs --test test/proof/laravel-live.test.mjs` (needs `php`, `composer` and the network; skipped without the flag, so `npm test` never needs them; POSIX only — it was run on macOS, not on Windows). Each check builds the skeleton, commits it with a broken `app/Support/Clamp.php`, adds the block's owned files (the fixed `Clamp` and its unit test, a new `app/` class and its unit test, a feature test that writes to the database), breaks a file the block does not own in the main tree (the main tree goes red), builds the export with the default lists and runs the detected gate `php artisan test --compact` there: exit 0, 6 tests passed (the skeleton's 2 and the block's 4). What a Laravel export needs:
|
|
69
138
|
- `vendor` in `proof.export.link_dirs` (the default): it is **copied**, not linked, as a Composer vendor dir (§3 (e)). Run `composer install` in the main tree first; the export never installs anything. The copy has a cost on every export: about 5 s and 69 MB for the Laravel skeleton where the file system cannot clone; code-forge asks for a copy-on-write clone (`COPYFILE_FICLONE`), which is near-free where it is supported (APFS, Btrfs, XFS). An absolute symlink inside `vendor` that points into the main tree (a Composer path repository: `vendor/acme/pkg -> <main>/packages/pkg`) is pointed at the same path in the export; relative links and links outside the main tree are kept.
|
|
70
139
|
- `.env` (and `.env.testing` when the project has one) in `proof.export.copy_untracked` (the default): Laravel needs its `APP_KEY`.
|
|
71
140
|
- A database the export can own: SQLite in memory. The skeleton's `phpunit.xml` already sets `DB_CONNECTION=sqlite` and `DB_DATABASE=:memory:`; when yours does not, set them as gate env: `test: {argv: [php, artisan, test, --compact], env: {DB_CONNECTION: sqlite, DB_DATABASE: ":memory:"}}`. A suite that needs a running database server is not isolated by the export — use `proof.isolation: lock`.
|
|
72
141
|
|
|
73
142
|
Detection: `composer.json` + `vendor/bin/pest` is `php-pest`; `composer.json` + `vendor/bin/phpunit` without Pest is `php-phpunit`. With an `artisan` file either runs `php artisan test --compact`; without one, `vendor/bin/pest` or `vendor/bin/phpunit` (a plain PHP project; detected, not run live).
|
|
74
143
|
|
|
144
|
+
**PHP (PHPUnit and Pest, Laravel) red→green under export: VERIFIED** on 2026-10-13 with PHP 8.4.25, Composer 2.10.2 and Laravel Framework 13.35.0, on PHPUnit 12.5.39 and on Pest 4.7.8 (PHPUnit 12.5.33 under it), macOS on arm64, through the same live check (`CODE_FORGE_LIVE_PHP=1`, after its gate part). The runner is `php artisan test` (`[php, artisan, test, "{test_file}", --filter, "{test_filter}", --log-junit, "{report}"]`, §2 PHP): it passes `--log-junit` through to PHPUnit and to Pest, and both write the report. `proof red-green` on the Clamp test's failing case (`::test_caps_high`, Pest `::caps high`) gives RED `assertion`, then GREEN, exit 0 and a signed row with `runner: junit`, exactly that test in `failed_tests`, `env_keys` `DB_CONNECTION` and `DB_DATABASE`, covering `app/Support/Clamp.php` and `app/Support/Greeting.php`; a test of the class the block added gives `RED_INVALID` `import` (`proof red-green` exits 1; the runner itself exits 2), from a plain terminal and under an AI agent (both seen live). The main tree is unchanged. A full-config run (`php artisan test --log-junit <report>`, no `--filter`) writes nested suites: a root suite named after the `phpunit.xml` path, then `Unit` and `Feature`, then the classes; the reader gives the same ids whatever the root suite is named. A step takes about 0.3 s; a whole proof, export included, about 4 to 5 s. No `<warning>`, no element and no `type` the reader did not know appeared. The captured reports are the PHP fixtures in `test/fixtures/junit/`.
|
|
145
|
+
|
|
146
|
+
**Swift (`swift test`, XCTest and Swift Testing) red→green under export: VERIFIED** on 2026-10-13 with Apple Swift 6.3.3 (swiftlang-6.3.3.1.3), Xcode 26.6, macOS 26 on arm64. Re-run it with `CODE_FORGE_LIVE_SWIFT=1 node --import ./test/helpers/isolate.mjs --test test/proof/swift-live.test.mjs` (needs `swift` on PATH, no network; skipped with the reason without the flag, so `npm test` never needs Swift; XCTest is skipped with the reason when only the Command Line Tools are selected). It runs `swift package init --type library --name Kit` in a temp dir, commits a broken `clamp`, fixes it in the block and adds an XCTest class, a Swift Testing suite and two near-name suites that always fail (`ClampXCTestsExtra`, `MoreClampSwiftTests`); `proof red-green` with the runner of §2 gives RED `assertion`, then GREEN, exit 0 and a signed row covering `Sources/Kit/Kit.swift` for the XCTest file (no case), the Swift Testing file (no case) and one case of each (`::ClampXCTests/testCapsHigh`, `::ClampSwiftTests/capsHigh()`), each with exactly that test in `failed_tests`: neither anchored filter selects a near-name suite, also not one with a test of the same name (`MoreClampXCTests/testCapsHigh`, `MoreClampSwiftTests/capsHigh()`, which always fail); SwiftPM builds in the export (`.code-forge/export/A/.build`), never in the package; without XCTest no XCTest file is written; a test that crashes at base (`fatalError`) gives `RED_INVALID error`; a file whose suite is not named after it selects no test and gives `NOT_RED`; a test of a new function gives `RED_INVALID compile` (exit 1), which needs the human waiver (§2 Swift). A step takes 11 to 16 s when the export builds from scratch and about 1.2 s once it is built (each proof's export starts without `.build`). The captured reports are the Swift fixtures in `test/fixtures/junit/`.
|
|
147
|
+
|
|
148
|
+
**Xcode (`xcodebuild test`, XCTest and Swift Testing on an iOS simulator) red→green under export: VERIFIED** on 2026-10-13 with Xcode 26.6 (17F113) and the iPhone 17 simulator (iOS 26.5), macOS 26 on arm64. Re-run it with `CODE_FORGE_LIVE_XCODE=1 node --import ./test/helpers/isolate.mjs --test test/proof/xcodebuild-live.test.mjs` (needs a full Xcode and an available iPhone simulator, no network; skipped with the reason without the flag, when `xcode-select -p` points at the Command Line Tools, or when `xcrun simctl list devices available -j` lists no iPhone on an iOS runtime, so `npm test` never needs Xcode). It uses an already booted iPhone simulator, else the first available one, and never boots, creates, erases or deletes one itself (`xcodebuild` may boot the one it is given). It runs `swift package init --type library --name Kit` in a temp dir with `platforms: [.iOS(.v17), .macOS(.v14)]`, commits a broken `clamp`, fixes it in the block and adds an XCTest class, a Swift Testing suite and two near-name suites that always fail (`ClampXCTestsExtra`, `MoreClampSwiftTests`); the runner is `[xcodebuild, test, -scheme, "{scheme}", -destination, "{destination}", "{only_testing}", -derivedDataPath, <dir>, -resultBundlePath, "{result_bundle}"]` with `xcode: {scheme: Kit, destination: "platform=iOS Simulator,name=<simulator>,OS=<iOS version>", test_target: KitTests}`, the destination filled in from the chosen simulator's name and iOS version (on the verified run `platform=iOS Simulator,name=iPhone 17,OS=26.5`) (a Swift package needs no `workspace` or `project`; `-derivedDataPath` keeps the build out of the home folder). `proof red-green` gives RED `assertion`, then GREEN, exit 0 and a signed row with `runner: xcodebuild` covering `Sources/Kit/Kit.swift` for the XCTest file (no case), the Swift Testing file (no case) and one case of each (`::ClampXCTests/testCapsHigh`, `::ClampSwiftTests/capsHigh()`), each with exactly that test in `failed_tests` (`KitTests/ClampXCTests/testCapsHigh()`, `KitTests/ClampSwiftTests/capsHigh()`): `-only-testing` selects no near-name suite; `xcodebuild` exits 65 at RED and 0 at GREEN, and writes a result bundle at both; a test of a new function gives `RED_INVALID compile` (exit 1; `xcodebuild` exits 65 with `Testing cancelled because the build failed.` and a bundle with no test), which needs the human waiver (§2 Swift); a runner whose destination names a simulator that does not exist gives exit 1 `destination-unavailable` before the export is built: no row, no `xcodebuild` run, the package tree unchanged (same hash). Every step's `xcresulttool` JSON has the node shape of the B102 fixture of the same run and reads to the same tests (Swift Testing lists a suite's tests in a different order from run to run; the reader does not depend on it), so the B102 fixtures in `test/fixtures/xcresult/` are kept as captured. A RED step takes about 23 to 30 s (it rebuilds after the revert), a GREEN step about 4 to 7 s, the compile step about 3 s, the precheck refusal about 0.2 s; the whole check about 130 s.
|
|
149
|
+
|
|
75
150
|
A Vitest stack is **UNVERIFIED** until a real second consumer runs it; `proof.isolation: lock` is the documented fallback until then. `doctor` says which gate commands are on PATH.
|