rungate 0.8.0 → 0.8.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/gates/.gate-salt +1 -0
- package/gates/SCHEMA-GUIDE.md +485 -0
- package/gates/ac-quality.test.ts +106 -0
- package/gates/adversarial.test.ts +283 -0
- package/gates/brief-assembler.test.ts +487 -0
- package/gates/brief-assembler.ts +448 -0
- package/gates/bunfig.toml +2 -0
- package/gates/chain.test.ts +91 -0
- package/gates/e2e-smoke.test.ts +234 -0
- package/gates/error-classifier.test.ts +250 -0
- package/gates/error-classifier.ts +99 -0
- package/gates/gate-executor.ts +3071 -0
- package/gates/orchestrator.test.ts +968 -0
- package/gates/orchestrator.ts +827 -0
- package/gates/preload.ts +89 -0
- package/gates/prompts/ac-adversary.md +20 -0
- package/gates/prompts/evidence-validator.md +22 -0
- package/gates/prompts/prove-reproducer.md +53 -0
- package/gates/prompts.sha256 +3 -0
- package/gates/prove.test.ts +164 -0
- package/gates/rook-state.test.ts +108 -0
- package/gates/run-attribution.ts +290 -0
- package/gates/run-gate.ts +163 -0
- package/gates/schema-parity.test.ts +378 -0
- package/gates/schema.ts +1209 -0
- package/gates/self-heal.test.ts +511 -0
- package/gates/self-heal.ts +125 -0
- package/gates/ship-orchestrator.test.ts +399 -0
- package/gates/ship-orchestrator.ts +384 -0
- package/gates/spec-compliance-probe.ts +159 -0
- package/gates/test-fixtures/broken-no-environments.json +110 -0
- package/gates/test-fixtures/consumer-directory-layout.json +90 -0
- package/gates/test-fixtures/valid-standard.json +91 -0
- package/gates/test-utils.ts +43 -0
- package/gates/typecheck-gate.ts +111 -0
- package/gates/witness.ts +349 -0
- package/gates/workflow.test.ts +2018 -0
- package/hooks/BashWriteGuard.hook.ts +19 -0
- package/hooks/GateResultProtection.hook.ts +27 -1
- package/hooks/VerifyPhaseLock.hook.ts +51 -11
- package/hooks/WorkflowStateGuard.hook.ts +24 -0
- package/lib/ai-review-rounds.ts +462 -0
- package/lib/bash-file-read.ts +17 -3
- package/lib/bash-write-guard.ts +1285 -50
- package/lib/distribution-standing.ts +75 -4
- package/lib/failure-intake.ts +12 -4
- package/lib/file-extensions.ts +81 -0
- package/lib/finding-arbiter.ts +1015 -0
- package/lib/finding-jury.ts +537 -0
- package/lib/gate-enforcement.ts +171 -43
- package/lib/gate-result-protection.ts +12 -2
- package/lib/generators/code-map.ts +55 -4
- package/lib/generators/types.ts +27 -1
- package/lib/github.ts +55 -0
- package/lib/interpreters.ts +64 -0
- package/lib/issue-create-override.ts +69 -8
- package/lib/issue-filing-gate.ts +605 -0
- package/lib/js-top-level-code.ts +20 -2
- package/lib/junit-checks.ts +239 -1
- package/lib/package-closure.ts +114 -0
- package/lib/publish-window.ts +287 -0
- package/lib/scaffold/ai-review-findings.ts +142 -3
- package/lib/scaffold/ai-review-step.ts +1012 -18
- package/lib/scaffold/managed-workflow.ts +163 -4
- package/lib/scaffold/steps.ts +498 -22
- package/lib/scanner.ts +13 -4
- package/lib/ticked-sc-symbols.ts +361 -0
- package/lib/transcript-checker.ts +6 -0
- package/lib/workflow-actions.ts +18 -0
- package/package.json +3 -1
- package/scripts/arbitrate-finding.ts +224 -0
- package/scripts/generate-code-map.ts +18 -10
- package/scripts/sync-sc-status.ts +82 -8
|
@@ -26,6 +26,20 @@
|
|
|
26
26
|
* `test/ai-review-findings.test.ts` fails the suite if a second spelling of it
|
|
27
27
|
* appears anywhere in this module.
|
|
28
28
|
*
|
|
29
|
+
* ## Why the payload contract is read back (#1166)
|
|
30
|
+
*
|
|
31
|
+
* The prompt states a contract — "every one of those member names is required
|
|
32
|
+
* on every finding" — and until #1166 nothing checked it. This module
|
|
33
|
+
* validated a severity and a verdict word; `AI_REVIEW_FINDING_FIELDS` was
|
|
34
|
+
* consumed only by `renderFinding`, which printed `(no file recorded)` and
|
|
35
|
+
* carried on. So a BLOCKING finding with a fabricated `failure_scenario`, no
|
|
36
|
+
* `file` and no `line` adjudicated identically to one with real evidence
|
|
37
|
+
* under it, and the module's own philosophy — do not take the agent's word
|
|
38
|
+
* for it — reached one level short of the thing the agent is asked to write.
|
|
39
|
+
*
|
|
40
|
+
* It is read back on blocking findings and nowhere else. See
|
|
41
|
+
* `blockingContractViolations` for why that line and not a stricter one.
|
|
42
|
+
*
|
|
29
43
|
* ## Why so many ways to be UNMEASURED
|
|
30
44
|
*
|
|
31
45
|
* `0`, `{}`, `null` and `SKIP` are indistinguishable from success; UNMEASURED
|
|
@@ -130,6 +144,44 @@ export function renderFinding(index: number, finding: Record<string, unknown>):
|
|
|
130
144
|
return line.length > AI_REVIEW_DETAIL_MAX ? `${line.slice(0, AI_REVIEW_DETAIL_MAX)}…` : line;
|
|
131
145
|
}
|
|
132
146
|
|
|
147
|
+
/**
|
|
148
|
+
* EVERY finding a reading counted, rendered — not only the blocking ones.
|
|
149
|
+
*
|
|
150
|
+
* `detail` is not this list and must not become it. It is populated inside
|
|
151
|
+
* `if (blocks)` in the loop below, which is correct for a BLOCKED refusal:
|
|
152
|
+
* that annotation names what is stopping the pull request, and enumerating
|
|
153
|
+
* the advisory tail beside it buries the thing a maintainer has to act on.
|
|
154
|
+
*
|
|
155
|
+
* But it is wrong for a CONTRADICTION (#1178). The shape actually seen in CI
|
|
156
|
+
* — `BLOCK over 0 of 1 blocking finding(s)`, on runs 39770912983 and
|
|
157
|
+
* 39795440261 and at roughly 6 of the last 20 runs — is a verdict of BLOCK
|
|
158
|
+
* over a payload whose findings are all advisory. Every one of them is
|
|
159
|
+
* filtered out of `detail`, so the refusal printed the arithmetic and
|
|
160
|
+
* nothing else, and the one observation the reviewer made was destroyed by
|
|
161
|
+
* the code reporting it. A re-run returns CLEAN and writes no contradiction,
|
|
162
|
+
* so the loss is permanent: #1170's finding is gone.
|
|
163
|
+
*
|
|
164
|
+
* Rendered through `renderFinding` at the same index, so this list is a
|
|
165
|
+
* SUPERSET of `detail` built by the same function rather than a second
|
|
166
|
+
* renderer that can drift — and the contradiction arm uses it INSTEAD of
|
|
167
|
+
* `detail`, never alongside it, which is what keeps a blocking finding from
|
|
168
|
+
* being printed twice.
|
|
169
|
+
*
|
|
170
|
+
* Every item is already known to be an object carrying a declared severity
|
|
171
|
+
* by the time a reading is anything other than UNMEASURED, so the fallback
|
|
172
|
+
* arm is unreachable from `aiReviewRefusal`. It exists because the function
|
|
173
|
+
* is exported and `unknown[]` is what it is handed: a renderer that throws
|
|
174
|
+
* on a shape it was told it would never see is a refusal nobody can read.
|
|
175
|
+
*/
|
|
176
|
+
export function renderCountedFindings(reported: unknown[] | null): string[] {
|
|
177
|
+
if (reported === null) return [];
|
|
178
|
+
return reported.map((item, i) =>
|
|
179
|
+
item !== null && typeof item === "object" && !Array.isArray(item)
|
|
180
|
+
? renderFinding(i, item as Record<string, unknown>)
|
|
181
|
+
: `finding ${i} is ${JSON.stringify(item)}`,
|
|
182
|
+
);
|
|
183
|
+
}
|
|
184
|
+
|
|
133
185
|
/** The annotation title the script prints its refusal under. */
|
|
134
186
|
export const AI_REVIEW_FINDINGS_TITLE = "AI review findings";
|
|
135
187
|
|
|
@@ -147,9 +199,54 @@ export type AiReviewUnmeasuredReason =
|
|
|
147
199
|
| "FINDING_NOT_AN_OBJECT"
|
|
148
200
|
| "SEVERITY_ABSENT"
|
|
149
201
|
| "SEVERITY_UNRECOGNISED"
|
|
202
|
+
| "BLOCKING_FINDING_INCOMPLETE"
|
|
150
203
|
| "VERDICT_ABSENT"
|
|
151
204
|
| "VERDICT_UNRECOGNISED";
|
|
152
205
|
|
|
206
|
+
/**
|
|
207
|
+
* The fields a BLOCKING finding must carry, derived from the one contract map
|
|
208
|
+
* rather than re-listed (#1166).
|
|
209
|
+
*
|
|
210
|
+
* `severity` is excluded because it is read before this runs and has its own
|
|
211
|
+
* two reasons; a second check of it would be the fourth shape in
|
|
212
|
+
* `.claude/rules/checks-must-be-able-to-fail.md` — two deciders agreeing about
|
|
213
|
+
* one input, of which a mutation leaves either one holding the case up.
|
|
214
|
+
*/
|
|
215
|
+
export const AI_REVIEW_BLOCKING_REQUIRED_FIELDS: readonly string[] = Object.entries(
|
|
216
|
+
AI_REVIEW_FINDING_FIELDS,
|
|
217
|
+
)
|
|
218
|
+
.filter(([key]) => key !== "severity")
|
|
219
|
+
.map(([, name]) => name);
|
|
220
|
+
|
|
221
|
+
/**
|
|
222
|
+
* The contract fields a blocking finding failed to supply, in contract order.
|
|
223
|
+
*
|
|
224
|
+
* ONE binding, exported, so a test can mutate it and prove the refusal below
|
|
225
|
+
* is what the case observed rather than an unrelated reason firing.
|
|
226
|
+
*
|
|
227
|
+
* Absent, `null`, and blank are the same fact: the annotation a maintainer
|
|
228
|
+
* acts on names no code either way. `renderFinding` has printed
|
|
229
|
+
* `(no file recorded)` for this since #631 and carried on, so a blocking
|
|
230
|
+
* finding with a fabricated `failure_scenario`, no `file` and no `line`
|
|
231
|
+
* adjudicated IDENTICALLY to one with evidence under it — BLOCKED is BLOCKED,
|
|
232
|
+
* and the prompt's "every one of those member names is required on every
|
|
233
|
+
* finding" was enforced by nothing.
|
|
234
|
+
*
|
|
235
|
+
* Measured instance, 2026-10-22, #1137: a HIGH claimed `PROTECTED_GATES` was
|
|
236
|
+
* missing three gate names. Two of the three were real, the finding cited no
|
|
237
|
+
* file and no line, and the names are never written as literals anywhere —
|
|
238
|
+
* the write is `state.gates[gate] = …` inside `writeGateResult`. Requiring a
|
|
239
|
+
* file and a line would have forced the reviewer to produce the site or drop
|
|
240
|
+
* the finding; what happened instead was a round of argument from a grep.
|
|
241
|
+
*/
|
|
242
|
+
export function blockingContractViolations(finding: Record<string, unknown>): string[] {
|
|
243
|
+
return AI_REVIEW_BLOCKING_REQUIRED_FIELDS.filter(name => {
|
|
244
|
+
const value = finding[name];
|
|
245
|
+
if (value === undefined || value === null) return true;
|
|
246
|
+
return typeof value === "string" && value.trim() === "";
|
|
247
|
+
});
|
|
248
|
+
}
|
|
249
|
+
|
|
153
250
|
export interface AiReviewFindingsReading {
|
|
154
251
|
/** The single discriminator. Everything else is a projection of the same pass. */
|
|
155
252
|
outcome: AiReviewOutcome;
|
|
@@ -276,6 +373,23 @@ export function readAiReviewFindings(raw: string | null): AiReviewFindingsReadin
|
|
|
276
373
|
return;
|
|
277
374
|
}
|
|
278
375
|
if (blocks) {
|
|
376
|
+
// #1166, and BLOCKING findings only. A non-blocking finding costs a
|
|
377
|
+
// line in an annotation and nothing else, so holding it to the same
|
|
378
|
+
// contract would redden runs over the cheap tail the module is already
|
|
379
|
+
// losing (#1162) — and an incomplete `info` is not a decision anybody
|
|
380
|
+
// acts on. What a blocking finding buys is a red required check, which
|
|
381
|
+
// is the one output of this module worth forging.
|
|
382
|
+
const missing = blockingContractViolations(finding);
|
|
383
|
+
if (missing.length > 0) {
|
|
384
|
+
reasons.push("BLOCKING_FINDING_INCOMPLETE");
|
|
385
|
+
detail.push(
|
|
386
|
+
`finding ${i} blocks (severity ${JSON.stringify(severity)}) and carries no ` +
|
|
387
|
+
`${missing.join(", ")}. A blocking finding that names no code is not a ` +
|
|
388
|
+
`finding a maintainer can act on or refute, and the contract the ` +
|
|
389
|
+
`reviewer was given requires every field on every finding`,
|
|
390
|
+
);
|
|
391
|
+
return;
|
|
392
|
+
}
|
|
279
393
|
blocking.push(item);
|
|
280
394
|
detail.push(renderFinding(i, finding));
|
|
281
395
|
}
|
|
@@ -354,14 +468,39 @@ export function aiReviewRefusal(reading: AiReviewFindingsReading): string | null
|
|
|
354
468
|
`The AI review reported ${reading.blockingReported?.length ?? 0} blocking ` +
|
|
355
469
|
`finding(s). ${detail}`.trimEnd()
|
|
356
470
|
);
|
|
357
|
-
case "CONTRADICTION":
|
|
471
|
+
case "CONTRADICTION": {
|
|
472
|
+
// THE WHOLE PAYLOAD, not `detail` (#1178). `detail` holds only the
|
|
473
|
+
// findings that block, and the contradiction this issue is about —
|
|
474
|
+
// `BLOCK over 0 of 1` — is one where none of them do, so `detail` is
|
|
475
|
+
// empty and the refusal was arithmetic with the evidence stripped out.
|
|
476
|
+
//
|
|
477
|
+
// The annotation is the right place for it and the only durable one.
|
|
478
|
+
// The payload file lives under `$RUNNER_TEMP` and dies with the runner;
|
|
479
|
+
// three attempts to retrieve #1170's from there returned nothing. This
|
|
480
|
+
// string reaches `::error` at the single emission site in `main()`, the
|
|
481
|
+
// generated step captures the reader with `2>&1` and `printf`s the
|
|
482
|
+
// capture, and GitHub records the result against the check run — so it
|
|
483
|
+
// outlives the job, and it outlives a later re-run that comes back
|
|
484
|
+
// CLEAN. Which is the point: a contradiction that resolves on retry
|
|
485
|
+
// still has to leave something countable behind.
|
|
486
|
+
//
|
|
487
|
+
// Used INSTEAD of `detail` rather than appended to it. The two overlap
|
|
488
|
+
// exactly on the blocking findings, so appending would print those
|
|
489
|
+
// twice, and a duplicate reads as two findings.
|
|
490
|
+
const counted = renderCountedFindings(reading.reported);
|
|
491
|
+
const payload = counted.length > 0
|
|
492
|
+
? `${counted.join("; ")}. `
|
|
493
|
+
: `The payload records no findings at all. `;
|
|
358
494
|
return (
|
|
359
495
|
`The AI review's verdict ${JSON.stringify(reading.verdict)} contradicts its ` +
|
|
360
496
|
`own payload: ${reading.blockingReported?.length ?? 0} of ` +
|
|
361
|
-
`${reading.reported?.length ?? 0} finding(s) block. ${
|
|
497
|
+
`${reading.reported?.length ?? 0} finding(s) block. ${payload}` +
|
|
362
498
|
`A verdict that disagrees with the findings beneath it is a contradiction, ` +
|
|
363
|
-
`not a result
|
|
499
|
+
`not a result. The findings above are what the reviewer actually wrote; ` +
|
|
500
|
+
`they are reported here because this is the only place they survive the ` +
|
|
501
|
+
`job, and a re-run that comes back clean will not produce them again.`
|
|
364
502
|
);
|
|
503
|
+
}
|
|
365
504
|
}
|
|
366
505
|
}
|
|
367
506
|
|