@fro.bot/systematic 3.18.3 → 3.18.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/HARNESSES.md +8 -4
- package/dist/ce-review-validator.d.ts +43 -2
- package/dist/cli.js +10 -16
- package/dist/lib/review-artifact-schema.d.ts +18 -1
- package/dist/lib/review-pipeline-contract.d.ts +1446 -0
- package/dist/lib/review-pipeline.d.ts +812 -0
- package/dist/lib/review-return-validator.d.ts +19 -0
- package/package.json +1 -1
- package/skills/ce-review/SKILL.md +54 -83
- package/skills/ce-review/references/pipeline-invocation.md +263 -0
- package/skills/ce-review/references/review-output-template.md +2 -1
- package/skills/ce-review/references/review-pipeline-schema.json +391 -0
- package/skills/ce-review/references/subagent-template.md +1 -1
- package/skills/ce-review/references/synthesis-artifact-contract.md +57 -68
- package/skills/ce-review/scripts/validate-review.mjs +3277 -58
|
@@ -47,6 +47,25 @@ export interface ReviewReturnValidatorOptions {
|
|
|
47
47
|
readonly outputSink?: (message: string) => void;
|
|
48
48
|
readonly errorSink?: (message: string) => void;
|
|
49
49
|
}
|
|
50
|
+
export type StdinRead = {
|
|
51
|
+
readonly status: 'ok';
|
|
52
|
+
readonly buffer: Buffer;
|
|
53
|
+
} | {
|
|
54
|
+
readonly status: 'oversized';
|
|
55
|
+
} | {
|
|
56
|
+
readonly status: 'read-error';
|
|
57
|
+
};
|
|
58
|
+
export declare function defaultReadChunk(fd: number, buffer: Buffer, offset: number, length: number, position: number | null): number;
|
|
59
|
+
/**
|
|
60
|
+
* Read stdin in bounded chunks, stopping at the cap plus one byte so an
|
|
61
|
+
* oversized payload is rejected without buffering the whole document.
|
|
62
|
+
*
|
|
63
|
+
* `maxBytes` defaults to {@link MAX_REVIEW_RETURN_BYTES} (the `return`/`screen`
|
|
64
|
+
* 1 MiB cap); callers with a different bound -- such as `prepare`'s larger
|
|
65
|
+
* aggregate envelope cap -- pass it explicitly. The default preserves
|
|
66
|
+
* `return` and `screen`'s existing behavior byte-for-byte.
|
|
67
|
+
*/
|
|
68
|
+
export declare function readBoundedStdin(fd: number, readChunk: ReadChunk, maxBytes?: number): StdinRead;
|
|
50
69
|
/**
|
|
51
70
|
* Validate exactly one raw persona return from stdin.
|
|
52
71
|
*
|
package/package.json
CHANGED
|
@@ -55,7 +55,7 @@ All tokens are optional. Each one present means one less thing to infer. When ab
|
|
|
55
55
|
|
|
56
56
|
- **Skip all user questions.** Infer intent conservatively if the diff metadata is thin.
|
|
57
57
|
- **Never edit files or externalize work.** Do not write `.context/systematic/ce-review/<run-id>/`, do not create todo files, and do not commit, push, or create a PR.
|
|
58
|
-
- **Report-only runs in memory.** Run raw-return structural validation,
|
|
58
|
+
- **Report-only runs in memory.** Run raw-return structural validation, synthesis, and reporting without writing a run directory, artifact, or ignore file.
|
|
59
59
|
- **Safe for parallel read-only verification.** `mode:report-only` is the only mode that is safe to run concurrently with browser testing on the same checkout.
|
|
60
60
|
- **Do not switch the shared checkout.** If the caller passes an explicit PR or branch target, `mode:report-only` must run in an isolated checkout/worktree or stop instead of running `gh pr checkout` / `git checkout`.
|
|
61
61
|
- **Do not overlap mutating review with browser testing on the same checkout.** If a future orchestrator wants fixes, run the mutating review phase after browser testing or in an isolated checkout/worktree.
|
|
@@ -484,104 +484,75 @@ Returning the detail tier inline increases parent context per persona. The previ
|
|
|
484
484
|
|
|
485
485
|
**CE conditional agents** (deployment-verification-agent) are also dispatched as standard Agent calls when applicable. Pass the same review context bundle plus the selection reason and triggering surface (for example, which migration files triggered the agent). Their output is unstructured and must be preserved for Stage 6 synthesis just like the other CE conditional agents.
|
|
486
486
|
|
|
487
|
-
####
|
|
487
|
+
#### Screen each return (all modes)
|
|
488
488
|
|
|
489
|
-
Before parsing a persona return into fields,
|
|
490
|
-
|
|
491
|
-
```bash
|
|
492
|
-
# Resolve the validator relative to this skill's directory.
|
|
493
|
-
SKILL_DIR="<skill directory stated when this skill loads>";
|
|
494
|
-
node "$SKILL_DIR/scripts/validate-review.mjs" return <<'REVIEW_RETURN_A1B2C3D4'
|
|
495
|
-
<the persona's returned JSON payload, copied verbatim>
|
|
496
|
-
REVIEW_RETURN_A1B2C3D4
|
|
497
|
-
```
|
|
498
|
-
|
|
499
|
-
Before each invocation, choose a fresh delimiter for that exact raw payload over a safe token alphabet (`A-Z`, `0-9`, `_`), for example a random hex token. Verify the delimiter is absent as a complete line in that exact raw payload before running. The `REVIEW_RETURN_A1B2C3D4` token above is only an illustration; never reuse a fixed delimiter, and choose a new token for every payload. Open the heredoc with a single-quoted heredoc opener (`<<'DELIM'`) so the payload is never interpolated, and close it with a line containing exactly that delimiter. Feed the payload on stdin (never as a command argument) so it cannot appear in argv or a process listing; never use unquoted interpolation or command substitution to pass the payload, and never write it to a temp file. This block is self-contained for one-block execution: each fenced block re-assigns `SKILL_DIR` and terminates the assignment with `;`.
|
|
489
|
+
Before parsing a persona return into fields, assessing evidence, synthesizing, or persisting anything, admit it with the packaged structural validator's `screen` phase. `screen` replaces the former separate raw-return-admission and dispatch-identity-binding steps with one call: it structurally admits the return and binds it to the dispatched persona in the same pass. Invoke it through this skill's own installed directory (`node "$SKILL_DIR/scripts/validate-review.mjs" screen --reviewer <persona> --harness <opencode|pi|claude-code>`, the raw return on stdin through a fresh single-quoted heredoc delimiter) so every harness resolves the same committed bytes. The full envelope shape, the fresh-delimiter rules, and the invocation block are in [pipeline invocation: screen](./references/pipeline-invocation.md#screen).
|
|
500
490
|
|
|
501
491
|
Read the exit status:
|
|
502
492
|
|
|
503
|
-
- **exit 0** — structurally admitted. Parse the already structurally validated JSON without logging the raw text
|
|
504
|
-
- **exit 1** — the whole return is `dispatch_outcome: "malformed"
|
|
493
|
+
- **exit 0** — structurally admitted. Parse the already structurally validated JSON without logging the raw text; the parent adds `harness` and `dispatch_outcome` from the result to the persisted per-agent dispatch record. `exit 0` with zero findings is `dispatch_outcome: "empty"`; `exit 0` with findings is `dispatch_outcome: "findings"`, each finding already carrying a stable `input_id` and `disposition: "surviving"`.
|
|
494
|
+
- **exit 1** — the whole return is `dispatch_outcome: "malformed"`, and the parent must never parse or persist its payload fields or values; retain only the bounded validator diagnostic in Coverage. This covers malformed JSON, a schema violation, and a dispatch identity mismatch alike.
|
|
505
495
|
- **exit 2**, a missing or unreadable helper, or a command launch failure — validation unavailable. Withhold the return and report the exact unavailability and what was withheld. Update that selected persona's preinitialized dispatch entry from `never_returned` to `dispatch_outcome: "validation_unavailable"` with `input_finding_count: 0` and, optionally, a safe `rejection_reason` naming the exit status, missing helper, or launch failure without payload values; set `run_status` to `degraded`. A run that contains `validation_unavailable` evidence can never finalize as `completed`, and that persona must not have an admitted input finding. Never omit the dispatch entry, never leave it as `never_returned`, never label it `malformed`, never admit the payload, and never fabricate a reviewer record or a rejected-summary ledger row. The word `unavailable` also names the artifact-level self-validation status, a different object and phase; never repurpose the artifact-level `validation` fields.
|
|
506
496
|
|
|
507
|
-
**Dispatch identity binding.**
|
|
497
|
+
**Dispatch identity binding.** `screen` confirms the returned `reviewer` field matches the dispatched persona before it admits anything; an identity mismatch is rejected as `dispatch_outcome: "malformed"` and degrades the run. This comparison happens inside the same exit 0/exit 1 decision above, before the parent parses a single finding field -- structural admission never proves identity on its own.
|
|
508
498
|
|
|
509
499
|
A task that did not return is `never_returned`: a task-lifecycle fact recorded without invoking the validator. Validation unavailable is not malformed and is not never_returned; they are distinct coverage states. The public `systematic validate-review-return` command is an operator/development fallback selected before invocation, never a fallback chosen because a validator run exited 1 or 2.
|
|
510
500
|
|
|
511
|
-
Structural validity never implies evidence validity. A return that passes
|
|
512
|
-
|
|
513
|
-
### Stage 5: Merge findings
|
|
501
|
+
Structural validity never implies evidence validity. A return that passes `screen` is admitted structurally only. The parent must still assess evidence for its claims, and only then add parent annotations, persist, or synthesize -- a wrong-checkout or unsupported citation remains unverified until current-target evidence resolves it.
|
|
514
502
|
|
|
515
|
-
|
|
503
|
+
A helper failure is never permission to hand-synthesize an admitted finding, a dispatch outcome, or a rejected-summary row; see [Never-bypass](./references/pipeline-invocation.md#never-bypass) for the retry and correction protocol every phase shares.
|
|
516
504
|
|
|
517
|
-
|
|
505
|
+
### Stage 5: Merge findings
|
|
518
506
|
|
|
519
|
-
|
|
507
|
+
The parent-owned artifact and its reconciliation rules are defined in the [synthesis artifact contract](./references/synthesis-artifact-contract.md). Stage 5 no longer computes the confidence gate, candidate grouping, cross-reviewer agreement, route narrowing, partitioning, sorting, or coverage union by hand -- the `prepare` and `merge` phases of the packaged validator compute all of it deterministically. The model's remaining job is adjudication: deciding, for each file's candidate group, which findings describe the same underlying defect, and proposing each merged finding's narrative and route.
|
|
520
508
|
|
|
521
|
-
1. **
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
- **Schema constraints:** enforce every enum, type, confidence, line, path, evidence count, evidence length, and explicit overflow-marker bound from the schema. Empty evidence, absolute paths, and over-bound evidence are rejection cases, not truncation cases.
|
|
525
|
-
- **Environment-value detection:** JSON Schema cannot determine where a string came from, so recursively inspect every string leaf in the parsed payload before writing. Apply the environment-value matching, structural-detector, and finding-granularity rules in the [synthesis artifact contract](./references/synthesis-artifact-contract.md). This detector is an additional parent-side check, not a schema claim.
|
|
526
|
-
- **Safe rejection message:** Use the safe rejection message rule in the [synthesis artifact contract](./references/synthesis-artifact-contract.md); never include the offending value, raw return, or validator parameters.
|
|
527
|
-
- **No partial writes:** Do not write or merge a finding until it passes the parent-side validation rules. Apply the admitted-finding persistence and rejected-payload ledger rules in the [synthesis artifact contract](./references/synthesis-artifact-contract.md). A valid admitted finding is then annotated by the parent with `harness` and `dispatch_outcome` and written by the parent only. Revalidate the enriched record before persistence.
|
|
528
|
-
- **Dispatch outcome:** Record the parent-owned dispatch outcomes and ledger dispositions according to the [synthesis artifact contract](./references/synthesis-artifact-contract.md); keep dispatch outcomes separate from finding dispositions.
|
|
529
|
-
- **Rejection policy: degrade, do not fail the whole review.** Continue merging conforming returns when a persona or finding is rejected; record the rejection and apply the risk-aware verdict according to the [synthesis artifact contract](./references/synthesis-artifact-contract.md). If every persona fails or times out, use the existing degraded-review behavior.
|
|
530
|
-
2. **Confidence gate.** Suppress findings below 0.60 confidence. Exception: P0 findings at 0.50+ confidence survive the gate -- critical-but-uncertain issues must not be silently dropped. Record the suppressed finding's original confidence and an explicit reason in the input ledger. A retained P0 at 0.50+ is recorded as `surviving` unless it later participates in a deduplication merge. This matches the persona instructions and the schema's confidence thresholds.
|
|
531
|
-
3. **Deduplicate.** Group all gated findings by `normalize(file)`. A file with two or more findings from different personas forms a candidate group. Do not use line number to form groups. Sort findings within each group by line. Adjudicate each candidate group: merge findings judged to describe the same underlying defect; keep genuinely different defects separate. Adjacency creates a candidate, not a conclusion -- findings on the same line that describe different defects must not merge. For each resulting finding, derive the fingerprint from its file and line as `normalize(file) + "|" + line`. Keep highest severity, keep highest confidence, and retain the input IDs that produced the merged entry. Record each declined merge in the artifact's optional `declined_merges` field with the normalized file, input finding IDs considered but not merged, and a brief reason. A singleton that passes the gate is `surviving`; each input in a multi-input merge is provisionally `merged`.
|
|
509
|
+
1. **Assemble every screen result.** Collect the `screen` output for every persona that returned (including `malformed`, `never_returned`, and `validation_unavailable` entries), plus the Stage 3 selection record (`selected_dispatches`), into the `prepare` input envelope. See [pipeline invocation: prepare](./references/pipeline-invocation.md#prepare) for the exact shape and invocation.
|
|
510
|
+
2. **Run `prepare`.** It applies the confidence gate (suppress below 0.60, except P0 at 0.50+ survives), groups admitted findings into candidate groups by `normalize(file)` (never by line), sorts each group's members by line, and unions selection-surface coverage. Its output's `singletons` need no adjudication; its `candidate_groups` do.
|
|
511
|
+
3. **Adjudicate every candidate group.** For each group, decide whether its members describe the same underlying defect (merge) or genuinely different defects (decline). Adjacency creates a candidate, not a conclusion -- findings on the same line describing different defects must stay declined. For a merge decision, write the merged finding's `title`, `why_it_matters`, `evidence`, and `line`, plus (when the route should narrow) a `proposed_route` with a `route_narrowing_reason`; optionally note `disagreement_facts` when reviewers disagreed on severity/autofix_class/owner, and `eligible_agreement_credit` for personas that agree without their own input finding in the group. For a decline decision, write a `declined_reason`. Every candidate-group member must be cited by exactly one decision -- no omissions, no double-citations.
|
|
532
512
|
|
|
533
513
|
Worked example: at `src/lib/model-availability.ts:139`, reliability's `Config hook awaits providers API without a timeout` and adversarial's `Config startup can hang forever behind a stalled /config` describe the same underlying defect in different words, so they merge.
|
|
534
|
-
4. **
|
|
535
|
-
5. **
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
8. **Partition the work.** Build three sets:
|
|
539
|
-
- in-skill fixer queue: only `safe_auto -> review-fixer`
|
|
540
|
-
- residual actionable queue: unresolved `gated_auto` or `manual` findings whose owner is `downstream-resolver`
|
|
541
|
-
- report-only queue: `advisory` findings plus anything owned by `human` or `release`
|
|
542
|
-
9. **Sort.** Order by severity (P0 first) -> confidence (descending) -> file path -> line number.
|
|
543
|
-
10. **Collect coverage data.** Union residual_risks and testing_gaps across reviewers.
|
|
544
|
-
11. **Preserve CE agent artifacts.** Keep the outputs of the selected learnings, agent-native, schema-drift, and deployment-verification agents alongside the merged finding set. Do not drop unstructured agent output just because it does not match the persona JSON schema.
|
|
545
|
-
12. **Keep the input ledger complete.** Reconcile admitted findings and rejected-payload summaries according to the [synthesis artifact contract](./references/synthesis-artifact-contract.md).
|
|
514
|
+
4. **Run `merge`.** It applies the adjudication envelope to `prepare`'s output, deriving each merged finding's severity, the cross-reviewer-agreement-boosted confidence (+0.10, capped at 1.0, for a merge with 2+ independent submitters), the fingerprint (`normalize(file) + "|" + line`), and the conservatively narrowed `autofix_class`/`owner`/`requires_verification` -- synthesis may narrow a route, never widen it without new evidence. It also returns `validator_requests`: the merged findings that need Stage 5b validation. See [pipeline invocation: merge](./references/pipeline-invocation.md#merge).
|
|
515
|
+
5. **Preserve CE agent artifacts.** Keep the outputs of the selected learnings, agent-native, schema-drift, and deployment-verification agents alongside the merged finding set for Stage 6 rendering. Do not drop unstructured agent output just because it does not match the persona JSON schema.
|
|
516
|
+
|
|
517
|
+
A helper failure (`prepare` or `merge` exiting 1 or 2) is never permission to hand-assemble a merged finding, a route, or a confidence value; see [Never-bypass](./references/pipeline-invocation.md#never-bypass).
|
|
546
518
|
|
|
547
519
|
### Stage 5b: Validation pass
|
|
548
520
|
|
|
549
|
-
|
|
521
|
+
Dispatch validators for exactly the findings `merge`'s `validator_requests` names -- the P0/P1-or-`requires_verification` gating band is already computed; Stage 5b no longer identifies the gated set by hand.
|
|
550
522
|
|
|
551
|
-
|
|
523
|
+
1. For each entry in `validator_requests`, spawn one validator subagent in parallel using the validator template at `references/validator-template.md`. Look up that finding's full fields (title, why_it_matters, evidence, file, line, severity, autofix_class, owner, suggested_fix) from `merge`'s `merged_findings` by `finding_id`, and pass them along with the intent summary, file list, and full diff.
|
|
524
|
+
2. Collect `{outcome: 'true'|'false'|'failed'|'unavailable', reason?}` from each validator, keyed by `finding_id`. `outcome: 'true'` needs no reason; the other three outcomes each require one.
|
|
525
|
+
3. Carry every result forward as `validator_lifecycle_results` into `finalize` (Stage 6). Do not reconcile filtered findings, update ledger dispositions, or recompute the "Filtered (not validated)" group by hand -- `finalize` derives all of it from these results plus `merge`'s output.
|
|
552
526
|
|
|
553
|
-
**
|
|
527
|
+
**Outcome semantics** (enforced by `finalize`, not the model): `true` validates the finding -- it flows to Stage 6 unchanged. `false` filters the finding -- it drops out of the surviving/actioned set, receives disposition `filtered` with the validator's exact one-sentence reason, and appears in the "Filtered (not validated)" group. `failed` and `unavailable` leave the finding actionable and unvalidated (no `validated` annotation; it appears in the normal severity tables) but each records a lifecycle failure that marks the run `degraded` and blocks a clean verdict -- report the failed/unavailable validator in Coverage rather than silently treating it as validated.
|
|
554
528
|
|
|
555
|
-
|
|
556
|
-
2. For each gated finding, spawn one validator subagent in parallel using the validator template at `references/validator-template.md`. Pass the finding fields, the intent summary, the file list, and the full diff.
|
|
557
|
-
3. Collect `{validated, reason}` from each validator. Attach both fields to the finding.
|
|
558
|
-
4. **Reconcile filtered inputs.** A finding with `validated: false` moves to the "Filtered (not validated)" presentation group in Stage 6, and every input ID contributing to that merged finding is updated to disposition `filtered` with the validator's exact one-sentence reason. Apply the remaining ledger rules from the [synthesis artifact contract](./references/synthesis-artifact-contract.md).
|
|
559
|
-
5. Findings with `validated: true` flow through to Stage 6 unchanged — they appear in the normal severity tables.
|
|
560
|
-
6. Findings outside the gating band carry no `validated` annotation and appear in Stage 6 severity tables unchanged.
|
|
529
|
+
Findings outside the gating band (no `validator_requests` entry) carry no `validated` annotation and appear in Stage 6 severity tables unchanged.
|
|
561
530
|
|
|
562
|
-
|
|
531
|
+
### Stage 6: Synthesize and present
|
|
563
532
|
|
|
564
|
-
|
|
533
|
+
Call `finalize` with `applied_fixes: []` to synthesize the run's report projection -- verdict, findings, coverage, disposition counts, and every queue (`fixer`, `residual`, `report_only`) -- from `merge`'s output, `prepare`'s output, the screen results, the dispatch records, the Stage 5b validator lifecycle results, and the Stage 2b plan assessment below. See [pipeline invocation: finalize](./references/pipeline-invocation.md#finalize) for the exact envelope. This first call's `report.queues` feeds the post-review action sets directly (see Step 1 under [After Review](#after-review)); a fix-applying mode calls `finalize` again after fixes land with the real `applied_fixes`, and only that second call is persisted.
|
|
565
534
|
|
|
566
|
-
|
|
535
|
+
**Plan assessment.** Read the plan's Requirements Trace and Implementation Units located in Stage 2b, and check each one against the diff: met / not addressed / partially addressed. For every requirement or unit not clearly met, produce one plan-assessment result:
|
|
567
536
|
|
|
568
|
-
|
|
537
|
+
- **`explicit_unmet_requirement`** -- the plan is `plan_source: explicit` (caller-provided or an unambiguous PR body match) and a stated requirement is unaddressed. Routes to residual actionable work and blocks a clean verdict.
|
|
538
|
+
- **`inferred_gap`** -- the plan is `plan_source: inferred` (auto-discovered), or the gap is a suspicion rather than a stated requirement. Routes to advisory-only output and never blocks the verdict by itself.
|
|
539
|
+
|
|
540
|
+
Neither kind becomes a finding -- `finalize` routes `results` directly into `residual_actionable_work` or `advisory_outputs` strings, never into the severity tables. Omit plan assessment entirely when no plan was found in Stage 2b -- do not mention the absence of a plan, and pass an empty `results` array so `finalize` neither fabricates a gap nor silently relaxes the verdict gate.
|
|
541
|
+
|
|
542
|
+
Assemble the final report using **pipe-delimited markdown tables for findings** from the review output template included below, rendering `finalize`'s report projection directly -- do not recompute any of the fields it already derived. The table format is mandatory for finding rows in interactive mode — do not render findings as freeform text blocks or horizontal-rule-separated prose. Other report sections (Applied Fixes, Learnings, Coverage, etc.) use bullet lists and the `---` separator before the verdict, as shown in the template.
|
|
569
543
|
|
|
570
544
|
1. **Header.** Scope, intent, mode, harness, reviewer team with per-conditional justifications.
|
|
571
|
-
2. **Findings.** Rendered as pipe-delimited tables grouped by severity (`### P0 -- Critical`, `### P1 -- High`, `### P2 -- Moderate`, `### P3 -- Low`). Each finding row shows `#`, file, issue, reviewer(s), confidence, and synthesized route. Omit empty severity levels. Never render findings as freeform text blocks or numbered lists. Only findings with `validated: true` (or no `validated` annotation) appear in these tables.
|
|
572
|
-
3. **Requirements Completeness.** Include only when a plan was found in Stage 2b.
|
|
573
|
-
- **`explicit`** (caller-provided or PR body): Flag unaddressed requirements as P1 findings with `autofix_class: manual`, `owner: downstream-resolver`. These enter the residual actionable queue and can become todos.
|
|
574
|
-
- **`inferred`** (auto-discovered): Flag unaddressed requirements as P3 findings with `autofix_class: advisory`, `owner: human`. These stay in the report only — no todos, no autonomous follow-up. An inferred plan match is a hint, not a contract.
|
|
575
|
-
Omit this section entirely when no plan was found — do not mention the absence of a plan.
|
|
545
|
+
2. **Findings.** Rendered as pipe-delimited tables grouped by severity (`### P0 -- Critical`, `### P1 -- High`, `### P2 -- Moderate`, `### P3 -- Low`) from `report.findings`. Each finding row shows `#`, file, issue, reviewer(s), confidence, and synthesized route. Omit empty severity levels. Never render findings as freeform text blocks or numbered lists. Only findings with `validated: true` (or no `validated` annotation) appear in these tables.
|
|
546
|
+
3. **Requirements Completeness.** Include only when a plan was found in Stage 2b. Render the met/not-addressed/partially-addressed checklist from the plan-assessment step above, then list `report.residual_actionable_work` and `report.advisory_outputs` as their own bullet lists -- these are plain descriptions, not findings, and never gain a file/line/route. Omit this section entirely when no plan was found.
|
|
576
547
|
4. **Applied Fixes.** Include only if a fix phase ran in this invocation.
|
|
577
|
-
5. **Residual Actionable Work.**
|
|
578
|
-
6. **Pre-existing.** Separate section, does not count toward verdict.
|
|
579
|
-
7. **Filtered (not validated).** Include when
|
|
548
|
+
5. **Residual Actionable Work.** Render `report.queues.residual` as a table of findings whose owner is `downstream-resolver`, using `report.input_dispositions` to resolve each entry's fields.
|
|
549
|
+
6. **Pre-existing.** Render `report.pre_existing_findings`. Separate section, does not count toward verdict.
|
|
550
|
+
7. **Filtered (not validated).** Include when `report.findings` contains any entry with `validated: false`. Render as a pipe-delimited table with columns `#`, `File`, `Issue`, `Reviewer`, `Confidence`, `Validator reason`. These findings are surfaced for human review — they are not removed from the report. The validator found evidence that the issue may not be real in the code as written, was not introduced by this diff, or is already handled elsewhere; the human reviewer makes the final call. Omit this section when no findings were filtered.
|
|
580
551
|
8. **Learnings & Past Solutions.** Render only when CE `learnings-researcher` was selected and returned relevant output: if past solutions are relevant, flag them as "Known Pattern" with links to docs/solutions/ files. Omit the section otherwise.
|
|
581
552
|
9. **Agent-Native Gaps.** Render only when CE `agent-native-reviewer` was selected and returned relevant output. Omit the section otherwise.
|
|
582
553
|
10. **Deployment Notes.** If deployment-verification-agent ran, surface the key Go/No-Go items: blocking pre-deploy checks, the most important verification queries, rollback caveats, and monitoring focus areas. Keep the checklist actionable rather than dropping it into Coverage.
|
|
583
|
-
11. **Coverage.**
|
|
584
|
-
12. **Verdict.** Ready to merge / Ready with fixes / Not ready
|
|
554
|
+
11. **Coverage.** Render `report.coverage` directly: suppressed count, residual risks, testing gaps, failed/timed-out reviewers, validator lifecycle failures (`failed`/`unavailable` outcomes from Stage 5b), risk-coverage entries with citing input finding IDs and exit conditions for blocked entries, and any intent uncertainty carried by non-interactive modes. For raw returns, state each selected persona's admission state — `findings`, `empty`, `malformed`, `never_returned`, `validation_unavailable` (the persisted raw dispatch outcome; distinct from the artifact-level `validation.status: "unavailable"`) — and what was admitted or withheld. Report admission states here only; do not add fields to `review-summary.v1`. Distinguish core reviewers, each selected conditional with its one-line rationale and triggering repository-relative paths, an explicit "no conditional selected" case, and any selected-but-failed/malformed/validation-unavailable reviewer; never label an unselected reviewer as failed.
|
|
555
|
+
12. **Verdict.** Render `report.verdict` directly: Ready to merge / Ready with fixes / Not ready, with fix order if applicable. `finalize` already applies the risk-aware degraded verdict rule and the plan-assessment gate from the [synthesis artifact contract](./references/synthesis-artifact-contract.md) -- do not recompute or override it.
|
|
585
556
|
|
|
586
557
|
Do not include time estimates.
|
|
587
558
|
|
|
@@ -656,11 +627,7 @@ Coverage:
|
|
|
656
627
|
Review complete
|
|
657
628
|
```
|
|
658
629
|
|
|
659
|
-
**Detail enrichment (headless only):** The headless envelope includes `Why:`, `Evidence:`, and `Suggested fix:` lines.
|
|
660
|
-
- **Field tiers:** `Why:` and `Evidence:` are detail-tier and are already present in the validated inline return. `Suggested fix:` is also available directly from that return and survives merge as optional fix context.
|
|
661
|
-
- **In-memory matching:** For each surviving finding, look up its detail-tier fields in the validated returns of the contributing reviewers. Use the merged finding's `input_finding_ids` (`<reviewer>#<1-based finding index>`) to identify the contributing return and source finding. When an input ID cannot be resolved or multiple candidates remain, match on normalized `file`, then use line and `normalize(title)` only as tie-breakers.
|
|
662
|
-
- **Reviewer order:** Try contributing reviewers in the order they appear in the merged finding's reviewer list; use the first validated match.
|
|
663
|
-
- **No-match fallback:** If no validated in-memory return contains a match, omit the `Why:` and `Evidence:` lines for that finding and note the gap in Coverage. This should indicate a synthesis/matching gap, not a failed artifact-file write. Never re-read per-agent files to recover detail.
|
|
630
|
+
**Detail enrichment (headless only):** The headless envelope includes `Why:`, `Evidence:`, and `Suggested fix:` lines. `finalize`'s report projection already carries `why_it_matters`, `evidence`, and `suggested_fix` inline on every finding in `report.findings` -- render them directly. No in-memory matching against persona returns is needed or performed; that matching step is obsolete now that `merge` derives every detail-tier field as part of the merged finding itself.
|
|
664
631
|
|
|
665
632
|
**Formatting rules:**
|
|
666
633
|
- The `[needs-verification]` marker appears only on findings where `requires_verification: true`.
|
|
@@ -698,11 +665,13 @@ After presenting findings and verdict (Stage 6), route the next steps by mode. R
|
|
|
698
665
|
|
|
699
666
|
#### Step 1: Build the action sets
|
|
700
667
|
|
|
701
|
-
|
|
702
|
-
|
|
703
|
-
- **
|
|
704
|
-
- **
|
|
705
|
-
- **
|
|
668
|
+
Call `finalize` with `applied_fixes: []` (see [Stage 6](#stage-6-synthesize-and-present)) and read the action sets directly from `report.queues` -- do not recompute them by hand.
|
|
669
|
+
|
|
670
|
+
- **Clean review** means `report.queues.fixer`, `report.queues.residual`, and `report.queues.report_only` are all empty. Skip the fix/handoff phase when the review is clean.
|
|
671
|
+
- **Fixer queue:** `report.queues.fixer` -- findings routed to `safe_auto -> review-fixer`.
|
|
672
|
+
- **Residual actionable queue:** `report.queues.residual` -- unresolved `gated_auto` or `manual` findings whose final owner is `downstream-resolver`.
|
|
673
|
+
- **Report-only queue:** `report.queues.report_only` -- `advisory` findings and any outputs owned by `human` or `release`.
|
|
674
|
+
- **Never convert advisory-only outputs into fix work or todos.** Deployment notes, residual risks, `report.advisory_outputs`, and release-owned items stay in the report.
|
|
706
675
|
|
|
707
676
|
#### Step 2: Choose policy by mode
|
|
708
677
|
|
|
@@ -765,14 +734,16 @@ After presenting findings and verdict (Stage 6), route the next steps by mode. R
|
|
|
765
734
|
#### Step 4: Emit artifacts and downstream handoff
|
|
766
735
|
|
|
767
736
|
- In interactive, autofix, and headless modes, write **`review-summary.json` unconditionally** under `.context/systematic/ce-review/<run-id>`; `mode:report-only` remains the deliberate no-write exception.
|
|
768
|
-
- `
|
|
769
|
-
-
|
|
770
|
-
-
|
|
771
|
-
-
|
|
772
|
-
-
|
|
737
|
+
- If a fix phase ran (Step 3), call `finalize` again with the exact applied-fix outcomes in `parent_run_metadata.applied_fixes` and the same `validator_lifecycle_results` used in Step 1; only this second call's output is persisted. If no fix phase ran, Step 1's call already is the output to persist -- do not call `finalize` a third time.
|
|
738
|
+
- `review-summary.json` is the persisted call's `artifact` value, written verbatim. `finalize`'s writing-mode stdout is the wrapper `{ kind: 'writing', artifact, report }`, so extract only its `artifact` member -- never the whole wrapper -- and write that JSON to a same-directory temp file created exclusively with owner-only permissions inside `.context/systematic/ce-review/<run-id>`, then atomically rename it over `review-summary.json`; remove the temp file on any non-success. See [Persisting the artifact](./references/pipeline-invocation.md#persisting-the-artifact).
|
|
739
|
+
- After the rename, run the existing `artifact` subcommand on the persisted file exactly as before; see the [artifact validation and failure path](./references/synthesis-artifact-contract.md#artifact-validation).
|
|
740
|
+
- `review-summary.json`'s lifecycle, dispatch outcomes, complete input ledger, synthesized and filtered findings with provenance, disposition counts, and downstream work are defined in the [canonical synthesis artifact contract](./references/synthesis-artifact-contract.md), whose vocabulary and bounds are executable in [`findings-schema.json`](./references/findings-schema.json).
|
|
741
|
+
- Capture `branch` and `head_sha` at dispatch time, before any autofixes land, and pass them in `parent_run_metadata` with `completed_at` when the verdict is finalized; see the [canonical synthesis artifact contract](./references/synthesis-artifact-contract.md) for the provenance semantics.
|
|
742
|
+
- In autofix mode, create durable todo files only for unresolved actionable findings whose final owner is `downstream-resolver` (`report.queues.residual`). Load the `todos` skill (Create section) for the canonical directory path, naming convention, YAML frontmatter structure, and template. Each todo should map the finding's severity to the todo priority (`P0`/`P1` -> `p1`, `P2` -> `p2`, `P3` -> `p3`) and set `status: ready` since these findings have already been triaged by synthesis.
|
|
773
743
|
- Do not create todos for `advisory` findings, `owner: human`, `owner: release`, or protected-artifact cleanup suggestions.
|
|
774
744
|
- If only advisory outputs remain, create no todos.
|
|
775
745
|
- Interactive mode may offer to externalize residual actionable work after fixes, but it is not required to finish the review.
|
|
746
|
+
- Report-only mode never runs this step (no ignore preparation, no run directory, no temp file, no artifact write, no `artifact` subcommand validation): it calls `finalize` once (Step 1) and renders `report` directly, in memory.
|
|
776
747
|
|
|
777
748
|
#### Step 5: Final next steps
|
|
778
749
|
|
|
@@ -0,0 +1,263 @@
|
|
|
1
|
+
# Pipeline Phase Invocation
|
|
2
|
+
|
|
3
|
+
This is the canonical invocation detail for the four pure stdin phases of the
|
|
4
|
+
packaged validator shim that `ce:review`'s Stage 4-6 and post-review handoff
|
|
5
|
+
call at their decision boundaries: `screen`, `prepare`, `merge`, and
|
|
6
|
+
`finalize`. `SKILL.md` keeps
|
|
7
|
+
terse call sites; this document carries the envelope shapes and the full
|
|
8
|
+
invocation blocks. `return` and `artifact` are unchanged and documented at
|
|
9
|
+
their existing call sites (Stage 4 raw-return history and the
|
|
10
|
+
[synthesis artifact contract](./synthesis-artifact-contract.md)).
|
|
11
|
+
|
|
12
|
+
Every block below reassigns `SKILL_DIR` in the same fenced block, terminated
|
|
13
|
+
with `;`, and invokes `node "$SKILL_DIR/scripts/validate-review.mjs" <phase>`
|
|
14
|
+
-- never a bare relative path, and never a Claude-only path substitution
|
|
15
|
+
(off-Claude harnesses would silently expand it to nothing).
|
|
16
|
+
|
|
17
|
+
## Never-bypass
|
|
18
|
+
|
|
19
|
+
A helper failure (exit 1 or exit 2) is never permission to hand-synthesize
|
|
20
|
+
the phase's output. On exit 2 (usage error, TTY input, or a stdin read
|
|
21
|
+
failure), retry the launch exactly once with byte-identical input; a second
|
|
22
|
+
exit 2 is a launch failure, not a payload problem. On exit 1 (the input
|
|
23
|
+
was structurally rejected), the parent may correct its own JSON envelope
|
|
24
|
+
exactly once -- fixing a genuine encoding mistake, never reshaping the
|
|
25
|
+
envelope to force acceptance -- and retry; a second exit 1 stops the run
|
|
26
|
+
visibly with degraded or abnormal status. Never fall back to writing a
|
|
27
|
+
finding, a merged finding, a queue, a disposition count, or a report section
|
|
28
|
+
by hand because a phase call failed.
|
|
29
|
+
|
|
30
|
+
## screen
|
|
31
|
+
|
|
32
|
+
Structurally admits one persona's raw return and binds it to the dispatched
|
|
33
|
+
persona, replacing the former separate raw-return-admission and
|
|
34
|
+
dispatch-identity-binding steps with one call. Feed the persona's raw JSON
|
|
35
|
+
return on stdin through a fresh single-quoted heredoc delimiter -- never
|
|
36
|
+
argv, command substitution, or a temp file -- exactly as the packaged
|
|
37
|
+
validator's `return` subcommand was invoked before this phase replaced it.
|
|
38
|
+
|
|
39
|
+
**Output** (`exit 0`): `{ dispatch_outcome, admitted_findings[{input_id, ...finding}], rejected_summary?{dispatch_outcome, rejected_finding_count, rejected_severities, reason}, residual_risks[], testing_gaps[], harness }`.
|
|
40
|
+
`admitted_findings` carries each finding with a stable `<reviewer>#<index>`
|
|
41
|
+
`input_id` and `disposition: "surviving"` pre-assigned. A whole-payload
|
|
42
|
+
rejection (malformed JSON, schema violation, or an identity mismatch between
|
|
43
|
+
the return's `reviewer` field and the dispatched persona) never admits any
|
|
44
|
+
finding from that return.
|
|
45
|
+
|
|
46
|
+
Before each invocation, choose a fresh delimiter for that exact raw payload
|
|
47
|
+
over a safe token alphabet (`A-Z`, `0-9`, `_`), for example a random hex
|
|
48
|
+
token, and verify the delimiter is absent as a complete line in that exact
|
|
49
|
+
raw payload before running. Never reuse a fixed delimiter across payloads.
|
|
50
|
+
Open the heredoc with a single-quoted heredoc opener (`<<'DELIM'`) so the
|
|
51
|
+
payload is never interpolated, and close it with a line containing exactly
|
|
52
|
+
that delimiter.
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
# Resolve the validator relative to the skill's own directory.
|
|
56
|
+
SKILL_DIR="<skill directory stated when this skill loads>";
|
|
57
|
+
node "$SKILL_DIR/scripts/validate-review.mjs" screen --reviewer <persona> --harness <opencode|pi|claude-code> <<'SCREEN_INPUT_A1B2C3D4'
|
|
58
|
+
<the persona's returned JSON payload, copied verbatim>
|
|
59
|
+
SCREEN_INPUT_A1B2C3D4
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Read the exit status:
|
|
63
|
+
|
|
64
|
+
- **exit 0** -- structurally admitted. Parse the already structurally
|
|
65
|
+
validated JSON without logging the raw text; the parent may then attach
|
|
66
|
+
both `harness` and `dispatch_outcome` from this result to the persisted
|
|
67
|
+
per-agent dispatch record. `dispatch_outcome: "empty"` means zero
|
|
68
|
+
findings; `"findings"` means one or more admitted findings.
|
|
69
|
+
- **exit 1** -- the whole return is `dispatch_outcome: "malformed"`, covering
|
|
70
|
+
malformed JSON, a schema violation, and a `reviewer`-field identity
|
|
71
|
+
mismatch alike. Retain only the bounded validator diagnostic on stderr in
|
|
72
|
+
Coverage; never parse or persist payload fields or values.
|
|
73
|
+
- **exit 2**, a missing or unreadable helper, or a command launch failure --
|
|
74
|
+
validation unavailable. Withhold the return and report the exact
|
|
75
|
+
unavailability and what was withheld. Update that selected persona's
|
|
76
|
+
preinitialized dispatch entry from `never_returned` to `dispatch_outcome:
|
|
77
|
+
"validation_unavailable"` with `input_finding_count: 0`, and set
|
|
78
|
+
`run_status` to `degraded`. A run containing `validation_unavailable`
|
|
79
|
+
evidence can never finalize as `completed`, and that persona must not have
|
|
80
|
+
an admitted input finding. `validation_unavailable` is not `malformed` and
|
|
81
|
+
is not `never_returned`; they are distinct coverage states. `never_returned`
|
|
82
|
+
is a task-lifecycle fact for a task that did not return at all, recorded
|
|
83
|
+
without invoking the validator.
|
|
84
|
+
|
|
85
|
+
Structural validity never implies evidence validity. A return that passes
|
|
86
|
+
`screen` is admitted structurally only; its claims still require evidence
|
|
87
|
+
assessment during adjudication.
|
|
88
|
+
|
|
89
|
+
## prepare
|
|
90
|
+
|
|
91
|
+
Applies the confidence gate, forms dedup candidate groups, and unions
|
|
92
|
+
selection-surface coverage across every screened return -- deterministic
|
|
93
|
+
parent-owned bookkeeping that no longer needs model recomputation.
|
|
94
|
+
|
|
95
|
+
**Input:** `{ screen_results[{reviewer, result:<screen output>}], selected_dispatches[{persona, dispatch_outcome, selection_surface?}] }`.
|
|
96
|
+
Assemble `screen_results` from every `screen` call made in this run, and
|
|
97
|
+
`selected_dispatches` from the Stage 3 selection record (including any
|
|
98
|
+
persona whose dispatch never produced a screen result, so `never_returned`
|
|
99
|
+
and `validation_unavailable` personas are represented too).
|
|
100
|
+
|
|
101
|
+
**Output:** `{ confidence_dispositions, coverage_union, singletons, candidate_groups[{file, members[{input_id,line}]}], surviving_findings }`.
|
|
102
|
+
`candidate_groups` are the file-grouped, line-sorted sets of two or more
|
|
103
|
+
admitted findings from different personas that the model must adjudicate in
|
|
104
|
+
`merge`. `singletons` are admitted findings that passed the confidence gate
|
|
105
|
+
but did not land in any candidate group -- they need no adjudication and flow
|
|
106
|
+
straight through. `confidence_dispositions` and `coverage_union` are the
|
|
107
|
+
suppressed-finding ledger and the unioned selection-surface coverage list
|
|
108
|
+
respectively; both feed later phases and Coverage reporting without further
|
|
109
|
+
recomputation.
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
# Resolve the validator relative to the skill's own directory.
|
|
113
|
+
SKILL_DIR="<skill directory stated when this skill loads>";
|
|
114
|
+
node "$SKILL_DIR/scripts/validate-review.mjs" prepare <<'PREPARE_INPUT_A1B2C3D4'
|
|
115
|
+
{ "screen_results": [...], "selected_dispatches": [...] }
|
|
116
|
+
PREPARE_INPUT_A1B2C3D4
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Choose a fresh delimiter the same way as `screen`'s. Read the exit status:
|
|
120
|
+
|
|
121
|
+
- **exit 0** -- the aggregate envelope validated; parse the JSON result and
|
|
122
|
+
carry it into adjudication.
|
|
123
|
+
- **exit 1** -- the aggregate envelope was rejected (malformed JSON or a
|
|
124
|
+
structural violation against the screen results and selected dispatches
|
|
125
|
+
supplied). Never hand-assemble a substitute `prepare` output.
|
|
126
|
+
- **exit 2**, a missing helper, or a launch failure -- unavailable; retry
|
|
127
|
+
once with identical bytes per the never-bypass rule above, then stop
|
|
128
|
+
visibly.
|
|
129
|
+
|
|
130
|
+
## merge
|
|
131
|
+
|
|
132
|
+
Applies the model's adjudication decisions to `prepare`'s candidate groups,
|
|
133
|
+
deriving each merged finding's severity, confidence (including the
|
|
134
|
+
cross-reviewer agreement boost), fingerprint, submitters, and conservatively
|
|
135
|
+
narrowed route -- the model supplies judgment per candidate group; the helper
|
|
136
|
+
supplies the arithmetic and the narrowing rule.
|
|
137
|
+
|
|
138
|
+
**Input:** `{ prepared:<prepare output>, adjudication:<model envelope with decisions[]> }`.
|
|
139
|
+
Build one `decisions[]` entry for every candidate-group member: a `merged`
|
|
140
|
+
decision citing 2+ input IDs from the same group with the merged finding's
|
|
141
|
+
`title`, `why_it_matters`, `evidence`, `line`, `proposed_route` (when the
|
|
142
|
+
merge's route should narrow), `route_narrowing_reason` (required whenever
|
|
143
|
+
`proposed_route` is present), and optional `disagreement_facts` and
|
|
144
|
+
`eligible_agreement_credit`; or a `declined` decision citing exactly one
|
|
145
|
+
input ID with a `declined_reason` explaining why it stays a separate defect.
|
|
146
|
+
Every candidate-group member must be cited by exactly one decision -- no
|
|
147
|
+
omissions, no double-citations. `prepare`'s true singletons need no decision
|
|
148
|
+
at all.
|
|
149
|
+
|
|
150
|
+
**Output:** `{ merged_findings, validator_requests[{finding_id,file,line}], disagreement_facts }`.
|
|
151
|
+
`validator_requests` names the merged findings that fall inside the Stage 5b
|
|
152
|
+
gating band (P0/P1 severity, or `requires_verification: true`) -- dispatch
|
|
153
|
+
exactly one validator subagent per entry, looking up that finding's full
|
|
154
|
+
fields from `merged_findings` by `finding_id`.
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
# Resolve the validator relative to the skill's own directory.
|
|
158
|
+
SKILL_DIR="<skill directory stated when this skill loads>";
|
|
159
|
+
node "$SKILL_DIR/scripts/validate-review.mjs" merge <<'MERGE_INPUT_A1B2C3D4'
|
|
160
|
+
{ "prepared": { ... }, "adjudication": { "decisions": [...] } }
|
|
161
|
+
MERGE_INPUT_A1B2C3D4
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Choose a fresh delimiter the same way as `screen`'s. Read the exit status:
|
|
165
|
+
|
|
166
|
+
- **exit 0** -- the adjudication envelope validated against the prepared
|
|
167
|
+
candidate set; parse the JSON result and carry `merged_findings` and
|
|
168
|
+
`validator_requests` forward.
|
|
169
|
+
- **exit 1** -- the envelope was rejected: malformed JSON, a schema
|
|
170
|
+
violation, an omitted or double-cited candidate member, or a merged
|
|
171
|
+
decision inconsistent with its cited members. Correct the envelope once
|
|
172
|
+
per the never-bypass rule, then stop visibly if it is rejected again.
|
|
173
|
+
Never hand-assemble a substitute merged-finding set.
|
|
174
|
+
- **exit 2**, a missing helper, or a launch failure -- unavailable; retry
|
|
175
|
+
once with identical bytes, then stop visibly.
|
|
176
|
+
|
|
177
|
+
## finalize
|
|
178
|
+
|
|
179
|
+
Synthesizes the run's final report and, in writing modes, the persistable
|
|
180
|
+
artifact: reconciles validator lifecycle results against `merge`'s
|
|
181
|
+
`validator_requests`, routes the model's plan-assessment results into
|
|
182
|
+
residual actionable work and advisory output, derives the risk-aware
|
|
183
|
+
verdict, and computes every queue, disposition count, and coverage entry.
|
|
184
|
+
Called once with `applied_fixes: []` to obtain the queues that drive fix
|
|
185
|
+
dispatch; fix-applying modes call it again with the exact applied-fix
|
|
186
|
+
outcomes and the same validator results, and only that second call is
|
|
187
|
+
persisted. Report-only calls it once and writes nothing.
|
|
188
|
+
|
|
189
|
+
**Input:** `{ merge:<merge output>, prepared:<prepare output>, screen_results:<same array given to prepare>, dispatch_records:<same selected_dispatches>, validator_lifecycle_results[{finding_id, result:{outcome:'true'|'false'|'failed'|'unavailable', reason?}}], plan_assessment:{ verdict, results[{kind:'explicit_unmet_requirement'|'inferred_gap', description}] }, parent_run_metadata:{ run_id, mode:'interactive'|'autofix'|'headless'|'report-only', harness, branch, head_sha, selected_dispatches, timestamps:{started_at, completed_at}, validation:{status, reason?}, applied_fixes[] } }`.
|
|
190
|
+
Reuse `prepared`, `screen_results`, and `dispatch_records` wholesale from the
|
|
191
|
+
earlier phases rather than restating them; `finalize` re-derives the ledger,
|
|
192
|
+
rejected-payload weights, coverage notes, and reviewer ownership from this
|
|
193
|
+
carried state. `validator_lifecycle_results` carries one entry per
|
|
194
|
+
`validator_requests` finding ID -- `outcome: "true"` needs no reason;
|
|
195
|
+
`"false"`, `"failed"`, and `"unavailable"` each require one. `plan_assessment`
|
|
196
|
+
is the model's Stage 2b requirements check: each result's `kind` routes it --
|
|
197
|
+
`explicit_unmet_requirement` becomes residual actionable work and blocks a
|
|
198
|
+
clean verdict; `inferred_gap` becomes advisory-only output and never blocks
|
|
199
|
+
the verdict by itself. Neither kind becomes a finding. `parent_run_metadata`
|
|
200
|
+
carries the run's own identity and mode; `validation` is the artifact
|
|
201
|
+
self-validation envelope, but finalize is called before that
|
|
202
|
+
[artifact validation](./synthesis-artifact-contract.md#artifact-validation)
|
|
203
|
+
step can run, so the only truthful value at that point is
|
|
204
|
+
`{status: "not_attempted", reason}` -- finalize rejects any other value.
|
|
205
|
+
The post-write validation status is reported by the parent in the rendered
|
|
206
|
+
Coverage section, never rewritten into the persisted artifact.
|
|
207
|
+
|
|
208
|
+
**Output:** `{ kind:'writing', artifact, report }` or `{ kind:'report_only', ...report }`.
|
|
209
|
+
`report` carries `verdict`, `findings`, `applied_fixes`,
|
|
210
|
+
`residual_actionable_work`, `advisory_outputs`, `coverage`,
|
|
211
|
+
`input_dispositions`, `disposition_counts`, `queues{fixer,residual,report_only}`,
|
|
212
|
+
`pre_existing_findings`, and `risk_coverage` -- render Stage 6 directly from
|
|
213
|
+
this projection rather than recomputing any of it. In writing modes,
|
|
214
|
+
`artifact` is the exact `review-summary.json` payload to persist.
|
|
215
|
+
|
|
216
|
+
```bash
|
|
217
|
+
# Resolve the validator relative to the skill's own directory.
|
|
218
|
+
SKILL_DIR="<skill directory stated when this skill loads>";
|
|
219
|
+
node "$SKILL_DIR/scripts/validate-review.mjs" finalize <<'FINALIZE_INPUT_A1B2C3D4'
|
|
220
|
+
{ "merge": { ... }, "prepared": { ... }, "screen_results": [...], "dispatch_records": [...], "validator_lifecycle_results": [...], "plan_assessment": { ... }, "parent_run_metadata": { ... } }
|
|
221
|
+
FINALIZE_INPUT_A1B2C3D4
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Choose a fresh delimiter the same way as `screen`'s. Read the exit status:
|
|
225
|
+
|
|
226
|
+
- **exit 0** -- parse the JSON result. In writing modes, capture the exact
|
|
227
|
+
stdout bytes to a same-directory temp file (see
|
|
228
|
+
[Persisting the artifact](#persisting-the-artifact) below) before rendering
|
|
229
|
+
the report; report-only renders `report` directly and writes nothing.
|
|
230
|
+
- **exit 1** -- the aggregate envelope was rejected: a shape mismatch against
|
|
231
|
+
`merge`/`prepared`, a missing or extra validator-lifecycle result, an
|
|
232
|
+
invalid plan-assessment envelope, or an artifact that failed its own
|
|
233
|
+
internal schema check. Correct the envelope once per the never-bypass
|
|
234
|
+
rule, then stop visibly if it is rejected again. Never hand-assemble a
|
|
235
|
+
substitute report or artifact.
|
|
236
|
+
- **exit 2**, a missing helper, or a launch failure -- unavailable; retry
|
|
237
|
+
once with identical bytes, then stop visibly.
|
|
238
|
+
|
|
239
|
+
### Persisting the artifact
|
|
240
|
+
|
|
241
|
+
Writing-mode `finalize` stdout is the wrapper
|
|
242
|
+
`{ kind: 'writing', artifact, report }`, not the artifact by itself. In
|
|
243
|
+
interactive, autofix, and headless modes, after the persisted `finalize` call
|
|
244
|
+
succeeds, extract only the captured stdout's `artifact` member -- never the
|
|
245
|
+
whole wrapper -- and write that JSON to a temp file created exclusively with
|
|
246
|
+
owner-only permissions in the same `.context/systematic/ce-review/<run-id>`
|
|
247
|
+
directory as the final artifact, then atomically rename it over
|
|
248
|
+
`review-summary.json`. Render the report from the same captured stdout's
|
|
249
|
+
`report` member. No `jq` dependency is assumed; Node performs the extraction:
|
|
250
|
+
|
|
251
|
+
```bash
|
|
252
|
+
node -e 'const r=JSON.parse(require("fs").readFileSync(0,"utf8"));process.stdout.write(JSON.stringify(r.artifact))' \
|
|
253
|
+
< "$FINALIZE_STDOUT" > "$TEMP_FILE"
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
Remove the temp file on any non-success (a rejected `finalize` call, a write
|
|
257
|
+
failure, or an interrupted run) instead of leaving a partial file behind.
|
|
258
|
+
Only after the rename succeeds does the parent run the existing `artifact`
|
|
259
|
+
subcommand (see the
|
|
260
|
+
[synthesis artifact contract](./synthesis-artifact-contract.md#artifact-validation))
|
|
261
|
+
against the persisted path. Report-only never creates the run directory,
|
|
262
|
+
never writes a temp file, and never runs `artifact` validation -- it has no
|
|
263
|
+
artifact to validate.
|
|
@@ -130,6 +130,7 @@ This fails because: no pipe-delimited tables, no severity-grouped `###` headers,
|
|
|
130
130
|
- **Route column** shows the synthesized handling decision as ``<autofix_class> -> <owner>``.
|
|
131
131
|
- **Header includes** scope, intent, and reviewer team with per-conditional justifications
|
|
132
132
|
- **Mode line** -- include `interactive`, `autofix`, `report-only`, or `headless`
|
|
133
|
+
- **Requirements Completeness section** -- include only when a plan was found in Stage 2b. Render the met/not-addressed/partially-addressed checklist, then list the plan-assessment routing output as its own bullet lists: `explicit_unmet_requirement` results appear as `report.residual_actionable_work` bullets (they gate the verdict); `inferred_gap` results appear as `report.advisory_outputs` bullets (they never gate the verdict alone). Neither kind is a finding -- they never gain a file, line, reviewer, confidence, or route, and never appear in a severity table. Omit the section entirely when no plan was found.
|
|
133
134
|
- **Applied Fixes section** -- include only when a fix phase ran in this review invocation
|
|
134
135
|
- **Residual Actionable Work section** -- include only when unresolved actionable findings were handed off for later work
|
|
135
136
|
- **Pre-existing section** -- separate table, no confidence column (these are informational)
|
|
@@ -137,7 +138,7 @@ This fails because: no pipe-delimited tables, no severity-grouped `###` headers,
|
|
|
137
138
|
- **Learnings & Past Solutions section** -- render only when CE `learnings-researcher` was selected and returned relevant output; results with links to docs/solutions/ files. Omit otherwise.
|
|
138
139
|
- **Agent-Native Gaps section** -- render only when CE `agent-native-reviewer` was selected and returned relevant output. Omit otherwise.
|
|
139
140
|
- **Deployment Notes section** -- key checklist items from deployment-verification-agent. Omit if the agent did not run.
|
|
140
|
-
- **Coverage section** -- suppressed count with original confidences, residual risks, testing gaps, failed reviewers, disposition reconciliation, and risk-coverage entries with their citing input finding IDs and blocked-entry exit conditions. For raw returns, distinguish `findings`, `empty`, `malformed`, `never_returned`, `validation_unavailable` (the persisted raw dispatch outcome, distinct from the artifact-level `validation.status: "unavailable"`), and
|
|
141
|
+
- **Coverage section** -- suppressed count with original confidences, residual risks, testing gaps, failed reviewers, disposition reconciliation, and risk-coverage entries with their citing input finding IDs and blocked-entry exit conditions. For raw returns, distinguish `findings`, `empty`, `malformed`, `never_returned`, and `validation_unavailable` (the persisted raw dispatch outcome, distinct from the artifact-level `validation.status: "unavailable"`), and state what was admitted or withheld
|
|
141
142
|
- **Summary uses blockquotes** for verdict, reasoning, and fix order
|
|
142
143
|
- **Horizontal rule** (`---`) separates findings from verdict
|
|
143
144
|
- **`###` headers** for each section -- never plain text headers
|