@mrciphersmith/keryx 0.3.84 → 0.3.86
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -1
- package/dist/cli.js +2552 -2309
- package/dist/core.js +1145 -1034
- package/package.json +1 -1
- package/src/gdskills/bundled/rules/core/model-selection.mdc +6 -0
- package/src/gdskills/bundled/skills/orchestration/flow-orchestrator/SKILL.md +22 -22
- package/src/gdskills/bundled/skills/orchestration/job-orchestrator/SKILL.md +28 -36
- package/src/gdskills/bundled/skills/planning/autodoc-orchestrator/SKILL.md +1 -2
- package/src/gdskills/bundled/skills/review/review-orchestrator/SKILL.detail.md +289 -0
- package/src/gdskills/bundled/skills/review/review-orchestrator/SKILL.md +116 -213
- package/src/gdskills/bundled/skills/review/review-orchestrator/review-context.schema.json +57 -0
- package/src/gdskills/bundled/skills/review/review-orchestrator/templates/review-report.md +1 -1
|
@@ -18,14 +18,16 @@ metadata:
|
|
|
18
18
|
compatible_harnesses: "cursor,codex,zed,opencode,claude"
|
|
19
19
|
license: "MIT"
|
|
20
20
|
---
|
|
21
|
-
|
|
22
21
|
# Review Orchestrator
|
|
23
22
|
|
|
24
|
-
|
|
25
|
-
dispatches specialized reviewers in parallel, then consolidates their findings into one
|
|
26
|
-
unified report sorted by severity. It does not perform any review logic itself.
|
|
23
|
+
**You are the orchestrator. Run the round yourself, inline, in the main session that loaded this skill; never hand the whole round to one subagent.** This skill routes: it detects scope, dispatches specialized reviewers, and consolidates their findings into one report sorted by severity. The reviewers do the reviewing.
|
|
27
24
|
|
|
28
|
-
|
|
25
|
+
## Preflight — before Step 0, in this order
|
|
26
|
+
|
|
27
|
+
1. **Can it spawn?** The round needs the `Agent` tool (or the host's equivalent) and depth budget left: Claude Code nests subagents at most 3 deep, and at the limit `Agent` is withheld. If it cannot spawn, the whole output is this line, with no tool calls and no review of your own:
|
|
28
|
+
`STATUS: BLOCKED nested_dispatch_unavailable — run this skill in the main session`
|
|
29
|
+
2. **Is a round needed?** A diff of only docs, markdown, comments, formatting, lockfiles or generated files, with no CI, auth, migration or rule-bearing config, ends here with one line, `round not needed: <reason>`, and no reviewer. `--all`, an explicit reviewer list or a gating round forces a round.
|
|
30
|
+
3. **Round ceiling.** 5 rounds (`REVIEW_ROUND_CAP`) over the same `merge-base..HEAD`. Past it: interactive, ask the operator; unattended, `STATUS: BLOCKED round_cap`.
|
|
29
31
|
|
|
30
32
|
## Workflow
|
|
31
33
|
|
|
@@ -38,18 +40,18 @@ Review Orchestrator Progress:
|
|
|
38
40
|
- [ ] Step 3: Build the bounded scope with `keryx review scope` — never by hand
|
|
39
41
|
- [ ] Step 3b: On a deep round, compute scope B with `keryx review blast-radius` — never by browsing — and KEEP the `--json` file; `review ingest --blast-radius <file>` is refused without it
|
|
40
42
|
- [ ] Step 4: Parse flags / auto-detect domain from scope
|
|
41
|
-
- [ ] Step 5:
|
|
43
|
+
- [ ] Step 5: Start questions, once, after Steps 1 and 3 — reviewers (default `--all`), counterpart verification (default on), model plan; unattended takes the defaults (SKILL.detail.md § "Start questions"; legacy/profile reviewers stay flag-only)
|
|
42
44
|
- [ ] Step 5c: When `review.jev.select` is on, run `keryx review jev-select` over the finalized candidate set and drop its `skip` decisions from Wave A/B — see "Reviewer selection with Jev"
|
|
43
|
-
- [ ] Step 6: Plan sub-agent dispatch and token budgets
|
|
45
|
+
- [ ] Step 6: Plan sub-agent dispatch and token budgets; compute each dispatch's model with `keryx review tier` (6a: publish this plan)
|
|
44
46
|
- [ ] Step 7: Stage 1 gate - spec compliance check (if issue/task provided)
|
|
45
|
-
- [ ] Step 8: Dispatch selected reviewers in
|
|
47
|
+
- [ ] Step 8: Dispatch selected reviewers in parallel with the reviewer-input schema — see "How to spawn"
|
|
46
48
|
- [ ] Step 9: Collect reviewer-finding schema results and handle NEEDS_CONTEXT
|
|
47
49
|
- [ ] Step 9b: `jev-triage` — advisory, annotate-only severity/duplicate/verify-order annotations over the consolidated findings (opt-in `review.jev.triage`)
|
|
48
|
-
- [ ] Step 10: Wave C — dispatch `review-verifier` over the consolidated findings
|
|
50
|
+
- [ ] Step 10: Wave C — when blockers/majors exist, `--verify` is set, or the PR is high-risk, dispatch `review-verifier` over the consolidated findings
|
|
49
51
|
- [ ] Step 11: Sort by severity, deduplicate, emit unified report
|
|
50
52
|
- [ ] Step 12: Emit the machine-readable `keryx:findings` block alongside the report
|
|
51
53
|
- [ ] Step 13: Report the stage counts: dropped by pre-filter, refuted by the verifier, retained
|
|
52
|
-
- [ ] Step 14:
|
|
54
|
+
- [ ] Step 14: Managed rounds only (not `lightweight`, which is report-only), after the final round, on an open pull request at the head you reviewed — answer every external comment once with `keryx review comments reply --final` (render with `--dry-run` and get the user's explicit approval first; unattended: do not publish, hand the rendered body back) — never against a pull request the dispatch named as the caller's
|
|
53
55
|
```
|
|
54
56
|
|
|
55
57
|
Step 0 runs on **every** round. Step 14 runs **once**, after the last one. They are
|
|
@@ -59,22 +61,28 @@ one comment.
|
|
|
59
61
|
|
|
60
62
|
### Step 6 — the model is computed, not chosen
|
|
61
63
|
|
|
62
|
-
Before dispatching each reviewer, run `keryx review tier` with the signals you
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
64
|
+
Before dispatching each reviewer, run `keryx review tier` with the signals you hold (`--scope`, `--findings`, `--diff-lines`, `--fix-attempt`, `--verifier`, `--security`, `--forced-strategy-change`) and paste the `model` block it prints into that dispatch. The tier comes from the command, not from reading `rules/core/model-selection.mdc`.
|
|
65
|
+
|
|
66
|
+
It names a model only if you gave it your session (`--session-provider`/`--session-model` or `KERYX_SESSION_*`) and discovery found another. Otherwise it prints `inherit: true` with the tier — **adaptive**: pick your own runtime's model for that tier (`standard` = your session model; `light`/`deep` = your runtime's lighter/most capable class if its dispatch tool offers one, else the session model). A model id never goes into a dispatch by hand.
|
|
67
|
+
|
|
68
|
+
### Step 6a — the plan bridge
|
|
69
|
+
|
|
70
|
+
Publish this checklist with `plan_set` under the ids `step-0`…`step-14` once, here, when scope and dispatch are fixed. Publish each of steps 0–5 that ran as `completed` and each that did not run as `skipped`; publish steps 6–14 as `pending` (`in_progress` for the one starting now). From then on move each item with `plan_update` as its step completes — see the `session-plan-bridge` rule. `completed` means the step's artifact exists: steps 8–10 become `completed` (in the plan and in the report) only after `keryx review ingest` has recorded their results, and a step whose reviewers were `BLOCKED` is `blocked`.
|
|
66
71
|
|
|
67
|
-
|
|
68
|
-
Working it out in your head is exactly the mechanical step that rule moves into
|
|
69
|
-
code — and it is the step that was documented as running for a whole release
|
|
70
|
-
while nothing called it.
|
|
72
|
+
## How to spawn (Steps 8 and 10)
|
|
71
73
|
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
74
|
+
- One `Agent` call per reviewer, all calls of a wave in a single message, so the wave runs in parallel.
|
|
75
|
+
- Use the reviewer's exact `subagent_type`. If the runtime has no such type, use `general-purpose` and name the reviewer's `SKILL.md` path in the prompt. Neither available: that reviewer is `BLOCKED`.
|
|
76
|
+
- Prompts carry pointers, not pasted files — the diff, the SHAs, the PR, the schema paths — and never paraphrase a contract.
|
|
77
|
+
- A reply is a result only if it starts with a `STATUS:` line and carries a `REVIEW_RESULT` block matching `reviewer-finding.schema.json`; anything else is `BLOCKED`.
|
|
78
|
+
- The model comes from `keryx review tier` (Step 6).
|
|
79
|
+
|
|
80
|
+
## Control
|
|
81
|
+
|
|
82
|
+
- **Profiles.** The default is read-only: one round, no edits. `--fix` hands the findings to `flow-orchestrator` or `task-implementer`; the orchestrator never edits code. Control returns to this main session after each hand-off, which loads this skill again (never the implementer, which would nest); each re-review counts toward the ceiling (it adds 1 to the count).
|
|
83
|
+
- **Waves.** Split by diff domain (`core` = logic, security, architecture; `domain` = frontend, backend, conventions; `support` = style, tests), in that order; A/B/C below keep their dependency order. A full or `--all` round runs at most 10 reviewers per wave (`MAX_REVIEWERS_PER_WAVE_FULL`).
|
|
84
|
+
- **Rate limits.** On a reviewer dispatch that fails with a rate-limit or overloaded error (reported as `STATUS: RATE_LIMITED`, a harness signal, not a schema status), halve the wave, requeue the rest and honour `retry-after`. After at most two halvings (`MAX_RATE_HALVINGS`) stop with `STATUS: BLOCKED rate_limited`, naming the queue left.
|
|
85
|
+
- **Done when** every dispatched reviewer's result is recorded with `keryx review ingest`, the stage counts and **Not run** are in the report, and `keryx review complete` has run.
|
|
78
86
|
|
|
79
87
|
---
|
|
80
88
|
|
|
@@ -144,7 +152,7 @@ which of its inputs were recovered and which were never written down.
|
|
|
144
152
|
| `context_doc` | string | no | Path to job context document (e.g., `.metaproject/jobs/<job>/ai/context.md`). |
|
|
145
153
|
| `context_mode` | string | no | `none`, `light`, or `full`. Default: `light` for PR review, `none` for small path reviews. `full` may call `context-collector` before dispatch. |
|
|
146
154
|
| `token_budget` | object | no | Optional budget controls: `{total, per_reviewer, diff_max_chars, file_max_chars}`. |
|
|
147
|
-
| `model_strategy` | string | no | `ask` or `adaptive`. Default: `adaptive
|
|
155
|
+
| `model_strategy` | string | no | `ask` or `adaptive`. Default: `ask` in an interactive round (shown once as Start question 3), `adaptive` unattended; the model per dispatch is computed by `keryx review tier`, never chosen by hand (see Step 6). |
|
|
148
156
|
| `managed_review` | object | no | Optional managed review mode: `{mode, target, target_ref, flow_id, reviewers}` where mode is `lightweight`, `attach-review`, `review-flow`, or `ingest`. |
|
|
149
157
|
| `verification_mode` | string | no | `off`, `annotate`, or `filter`. Default `annotate` — verdicts are recorded and nothing is removed. See Wave C. |
|
|
150
158
|
| `pr_comments` | object | no | `{enabled, max_replies_total, max_sentences_per_reply}`. Defaults: enabled when a PR exists, `30`, `2`. Collect every round, reply once at the end. See External PR comments. |
|
|
@@ -522,9 +530,10 @@ The rule that follows for reviewers, and that belongs in the dispatch prompt:
|
|
|
522
530
|
> producer disproves, and a defect missed because the producer's actual default was
|
|
523
531
|
> assumed rather than read.
|
|
524
532
|
|
|
525
|
-
If the
|
|
526
|
-
|
|
527
|
-
|
|
533
|
+
If you cannot locate the producer, ask the operator where it is (Start question 2); record `state: unavailable` only when it cannot be read, and leave the dependent findings at `info`.
|
|
534
|
+
An unavailable producer is a result. Assuming one is not. An input rule records the producer's full check set and is verified every round,
|
|
535
|
+
paired PR or not — SKILL.detail.md § "Counterpart verification". Unattended, nobody is asked: record
|
|
536
|
+
`state: unavailable` with the reason and continue.
|
|
528
537
|
|
|
529
538
|
A finding whose evidence lives in the producer sets `repo` on the finding itself.
|
|
530
539
|
`file` and `line` alone name a path in the repository under review, so without
|
|
@@ -600,8 +609,8 @@ Context modes:
|
|
|
600
609
|
- `full`: for large/high-risk PRs or user request. Invoke `context-collector` first, then pass the resulting context path and summary to reviewers.
|
|
601
610
|
|
|
602
611
|
High-risk triggers for `full` recommendation:
|
|
603
|
-
- Auth, permissions, API contracts, migrations, shared core, state management, graph/flow, security, performance-critical paths.
|
|
604
|
-
- More than 20 changed source files or more than 2,000 changed lines.
|
|
612
|
+
- Auth, permissions, API contracts, a client-side mirror of a server rule (validation, limit, permission, uniqueness, reserved names), migrations, shared core, state management, graph/flow, security, performance-critical paths.
|
|
613
|
+
- More than 20 changed source files or more than 2,000 changed lines. Size never lowers depth below the risk class (SKILL.detail.md § "Depth floor").
|
|
605
614
|
- Missing or ambiguous linked requirements.
|
|
606
615
|
|
|
607
616
|
If `full` context would be useful but was not explicitly requested, ask once:
|
|
@@ -615,6 +624,8 @@ This PR touches high-risk areas. Build full review context before dispatching re
|
|
|
615
624
|
> pick a letter (default: A)
|
|
616
625
|
```
|
|
617
626
|
|
|
627
|
+
Unattended: take A and record it in the review scope.
|
|
628
|
+
|
|
618
629
|
---
|
|
619
630
|
|
|
620
631
|
## Token and Context Budget Management
|
|
@@ -643,31 +654,22 @@ Default budget guidance:
|
|
|
643
654
|
|
|
644
655
|
## Model Strategy
|
|
645
656
|
|
|
646
|
-
The model
|
|
647
|
-
|
|
648
|
-
the `model` block it prints into that dispatch. Do not assign a model class by
|
|
649
|
-
reading a table or by hand; that mechanical step is exactly what the command
|
|
650
|
-
replaced (see Step 6 above and `job-orchestrator` Step 5, which pins the same
|
|
651
|
-
rule: `model_strategy: "current"` is gone).
|
|
652
|
-
|
|
653
|
-
`model_strategy` now governs only how the *computed* model is surfaced, not
|
|
654
|
-
whether it is computed:
|
|
657
|
+
The model per dispatch is computed by `keryx review tier` (Step 6) and never written by hand.
|
|
658
|
+
`model_strategy` governs only how the computed model is surfaced:
|
|
655
659
|
|
|
656
660
|
| `model_strategy` | Behavior |
|
|
657
661
|
|---|---|
|
|
658
|
-
| `ask` | Present the `keryx review tier` model plan once before dispatch. |
|
|
659
|
-
| `adaptive` (default) | Dispatch on the computed model directly; record the chosen model per reviewer in the final report metadata. |
|
|
662
|
+
| `ask` (default, interactive) | Present the `keryx review tier` model plan once before dispatch — Start question 3; the operator may accept it, take all `deep`, or set it. |
|
|
663
|
+
| `adaptive` (default, unattended) | Dispatch on the computed model directly; record the chosen model per reviewer in the final report metadata. |
|
|
660
664
|
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
- `inherit: true` is the adaptive answer: dispatch on your runtime's model for the block's `tier` (Step 6). Record `model_assignment: adaptive` and the model you actually chose — not a failure.
|
|
664
|
-
- With `model_strategy: ask`, present the model plan once before dispatch, then proceed with the computed model.
|
|
665
|
+
`inherit: true` is the adaptive answer: dispatch on your runtime's model for the block's `tier`, and
|
|
666
|
+
record `model_assignment: adaptive` with the model you chose.
|
|
665
667
|
|
|
666
668
|
---
|
|
667
669
|
|
|
668
670
|
## Scope Detection
|
|
669
671
|
|
|
670
|
-
###
|
|
672
|
+
### Fix round check (Step 1a)
|
|
671
673
|
|
|
672
674
|
A **fix round** is any review of work produced to answer earlier findings. Set
|
|
673
675
|
`is_fix_round: true` on every reviewer input, and populate `prior_findings` with
|
|
@@ -706,63 +708,14 @@ Recording the enumeration matters as much as doing it: a round that searched and
|
|
|
706
708
|
found nothing is a different fact from a round that never searched, and only the
|
|
707
709
|
recorded list distinguishes them.
|
|
708
710
|
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
715
|
-
| Disposition | Meaning |
|
|
716
|
-
|---|---|
|
|
717
|
-
| `closed` | Checked against the code, not against the commit message, and the defect is gone |
|
|
718
|
-
| `open` | The fix does not reach the defect; say what is still true |
|
|
719
|
-
| `partial` | One site of the class was fixed and the enumeration named others |
|
|
720
|
-
| `regressed` | The fix removed this defect and introduced another — file the new one separately |
|
|
721
|
-
| `withdrawn` | The finding was wrong. See below |
|
|
722
|
-
|
|
723
|
-
**Check the code, not the commit message.** A commit titled *"report an abandoned
|
|
724
|
-
sync as abandoned"* is a claim; the disposition is whether the branch it renamed
|
|
725
|
-
is reachable and pinned. On a recorded round, two such commits asserted behaviour
|
|
726
|
-
on lines no test could reach.
|
|
727
|
-
|
|
728
|
-
#### A fix is a change, and changes get reviewed
|
|
729
|
-
|
|
730
|
-
The most expensive class in a multi-round review is **the defect the fix
|
|
731
|
-
introduced**. It is systematically under-found, for a structural reason: the fix
|
|
732
|
-
arrives framed as the answer to a finding, so it is read as an answer rather than
|
|
733
|
-
as new code. It is new code.
|
|
734
|
-
|
|
735
|
-
So scope A of a fix round includes the fix, reviewed on its own merits, and the
|
|
736
|
-
report carries its own section:
|
|
737
|
-
|
|
738
|
-
```markdown
|
|
739
|
-
## Regressions the fixes introduced
|
|
740
|
-
<[F-NNN] — the finding it was answering, and the new defect it created>
|
|
741
|
-
```
|
|
711
|
+
Every prior finding leaves the round with a disposition — `closed`, `open`, `partial`,
|
|
712
|
+
`regressed` or `withdrawn` — one line each in the report before the new findings, checked against
|
|
713
|
+
the code and not the commit message. A fix is new code: scope A includes it, and the report carries
|
|
714
|
+
a `## Regressions the fixes introduced` section. A finding this round disproves is withdrawn at the
|
|
715
|
+
top of the report with its evidence. The table, recorded shapes and rationale:
|
|
716
|
+
`SKILL.detail.md` § "Fix rounds — dispositions, regressions, withdrawal".
|
|
742
717
|
|
|
743
|
-
|
|
744
|
-
an early return added to stop a fall-through, which then skipped the work the
|
|
745
|
-
caller needed; a persistence call added to save expanded state, which then
|
|
746
|
-
persisted the broken state on the error path too. Both were closed correctly and
|
|
747
|
-
both shipped a new bug in the same commit.
|
|
748
|
-
|
|
749
|
-
#### Withdrawing your own earlier finding
|
|
750
|
-
|
|
751
|
-
A finding from a previous round that this round disproves is **withdrawn**,
|
|
752
|
-
explicitly, at the top of the report, with the evidence — before any new finding.
|
|
753
|
-
|
|
754
|
-
This is not a courtesy. An uncorrected wrong finding costs the author a fix they
|
|
755
|
-
did not need, and it stays in `prior_findings` steering later rounds. Withdrawal
|
|
756
|
-
is also the one self-correction this pipeline permits, and it is permitted because
|
|
757
|
-
it is asymmetric: it *deletes* a claim, so it cannot inflate the finding count,
|
|
758
|
-
which is the failure mode that removed the re-scoring pass from Wave C.
|
|
759
|
-
|
|
760
|
-
State what made the original claim wrong, in one sentence, and if the same
|
|
761
|
-
reasoning error has now happened twice in one review, say that too. A reviewer
|
|
762
|
-
that names its own recurring error is calibrating; one that quietly drops a
|
|
763
|
-
finding is hiding a result.
|
|
764
|
-
|
|
765
|
-
### Step 1: Determine Review Mode
|
|
718
|
+
### Step 2: Determine Review Mode
|
|
766
719
|
|
|
767
720
|
Before anything else, determine whether the request is **diff mode** or **path mode**:
|
|
768
721
|
|
|
@@ -836,72 +789,23 @@ keryx review blast-radius --ref "${BASE_SHA}" --brief # wha
|
|
|
836
789
|
scope-B finding arrives without it — pass it back as
|
|
837
790
|
`review ingest ... --blast-radius blast-radius.json`.
|
|
838
791
|
|
|
839
|
-
|
|
840
|
-
|
|
841
|
-
file
|
|
842
|
-
built graph — run `keryx gdgraph build` if it refuses.
|
|
843
|
-
|
|
844
|
-
**Do not pick the files yourself, and do not widen it.** "Review the
|
|
845
|
-
functionality so nothing breaks" naively means "review the whole repository every
|
|
846
|
-
round", which is unaffordable *and* actively harmful: review quality decays as
|
|
847
|
-
context grows — measured F1 0.65 at round 2 falling to 0.29 at round 10. An
|
|
848
|
-
unbounded scope B makes later rounds worse than earlier ones.
|
|
849
|
-
|
|
850
|
-
The bounds are measured on this repository, not guessed: at depth 2 the set is a
|
|
851
|
-
median of 19 files (p90 65); depth 3 buys eight more in the median and doubles
|
|
852
|
-
the p90. The 40-file cap fires on 25% of commits and removes only hop-2 entries
|
|
853
|
-
on all but 2 of 80, so it almost never costs a direct dependent — and when it
|
|
854
|
-
does, it says so.
|
|
855
|
-
|
|
856
|
-
**Record the whole thing.** `--out "<review-package>/blast-radius.md"` writes the
|
|
857
|
-
set, the depth, and **every file the cap removed**. A truncation nobody can see
|
|
858
|
-
reads afterwards as "we checked everything", which is the claim this pipeline
|
|
859
|
-
exists to stop making. An empty radius is reported as `unresolved`, not as clean:
|
|
860
|
-
the graph indexes code, so a change to a skill, a rule or a schema has no blast
|
|
861
|
-
radius at all and that is a different fact from "nothing depends on it".
|
|
862
|
-
|
|
863
|
-
#### The scope-B question, and what is rejected
|
|
864
|
-
|
|
865
|
-
> Does this change break an existing behaviour **at these sites**?
|
|
866
|
-
|
|
867
|
-
Nothing else. The blast-radius set is **under regression check, not under
|
|
868
|
-
review**. A finding about style, naming or architecture in code the change did
|
|
869
|
-
not touch is refused **by the orchestrator in code** — not discouraged here —
|
|
870
|
-
under three rules, every one of them a fact about the claim rather than about who
|
|
871
|
-
made it:
|
|
872
|
-
|
|
873
|
-
| Rule | Refused because |
|
|
874
|
-
|---|---|
|
|
875
|
-
| `outside-set` | the file is neither in the computed set nor in the changed set; the reviewer went browsing |
|
|
876
|
-
| `non-regression-severity` | below `major`. Under the canonical rubric `minor` states the code behaves correctly and `info` names neither trigger nor outcome; neither can be a claim that something broke |
|
|
877
|
-
| `no-link-to-change` | nothing in the finding names a changed file, module or symbol. A regression claim says THE CHANGE broke this site |
|
|
878
|
-
|
|
879
|
-
Rejections are **recorded, not deleted** — raise the observation under scope A or
|
|
880
|
-
as a separate review. Pass `--brief` output verbatim into the scope-B dispatch:
|
|
881
|
-
the code rejection is the enforcement, but a reviewer told afterwards has already
|
|
882
|
-
spent the round producing findings that will all be refused.
|
|
883
|
-
|
|
884
|
-
`class_scope` on a scope-B finding names the **caller that breaks**, not the
|
|
885
|
-
changed line, because that is the site a human has to look at.
|
|
792
|
+
Scope B is bounded and computed: distance ≤ 2 from every changed file, at most 40 files, closest
|
|
793
|
+
first. Do not pick the files or widen the set. Record it with `--out "<review-package>/blast-radius.md"`,
|
|
794
|
+
including every file the cap removed; an empty radius is `unresolved`, not clean.
|
|
886
795
|
|
|
887
|
-
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
| 2..N | yes | recomputed only if the changed-file set moved |
|
|
893
|
-
| final | yes | **yes, always** |
|
|
796
|
+
A scope-B reviewer answers one question: does this change break an existing behaviour **at these
|
|
797
|
+
sites**? The orchestrator refuses, in code, a finding that is `outside-set`, below `major`
|
|
798
|
+
(`non-regression-severity`), or names nothing the change touched (`no-link-to-change`) — pass
|
|
799
|
+
`--brief` verbatim so no reviewer spends the round on findings that will be refused. `class_scope`
|
|
800
|
+
names the caller that breaks.
|
|
894
801
|
|
|
895
|
-
|
|
802
|
+
Recompute scope B on rounds 2..N only when the changed-file set moved, and always on the final round:
|
|
896
803
|
|
|
897
804
|
```bash
|
|
898
805
|
keryx review blast-radius --ref "${BASE_SHA}" --previous blast-radius.json [--final]
|
|
899
806
|
```
|
|
900
807
|
|
|
901
|
-
|
|
902
|
-
nothing moved. The final round recomputes whatever the file set did — otherwise a
|
|
903
|
-
fix introduced in round 3 gets no regression check at all, and the round that
|
|
904
|
-
certifies the flow is the one that checked the least.
|
|
808
|
+
The measurements, the rule table and the recompute table: `SKILL.detail.md` § "Scope B — bounds, rejections and recompute".
|
|
905
809
|
|
|
906
810
|
### Path Mode
|
|
907
811
|
|
|
@@ -909,10 +813,10 @@ When a path or target is named, collect the candidate files:
|
|
|
909
813
|
|
|
910
814
|
```bash
|
|
911
815
|
# If a directory path is given:
|
|
912
|
-
|
|
816
|
+
keryx ctx rg --files -g '*.{ts,tsx,js,jsx}' <path> | sort
|
|
913
817
|
|
|
914
818
|
# If a module name is given (e.g. "UserStore", "pipelines module"):
|
|
915
|
-
|
|
819
|
+
keryx ctx rg --files -g '*<name>*.{ts,tsx}'
|
|
916
820
|
# Also check common locations: src/stores/, src/modules/, src/components/
|
|
917
821
|
```
|
|
918
822
|
|
|
@@ -1052,30 +956,30 @@ what to do with it: `SKILL.detail.md` beside this file, section "Project-local
|
|
|
1052
956
|
reviewers". A description saying another entry point dispatches it is its
|
|
1053
957
|
author's routing note, not a restriction.
|
|
1054
958
|
|
|
1055
|
-
###
|
|
959
|
+
### Start questions (Step 5)
|
|
1056
960
|
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
`--flow-graph`, or `--all`, ask before dispatch:
|
|
961
|
+
Ask once, after the context pack (Step 1) and the scope (Step 3), in one interaction — one
|
|
962
|
+
`AskUserQuestion` call on Claude Code, the lettered prompt in SKILL.detail.md elsewhere:
|
|
1060
963
|
|
|
1061
|
-
|
|
1062
|
-
|
|
1063
|
-
|
|
1064
|
-
|
|
1065
|
-
|
|
1066
|
-
|
|
964
|
+
1. **Reviewers** — default `--all`; detected convention reviewers are included. Narrowing
|
|
965
|
+
(tests only, one domain, custom) is the operator's explicit choice. Passed flags answer it.
|
|
966
|
+
Legacy/profile reviewers stay flag-only and are never prompted.
|
|
967
|
+
2. **Counterpart** — show what the context pack found for the other side of the contract
|
|
968
|
+
(endpoints, mirrored rules, a paired PR, or "no link found") and a checkbox to verify it,
|
|
969
|
+
ON by default even with no link. If the round cannot locate that code, it asks where it is.
|
|
970
|
+
Verification is not limited to a paired PR: contracts and logic are re-read every round.
|
|
971
|
+
3. **Models** — complexity, the per-role tier from `keryx review tier`, the model each tier
|
|
972
|
+
resolves to on this host; accept (default), all `deep`, or custom. Once per round.
|
|
1067
973
|
|
|
1068
|
-
|
|
1069
|
-
|
|
1070
|
-
|
|
1071
|
-
|
|
1072
|
-
- review-flow-graph: <why detected, or omit if not detected>
|
|
1073
|
-
```
|
|
974
|
+
Unattended rounds ask nothing: `--all`, counterpart verified, computed models — recorded in
|
|
975
|
+
`### How this review was run`. In a `job-orchestrator` pipeline the job setting
|
|
976
|
+
`convention_reviewers` still answers question 1 (unresolved: include all detected reviewers
|
|
977
|
+
and record that choice in the review scope).
|
|
1074
978
|
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
979
|
+
A gating round runs what these answers selected; a host that cannot dispatch subagents
|
|
980
|
+
returns `STATUS: BLOCKED` (`nested_dispatch_unavailable`) instead of reviewing alone (Preflight 1). Every
|
|
981
|
+
round also carries the always-on obligations — SKILL.detail.md § "Depth floor",
|
|
982
|
+
§ "Counterpart verification", § "Round obligations".
|
|
1079
983
|
|
|
1080
984
|
### Reviewer selection with Jev (`jev-select`) — advisory, before dispatch (Step 5c)
|
|
1081
985
|
|
|
@@ -1096,8 +1000,8 @@ Legacy/profile reviewers are specialized review profiles that predate the review
|
|
|
1096
1000
|
| `--mobx-store` | `code-mobx-store-review` |
|
|
1097
1001
|
| `*.store.ts`, `makeObservable`, `observable`, `computed`, `action.bound` | suggest `code-mobx-store-review` as optional profile reviewer |
|
|
1098
1002
|
|
|
1099
|
-
Legacy/profile reviewers are never auto-included and never prompted for — do not
|
|
1100
|
-
about them. They are exempt from the finding contract (for example `code-ai-review` emits
|
|
1003
|
+
Legacy/profile reviewers are never auto-included and never prompted for, interactive or unattended — do not
|
|
1004
|
+
ask about them. They are exempt from the finding contract (for example `code-ai-review` emits
|
|
1101
1005
|
free-prose Russian with no per-finding severity field, so its output cannot be normalised into
|
|
1102
1006
|
the unified report), which is why inclusion must be a deliberate, explicit act rather than a
|
|
1103
1007
|
default the user has to opt out of on every review. Dispatch them ONLY when the user passes one
|
|
@@ -1185,7 +1089,11 @@ whoever reads the merge commit a year later reads the body. When `review.jev.con
|
|
|
1185
1089
|
|
|
1186
1090
|
## Dispatching Reviewers
|
|
1187
1091
|
|
|
1188
|
-
Dispatch selected
|
|
1092
|
+
**Dispatch integrity is a completion prerequisite.** Use one distinct execution per selected reviewer role; do not combine multiple roles in one child or count sections of one response as independent runs. When budgets are tight, use sequential waves, not role batching. Resolve every project-rule anchor before dispatch; missing rules require context recovery or a blocked result, never silently skipped checks. A read-only pass cannot discharge executable obligations: dispatch a capable execution pass and retain its evidence. Partial scope and NEEDS_CONTEXT remain incomplete until rerun over the outstanding scope.
|
|
1093
|
+
|
|
1094
|
+
New managed packages declare `reviewerDispatch: "research.json"`. Alongside the research ledger, provide `dispatch: { version: 1, selected: [...], unresolvedRules: [], runs: [...] }`. Preserve the actual computed reviewer inventory. Each run has `reviewer`, unique `executionId`, `status: "complete"`, `scopeComplete: true`, `rawEvidence`, nonempty `ruleEvidence`, and boolean `executionRequired`. Required execution needs nonempty `executionEvidence` (command/output references); a source-only scope requires `executionReason`. Do not fabricate IDs or mark truncated responses complete. Runtime validates these declarations at closure and the flow gate, not their truth: independently audit evidence and inventory before freezing. Legacy artifact absence is compatibility, not proof of a correct launch. Research closure is mandatory: see [Runtime research-completion ledger](SKILL.detail.md#runtime-research-completion-ledger).
|
|
1095
|
+
|
|
1096
|
+
Dispatch selected reviewers in parallel when independent — this engine dispatches them itself (see "How to spawn"); a host without a dispatch tool returns `STATUS: BLOCKED` (`nested_dispatch_unavailable`) and never reviews alone — Preflight 1. Waves follow the diff domains in "Control" and keep their dependency order:
|
|
1189
1097
|
|
|
1190
1098
|
1. Wave A - core correctness/risk reviewers: logic, architecture, security/highload when selected.
|
|
1191
1099
|
2. Wave B - domain reviewers: frontend/backend/testing/convention reviewers filtered to relevant files. `review-jev-rules` (flow 330), `review-jev-risk`/`review-jev-scenarios` (flow 332), `review-jev-docs`/`review-jev-comments` (flow 333), `review-jev-contract` (flow 335) also run here, CLI-engine not sub-agent, `"engine": "jev"` in `keryx review reviewers --json`, gated on their own opt-in and a resolvable Jev/OpenRouter credential — measured verdicts per reviewer (keep off by default, experimental, etc.) in `SKILL.detail.md` § "CLI-engine reviewers".
|
|
@@ -1204,34 +1112,17 @@ not something Wave C can hand you.** If Wave B skips its mutation pass, that
|
|
|
1204
1112
|
finding class is unreachable for the entire round, and no amount of verification
|
|
1205
1113
|
downstream recovers it. When a testing reviewer returns without a mutation table,
|
|
1206
1114
|
treat it the way you would treat a reviewer that returned without findings *and*
|
|
1207
|
-
without evidence: ask once, then record that the pass did not run.
|
|
1115
|
+
without evidence: ask once, then record that the pass did not run. Unattended: record
|
|
1116
|
+
`Not run: mutation pass` without asking.
|
|
1208
1117
|
|
|
1209
1118
|
### Wave C — verification, and what it replaced
|
|
1210
1119
|
|
|
1211
|
-
Wave C
|
|
1212
|
-
|
|
1213
|
-
|
|
1214
|
-
|
|
1215
|
-
|
|
1216
|
-
-
|
|
1217
|
-
**GPT-3.5 on CommonSenseQA: 75.8 → 38.1.** Among the answers that changed,
|
|
1218
|
-
correct → incorrect exceeded incorrect → correct (Huang et al., *Large Language
|
|
1219
|
-
Models Cannot Self-Correct Reasoning Yet*, ICLR 2024, arXiv:2310.01798).
|
|
1220
|
-
- **Self-Refine (arXiv:2303.17651): +49.2 on dialogue response generation, +0.2
|
|
1221
|
-
on maths.** Self-refinement gains are on subjective tasks and vanish on
|
|
1222
|
-
verifiable reasoning. Judging whether a null-guard is missing is verifiable
|
|
1223
|
-
reasoning.
|
|
1224
|
-
|
|
1225
|
-
Re-scoring a finding by re-reading it is therefore not a rigour pass; it is a
|
|
1226
|
-
coin flip weighted toward more findings. **Do not restore it because it looks
|
|
1227
|
-
obviously useful — it looked obviously useful the first time.**
|
|
1228
|
-
|
|
1229
|
-
`review-verifier` occupies the slot and differs in exactly one way that matters:
|
|
1230
|
-
**it runs something.** Verification that executes rejects 85–96% of false reports
|
|
1231
|
-
against 4–15% unaided while finding 30–44% more true bugs (AnyPoC,
|
|
1232
|
-
arXiv:2604.11950); Meta's TestGen-LLM funnel discards 75% of its own output
|
|
1233
|
-
(75% build → 57% build and pass → 25% improve coverage) and the surviving quarter
|
|
1234
|
-
reaches 73% human acceptance (arXiv:2402.09171).
|
|
1120
|
+
Wave C replaced `review-strict`, a meta-pass that re-read the findings and adjusted severity with no new evidence. It was **removed, not improved**, on measured evidence:
|
|
1121
|
+
self-correction fell 95.5 → 91.5 → 89.0 (GPT-4, GSM8K) and 75.8 → 38.1 (GPT-3.5, CommonSenseQA), arXiv:2310.01798; Self-Refine's +49.2 on dialogue response generation is +0.2 on maths
|
|
1122
|
+
(arXiv:2303.17651), so re-reading helps subjective tasks and not verifiable ones. A verifier that **runs something** rejects most false reports.
|
|
1123
|
+
Do not restore the re-score because it looks useful. Full figures: `SKILL.detail.md` § "Why Wave C verifies instead of re-scoring".
|
|
1124
|
+
|
|
1125
|
+
`review-verifier` occupies the slot and differs in one way that matters: it runs something.
|
|
1235
1126
|
|
|
1236
1127
|
It also **never votes.** 80+ agents unanimously endorsed a padding-oracle
|
|
1237
1128
|
vulnerability that did not exist, and a single empirical test killed it: consensus
|
|
@@ -1346,12 +1237,21 @@ Each reviewer must return a `REVIEW_RESULT` object matching `.metaproject/skills
|
|
|
1346
1237
|
## Sub-Agent Report Quality Gate
|
|
1347
1238
|
|
|
1348
1239
|
Before consolidation, validate every reviewer result:
|
|
1349
|
-
-
|
|
1240
|
+
- Every reply starts with a `STATUS:` line, and each status has one action:
|
|
1241
|
+
|
|
1242
|
+
| Status | Action |
|
|
1243
|
+
|---|---|
|
|
1244
|
+
| `DONE` | Consolidate. |
|
|
1245
|
+
| `DONE_WITH_CONCERNS` | Consolidate and carry the concerns into the report. |
|
|
1246
|
+
| `NEEDS_CONTEXT` | One targeted context refill; if still unresolved, keep it as an explicit open question, not a blocker. |
|
|
1247
|
+
| `BLOCKED` | Re-dispatch once; if it fails again, list the pass under **Not run**. |
|
|
1248
|
+
| `RATE_LIMITED` | Halve the wave and requeue (Control). |
|
|
1249
|
+
| anything else, or no result | Treat as `BLOCKED`. |
|
|
1250
|
+
|
|
1350
1251
|
- Required finding fields: id, severity, file, `quote`, problem, impact, suggested_fix, evidence, confidence, reviewer. **`quote`, not `line`:** copy the code the finding is about verbatim out of `file` and do not count lines — `keryx review ingest` locates the quote at the round's head and derives `line` from where it matched, so a number reported here is overwritten. A model is poor at counting lines and good at repeating text it has just read; this asks only for the second. Quote enough to be unambiguous: a snippet appearing twice in the file is recorded `unlocatable` rather than anchored to the first hit. A finding about the round rather than a site carries neither.
|
|
1351
1252
|
- Every blocker must include evidence and a concrete suggested fix.
|
|
1352
1253
|
- Findings without evidence are downgraded to `info` or returned to the reviewer for clarification.
|
|
1353
1254
|
- Duplicate findings are merged by `dedupe_key` or by `(file, quote, problem)`.
|
|
1354
|
-
- `NEEDS_CONTEXT` triggers one targeted context refill. If still unresolved, keep it as an explicit open question, not as a blocker.
|
|
1355
1255
|
- **A reply that is not a result is `BLOCKED`.** That covers an empty reply (`(subagent produced no text)`), a reply without a valid `REVIEW_RESULT` block, and a `spawn_subagent` output whose first line is `status: BudgetExhausted (…)` or `status: NoProgress …` — the reviewer was cut short, so whatever fragment it left is not its review. Re-dispatch it exactly once with a larger budget, telling it to return its report before the budget ends (and the same `cwd` when the round reviews a worktree). If that also fails, the pass is **not run**: list it under **Not run** with the reason, and never write its findings yourself — a review nobody ran reported as run is worse than a gap the operator can see.
|
|
1356
1256
|
- **The verifier is held to the same rule.** A finding without an applied verifier verdict is `unverified`; the orchestrator never verifies one itself, and a verifier that returned nothing leaves every finding `unverified`, stated in the stage counts.
|
|
1357
1257
|
- If a reviewer exceeds `max_findings`, keep blockers/majors first and summarize lower severity findings.
|
|
@@ -1652,6 +1552,8 @@ Automation values, names unchanged:
|
|
|
1652
1552
|
- `comment-and-ai-artifact` -> the PR review plus the follow-up file.
|
|
1653
1553
|
- `none` or legacy `false` -> do not publish.
|
|
1654
1554
|
|
|
1555
|
+
Unattended: do not publish; hand the rendered body back to the caller.
|
|
1556
|
+
|
|
1655
1557
|
Default is do not publish. No resolvable PR number means skip and say so.
|
|
1656
1558
|
|
|
1657
1559
|
The review body is the report rendered from `templates/review-report.md`, English, with the domain file chosen above (`templates/pr-comment-frontend.md` / `templates/pr-comment-backend.md`) — never a summary written freehand. It does not use a tool heading, a finding table, or a meta table. It does not carry a co-author line, a `Generated with` trailer, or any sentence that names a vendor or a product as the author. Name the workflow and reviewers that actually ran, with models, tools, skills and subagents. Never identify the operator from `gh auth status`, git config or a credential; do not sign the review as a person, reviewer or model. A project overlay adds content only through the template's overlay slots.
|
|
@@ -1707,6 +1609,7 @@ If absent, proceed normally — context is optional and non-blocking.
|
|
|
1707
1609
|
| "The spec-gate finding is mine, so `reviewer: review-orchestrator`" | That name makes the finding unroutable in Wave C. File it under the reviewer that ran the comparison |
|
|
1708
1610
|
| "Steps 8–10 are done, I'll tick them now and ingest later" | `completed` means the ingest record exists. A plan that runs ahead of the artifacts is how an unrun review looked finished |
|
|
1709
1611
|
| "The user said publish, so I can post the comment" | That chose whether, not what. Show the rendered body and get approval of it before any GitHub write |
|
|
1612
|
+
| "I cannot spawn reviewers, so I will review it myself" | A review nobody dispatched is not the review the report names. Output the Preflight 1 `BLOCKED` line and stop |
|
|
1710
1613
|
| "No flags means no reviewers" | No flags → run auto-detection; never produce an empty review |
|
|
1711
1614
|
| "User named a module so I'll use diff mode" | Named module/component/store → path mode; diff mode is only for branch changes |
|
|
1712
1615
|
| "Path mode should only show lines I'd flag in diff mode" | Path mode reviews the entire file — all findings apply, not just added lines |
|
|
@@ -205,6 +205,51 @@
|
|
|
205
205
|
"null"
|
|
206
206
|
],
|
|
207
207
|
"description": "The last round the producer was re-read. An entry with `state: open` that was pinned in round 1 and never revalidated is stale by round 3: a parallel pull request is precisely the thing that moves between rounds."
|
|
208
|
+
},
|
|
209
|
+
"checks": {
|
|
210
|
+
"type": "array",
|
|
211
|
+
"minItems": 1,
|
|
212
|
+
"description": "For an input rule the diff mirrors or depends on (what the producer accepts for a name, a value, a role): the producer's COMPLETE check set, not a summary of it. Flow 405: a dispatch that carried only the producer's regex let two rounds approve a change that missed a case-sensitive prefix check, a case-sensitive uniqueness check and a column-width bound. The set is a starting point for reviewers, never a boundary.",
|
|
213
|
+
"items": {
|
|
214
|
+
"type": "object",
|
|
215
|
+
"required": [
|
|
216
|
+
"check",
|
|
217
|
+
"source",
|
|
218
|
+
"paths"
|
|
219
|
+
],
|
|
220
|
+
"properties": {
|
|
221
|
+
"check": {
|
|
222
|
+
"type": "string",
|
|
223
|
+
"minLength": 1,
|
|
224
|
+
"description": "What the producer refuses, in one line — including a storage bound such as a column width."
|
|
225
|
+
},
|
|
226
|
+
"source": {
|
|
227
|
+
"type": "string",
|
|
228
|
+
"minLength": 1,
|
|
229
|
+
"description": "`file:line` at the pinned ref."
|
|
230
|
+
},
|
|
231
|
+
"case_sensitive": {
|
|
232
|
+
"type": [
|
|
233
|
+
"boolean",
|
|
234
|
+
"null"
|
|
235
|
+
]
|
|
236
|
+
},
|
|
237
|
+
"paths": {
|
|
238
|
+
"type": "array",
|
|
239
|
+
"minItems": 1,
|
|
240
|
+
"items": {
|
|
241
|
+
"type": "string"
|
|
242
|
+
},
|
|
243
|
+
"description": "The producer paths this check runs on — create, update, rename, copy, import, move, …"
|
|
244
|
+
}
|
|
245
|
+
},
|
|
246
|
+
"additionalProperties": true
|
|
247
|
+
}
|
|
248
|
+
},
|
|
249
|
+
"enumeration_method": {
|
|
250
|
+
"type": "string",
|
|
251
|
+
"minLength": 1,
|
|
252
|
+
"description": "How `checks` was found to be complete — the producer entry points read and the searches run. Required whenever `checks` is present."
|
|
208
253
|
}
|
|
209
254
|
},
|
|
210
255
|
"allOf": [
|
|
@@ -241,6 +286,18 @@
|
|
|
241
286
|
"pr"
|
|
242
287
|
]
|
|
243
288
|
}
|
|
289
|
+
},
|
|
290
|
+
{
|
|
291
|
+
"if": {
|
|
292
|
+
"required": [
|
|
293
|
+
"checks"
|
|
294
|
+
]
|
|
295
|
+
},
|
|
296
|
+
"then": {
|
|
297
|
+
"required": [
|
|
298
|
+
"enumeration_method"
|
|
299
|
+
]
|
|
300
|
+
}
|
|
244
301
|
}
|
|
245
302
|
],
|
|
246
303
|
"additionalProperties": true
|
|
@@ -160,7 +160,7 @@ a tool, skill or model is unavailable; use `none` only when the run proves none.
|
|
|
160
160
|
- **Subagents:** <dispatched subagents, their reviewer roles and actual models; or none when confirmed, otherwise not recorded>.
|
|
161
161
|
- **Fallback:** <reviewer> via `general-purpose` because <native agent type unavailable | no bundled agent>. Or `none`.
|
|
162
162
|
- **Not run:** <reviewer> — <short reason>. Or `none`.
|
|
163
|
-
- **Selection:** <
|
|
163
|
+
- **Selection:** <explicit flags | start questions: reviewers <`--all` | narrowed to …>, counterpart <verified at <sha> | skipped: reason>, models <accepted | all deep | custom> | defaults (unattended)>
|
|
164
164
|
- **Verification:** <mode>. Confirmed N, refuted N, unverifiable N, unverified N. Removed N (always 0 outside `filter`).
|
|
165
165
|
- **Stage counts:** pre-filter dropped <files>/<blocks>/<lines> (or `not recorded`). Retained N.
|
|
166
166
|
- **Context:** <job/context path, or none>
|