@mrciphersmith/keryx 0.3.85 → 0.3.87
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -1
- package/dist/cli.js +1835 -688
- package/dist/core.js +9 -5
- package/package.json +1 -1
- package/src/gdskills/bundled/skills/orchestration/flow-orchestrator/SKILL.md +18 -18
- package/src/gdskills/bundled/skills/orchestration/job-orchestrator/SKILL.md +28 -36
- package/src/gdskills/bundled/skills/planning/autodoc-orchestrator/SKILL.md +1 -2
- package/src/gdskills/bundled/skills/review/review-orchestrator/SKILL.detail.md +171 -1
- package/src/gdskills/bundled/skills/review/review-orchestrator/SKILL.md +101 -199
- package/src/gdskills/bundled/skills/review/review-orchestrator/reviewer-input.schema.json +9 -1
|
@@ -13,16 +13,22 @@ triggers:
|
|
|
13
13
|
- "review --legacy-profiles"
|
|
14
14
|
metadata:
|
|
15
15
|
author: "MrCipherSmith"
|
|
16
|
-
version: "1.
|
|
16
|
+
version: "1.11.0"
|
|
17
17
|
category: "review"
|
|
18
18
|
compatible_harnesses: "cursor,codex,zed,opencode,claude"
|
|
19
19
|
license: "MIT"
|
|
20
20
|
---
|
|
21
21
|
# Review Orchestrator
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
22
|
+
|
|
23
|
+
**You are the orchestrator. Run the round yourself, inline, in the main session that loaded this skill; never hand the whole round to one subagent.** This skill routes: it detects scope, dispatches specialized reviewers, and consolidates their findings into one report sorted by severity. The reviewers do the reviewing.
|
|
24
|
+
|
|
25
|
+
## Preflight — before Step 0, in this order
|
|
26
|
+
|
|
27
|
+
1. **Can it spawn?** The round needs the `Agent` tool (or the host's equivalent) and depth budget left: Claude Code nests subagents at most 3 deep, and at the limit `Agent` is withheld. If it cannot spawn, the whole output is this line, with no tool calls and no review of your own:
|
|
28
|
+
`STATUS: BLOCKED nested_dispatch_unavailable — run this skill in the main session`
|
|
29
|
+
2. **Is a round needed?** A diff of only docs, markdown, comments, formatting, lockfiles or generated files, with no CI, auth, migration or rule-bearing config, ends here with one line, `round not needed: <reason>`, and no reviewer. `--all`, an explicit reviewer list or a gating round forces a round.
|
|
30
|
+
3. **Round ceiling.** 5 rounds (`REVIEW_ROUND_CAP`) over the same `merge-base..HEAD`. Past it: interactive, ask the operator; unattended, `STATUS: BLOCKED round_cap`.
|
|
31
|
+
|
|
26
32
|
## Workflow
|
|
27
33
|
|
|
28
34
|
```
|
|
@@ -36,16 +42,16 @@ Review Orchestrator Progress:
|
|
|
36
42
|
- [ ] Step 4: Parse flags / auto-detect domain from scope
|
|
37
43
|
- [ ] Step 5: Start questions, once, after Steps 1 and 3 — reviewers (default `--all`), counterpart verification (default on), model plan; unattended takes the defaults (SKILL.detail.md § "Start questions"; legacy/profile reviewers stay flag-only)
|
|
38
44
|
- [ ] Step 5c: When `review.jev.select` is on, run `keryx review jev-select` over the finalized candidate set and drop its `skip` decisions from Wave A/B — see "Reviewer selection with Jev"
|
|
39
|
-
- [ ] Step 6: Plan sub-agent dispatch and token budgets
|
|
45
|
+
- [ ] Step 6: Plan sub-agent dispatch and token budgets; compute each dispatch's model with `keryx review tier` (6a: publish this plan)
|
|
40
46
|
- [ ] Step 7: Stage 1 gate - spec compliance check (if issue/task provided)
|
|
41
|
-
- [ ] Step 8: Dispatch selected reviewers in
|
|
47
|
+
- [ ] Step 8: Dispatch selected reviewers in parallel with the reviewer-input schema — see "How to spawn"
|
|
42
48
|
- [ ] Step 9: Collect reviewer-finding schema results and handle NEEDS_CONTEXT
|
|
43
49
|
- [ ] Step 9b: `jev-triage` — advisory, annotate-only severity/duplicate/verify-order annotations over the consolidated findings (opt-in `review.jev.triage`)
|
|
44
|
-
- [ ] Step 10: Wave C — dispatch `review-verifier` over the consolidated findings
|
|
50
|
+
- [ ] Step 10: Wave C — when blockers/majors exist, `--verify` is set, or the PR is high-risk, dispatch `review-verifier` over the consolidated findings
|
|
45
51
|
- [ ] Step 11: Sort by severity, deduplicate, emit unified report
|
|
46
52
|
- [ ] Step 12: Emit the machine-readable `keryx:findings` block alongside the report
|
|
47
53
|
- [ ] Step 13: Report the stage counts: dropped by pre-filter, refuted by the verifier, retained
|
|
48
|
-
- [ ] Step 14:
|
|
54
|
+
- [ ] Step 14: Managed rounds only (not `lightweight`, which is report-only), after the final round, on an open pull request at the head you reviewed — answer every external comment once with `keryx review comments reply --final` (render with `--dry-run` and get the user's explicit approval first; unattended: do not publish, hand the rendered body back) — never against a pull request the dispatch named as the caller's
|
|
49
55
|
```
|
|
50
56
|
|
|
51
57
|
Step 0 runs on **every** round. Step 14 runs **once**, after the last one. They are
|
|
@@ -55,22 +61,39 @@ one comment.
|
|
|
55
61
|
|
|
56
62
|
### Step 6 — the model is computed, not chosen
|
|
57
63
|
|
|
58
|
-
Before dispatching each reviewer, run `keryx review tier` with the signals you
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
64
|
+
Before dispatching each reviewer, run `keryx review tier` with the signals you hold (`--scope`, `--findings`, `--diff-lines`, `--fix-attempt`, `--verifier`, `--security`, `--forced-strategy-change`) and paste the `model` block it prints into that dispatch. The tier comes from the command, not from reading `rules/core/model-selection.mdc`.
|
|
65
|
+
|
|
66
|
+
It names a model only if you gave it your session (`--session-provider`/`--session-model` or `KERYX_SESSION_*`) and discovery found another. Otherwise it prints `inherit: true` with the tier — **adaptive**: pick your own runtime's model for that tier (`standard` = your session model; `light`/`deep` = your runtime's lighter/most capable class if its dispatch tool offers one, else the session model). A model id never goes into a dispatch by hand.
|
|
67
|
+
|
|
68
|
+
### Step 6a — the plan bridge
|
|
69
|
+
|
|
70
|
+
Publish this checklist with `plan_set` under the ids `step-0`…`step-14` once, here, when scope and dispatch are fixed. Publish each of steps 0–5 that ran as `completed` and each that did not run as `skipped`; publish steps 6–14 as `pending` (`in_progress` for the one starting now). From then on move each item with `plan_update` as its step completes — see the `session-plan-bridge` rule. `completed` means the step's artifact exists: steps 8–10 become `completed` (in the plan and in the report) only after `keryx review ingest` has recorded their results, and a step whose reviewers were `BLOCKED` is `blocked`.
|
|
71
|
+
|
|
72
|
+
## How to spawn (Steps 8 and 10)
|
|
73
|
+
|
|
74
|
+
- One `Agent` call per reviewer, all calls of a wave in a single message, so the wave runs in parallel.
|
|
75
|
+
- Use the reviewer's exact `subagent_type`. If the runtime has no such type, use `general-purpose` and name the reviewer's `SKILL.md` path in the prompt. Neither available: that reviewer is `BLOCKED`.
|
|
76
|
+
- Prompts carry pointers, not pasted files — the diff, the SHAs, the PR, the schema paths — and never paraphrase a contract.
|
|
77
|
+
- A reply is a result only if it starts with a `STATUS:` line and carries a `REVIEW_RESULT` block matching `reviewer-finding.schema.json`; anything else is `BLOCKED`.
|
|
78
|
+
- The model comes from `keryx review tier` (Step 6).
|
|
79
|
+
|
|
80
|
+
## Shell host (keryx shell, or any host with no file-reading subagent) — mandatory
|
|
81
|
+
|
|
82
|
+
A reviewer on this host cannot open a path: its file tool reads inside its working directory only, and a pointer it cannot follow becomes a round spent on a refusal. So the pointer rule above does not apply; the TEXT travels in the prompt, in slices small enough to finish.
|
|
62
83
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
84
|
+
1. **Slice first.** `keryx review slice --ref <base> --out <dir>` after `keryx review scope`. It writes `slice-NN.diff` files of at most 150000 bytes each and a `manifest.json`; ledgers (csv, lockfiles, snapshots, generated files) are listed under `omissions` with a reason — list them in the report as not reviewed, never as clean.
|
|
85
|
+
2. **Slice text in the prompt.** Each reviewer's prompt holds the content of the slices assigned to it (the file bodies, not their paths), plus the `slices` ids in its reviewer-input.
|
|
86
|
+
3. **Check before dispatch.** `keryx review dispatch-check --payload <file> --manifest <dir>/manifest.json` for the wave; exit 1 means a payload names no slice, is over the ceiling, or breaks `reviewer-input.schema.json`. Fix it; do not dispatch around it.
|
|
87
|
+
4. **One batch.** Dispatch the whole wave — up to 10 reviewers — in one message, not one at a time.
|
|
88
|
+
5. **Incomplete means retry once, then Not run.** On `INCOMPLETE` or `BLOCKED`, `keryx review retry-plan --manifest <dir>/manifest.json --result <result.json>` decides: it cuts the assigned slices in half and names the one retry, and after that prints the `- **Not run:** …` line to paste into the report. An unfinished reviewer is never counted as a clean pass.
|
|
89
|
+
6. **Machine artifacts by direct write.** Save slices, payloads and results by redirection or `keryx ctx run --raw -- <command>`; the summarized form of `keryx ctx run` drops the bytes a later step parses.
|
|
67
90
|
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
91
|
+
## Control
|
|
92
|
+
|
|
93
|
+
- **Profiles.** The default is read-only: one round, no edits. `--fix` hands the findings to `flow-orchestrator` or `task-implementer`; the orchestrator never edits code. Control returns to this main session after each hand-off, which loads this skill again (never the implementer, which would nest); each re-review counts toward the ceiling (it adds 1 to the count).
|
|
94
|
+
- **Waves.** Split by diff domain (`core` = logic, security, architecture; `domain` = frontend, backend, conventions; `support` = style, tests), in that order; A/B/C below keep their dependency order. A full or `--all` round runs at most 10 reviewers per wave (`MAX_REVIEWERS_PER_WAVE_FULL`).
|
|
95
|
+
- **Rate limits.** On a reviewer dispatch that fails with a rate-limit or overloaded error (reported as `STATUS: RATE_LIMITED`, a harness signal, not a schema status), halve the wave, requeue the rest and honour `retry-after`. After at most two halvings (`MAX_RATE_HALVINGS`) stop with `STATUS: BLOCKED rate_limited`, naming the queue left.
|
|
96
|
+
- **Done when** every dispatched reviewer's result is recorded with `keryx review ingest`, the stage counts and **Not run** are in the report, and `keryx review complete` has run.
|
|
74
97
|
|
|
75
98
|
---
|
|
76
99
|
|
|
@@ -520,7 +543,8 @@ The rule that follows for reviewers, and that belongs in the dispatch prompt:
|
|
|
520
543
|
|
|
521
544
|
If you cannot locate the producer, ask the operator where it is (Start question 2); record `state: unavailable` only when it cannot be read, and leave the dependent findings at `info`.
|
|
522
545
|
An unavailable producer is a result. Assuming one is not. An input rule records the producer's full check set and is verified every round,
|
|
523
|
-
paired PR or not — SKILL.detail.md § "Counterpart verification".
|
|
546
|
+
paired PR or not — SKILL.detail.md § "Counterpart verification". Unattended, nobody is asked: record
|
|
547
|
+
`state: unavailable` with the reason and continue.
|
|
524
548
|
|
|
525
549
|
A finding whose evidence lives in the producer sets `repo` on the finding itself.
|
|
526
550
|
`file` and `line` alone name a path in the repository under review, so without
|
|
@@ -611,6 +635,8 @@ This PR touches high-risk areas. Build full review context before dispatching re
|
|
|
611
635
|
> pick a letter (default: A)
|
|
612
636
|
```
|
|
613
637
|
|
|
638
|
+
Unattended: take A and record it in the review scope.
|
|
639
|
+
|
|
614
640
|
---
|
|
615
641
|
|
|
616
642
|
## Token and Context Budget Management
|
|
@@ -639,31 +665,22 @@ Default budget guidance:
|
|
|
639
665
|
|
|
640
666
|
## Model Strategy
|
|
641
667
|
|
|
642
|
-
The model
|
|
643
|
-
|
|
644
|
-
the `model` block it prints into that dispatch. Do not assign a model class by
|
|
645
|
-
reading a table or by hand; that mechanical step is exactly what the command
|
|
646
|
-
replaced (see Step 6 above and `job-orchestrator` Step 5, which pins the same
|
|
647
|
-
rule: `model_strategy: "current"` is gone).
|
|
648
|
-
|
|
649
|
-
`model_strategy` now governs only how the *computed* model is surfaced, not
|
|
650
|
-
whether it is computed:
|
|
668
|
+
The model per dispatch is computed by `keryx review tier` (Step 6) and never written by hand.
|
|
669
|
+
`model_strategy` governs only how the computed model is surfaced:
|
|
651
670
|
|
|
652
671
|
| `model_strategy` | Behavior |
|
|
653
672
|
|---|---|
|
|
654
673
|
| `ask` (default, interactive) | Present the `keryx review tier` model plan once before dispatch — Start question 3; the operator may accept it, take all `deep`, or set it. |
|
|
655
674
|
| `adaptive` (default, unattended) | Dispatch on the computed model directly; record the chosen model per reviewer in the final report metadata. |
|
|
656
675
|
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
- `inherit: true` is the adaptive answer: dispatch on your runtime's model for the block's `tier` (Step 6). Record `model_assignment: adaptive` and the model you actually chose — not a failure.
|
|
660
|
-
- With `model_strategy: ask`, present the model plan once before dispatch, then proceed with the computed model.
|
|
676
|
+
`inherit: true` is the adaptive answer: dispatch on your runtime's model for the block's `tier`, and
|
|
677
|
+
record `model_assignment: adaptive` with the model you chose.
|
|
661
678
|
|
|
662
679
|
---
|
|
663
680
|
|
|
664
681
|
## Scope Detection
|
|
665
682
|
|
|
666
|
-
###
|
|
683
|
+
### Fix round check (Step 1a)
|
|
667
684
|
|
|
668
685
|
A **fix round** is any review of work produced to answer earlier findings. Set
|
|
669
686
|
`is_fix_round: true` on every reviewer input, and populate `prior_findings` with
|
|
@@ -702,63 +719,14 @@ Recording the enumeration matters as much as doing it: a round that searched and
|
|
|
702
719
|
found nothing is a different fact from a round that never searched, and only the
|
|
703
720
|
recorded list distinguishes them.
|
|
704
721
|
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
| Disposition | Meaning |
|
|
712
|
-
|---|---|
|
|
713
|
-
| `closed` | Checked against the code, not against the commit message, and the defect is gone |
|
|
714
|
-
| `open` | The fix does not reach the defect; say what is still true |
|
|
715
|
-
| `partial` | One site of the class was fixed and the enumeration named others |
|
|
716
|
-
| `regressed` | The fix removed this defect and introduced another — file the new one separately |
|
|
717
|
-
| `withdrawn` | The finding was wrong. See below |
|
|
718
|
-
|
|
719
|
-
**Check the code, not the commit message.** A commit titled *"report an abandoned
|
|
720
|
-
sync as abandoned"* is a claim; the disposition is whether the branch it renamed
|
|
721
|
-
is reachable and pinned. On a recorded round, two such commits asserted behaviour
|
|
722
|
-
on lines no test could reach.
|
|
723
|
-
|
|
724
|
-
#### A fix is a change, and changes get reviewed
|
|
725
|
-
|
|
726
|
-
The most expensive class in a multi-round review is **the defect the fix
|
|
727
|
-
introduced**. It is systematically under-found, for a structural reason: the fix
|
|
728
|
-
arrives framed as the answer to a finding, so it is read as an answer rather than
|
|
729
|
-
as new code. It is new code.
|
|
730
|
-
|
|
731
|
-
So scope A of a fix round includes the fix, reviewed on its own merits, and the
|
|
732
|
-
report carries its own section:
|
|
722
|
+
Every prior finding leaves the round with a disposition — `closed`, `open`, `partial`,
|
|
723
|
+
`regressed` or `withdrawn` — one line each in the report before the new findings, checked against
|
|
724
|
+
the code and not the commit message. A fix is new code: scope A includes it, and the report carries
|
|
725
|
+
a `## Regressions the fixes introduced` section. A finding this round disproves is withdrawn at the
|
|
726
|
+
top of the report with its evidence. The table, recorded shapes and rationale:
|
|
727
|
+
`SKILL.detail.md` § "Fix rounds — dispositions, regressions, withdrawal".
|
|
733
728
|
|
|
734
|
-
|
|
735
|
-
## Regressions the fixes introduced
|
|
736
|
-
<[F-NNN] — the finding it was answering, and the new defect it created>
|
|
737
|
-
```
|
|
738
|
-
|
|
739
|
-
Recorded shapes, all from fixes that correctly closed the finding they answered:
|
|
740
|
-
an early return added to stop a fall-through, which then skipped the work the
|
|
741
|
-
caller needed; a persistence call added to save expanded state, which then
|
|
742
|
-
persisted the broken state on the error path too. Both were closed correctly and
|
|
743
|
-
both shipped a new bug in the same commit.
|
|
744
|
-
|
|
745
|
-
#### Withdrawing your own earlier finding
|
|
746
|
-
|
|
747
|
-
A finding from a previous round that this round disproves is **withdrawn**,
|
|
748
|
-
explicitly, at the top of the report, with the evidence — before any new finding.
|
|
749
|
-
|
|
750
|
-
This is not a courtesy. An uncorrected wrong finding costs the author a fix they
|
|
751
|
-
did not need, and it stays in `prior_findings` steering later rounds. Withdrawal
|
|
752
|
-
is also the one self-correction this pipeline permits, and it is permitted because
|
|
753
|
-
it is asymmetric: it *deletes* a claim, so it cannot inflate the finding count,
|
|
754
|
-
which is the failure mode that removed the re-scoring pass from Wave C.
|
|
755
|
-
|
|
756
|
-
State what made the original claim wrong, in one sentence, and if the same
|
|
757
|
-
reasoning error has now happened twice in one review, say that too. A reviewer
|
|
758
|
-
that names its own recurring error is calibrating; one that quietly drops a
|
|
759
|
-
finding is hiding a result.
|
|
760
|
-
|
|
761
|
-
### Step 1: Determine Review Mode
|
|
729
|
+
### Step 2: Determine Review Mode
|
|
762
730
|
|
|
763
731
|
Before anything else, determine whether the request is **diff mode** or **path mode**:
|
|
764
732
|
|
|
@@ -832,72 +800,23 @@ keryx review blast-radius --ref "${BASE_SHA}" --brief # wha
|
|
|
832
800
|
scope-B finding arrives without it — pass it back as
|
|
833
801
|
`review ingest ... --blast-radius blast-radius.json`.
|
|
834
802
|
|
|
835
|
-
|
|
836
|
-
|
|
837
|
-
file
|
|
838
|
-
built graph — run `keryx gdgraph build` if it refuses.
|
|
839
|
-
|
|
840
|
-
**Do not pick the files yourself, and do not widen it.** "Review the
|
|
841
|
-
functionality so nothing breaks" naively means "review the whole repository every
|
|
842
|
-
round", which is unaffordable *and* actively harmful: review quality decays as
|
|
843
|
-
context grows — measured F1 0.65 at round 2 falling to 0.29 at round 10. An
|
|
844
|
-
unbounded scope B makes later rounds worse than earlier ones.
|
|
845
|
-
|
|
846
|
-
The bounds are measured on this repository, not guessed: at depth 2 the set is a
|
|
847
|
-
median of 19 files (p90 65); depth 3 buys eight more in the median and doubles
|
|
848
|
-
the p90. The 40-file cap fires on 25% of commits and removes only hop-2 entries
|
|
849
|
-
on all but 2 of 80, so it almost never costs a direct dependent — and when it
|
|
850
|
-
does, it says so.
|
|
851
|
-
|
|
852
|
-
**Record the whole thing.** `--out "<review-package>/blast-radius.md"` writes the
|
|
853
|
-
set, the depth, and **every file the cap removed**. A truncation nobody can see
|
|
854
|
-
reads afterwards as "we checked everything", which is the claim this pipeline
|
|
855
|
-
exists to stop making. An empty radius is reported as `unresolved`, not as clean:
|
|
856
|
-
the graph indexes code, so a change to a skill, a rule or a schema has no blast
|
|
857
|
-
radius at all and that is a different fact from "nothing depends on it".
|
|
858
|
-
|
|
859
|
-
#### The scope-B question, and what is rejected
|
|
860
|
-
|
|
861
|
-
> Does this change break an existing behaviour **at these sites**?
|
|
862
|
-
|
|
863
|
-
Nothing else. The blast-radius set is **under regression check, not under
|
|
864
|
-
review**. A finding about style, naming or architecture in code the change did
|
|
865
|
-
not touch is refused **by the orchestrator in code** — not discouraged here —
|
|
866
|
-
under three rules, every one of them a fact about the claim rather than about who
|
|
867
|
-
made it:
|
|
868
|
-
|
|
869
|
-
| Rule | Refused because |
|
|
870
|
-
|---|---|
|
|
871
|
-
| `outside-set` | the file is neither in the computed set nor in the changed set; the reviewer went browsing |
|
|
872
|
-
| `non-regression-severity` | below `major`. Under the canonical rubric `minor` states the code behaves correctly and `info` names neither trigger nor outcome; neither can be a claim that something broke |
|
|
873
|
-
| `no-link-to-change` | nothing in the finding names a changed file, module or symbol. A regression claim says THE CHANGE broke this site |
|
|
874
|
-
|
|
875
|
-
Rejections are **recorded, not deleted** — raise the observation under scope A or
|
|
876
|
-
as a separate review. Pass `--brief` output verbatim into the scope-B dispatch:
|
|
877
|
-
the code rejection is the enforcement, but a reviewer told afterwards has already
|
|
878
|
-
spent the round producing findings that will all be refused.
|
|
879
|
-
|
|
880
|
-
`class_scope` on a scope-B finding names the **caller that breaks**, not the
|
|
881
|
-
changed line, because that is the site a human has to look at.
|
|
803
|
+
Scope B is bounded and computed: distance ≤ 2 from every changed file, at most 40 files, closest
|
|
804
|
+
first. Do not pick the files or widen the set. Record it with `--out "<review-package>/blast-radius.md"`,
|
|
805
|
+
including every file the cap removed; an empty radius is `unresolved`, not clean.
|
|
882
806
|
|
|
883
|
-
|
|
807
|
+
A scope-B reviewer answers one question: does this change break an existing behaviour **at these
|
|
808
|
+
sites**? The orchestrator refuses, in code, a finding that is `outside-set`, below `major`
|
|
809
|
+
(`non-regression-severity`), or names nothing the change touched (`no-link-to-change`) — pass
|
|
810
|
+
`--brief` verbatim so no reviewer spends the round on findings that will be refused. `class_scope`
|
|
811
|
+
names the caller that breaks.
|
|
884
812
|
|
|
885
|
-
|
|
886
|
-
|---|---|---|
|
|
887
|
-
| 1 (first after the draft PR) | yes | yes |
|
|
888
|
-
| 2..N | yes | recomputed only if the changed-file set moved |
|
|
889
|
-
| final | yes | **yes, always** |
|
|
890
|
-
|
|
891
|
-
Do not decide this by memory:
|
|
813
|
+
Recompute scope B on rounds 2..N only when the changed-file set moved, and always on the final round:
|
|
892
814
|
|
|
893
815
|
```bash
|
|
894
816
|
keryx review blast-radius --ref "${BASE_SHA}" --previous blast-radius.json [--final]
|
|
895
817
|
```
|
|
896
818
|
|
|
897
|
-
|
|
898
|
-
nothing moved. The final round recomputes whatever the file set did — otherwise a
|
|
899
|
-
fix introduced in round 3 gets no regression check at all, and the round that
|
|
900
|
-
certifies the flow is the one that checked the least.
|
|
819
|
+
The measurements, the rule table and the recompute table: `SKILL.detail.md` § "Scope B — bounds, rejections and recompute".
|
|
901
820
|
|
|
902
821
|
### Path Mode
|
|
903
822
|
|
|
@@ -905,10 +824,10 @@ When a path or target is named, collect the candidate files:
|
|
|
905
824
|
|
|
906
825
|
```bash
|
|
907
826
|
# If a directory path is given:
|
|
908
|
-
|
|
827
|
+
keryx ctx rg --files -g '*.{ts,tsx,js,jsx}' <path> | sort
|
|
909
828
|
|
|
910
829
|
# If a module name is given (e.g. "UserStore", "pipelines module"):
|
|
911
|
-
|
|
830
|
+
keryx ctx rg --files -g '*<name>*.{ts,tsx}'
|
|
912
831
|
# Also check common locations: src/stores/, src/modules/, src/components/
|
|
913
832
|
```
|
|
914
833
|
|
|
@@ -1069,7 +988,7 @@ Unattended rounds ask nothing: `--all`, counterpart verified, computed models
|
|
|
1069
988
|
and record that choice in the review scope).
|
|
1070
989
|
|
|
1071
990
|
A gating round runs what these answers selected; a host that cannot dispatch subagents
|
|
1072
|
-
returns `STATUS: BLOCKED` (`nested_dispatch_unavailable`) instead of reviewing alone. Every
|
|
991
|
+
returns `STATUS: BLOCKED` (`nested_dispatch_unavailable`) instead of reviewing alone (Preflight 1). Every
|
|
1073
992
|
round also carries the always-on obligations — SKILL.detail.md § "Depth floor",
|
|
1074
993
|
§ "Counterpart verification", § "Round obligations".
|
|
1075
994
|
|
|
@@ -1092,8 +1011,8 @@ Legacy/profile reviewers are specialized review profiles that predate the review
|
|
|
1092
1011
|
| `--mobx-store` | `code-mobx-store-review` |
|
|
1093
1012
|
| `*.store.ts`, `makeObservable`, `observable`, `computed`, `action.bound` | suggest `code-mobx-store-review` as optional profile reviewer |
|
|
1094
1013
|
|
|
1095
|
-
Legacy/profile reviewers are never auto-included and never prompted for — do not
|
|
1096
|
-
about them. They are exempt from the finding contract (for example `code-ai-review` emits
|
|
1014
|
+
Legacy/profile reviewers are never auto-included and never prompted for, interactive or unattended — do not
|
|
1015
|
+
ask about them. They are exempt from the finding contract (for example `code-ai-review` emits
|
|
1097
1016
|
free-prose Russian with no per-finding severity field, so its output cannot be normalised into
|
|
1098
1017
|
the unified report), which is why inclusion must be a deliberate, explicit act rather than a
|
|
1099
1018
|
default the user has to opt out of on every review. Dispatch them ONLY when the user passes one
|
|
@@ -1185,7 +1104,7 @@ whoever reads the merge commit a year later reads the body. When `review.jev.con
|
|
|
1185
1104
|
|
|
1186
1105
|
New managed packages declare `reviewerDispatch: "research.json"`. Alongside the research ledger, provide `dispatch: { version: 1, selected: [...], unresolvedRules: [], runs: [...] }`. Preserve the actual computed reviewer inventory. Each run has `reviewer`, unique `executionId`, `status: "complete"`, `scopeComplete: true`, `rawEvidence`, nonempty `ruleEvidence`, and boolean `executionRequired`. Required execution needs nonempty `executionEvidence` (command/output references); a source-only scope requires `executionReason`. Do not fabricate IDs or mark truncated responses complete. Runtime validates these declarations at closure and the flow gate, not their truth: independently audit evidence and inventory before freezing. Legacy artifact absence is compatibility, not proof of a correct launch. Research closure is mandatory: see [Runtime research-completion ledger](SKILL.detail.md#runtime-research-completion-ledger).
|
|
1187
1106
|
|
|
1188
|
-
Dispatch selected reviewers in parallel when independent — this engine dispatches them itself; a host without a dispatch tool returns `STATUS: BLOCKED` (`nested_dispatch_unavailable`) and never reviews alone
|
|
1107
|
+
Dispatch selected reviewers in parallel when independent — this engine dispatches them itself (see "How to spawn"); a host without a dispatch tool returns `STATUS: BLOCKED` (`nested_dispatch_unavailable`) and never reviews alone — Preflight 1. Waves follow the diff domains in "Control" and keep their dependency order:
|
|
1189
1108
|
|
|
1190
1109
|
1. Wave A - core correctness/risk reviewers: logic, architecture, security/highload when selected.
|
|
1191
1110
|
2. Wave B - domain reviewers: frontend/backend/testing/convention reviewers filtered to relevant files. `review-jev-rules` (flow 330), `review-jev-risk`/`review-jev-scenarios` (flow 332), `review-jev-docs`/`review-jev-comments` (flow 333), `review-jev-contract` (flow 335) also run here, CLI-engine not sub-agent, `"engine": "jev"` in `keryx review reviewers --json`, gated on their own opt-in and a resolvable Jev/OpenRouter credential — measured verdicts per reviewer (keep off by default, experimental, etc.) in `SKILL.detail.md` § "CLI-engine reviewers".
|
|
@@ -1204,34 +1123,17 @@ not something Wave C can hand you.** If Wave B skips its mutation pass, that
|
|
|
1204
1123
|
finding class is unreachable for the entire round, and no amount of verification
|
|
1205
1124
|
downstream recovers it. When a testing reviewer returns without a mutation table,
|
|
1206
1125
|
treat it the way you would treat a reviewer that returned without findings *and*
|
|
1207
|
-
without evidence: ask once, then record that the pass did not run.
|
|
1126
|
+
without evidence: ask once, then record that the pass did not run. Unattended: record
|
|
1127
|
+
`Not run: mutation pass` without asking.
|
|
1208
1128
|
|
|
1209
1129
|
### Wave C — verification, and what it replaced
|
|
1210
1130
|
|
|
1211
|
-
Wave C
|
|
1212
|
-
|
|
1213
|
-
|
|
1214
|
-
|
|
1215
|
-
|
|
1216
|
-
-
|
|
1217
|
-
**GPT-3.5 on CommonSenseQA: 75.8 → 38.1.** Among the answers that changed,
|
|
1218
|
-
correct → incorrect exceeded incorrect → correct (Huang et al., *Large Language
|
|
1219
|
-
Models Cannot Self-Correct Reasoning Yet*, ICLR 2024, arXiv:2310.01798).
|
|
1220
|
-
- **Self-Refine (arXiv:2303.17651): +49.2 on dialogue response generation, +0.2
|
|
1221
|
-
on maths.** Self-refinement gains are on subjective tasks and vanish on
|
|
1222
|
-
verifiable reasoning. Judging whether a null-guard is missing is verifiable
|
|
1223
|
-
reasoning.
|
|
1224
|
-
|
|
1225
|
-
Re-scoring a finding by re-reading it is therefore not a rigour pass; it is a
|
|
1226
|
-
coin flip weighted toward more findings. **Do not restore it because it looks
|
|
1227
|
-
obviously useful — it looked obviously useful the first time.**
|
|
1228
|
-
|
|
1229
|
-
`review-verifier` occupies the slot and differs in exactly one way that matters:
|
|
1230
|
-
**it runs something.** Verification that executes rejects 85–96% of false reports
|
|
1231
|
-
against 4–15% unaided while finding 30–44% more true bugs (AnyPoC,
|
|
1232
|
-
arXiv:2604.11950); Meta's TestGen-LLM funnel discards 75% of its own output
|
|
1233
|
-
(75% build → 57% build and pass → 25% improve coverage) and the surviving quarter
|
|
1234
|
-
reaches 73% human acceptance (arXiv:2402.09171).
|
|
1131
|
+
Wave C replaced `review-strict`, a meta-pass that re-read the findings and adjusted severity with no new evidence. It was **removed, not improved**, on measured evidence:
|
|
1132
|
+
self-correction fell 95.5 → 91.5 → 89.0 (GPT-4, GSM8K) and 75.8 → 38.1 (GPT-3.5, CommonSenseQA), arXiv:2310.01798; Self-Refine's +49.2 on dialogue response generation is +0.2 on maths
|
|
1133
|
+
(arXiv:2303.17651), so re-reading helps subjective tasks and not verifiable ones. A verifier that **runs something** rejects most false reports.
|
|
1134
|
+
Do not restore the re-score because it looks useful. Full figures: `SKILL.detail.md` § "Why Wave C verifies instead of re-scoring".
|
|
1135
|
+
|
|
1136
|
+
`review-verifier` occupies the slot and differs in one way that matters: it runs something.
|
|
1235
1137
|
|
|
1236
1138
|
It also **never votes.** 80+ agents unanimously endorsed a padding-oracle
|
|
1237
1139
|
vulnerability that did not exist, and a single empirical test killed it: consensus
|
|
@@ -1346,12 +1248,21 @@ Each reviewer must return a `REVIEW_RESULT` object matching `.metaproject/skills
|
|
|
1346
1248
|
## Sub-Agent Report Quality Gate
|
|
1347
1249
|
|
|
1348
1250
|
Before consolidation, validate every reviewer result:
|
|
1349
|
-
-
|
|
1251
|
+
- Every reply starts with a `STATUS:` line, and each status has one action:
|
|
1252
|
+
|
|
1253
|
+
| Status | Action |
|
|
1254
|
+
|---|---|
|
|
1255
|
+
| `DONE` | Consolidate. |
|
|
1256
|
+
| `DONE_WITH_CONCERNS` | Consolidate and carry the concerns into the report. |
|
|
1257
|
+
| `NEEDS_CONTEXT` | One targeted context refill; if still unresolved, keep it as an explicit open question, not a blocker. |
|
|
1258
|
+
| `BLOCKED` | Re-dispatch once; if it fails again, list the pass under **Not run**. |
|
|
1259
|
+
| `RATE_LIMITED` | Halve the wave and requeue (Control). |
|
|
1260
|
+
| anything else, or no result | Treat as `BLOCKED`. |
|
|
1261
|
+
|
|
1350
1262
|
- Required finding fields: id, severity, file, `quote`, problem, impact, suggested_fix, evidence, confidence, reviewer. **`quote`, not `line`:** copy the code the finding is about verbatim out of `file` and do not count lines — `keryx review ingest` locates the quote at the round's head and derives `line` from where it matched, so a number reported here is overwritten. A model is poor at counting lines and good at repeating text it has just read; this asks only for the second. Quote enough to be unambiguous: a snippet appearing twice in the file is recorded `unlocatable` rather than anchored to the first hit. A finding about the round rather than a site carries neither.
|
|
1351
1263
|
- Every blocker must include evidence and a concrete suggested fix.
|
|
1352
1264
|
- Findings without evidence are downgraded to `info` or returned to the reviewer for clarification.
|
|
1353
1265
|
- Duplicate findings are merged by `dedupe_key` or by `(file, quote, problem)`.
|
|
1354
|
-
- `NEEDS_CONTEXT` triggers one targeted context refill. If still unresolved, keep it as an explicit open question, not as a blocker.
|
|
1355
1266
|
- **A reply that is not a result is `BLOCKED`.** That covers an empty reply (`(subagent produced no text)`), a reply without a valid `REVIEW_RESULT` block, and a `spawn_subagent` output whose first line is `status: BudgetExhausted (…)` or `status: NoProgress …` — the reviewer was cut short, so whatever fragment it left is not its review. Re-dispatch it exactly once with a larger budget, telling it to return its report before the budget ends (and the same `cwd` when the round reviews a worktree). If that also fails, the pass is **not run**: list it under **Not run** with the reason, and never write its findings yourself — a review nobody ran reported as run is worse than a gap the operator can see.
|
|
1356
1267
|
- **The verifier is held to the same rule.** A finding without an applied verifier verdict is `unverified`; the orchestrator never verifies one itself, and a verifier that returned nothing leaves every finding `unverified`, stated in the stage counts.
|
|
1357
1268
|
- If a reviewer exceeds `max_findings`, keep blockers/majors first and summarize lower severity findings.
|
|
@@ -1652,6 +1563,8 @@ Automation values, names unchanged:
|
|
|
1652
1563
|
- `comment-and-ai-artifact` -> the PR review plus the follow-up file.
|
|
1653
1564
|
- `none` or legacy `false` -> do not publish.
|
|
1654
1565
|
|
|
1566
|
+
Unattended: do not publish; hand the rendered body back to the caller.
|
|
1567
|
+
|
|
1655
1568
|
Default is do not publish. No resolvable PR number means skip and say so.
|
|
1656
1569
|
|
|
1657
1570
|
The review body is the report rendered from `templates/review-report.md`, English, with the domain file chosen above (`templates/pr-comment-frontend.md` / `templates/pr-comment-backend.md`) — never a summary written freehand. It does not use a tool heading, a finding table, or a meta table. It does not carry a co-author line, a `Generated with` trailer, or any sentence that names a vendor or a product as the author. Name the workflow and reviewers that actually ran, with models, tools, skills and subagents. Never identify the operator from `gh auth status`, git config or a credential; do not sign the review as a person, reviewer or model. A project overlay adds content only through the template's overlay slots.
|
|
@@ -1666,19 +1579,7 @@ and name unchecked commits. Do not duplicate findings; cite extended ones.
|
|
|
1666
1579
|
|
|
1667
1580
|
## Job Context Awareness
|
|
1668
1581
|
|
|
1669
|
-
When
|
|
1670
|
-
|
|
1671
|
-
```
|
|
1672
|
-
JOB_NAME: <job-name>
|
|
1673
|
-
CONTEXT_PATH: .metaproject/jobs/<job-name>/ai/context.md
|
|
1674
|
-
```
|
|
1675
|
-
|
|
1676
|
-
If provided and the file exists, read the context document **before** running scope detection.
|
|
1677
|
-
Use it to understand:
|
|
1678
|
-
- Intentionally chosen libraries and patterns (do not flag as issues)
|
|
1679
|
-
- Architectural decisions already agreed upon, and acceptance criteria driving the Stage 1 spec compliance gate
|
|
1680
|
-
|
|
1681
|
-
If absent, proceed normally — context is optional and non-blocking.
|
|
1582
|
+
When a prompt carries `JOB_NAME` and `CONTEXT_PATH`, read that context document before scope detection (`SKILL.detail.md` § "Job Context Awareness"). It is optional and non-blocking.
|
|
1682
1583
|
|
|
1683
1584
|
---
|
|
1684
1585
|
|
|
@@ -1707,6 +1608,7 @@ If absent, proceed normally — context is optional and non-blocking.
|
|
|
1707
1608
|
| "The spec-gate finding is mine, so `reviewer: review-orchestrator`" | That name makes the finding unroutable in Wave C. File it under the reviewer that ran the comparison |
|
|
1708
1609
|
| "Steps 8–10 are done, I'll tick them now and ingest later" | `completed` means the ingest record exists. A plan that runs ahead of the artifacts is how an unrun review looked finished |
|
|
1709
1610
|
| "The user said publish, so I can post the comment" | That chose whether, not what. Show the rendered body and get approval of it before any GitHub write |
|
|
1611
|
+
| "I cannot spawn reviewers, so I will review it myself" | A review nobody dispatched is not the review the report names. Output the Preflight 1 `BLOCKED` line and stop |
|
|
1710
1612
|
| "No flags means no reviewers" | No flags → run auto-detection; never produce an empty review |
|
|
1711
1613
|
| "User named a module so I'll use diff mode" | Named module/component/store → path mode; diff mode is only for branch changes |
|
|
1712
1614
|
| "Path mode should only show lines I'd flag in diff mode" | Path mode reviews the entire file — all findings apply, not just added lines |
|
|
@@ -22,7 +22,7 @@
|
|
|
22
22
|
"string",
|
|
23
23
|
"null"
|
|
24
24
|
],
|
|
25
|
-
"description": "Absolute path of the reviewer's registered `<path>/SKILL.md`, set for every reviewer from the `project` half of `keryx review reviewers --json` and omitted for a bundled one. The file is the reviewer's definition: it overrides the built-in definition of a same-named agent type, whichever runtime branch dispatches it. A path that does not exist makes the reviewer BLOCKED, never substituted. Optional
|
|
25
|
+
"description": "Absolute path of the reviewer's registered `<path>/SKILL.md`, set for every reviewer from the `project` half of `keryx review reviewers --json` and omitted for a bundled one. The file is the reviewer's definition: it overrides the built-in definition of a same-named agent type, whichever runtime branch dispatches it. A path that does not exist makes the reviewer BLOCKED, never substituted. Optional. `keryx review dispatch-check --payload <file>` validates a payload against this schema and refuses a `diff` scope that names no slice or whose assigned bytes exceed the ceiling."
|
|
26
26
|
},
|
|
27
27
|
"scope_mode": {
|
|
28
28
|
"type": "string",
|
|
@@ -31,6 +31,14 @@
|
|
|
31
31
|
"path"
|
|
32
32
|
]
|
|
33
33
|
},
|
|
34
|
+
"slices": {
|
|
35
|
+
"type": "array",
|
|
36
|
+
"description": "Slice ids (or manifest-relative paths) from `keryx review slice` that this reviewer is assigned. A `diff` scope names its slices here or carries a `diff` that fits the slice ceiling; `keryx review dispatch-check` refuses a payload that does neither.",
|
|
37
|
+
"items": {
|
|
38
|
+
"type": "string",
|
|
39
|
+
"minLength": 1
|
|
40
|
+
}
|
|
41
|
+
},
|
|
34
42
|
"context_doc": {
|
|
35
43
|
"type": [
|
|
36
44
|
"string",
|