@mrciphersmith/keryx 0.3.2 → 0.3.5

package/src/gdskills/bundled/skills/review/review-jev-contract/SKILL.md

@@ -0,0 +1,193 @@
+---
+name: review-jev-contract
+model_tier: light
+description: |
+  Use when: an ADDITIONAL, machine-scored check of a PR description's own CLAIMS —
+  and, when a flow is linked, that flow's frozen acceptance criteria — against the diff
+  is wanted, never replacing any other reviewer. Dispatched by review-orchestrator in
+  Wave B, via the CLI (`keryx review jev-contract`), when `review.jev.contract: true` in
+  .metaproject/tasks.config.json and a Jev/OpenRouter credential is resolvable. It is not
+  an LLM sub-agent: there is nothing to dispatch through a platform-native agent
+  mechanism, and no prose is generated by a model — Jev only answers one `noul`
+  probability per claim; keryx writes every word of every finding. Replaces the
+  orchestrator's by-eye Stage 1 "description vs diff" judgement with a scored input when
+  the opt-in is on; the by-eye check stays the fallback when it is off.
+  NOT for: judging a flow's frozen criteria from scratch (that machinery is flow 328's
+  `check-ac.ts`, reused, not duplicated), and not a substitute for a human reviewer
+  reading the claim list it produces.
+triggers:
+  - "jev contract"
+  - "check the PR description against the diff"
+  - "review --jev-contract"
+metadata:
+  author: "MrCipherSmith"
+  version: "1.0.0"
+  category: "review"
+  compatible_harnesses: "cursor,codex,zed,opencode,claude"
+  engine: "jev"
+  origin: "authored"
+license: "MIT"
+---
+
+# Review — Jev Contract (PR-description claims and frozen acceptance criteria vs. the diff)
+
+An ADDITIONAL orchestrator reviewer, flow 335. It is a **keryx program**, not
+an LLM sub-agent: `review-orchestrator` never dispatches a platform-native
+agent for it, it runs `keryx review jev-contract` and reads the `--json`
+output. Every finding is composed by keryx; Jev ("System One" on OpenRouter)
+supplies only one `noul` probability per claim — never prose.
+
+---
+
+## What it does
+
+Two independent tracks, both advisory, both merged into one report:
+
+1. **Claims** — extracted deterministically from the PR description: every
+   bullet/numbered-list item is a claim, plus every prose sentence carrying a
+   verb cue (adds, fixes, removes, "does not change", tests). For each claim,
+   keryx computes deterministic facts FIRST — named files/symbols/flags
+   present in the diff, whether a test file was touched (for a "tests added"
+   claim), whether an EXPORTED symbol changed anywhere in the diff (for a "no
+   API change" claim) — then asks Jev **one `noul`**: "does the diff support
+   this claim?". A claim the facts directly CONTRADICT (e.g. "no API change"
+   with an exported symbol touched) is `major`, with `class_scope`,
+   regardless of what Jev answered — a contradiction is a fact, not an
+   opinion Jev could outvote. A claim scoring below threshold with no
+   contradiction is `minor` ("unsupported"). A claim at/above threshold with
+   no contradiction produces no finding.
+2. **Criteria** — the linked flow's frozen acceptance criteria, when
+   `--flow <id>` names one. This track reuses flow 328's
+   `src/flow/check-ac.ts` **wholesale**, through `runCheckAc`
+   (`src/commands/flow-check-ac.ts`) — the exact same diff acquisition, Jev
+   batching, cache and degrade path `keryx flow check-ac` already uses. No
+   criterion-checking logic is reimplemented for this reviewer.
+
+---
+
+## Input Contract
+
+Not dispatched with a prompt — invoked as a CLI command:
+
+```text
+keryx review jev-contract (--diff <ref> | --pr <n>) [--flow <id>]
+                          [--max-calls <n>] [--threshold <0..1>]
+                          [--repo <owner/repo>] [--model <jev-1.13|jev-latest>]
+                          [--fixtures <dir>] [--json]
+```
+
+`--pr <n>` is required to check claims at all — a `--diff <ref>` target has
+no PR description to extract claims from, so that track reports zero claims
+(honestly, not a refusal). `--flow <id>` is independent of either target: it
+names the flow whose frozen acceptance criteria are checked against the same
+diff, when one is linked.
+
+---
+
+## Opt-in and privacy
+
+Refuses before any read or network call unless `.metaproject/tasks.config.json`
+declares:
+
+```json
+{ "review": { "jev": { "contract": true } } }
+```
+
+Every claim and every matched diff hunk sent to Jev is redacted first through
+`src/security/service.ts` — the same floor `review conform`/`review
+jev-risk`/`review jev-rules` already apply. No cache or output file stores a
+credential.
+
+---
+
+## Output Contract
+
+Emits a `REVIEW_RESULT`-shaped object matching
+`.metaproject/skills/gdskills/review/review-orchestrator/reviewer-finding.schema.json`
+— `status`, `reviewer: "review-jev-contract"`, `summary`, `findings`, `stats`
+— under `--json`, plus two extras: `claims` (every extracted claim, its
+intent, and its `noul` score) and `budget` (`--max-calls`, claims scored,
+claims skipped). When `--flow` names a flow, an `acCheck` object is merged in
+(`flowId`, `verdicts`, `jevAsked`, `summary`) — the same shape `keryx flow
+check-ac` reports. Its findings merge into the consolidated array exactly
+like any other reviewer's: same Quality Gate, same dedup, same Wave C
+verification. A `major` finding always carries a schema-valid `class_scope`
+object (`sites`, `enumeration_method`) — never a bare label.
+
+---
+
+## Orchestrator integration
+
+- `keryx review reviewers --json` lists it under `bundled` with
+  `"engine": "jev"`.
+- `review-orchestrator` dispatches it in **Wave B**, alongside the domain/
+  convention reviewers and the other CLI-engine reviewers, when the opt-in is
+  on and a Jev/OpenRouter credential resolves. When either is false it is
+  **skipped with the stated reason**, recorded in `Skipped reviewers` — never
+  silently absent.
+- It is **ADDITIONAL**: it never replaces any other pass. **When the opt-in is
+  on, it REPLACES the orchestrator's by-eye Stage 1 "description vs diff"
+  judgement with this scored input** — see `review-orchestrator/SKILL.md`'s
+  "This gate owns the description-vs-diff comparison" and `SKILL.detail.md`'s
+  "CLI-engine reviewers" section. When the opt-in is off, the by-eye check
+  stays the fallback, unchanged.
+
+---
+
+### Shared laws (every reviewer)
+
+1. **A claim of runtime harm with no reproducible path is `info`.** If you cannot
+   name the input, call, or condition that reaches the code, you have an
+   observation, not a finding. Report it as `info` and say what would settle it.
+2. **Never flag the theoretical.** The path you describe must exist in the code
+   under review. Do not report a safe API because it could be misused, or a
+   pattern because it is often wrong elsewhere.
+3. **One finding per class, not one per occurrence.** When the same shape appears
+   at several sites, report it once and list every site. Ten findings that are one
+   finding hide the other nine problems.
+
+This reviewer satisfies all three by construction: `evidence` always names the
+matched artefacts/facts and the claim text (never an unreproducible claim); a
+finding is synthesized only against a claim actually extracted from the real
+description, never a hypothetical one; and each claim is its own site, so
+there is no repeated class across claims for this reviewer to collapse — a
+`major` finding's `class_scope` still enumerates every matched region, not
+just the first.
+
+---
+
+## Red Flags
+
+| Rationalization | Why it is wrong |
+|---|---|
+| "Jev said 0.9, so the claim is definitely supported" | A `noul` score is a probability the claim is evidenced, not a verdict — a contradiction detected by the deterministic facts overrides it regardless of the score |
+| "The description doesn't mention an API, so 'no change' can't be contradicted" | The contradiction check only runs when the claim's own text IS API-shaped (mentions api/public/export/interface/signature/contract) — a claim like "no change to behavior for CLI callers" is checked for tokens, but the exported-symbol contradiction path is intentionally narrower |
+| "No claims extracted means the description said nothing" | `--diff` targets have no PR description at all — zero claims there is a target-shape fact, not a description-quality one; check `budget.claimsSkipped` too before reading silence as clean |
+| "The AC track failed, so the whole review failed" | `acCheck` degrades to facts-only (never throws) exactly like `keryx flow check-ac` does — a Jev failure on the criteria track is advisory, same as on the claims track |
+
+---
+
+## Verification
+
+Before trusting a run's findings:
+
+1. Read `budget.claimsSkipped` in the `--json` output — a capped run checked
+   fewer than "every extracted claim" and the report should say so.
+2. Spot-check a `major` finding's `class_scope.sites` against the actual
+   diff: does the named region really touch an exported symbol the claim
+   said would not change?
+3. Confirm every finding carries `reviewer: "review-jev-contract"` and a
+   `dedupe_key` — both are required for the orchestrator's Quality Gate and
+   Wave C verification to route it correctly.
+
+---
+
+## Scope Boundaries
+
+| Concern | This skill | Use instead |
+|---------|------------|-------------|
+| Whether the PR description's own claims match the diff | YES | — |
+| Whether a flow's frozen acceptance criteria are likely met | YES (via `--flow`, reusing `check-ac.ts`) | `keryx flow check-ac` directly, for a standalone advisory run |
+| An actual security/concurrency finding | NO | `review-security-code` / `review-highload` |
+| Judging whether a reported finding is real | NO | `review-verifier` |
+| A ranked risk map of the diff's hunks | NO | `review-jev-risk` |

package/src/gdskills/bundled/skills/review/review-orchestrator/SKILL.detail.md

@@ -47,34 +47,45 @@ review: it is a fact about the review, not about the diff.
 ## CLI-engine reviewers — dispatched as a command, not a sub-agent
 
 `review-jev-rules` (flow 330), `review-jev-risk` and `review-jev-scenarios`
-(both flow 332), and `review-jev-docs` and `review-jev-comments` (flow 333)
-are ADDITIONAL reviewers, never replacing any other, and their dispatch
-mechanism differs from every reviewer named in SKILL.md's Routing Table:
-there is no platform-native agent to invoke, because each is a deterministic
-**keryx program**. Run them with `keryx review jev-rules --scope
-<scope.json> --json`, `keryx review jev-risk --scope <scope.json> --json`,
-and `keryx review jev-scenarios --scope <scope.json> --json` — the SAME
-`scope.json` every other Wave A/B reviewer's dispatch already reads, so each
-checks exactly the same hunks. `review-jev-docs` and `review-jev-comments`
-take a diff/PR target instead: `keryx review jev-docs (--diff <ref>|--pr
-<n>) --json` and `keryx review jev-comments --pr <n> --repo <owner/repo>
---json`. Read each `--json` output as a `REVIEW_RESULT` and merge its
+(both flow 332), `review-jev-docs` and `review-jev-comments` (flow 333), and
+`review-jev-contract` (flow 335) are ADDITIONAL reviewers, never replacing
+any other, and their dispatch mechanism differs from every reviewer named in
+SKILL.md's Routing Table: there is no platform-native agent to invoke,
+because each is a deterministic **keryx program**. Run them with `keryx
+review jev-rules --scope <scope.json> --json`, `keryx review jev-risk
+--scope <scope.json> --json`, and `keryx review jev-scenarios --scope
+<scope.json> --json` — the SAME `scope.json` every other Wave A/B reviewer's
+dispatch already reads, so each checks exactly the same hunks.
+`review-jev-docs` and `review-jev-comments` take a diff/PR target instead:
+`keryx review jev-docs (--diff <ref>|--pr <n>) --json` and `keryx review
+jev-comments --pr <n> --repo <owner/repo> --json`. `review-jev-contract`
+also takes a diff/PR target, plus an optional linked flow: `keryx review
+jev-contract (--diff <ref>|--pr <n>) [--flow <id>] --json` — a `--pr` target
+checks the description's own claims; `--flow <id>` additionally checks that
+flow's frozen acceptance criteria (reusing flow 328's `check-ac.ts`, see its
+own SKILL.md). Read each `--json` output as a `REVIEW_RESULT` and merge its
 `findings` into the consolidated array exactly like a sub-agent reviewer's:
 same Sub-Agent Report Quality Gate, same dedup, same Wave C verification.
 `review-jev-risk` additionally emits `ranked` and `routingHints`;
-`review-jev-scenarios` additionally emits `checklist` — see each reviewer's
-own SKILL.md for what to do with its extra.
+`review-jev-scenarios` additionally emits `checklist`; `review-jev-contract`
+additionally emits `claims`, `budget`, and (when `--flow` was given)
+`acCheck` — see each reviewer's own SKILL.md for what to do with its extra.
 
 Gate each BEFORE running its command, not after: skip it — recorded in
 `Skipped reviewers` with the reason, never silently absent — when its own
 opt-in (`review.jev.rules` / `review.jev.risk` / `review.jev.scenarios` /
-`review.jev.docs` / `review.jev.comments`) is not `true` in
-`.metaproject/tasks.config.json`, or when no Jev/OpenRouter credential is
-resolvable. Either gate failing means the command itself would refuse before
-any read or network call, so checking first saves a doomed dispatch. The
-five opt-ins are independent: any subset may be on. `review-jev-comments`
-additionally needs a comment ledger to already exist (`keryx review comments
-collect` run first) — it refuses, before any read, when there is none.
+`review.jev.docs` / `review.jev.comments` / `review.jev.contract`) is not
+`true` in `.metaproject/tasks.config.json`, or when no Jev/OpenRouter
+credential is resolvable. Either gate failing means the command itself would
+refuse before any read or network call, so checking first saves a doomed
+dispatch. The six opt-ins are independent: any subset may be on.
+`review-jev-comments` additionally needs a comment ledger to already exist
+(`keryx review comments collect` run first) — it refuses, before any read,
+when there is none. When `review.jev.contract` is on, dispatch
+`review-jev-contract` with `--pr` (never `--diff`, which has no description
+to check) and let its `findings` cover the description-vs-diff claim, in
+place of the by-eye Stage 1 judgement described in SKILL.md's "This gate owns
+the description-vs-diff comparison" — see that section's own note.
 
 `keryx review reviewers --json` marks a CLI-engine reviewer with
 `"engine": "jev"` on its `bundled` entry — the field's presence, not its
@@ -82,3 +93,52 @@ absence, is what distinguishes it from the default (an LLM sub-agent
 dispatch). A future engine-backed reviewer follows the same pattern: gate on
 its own opt-in and reachability, dispatch as a command, merge its `--json`
 output the same way.
+
+## `jev-triage` — advisory annotations, not a reviewer (Step 9b)
+
+`review-jev-triage` (flow 340) is a DIFFERENT shape from every CLI-engine
+reviewer above: it never produces a `findings` array of its own, and it is
+never merged into the consolidated array. It runs AFTER the Sub-Agent Report
+Quality Gate has already validated/merged/deduplicated the round's findings
+(Step 9) and BEFORE Wave C verification (Step 10) — Step 9b in the Workflow
+block — annotating the findings that already exist. It is advisory and
+annotate-only by construction: nothing it returns drops or demotes a finding,
+and no downstream step is permitted to treat its output as anything but a
+hint.
+
+Run it once per round, over the round's own consolidated findings:
+
+```bash
+keryx review jev-triage --report <this round's review package dir> --json
+```
+
+Gate it the same way as every other CLI-engine reviewer: skip it — recorded
+in `Skipped reviewers` with the reason — when `review.jev.triage` is not
+`true` in `.metaproject/tasks.config.json`, or when no Jev/OpenRouter
+credential is resolvable. Either gate failing means the command itself would
+refuse before any read or network call.
+
+Its `--json` output is `{status, reviewer: "review-jev-triage", summary,
+annotations, budget, tokens}` — never a `REVIEW_RESULT` merged into the
+findings array. `annotations` carries three tracks, each read and used
+differently in the report:
+
+| Track | Shape | What to do with it |
+|---|---|---|
+| `severity_check` | `{id, p, flagged}` per blocker/major finding — `flagged` means `p < 0.4` | Show `flagged` findings as a soft note next to their existing severity ("Jev's own trigger+outcome check scored this low"). Never change the finding's `severity` field from this alone. |
+| `merge_candidates` | `{id, a, b, reason, p}` per candidate pair | A high `p` is a suggestion to a human that `a` and `b` may be the same defect — surface it in the report as a note on both findings. Never merge, drop, or renumber either finding from this alone. |
+| `verify_order` | `{id, p}`, sorted lowest-`p`-first | Feed this order into Wave C's own dispatch — verify the lowest-plausibility findings first — never as a reason to skip verifying any of them. |
+
+`status` is `DONE_WITH_CONCERNS` — never `DONE` — when any `severity_check` is
+`flagged` OR any `merge_candidates` pair scores `p >= 0.5`
+(`LIKELY_DUPLICATE_THRESHOLD`, `src/commands/review-jev-triage.ts`). Live-check
+calibration found same-file pairs that were NOT duplicates scoring around
+0.6, so a merge candidate above the threshold is a prompt to look, never a
+merge — the status flip is a nudge to read the pair, not a verdict that the
+findings are duplicates.
+
+Show its annotations in the report as their own subsection, next to (not
+inside) the findings they annotate — same separation the finding schema
+itself draws between a reviewer's claim (`severity`, `problem`, …) and what
+became of it (`disposition`), which a reviewer never states and this pass
+does not either.

package/src/gdskills/bundled/skills/review/review-orchestrator/SKILL.md

@@ -42,6 +42,7 @@ Review Orchestrator Progress:
 - [ ] Step 7: Stage 1 gate - spec compliance check (if issue/task provided)
 - [ ] Step 8: Dispatch selected reviewers in PARALLEL with reviewer-input schema
 - [ ] Step 9: Collect reviewer-finding schema results and handle NEEDS_CONTEXT
+- [ ] Step 9b: `jev-triage` — advisory, annotate-only severity/duplicate/verify-order annotations over the consolidated findings (opt-in `review.jev.triage`)
 - [ ] Step 10: Wave C — dispatch `review-verifier` over the consolidated findings
 - [ ] Step 11: Sort by severity, deduplicate, emit unified report
 - [ ] Step 12: Emit the machine-readable `keryx:findings` block alongside the report
@@ -1168,7 +1169,7 @@ that already holds intent and diff side by side is this one.
 
 Run it on every round, not only the first. The drift the finding catches is
 created BY the rounds: the code moves to answer findings, the body does not, and
-whoever reads the merge commit a year later reads the body.
+whoever reads the merge commit a year later reads the body. When `review.jev.contract` is on, dispatch `review-jev-contract --pr` and read its `findings` as this comparison's scored result instead of judging it by eye; the by-eye judgement is the fallback when that opt-in is off — `SKILL.detail.md` § "CLI-engine reviewers".
 
 ---
 
@@ -1177,7 +1178,7 @@ whoever reads the merge commit a year later reads the body.
 Dispatch selected reviewers in parallel when independent. Use waves when token budget is tight or when one reviewer needs another result:
 
 1. Wave A - core correctness/risk reviewers: logic, architecture, security/highload when selected.
-2. Wave B - domain reviewers: frontend/backend/testing/convention reviewers filtered to relevant files. `review-jev-rules` (flow 330), `review-jev-risk`/`review-jev-scenarios` (flow 332), `review-jev-docs`/`review-jev-comments` (flow 333) also run here, CLI-engine not sub-agent, `"engine": "jev"` in `keryx review reviewers --json`, gated on their own opt-in and a resolvable Jev/OpenRouter credential — `SKILL.detail.md` § "CLI-engine reviewers".
+2. Wave B - domain reviewers: frontend/backend/testing/convention reviewers filtered to relevant files. `review-jev-rules` (flow 330), `review-jev-risk`/`review-jev-scenarios` (flow 332), `review-jev-docs`/`review-jev-comments` (flow 333), `review-jev-contract` (flow 335) also run here, CLI-engine not sub-agent, `"engine": "jev"` in `keryx review reviewers --json`, gated on their own opt-in and a resolvable Jev/OpenRouter credential — `SKILL.detail.md` § "CLI-engine reviewers".
 3. Wave C - **verification**: `review-verifier` over the consolidated findings, when blockers/majors
    exist, `--verify` is set, or the PR is high-risk. See below.
 
@@ -1691,8 +1692,7 @@ CONTEXT_PATH: .metaproject/jobs/<job-name>/ai/context.md
 If provided and the file exists, read the context document **before** running scope detection.
 Use it to understand:
 - Intentionally chosen libraries and patterns (do not flag as issues)
-- Architectural decisions already agreed upon
-- Acceptance criteria to drive the Stage 1 spec compliance gate
+- Architectural decisions already agreed upon, and acceptance criteria driving the Stage 1 spec compliance gate
 
 If absent, proceed normally — context is optional and non-blocking.
 

package/src/gdskills/bundled/stacks/c-cpp/agent-refs.json

@@ -0,0 +1,4 @@
+{
+  "agents": [],
+  "note": "honest gate (DeepSeek deepseek-chat runner+judge, strictness high, trials 10, flow 337) ran and failed for all four skills -- trigger accuracy, not behavior content (every behavior scenario scored 0.9 or 1.0). No generated pair ships; pack stays stability: experimental. See governance/eval.json for the recorded reports and W1-stack-catalog.md's Wave 4 batch 5 implementation notes for the diagnosis."
+}