@clipboard-health/ai-rules 2.37.2 → 2.38.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@clipboard-health/ai-rules",
-  "version": "2.37.2",
+  "version": "2.38.0",
   "description": "Pre-built AI agent rules for consistent coding standards.",
   "keywords": [
     "ai",

package/skills/blind-backtest/SKILL.md

@@ -0,0 +1,46 @@
+---
+name: blind-backtest
+description: Backtest an automated judgment stage (clustering, verdicts, routing, gating) against historical decisions with known outcomes. Use before any judgment skill goes live or into shadow operation, or when asked to grade an agent's decisions against a human's.
+---
+
+# Blind Backtest
+
+Before an automated judgment stage touches production state, prove it against history. The trap this skill exists to avoid: a single agent that can see how cases actually resolved will leak outcomes into its judgments and produce a worthless score. Blinding is enforced by splitting roles across separate agents that never share context.
+
+The coordinator (you, the session running this skill) holds the answer key and grades. Two subagents do the work:
+
+## Phase 1: Assembler (sees outcomes — never judges)
+
+Spawn a subagent that may read the historical record (Linear, GitHub, logs) to:
+
+- **Select a balanced cohort** (~20 cases is usually enough to expose error structure): both outcome classes represented, ideally ~60/40. If one class is rare, say so rather than padding with weak cases.
+- **Verify outcomes, don't assume them.** "Ticket Done" is weaker than "Done AND the PR merged"; "Canceled" is weaker than "canceled by a reviewer's judgment" (bulk sweeps and staleness cancellations are noisy labels — flag them in the key).
+- **Sanitize each case into an input file** containing only what existed _before_ the human verdict: the plan/content under judgment and its cited evidence. Strip everything outcome-revealing: states, completion/cancel dates, merge status, post-decision comments, and identifiers (ticket/PR numbers → `[redacted]`) that the judge could look up.
+- **Write** `inputs/` files (shuffled, opaque/neutral filenames like `case-01`, no class grouping) and `answer-key.json` (`{file, source, outcome, basis}`) to a scratch directory, then grep the inputs case-insensitively (`grep -i`) — contents, filenames, and any frontmatter — for leak terms (merged, closed, canceled, done, wontfix, approved, rejected, released, accepted, resolved) so capitalized status labels like `Done` or `Merged` aren't missed, and justify any survivor.
+- Report cohort size, balance, weak-label caveats, and any correlated cases (same family/mechanism) that could let the judge score unearned points.
+
+## Phase 2: Blind judge (sees nothing but the inputs)
+
+Spawn a separate subagent with an explicit blind protocol in its prompt:
+
+- May read ONLY: the judgment skill under test (and its references), and the `inputs/` files.
+- Forbidden, named explicitly: the answer key, the scratch directory's other files, Linear/GitHub/network, and anything that could reveal resolutions. Writes nothing.
+- Must judge each case per the skill under test, outputting exactly one structured line per case, keyed by the input's opaque filename: verdict, confidence, classification, cited rules, one-line justification quoting the decisive content.
+- Must flag cases it found genuinely borderline — those are where the grading discussion matters.
+
+Never reuse the assembler as the judge; an agent cannot unsee outcomes.
+
+## Phase 3: Grade (coordinator)
+
+- Join each verdict to `answer-key.json` by its opaque filename, never by output order. Require exactly one well-formed verdict per input before grading; missing, duplicate, or malformed records are protocol failures — re-run the affected cases under the same blind protocol, up to a small fixed retry budget (e.g., 2 attempts per case). If a case still lacks exactly one well-formed verdict after that, stop and report the run as a protocol failure rather than looping or grading a partial cohort. Never drop or exclude a case after seeing its verdict; outcome-dependent exclusion biases the matrix. Cohort membership is fixed before judging.
+- Build the confusion matrix against the answer key. Escalations (needs-human) on a correct-outcome case count as _safe non-matches_, not errors — bouncing up is designed behavior — but track them as a distinct outcome and report the escalation rate separately, so an agent that escalates everything cannot look artificially safe.
+- **Decompose every error before trusting the headline number.** The three buckets that recur:
+  1. **Era artifacts** — the skill enforces conventions that postdate the historical cases (will vanish on live traffic; consider scoring without them, but say so).
+  2. **Weak ground truth** — sweep-canceled, stale, or bulk-handled cases where the label doesn't reflect a judgment on quality.
+  3. **Genuine misses** — the only bucket that demands a rule change.
+- **Convert findings into amendments**: every genuine miss becomes a concrete edit to the skill/rubric under test, applied immediately and noted with date + "backtest amendment" provenance. Era artifacts are reported separately and become edits only when independently confirmed as live rule gaps — never amend the rubric for a systematic artifact alone, or you encode obsolete conventions and overfit future judgments.
+- A backtest validates the _mechanism_ (verdicts cite checkable rules) and surfaces rule gaps.
+
+## Output
+
+Report, in order: cohort summary (size, balance, label caveats); confusion matrix — raw score first, then the decomposed/adjusted view with its assumptions; error decomposition by bucket; amendments applied; borderline cases worth a human look; and what this backtest does and does not establish — **promotion to live enforcement still requires live shadow agreement**, since historical labels are too noisy to clear a high bar alone.

package/skills/cb-babysit/SKILL.md

@@ -19,9 +19,9 @@ Resolve bundled "./scripts" paths relative to SKILL.md.
 
 The skill uses two sentinels with visible footer lines.
 
-**Addressed sentinel**: `<sub>🤖 <code>cb-babysit:addressed v1 core@3.16.2</code></sub>`. Appended on its own line at the end of every reply the skill posts so re-runs know which threads and review-body comments are already handled. Dedupe also recognizes the legacy `babysit-pr:addressed v1` prefix from before this skill was renamed.
+**Addressed sentinel**: `<sub>🤖 <code>cb-babysit:addressed v1 core@3.17.0</code></sub>`. Appended on its own line at the end of every reply the skill posts so re-runs know which threads and review-body comments are already handled. Dedupe also recognizes the legacy `babysit-pr:addressed v1` prefix from before this skill was renamed.
 
-**Follow-up sentinel**: `<sub>🤖 <code>cb-babysit:followup v1 core@3.16.2</code></sub>`. Attached to replies that defer an out-of-scope comment as a tracked follow-up. The sentinel is additive: the post-reply scripts still append the `addressed` sentinel at the end.
+**Follow-up sentinel**: `<sub>🤖 <code>cb-babysit:followup v1 core@3.17.0</code></sub>`. Attached to replies that defer an out-of-scope comment as a tracked follow-up. The sentinel is additive: the post-reply scripts still append the `addressed` sentinel at the end.
 
 **Sentinel recency rules.** The script emits a per-thread `activityState` with three values. Step 6a owns the handling rules for each state.
 
@@ -247,7 +247,7 @@ Body templates (the script appends the `addressed` sentinel if missing):
 - **Agree**: `Addressed in <commit-url>. <one-line what-changed>.`
 - **Disagree**: `Leaving current behavior. <reasoning>.`
 - **Already fixed**: `Already handled by <commit-url-or-file:line>. <brief pointer>.`
-- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>🤖 <code>cb-babysit:followup v1 core@3.16.2</code></sub>`
+- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>🤖 <code>cb-babysit:followup v1 core@3.17.0</code></sub>`
 
 For Defer replies, include the follow-up sentinel on its own line as shown. The script will append the `addressed` sentinel after it on its own line, so the final body ends with the follow-up sentinel followed by a blank line followed by the `addressed` sentinel. `grep cb-babysit:followup` finds the deferral and `grep cb-babysit:addressed` still marks the thread handled for dedupe.
 
@@ -263,7 +263,7 @@ The PR-level summary should:
 
 - Group by source. Use `## Review-body findings` for step-7 work and `## Conversation-tab comments` for step-6b work. Omit a section if its list is empty.
 - Inside each section, group verdicts under **Agree / Disagree / Already fixed / Deferred (out of scope)** subheadings. Omit a subheading if its list is empty.
-- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>🤖 <code>cb-babysit:followup v1 core@3.16.2</code></sub>` so grep catches them individually.
+- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>🤖 <code>cb-babysit:followup v1 core@3.17.0</code></sub>` so grep catches them individually.
 - Include the commit URL for fixes.
 - End with a fenced fingerprint block listing every current fingerprint (addressed and deferred) one per line. Include both `reviewBodyComments[].fingerprint` (whole-body, one per automated review) and `activeIssueComments[].fingerprint` (per Conversation-tab comment). Future runs dedupe by matching these against `priorBabysitSentinels`.
 

package/skills/cb-babysit/scripts/_sentinel.sh

@@ -10,7 +10,7 @@
 
 SENTINEL_PREFIX='cb-babysit:addressed v1 '
 LEGACY_SENTINEL_PREFIX='babysit-pr:addressed v1 '
-SENTINEL='<sub>🤖 <code>cb-babysit:addressed v1 core@3.16.2</code></sub>'
+SENTINEL='<sub>🤖 <code>cb-babysit:addressed v1 core@3.17.0</code></sub>'
 
 # Bot author allowlist (JSON array literal). Used by unresolvedPrComments.sh
 # as a fallback when GraphQL's `author.__typename == "Bot"` misses a GitHub

package/skills/cb-review/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: cb-review
-description: Code review of a diff, branch, or PR, with findings posted as anchored PR comments. Use when the user asks to review a diff, branch, or PR, asks to check a change against its ticket/spec/PRD, or runs /cb-review [pr-number-or-url] [--effort low|high] [--report].
-argument-hint: "[pr-number-or-url] [--effort low|high] [--report]"
+description: Code review of a diff, branch, or PR, with findings posted as anchored PR comments. Use when the user asks to review a diff, branch, or PR, asks to check a change against its ticket/spec/PRD, or runs /cb-review [pr-number-or-url] [--effort low|high] [--report] [--spec-context <path-or-reference-or-text>].
+argument-hint: "[pr-number-or-url] [--effort low|high] [--report] [--spec-context <path-or-reference-or-text>]"
 ---
 
 # CB Review
@@ -17,6 +17,7 @@ Review a diff against one rubric, filter to the few findings worth raising, gate
 - `/cb-review <pr-number-or-url>` — review that PR without checking it out; forces reviewer mode. Accepts a bare number (current repo) or full GitHub URL (identifies owner/repo).
 - `--effort low|high` — pick the engine explicitly. Phrases also select: "quick"/"fast" → low; "deep"/"thorough"/"multi-perspective" → high.
 - `--report` — non-interactive: stop after Synthesize and return the findings to the caller. No user gates, no posting, no implementing. For agent callers (e.g. cb-ship's review step).
+- `--spec-context <path-or-reference-or-text>` — use the value only as the originating source of truth, never as the PR selector. Read a local path in full, fetch a ticket or spec reference, or preserve natural-language text verbatim. Use the resolved content for the Spec lens instead of reconstructing intent from commit or PR prose.
 
 **Effort auto-select** (no flag, no phrase): `high` when the diff exceeds 20 changed files or 600 changed lines, else `low`. Before reviewing, print one line — `Effort: <low|high> (<N> files, <M> lines; override with --effort <other>)` — so the user can interrupt.
 
@@ -50,7 +51,9 @@ Determine **mode**:
 - PR exists and authors differ → **reviewer mode**.
 - No PR → **author mode**.
 
-**Persistence:** both efforts persist for subagents into a fresh per-run directory — `RUN_DIR=$(mktemp -d "${TMPDIR:-/tmp}/cb-review.XXXXXX")` — so concurrent sessions never clobber each other: diff → `$RUN_DIR/diff.patch`, context → `$RUN_DIR/context.md`, changed files → `$RUN_DIR/files.txt`, metadata (PR number/url/base/author, viewer, head SHA, owner/repo, mode, `context_ref`) → `$RUN_DIR/meta.json`.
+**Persistence:** both efforts persist for subagents into a fresh per-run directory — `RUN_DIR=$(mktemp -d "${TMPDIR:-/tmp}/cb-review.XXXXXX")` — so concurrent sessions never clobber each other: diff → `$RUN_DIR/diff.patch`, context → `$RUN_DIR/context.md`, changed files → `$RUN_DIR/files.txt`, metadata (PR number/url/base/author, viewer, head SHA, owner/repo, mode, `context_ref`, `spec_source`) → `$RUN_DIR/meta.json`.
+
+When the caller supplied source-of-truth context, include it verbatim in `$RUN_DIR/context.md` and set `spec_source` so every dispatched reviewer uses it.
 
 ## Freshness preflight (mandatory before reading code)
 
@@ -121,7 +124,7 @@ Walk the changed-file list. Activate lenses that match:
 - **Security** triggers on: `routes/`, `controllers/`, `middleware*/`, files matching `auth*`/`*permission*`/`*acl*`/`*token*`/`*session*`, response serializers, OpenAPI/contract definitions, new API endpoint files.
 - **Database** triggers on: `migrations/`, `*.sql`, files matching `schema*`, Mongoose/Prisma model files (`models/`, `*.model.ts`, `*.schema.ts`), repository/DAL files, query builders.
 - **Frontend** triggers on: `*.tsx`, `*.jsx`, `*.css`, `*.scss`, `pages/`, `components/`, `hooks/`, or anything importing from `react`, `@tanstack/react-query`, or a design-system package.
-- **Spec** triggers when a spec source exists. Look in order: (1) issue/ticket references in the PR body or commit messages (`#123`, `Closes #45`, Linear/Jira keys) — fetch via `gh` or the tracker; (2) a path the user passed as an argument; (3) a plan/PRD file under `docs/`, `specs/`, or `.scratch/` matching the branch or feature name. Nothing found → skip the lens and note "no spec available" in Summary.
+- **Spec** triggers when a spec source exists. Look in order: (1) source-of-truth context supplied by the caller; (2) issue/ticket references in the PR body or commit messages (`#123`, `Closes #45`, Linear/Jira keys) — fetch via `gh` or the tracker; (3) a plan/PRD file under `docs/`, `specs/`, or `.scratch/` matching the branch or feature name. Nothing found → skip the lens and note "no spec available" in Summary.
 
 Always-on lenses: **Engineering**, **Minimalism**, **Conventions**, **AntiSlop**.
 
@@ -131,7 +134,7 @@ The rubric — severity ladder, `failure_mode` contract, do-not-raise list, NIT
 
 ### Low effort
 
-Dispatch **one** reviewer subagent — fresh eyes on the diff, and the bulk content stays out of your context. Its prompt carries the persisted file paths, the absolute path to references/review-rubric.md with the instruction to read it in full, the active lens list, and the two contracts from multi-agent.md §Dispatch mechanics (context-read, cross-repo evidence) with `<context_ref>` substituted and the moderator/Round-2 sentence replaced by: emit `evidence_required` findings capped at MAJOR; the dispatching agent resolves them in the Filter's cross-repo audit. Finding ids use an `R` prefix in place of roster letters. The subagent walks the diff, applying every active lens — the walk is done only when every hunk has been read under each active lens. Exhaustive reading, selective output: it returns _the smallest number of high-signal findings_ in the Round 1 output shape (multi-agent.md §Round 1), flagging anything that needs deeper investigation than it can do confidently rather than guessing. If the host cannot run subagents, do that same single pass yourself inline.
+Dispatch **one** reviewer subagent — fresh eyes on the diff, and the bulk content stays out of your context. Its prompt carries the persisted file paths, the absolute path to references/review-rubric.md with the instruction to read it in full, the active lens list, the spec source when the Spec lens is active, and the two contracts from multi-agent.md §Dispatch mechanics (context-read, cross-repo evidence) with `<context_ref>` substituted and the moderator/Round-2 sentence replaced by: emit `evidence_required` findings capped at MAJOR; the dispatching agent resolves them in the Filter's cross-repo audit. Finding ids use an `R` prefix in place of roster letters. The subagent walks the diff, applying every active lens — the walk is done only when every hunk has been read under each active lens. Exhaustive reading, selective output: it returns _the smallest number of high-signal findings_ in the Round 1 output shape (multi-agent.md §Round 1), flagging anything that needs deeper investigation than it can do confidently rather than guessing. If the host cannot run subagents, do that same single pass yourself inline.
 
 ### High effort
 

package/skills/cb-ship/SKILL.md

@@ -1,12 +1,13 @@
 ---
 name: cb-ship
 description: Ship changes. Simplify the diff, commit, review, push, and open or update a PR. Use when the user says 'ship it', 'commit and push', or wants a PR created or updated.
-argument-hint: "[--draft]"
+argument-hint: "[--draft] [--spec-context <path-or-reference-or-text>]"
 ---
 
 ## Setup
 
 - If `gh auth status` fails, stop and tell the user.
+- Treat `--spec-context` as source-of-truth input for review, never as a PR selector. Preserve and forward its value unchanged; `cb-review` owns resolution.
 - If `git fetch` or `git push` fails with public key or agent errors, retry the same operation with `git -c credential.helper='!gh auth git-credential'` while preserving the original remote and branch mapping. For `git fetch`, keep the configured remote (e.g., `git -c credential.helper='!gh auth git-credential' fetch origin <same arguments>`) so `origin/*` refs update; for `git push`, use the same destination and branch mapping, substituting `https://github.com/<org>/<repo>.git` only when the configured remote URL is SSH.
 - If `git rev-parse --verify origin/HEAD` fails, `origin/HEAD` is unset. Stop and tell the user to run `git remote set-head origin -a`.
 - If `git status --short`, `git log --oneline origin/HEAD..HEAD`, and `gh pr view --json url --jq .url 2>/dev/null` are all empty, stop and reply "nothing to ship."
@@ -18,7 +19,7 @@ Resolve bundled "./references" paths relative to SKILL.md.
 1. Create a new branch if on the default branch (e.g., `feat/add-user-validation`, `fix/null-check-in-parser`).
 2. Use ./references/simplify.md on the full PR diff: `git diff $(git merge-base HEAD origin/HEAD)..HEAD` and any uncommitted changes.
 3. Inspect `git status --short`, identify intended files, ask if ambiguous, then git add them. If uncommitted changes, create a conventional commit with `git commit --no-gpg-sign`.
-4. Invoke the `cb-review` skill with `--effort low --report` and triage each returned finding yourself: if it's real and in scope, apply it, rerun the repo's relevant checks, and commit; otherwise dismiss it with a one-line reason for the output. Skip this step when the session already ran cb-review over these same changes.
+4. Invoke the `cb-review` skill with `--effort low --report`, forwarding `--spec-context` unchanged when supplied so the Spec lens runs against the actual request. Triage each returned finding yourself: if it's real and in scope, apply it, rerun the repo's relevant checks, and commit; otherwise dismiss it with a one-line reason for the output. Skip this step when the session already ran cb-review over these same changes and the same source of truth.
 5. Push changes to origin.
 6. Create or update the PR using ./references/pr-template.md:
    a. If the host exposes a session ID (e.g., CODEX_THREAD_ID, CLAUDE_CODE_SESSION_ID), include the resume command in the PR body in backticks: ``Agent session: `codex resume <id>` `` or ``Agent session: `claude --resume <id>` ``. Preserve existing `Agent session:` lines, append only.

package/skills/cb-ship/references/pr-template.md

@@ -17,5 +17,5 @@ Concisely explain the user intent from session history and the meaningful behavi
 
 Optional, don't fabricate: ticket links, rollout plan, residual risk, or specific areas for reviewers to focus.
 
-<sub>🤖 <code>cb-ship:created v1 core@3.16.2</code></sub>
+<sub>🤖 <code>cb-ship:created v1 core@3.17.0</code></sub>
 ```

package/skills/cb-work/SKILL.md

@@ -27,12 +27,21 @@ The plan or request is the source of truth for scope:
 
 Run the relevant checks after each meaningful unit of work, not only at the end.
 
+For behavior changes in repositories with automated tests, use vertical red-green slices:
+
+1. Derive the public test seam from the source of truth. If the seam is unclear and the choice would materially affect scope or architecture, stop and ask; otherwise state the seam in a progress update and proceed.
+2. Write one behavior test through that seam. Avoid private interfaces, internal-collaborator assertions, and expected values computed with the implementation's algorithm.
+3. Run the focused test and confirm it fails for the intended reason.
+4. Implement only enough to pass, rerun the focused test, and repeat one slice at a time.
+
+Mock only at system boundaries such as third-party APIs, time, randomness, or external I/O. Prefer real controlled collaborators, including a test database when practical. Run the narrowest applicable typecheck regularly; if the repository exposes only a slow workspace-wide typecheck, leave it to CI unless the plan or user requires it. For documentation, metadata, or other changes without runtime behavior, do not invent a test; use the relevant validation instead.
+
 ## Validate
 
-If the plan names specific checks, use those; skip clearly slow or CI-only suites. Otherwise find validation commands in, e.g., `AGENTS.md`, `CLAUDE.md`, `CONTRIBUTING.md`, or pre-commit/pre-push hooks.
+If the plan names specific checks, use those. Otherwise find relevant validation commands in, e.g., `AGENTS.md`, `CLAUDE.md`, `CONTRIBUTING.md`, or pre-commit/pre-push hooks. Prefer checks scoped to the touched projects or files; skip clearly slow, resource-intensive, or CI-only suites unless the plan or user requires them.
 
-Done when every check passes or the user has explicitly accepted a failure.
+Done when every selected check passes or the user has explicitly accepted a failure.
 
 ## Hand off
 
-Invoke the `cb-ship` skill, passing `--draft` if it was passed to this skill. Relay `cb-ship`'s reply to the user.
+Invoke the `cb-ship` skill, passing `--draft` if it was passed to this skill. Pass `--spec-context` with the source of truth: the resolved absolute plan path, ticket or spec reference, or the natural-language implementation request. Preserve the original context instead of relying on the eventual commit or PR description to reconstruct it. Relay `cb-ship`'s reply to the user.