bigpowers 2.86.5 → 2.86.7

package/.qwen/skills/release-branch/SKILL.md

@@ -0,0 +1,302 @@
+---
+name: release-branch
+model: haiku
+description: "Make the merge/PR/keep/discard decision for a feature branch, verify coverage gates, create the PR with gh, and clean up the worktree. Use when a feature is done and ready to ship, or when user says \"release\", \"merge\", or \"open a PR\"."
+---
+
+<!-- story: e38s07 -->
+<!-- story: e45s15 -->
+<!-- story: e45s32 -->
+<!-- story: e45s39 -->
+<!-- story: e20s02 -->
+
+
+# Release Branch
+
+> **HARD GATE** — Do NOT merge or release if tests fail or if coverage gates are not met. If the branch is red, return to `develop-tdd` to fix regressions or add missing tests before proceeding.
+
+Finalize a completed feature branch: verify coverage gates, integrate onto `main`, and clean up the worktree.
+
+## Additional modes
+- `--hotfix`: Cherry-pick to main + tag. Skip PR in solo.
+- `--squash-state`: Squash `chore(state):` commits before merge.
+
+## Integrate mode
+
+Read `specs/state.yaml` key `workflow_mode` (`team-pr` | `solo-git`). Fall back to `profiles/solo-git.md`.
+
+| Mode | When | Ship path |
+|------|------|-----------|
+| **solo-local** | `workflow_mode: solo-git` | Auto: `scripts/land-branch.sh` if present, else fallback (Step 5) |
+| **team-pr** | `workflow_mode: team-pr` (default) | `gh pr create` → `gh pr merge --squash` |
+
+If unsure, prefer **solo-local**.
+
+## Process
+
+> **Timing:** `bash scripts/bp-timing.sh start release-branch` at invocation; `bash scripts/bp-timing.sh end release-branch` before handoff.
+
+### 1. Final verification
+
+```bash
+<full test command> && <typecheck command> && <lint command>
+git log main...HEAD --oneline | grep -vE "^[a-f0-9]+ (feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)(\(.+\))?!?: .+$" && echo "❌ Non-conventional commits found" || echo "✅ Commits verified"
+# Block AI agent attribution (P1)
+git log main...HEAD --format="%B" | grep -qiE 'co[- ]authored[- ]by' && echo "❌ Co-authored-by footer found — blocked" || echo "✅ No AI attribution"
+```
+
+- [ ] All tests pass, no type errors, no lint violations, all commits follow Conventional Commits
+- [ ] **NO `Co-authored-by` or `Co-Authored-By`** in any commit body — P1 rule (CONVENTIONS.md § Git Attribution). `land-branch.sh` blocks the merge if found.
+
+### 2. Coverage check
+
+- [ ] Overall coverage ≥ 80%; business logic coverage ≥ 95%
+
+### 2a. Security gate
+
+- [ ] `specs/security/REVIEW.md` exists and is fresh (matches current branch diff)
+- [ ] No unresolved HIGH findings with confidence ≥ 8 (or all documented in `specs/security/EXCEPTIONS.md` with sign-off rationale)
+
+If REVIEW.md is missing or stale → run `security-review` inline. Findings block the merge unless documented in EXCEPTIONS.md.
+
+### 2b. Traceability gate
+
+Run `gate-trace` before merge. FAIL blocks merge; CONCERNS requires explicit override in `specs/state.yaml` (`traceability_override: CONCERNS accepted, reason: <explanation>`). WAIVED if no matrix available.
+
+> **Adversarial refute framing (e45s32):** The final pre-merge check is **refute, not rubber-stamp**. Before declaring ready, actively try to disprove traceability completeness — missing story tags, absent verify evidence, stale security review. Only proceed when refutation fails.
+
+### 3. Diff review
+
+- [ ] All commits intentional, no secrets, CONVENTIONS.md compliance
+
+### 4. Decision
+
+Options: **Release (solo-local)** / **Open PR** / **Keep branch** / **Discard**
+
+### 5. Solo-local integrate
+
+Run `commit-message` for the squash subject, then land:
+
+```bash
+# Path A (preferred):
+bash scripts/land-branch.sh <task-slug> "feat(scope): description"
+# Path B (fallback if land-branch.sh missing): see REFERENCE.md
+```
+
+### 6. Create PR (team-pr only)
+
+Create the pull request with a **literal provenance marker** in the body so agent-generated PRs are identifiable and do not rot silently:
+
+```markdown
+<!-- bigpowers-provenance: agent-generated -->
+```
+
+Place the marker on its own line immediately after the `## Summary` heading. Then merge via `gh`:
+
+```bash
+gh pr create --title "..." --body "$(cat <<'EOF'
+## Summary
+<!-- bigpowers-provenance: agent-generated -->
+
+- ...
+
+## Test plan
+- [ ] ...
+
+EOF
+)"
+gh pr merge --squash --delete-branch
+```
+
+`semantic-release` auto-detects the commit, bumps SemVer, tags the repo, generates release notes.
+
+### 7a. Archive completed epic capsule
+
+> **HARD GATE** — When all epic stories are done (all `done` in `execution-status.yaml`), archive the capsule:
+
+```bash
+mv specs/epics/eNN-slug specs/epics/archive/
+```
+
+### 7b. CI verification & agent lock release (e39s02)
+
+> **HARD GATE** — Do NOT declare success until CI completes. **Three-independent-facts** (e45s15): commit landed, workflow green, registry visible — see [REFERENCE.md](REFERENCE.md#three-independent-facts-release).
+
+```bash
+bash scripts/wait-for-ci.sh --timeout 600 --interval 30
+```
+
+- [ ] CI passes; `release.ci_verified: true` in state.yaml
+- On failure: `handoff.next_skill = fix-bug`
+
+### 8. Clean up & return
+
+Worktree prune, branch delete, `git checkout main`. Cycle-time: see [REFERENCE.md](REFERENCE.md#cycle-time).
+
+Report: "Branch released."
+
+## Verify
+
+→ verify: `command -v gh >/dev/null 2>&1 && test -f specs/state.yaml && test -d skills/verify-work`
+
+---
+
+# Release Branch — Reference
+
+## Navigation
+
+| Lines | Section |
+|-------|---------|
+| 1 | Title |
+| 3–22 | Navigation |
+| 23–31 | PR body template (team-pr mode) |
+| 32–35 | Summary |
+| 36–41 | Verify |
+| 42–47 | specs/ artifacts |
+| 48–58 | Worktree cleanup details |
+| 59–81 | Cycle-time recording |
+| 82–100 | Why not story_start minus story_end? |
+| 101–121 | CI verification |
+| 122–127 | Solo-local fallback detail |
+| 128–134 | Handoff |
+| 135–159 | Reference block 1 |
+
+# Release Branch — Reference
+
+## PR body template (team-pr mode)
+
+```bash
+PR_TITLE="<type>(<scope>): <description>"
+echo "$PR_TITLE" | grep -vE "^(feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)(\(.+\))?!?: .+$" && echo "❌ ERROR: PR Title must follow Conventional Commits"
+
+gh pr create \
+  --title "$PR_TITLE" \
+  --body "$(cat <<'EOF'
+## Summary
+- [What this PR does]
+- [Key decisions made]
+
+## Verify
+- [ ] All tests pass
+- [ ] Coverage gates met (≥80% overall, ≥95% business logic)
+- [ ] CONVENTIONS.md compliance verified
+- [ ] PR Title follows Conventional Commits (for automated release)
+
+## specs/ artifacts
+- [List any specs/ files produced or updated]
+EOF
+)"
+```
+
+## Worktree cleanup details
+
+```bash
+# From the main repo root
+git worktree prune
+git worktree remove ../<branch-name> 2>/dev/null || true
+git branch -d <branch-name>
+```
+
+If `git worktree remove` fails due to uncommitted changes, ask: "There are uncommitted changes in the worktree. Force remove? (y/n)". If yes: `git worktree remove -f ../<branch-name>`.
+
+## Cycle-time recording
+
+After landing the branch, record delivery metrics using the git-derived,
+additive script (replaces hand-arithmetic):
+
+```bash
+bash scripts/record-cycle-time.sh append \
+  --story <story_id> --bcps <bcps> \
+  --range "$(git merge-base main HEAD)..HEAD" \
+  --file specs/metrics/cycle-times.yaml
+```
+
+This appends a row to the cycle-times ledger with two separated metrics:
+
+- **effort_hours** — ADDITIVE. Idle-stripped estimated effort from git commit
+  history (git-hours model: 120-min session threshold, 120-min first-commit pad).
+  Sums exactly to whole-repo effort. NO hand-arithmetic, NO wall-clock includes.
+- **lead_time_minutes** — calendar latency from first commit to merge.
+  Median-aggregated across stories; NEVER summed.
+
+The script also runs an additivity self-check: Σ(story effort) == whole-repo effort
+within rounding tolerance.
+
+### Why not story_start minus story_end?
+
+The previous hand-arithmetic approach (survey-context writes `story_start`,
+release-branch writes `story_end`, agent hand-computes `cycle_minutes`) was
+retired because:
+
+1. It was **agent-self-reported** — trivially fabricated or mis-subtracted.
+2. Wall-clock included **overnight/weekend/UAT gaps** — calendar latency,
+   not coding effort.
+3. The `bcp_per_hour` metric was **computationally meaningless** (velocity
+   derived from a latency measurement).
+
+The new approach derives effort from commit history (objective, reproducible)
+and lead time from first commit → merge (honest calendar latency). See
+`docs/references/bcp.md` for BCP sizing context and
+`scripts/record-cycle-time.sh` for the full algorithm.
+
+---
+
+## CI verification
+
+The CI polling logic has been extracted to `scripts/wait-for-ci.sh`.
+See the script's `--help` for usage. Step 7b of the main SKILL.md invokes it directly:
+
+```bash
+bash scripts/wait-for-ci.sh --timeout 600 --interval 30
+```
+
+**Exit codes:**
+- **0** — all workflows green. Set `release.ci_verified: true` in state.yaml.
+- **1** — at least one workflow failed. Prints failure URLs. Set `handoff.next_skill = fix-bug`.
+- **2** — timeout. CI did not complete. Retry or investigate.
+- **0 with warning** — `gh` CLI not available, git-only fallback confirmed push landed but CI status unverified.
+
+The script handles: auto-discovery of all workflows for the current
+branch/commit, polling until completion, success/failure/timeout exit codes,
+and git-only fallback when `gh` CLI is unavailable.
+
+---
+
+## Solo-local fallback detail
+
+The fallback sequence (Path B above) handles the "remote has moved" case with `git pull --rebase`. Use when `scripts/land-branch.sh` is absent.
+
+**Acceptance:** When fallback runs, main is updated, feature branch is deleted locally, and output states `"used fallback merge (land-branch.sh not found)"`.
+
+## Handoff
+
+Gate: READY -> next: survey-context
+Writes: state.yaml handoff.next_skill = survey-context
+
+---
+
+## Reference block 1
+
+```bash
+# Fallback: manual squash-merge when land-branch.sh is absent
+FEATURE_BRANCH=<task-slug>
+DEFAULT_BRANCH=$(git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's@^refs/remotes/origin/@@' || echo main)
+
+# Ensure we're on the feature branch
+if [ "$(git branch --show-current)" != "$FEATURE_BRANCH" ]; then
+  git checkout "$FEATURE_BRANCH"
+fi
+
+# Checkout default branch and update
+git checkout "$DEFAULT_BRANCH"
+git pull --rebase origin "$DEFAULT_BRANCH" 2>/dev/null || git pull origin "$DEFAULT_BRANCH"
+
+# Squash-merge the feature branch
+git merge --no-ff "$FEATURE_BRANCH" -m "<conventional-commit-message>"
+
+# Push
+git push origin "$DEFAULT_BRANCH"
+
+# Clean up local feature branch
+git branch -d "$FEATURE_BRANCH"
+```

package/.qwen/skills/request-review/SKILL.md

@@ -0,0 +1,113 @@
+---
+name: request-review
+model: opus
+description: "Dispatch a fresh reviewer agent with a clean context to critique the code after audit-code passes. The reviewer has no shared state with the coding agent and gives a genuine second opinion. Use after audit-code passes, before committing, or when user wants an independent code review."
+---
+
+# story: e45s28
+
+# Request Review
+
+Dispatch fresh reviewer agents with clean contexts. Reviewers have no shared state — they find what the coding agent missed.
+
+**Distinct from `audit-code`:** `audit-code` is self-review (internal). This skill dispatches external agents.
+
+**Solo developer note:** Reviewer agents replace the human reviewer.
+
+**Run `audit-code` first.** Don't waste reviewer attention on hygiene issues you could have caught yourself.
+
+## Santa Method — Dual-Blind AND Gate (e45s07)
+
+Use **two independent reviewers** (Reviewer A and Reviewer B) with **no shared context** between them or the coding agent.
+
+| Parameter | Value |
+|-----------|-------|
+| Reviewers | 2 (mandatory) |
+| `MAX_REVIEW_ITERATIONS` | **5** (hard cap — e45s28; iteration 6 forbidden) |
+| Pass rule | **AND-gate** — both reviewers must pass independently |
+| Blindness | Neither reviewer sees the other's report until both complete |
+
+**Iteration loop (max `MAX_REVIEW_ITERATIONS`):**
+
+1. Dispatch Reviewer A and Reviewer B in parallel with identical briefs but separate contexts.
+2. Collect both reports. Each categorizes findings: must-fix / should-fix / consider.
+3. **AND-gate:** If **either** reviewer has must-fix findings → FAIL round. Run `respond-review`, fix, re-dispatch both reviewers.
+4. If both pass (zero must-fix, score ≥ 94% each) → review complete.
+5. After **5** iterations without dual pass → **stop**; report "Review cap exhausted (5/5). Human decision required." Do not merge.
+
+> **HARD GATE** — Single-reviewer pass is insufficient. Partial agreement does not satisfy the AND-gate.
+
+## Process
+
+### 1. Prepare the review brief
+
+Write a self-contained brief for each reviewer. Include:
+
+- What was built (feature description, not implementation)
+- Which files changed (the diff context)
+- What `specs/` artifacts are relevant (active `epics/eNN-*.yaml`, `requirements/SCOPE_LATEST.yaml`, `bugs/BUG-*.md`)
+- What CONVENTIONS.md requires
+- What the verify command is
+- What you're most uncertain about (where you want fresh eyes)
+- **Security focus** — If the epic has a `specs/security/epics/<id>/THREAT_MODEL.md`, include the relevant vulnerability categories as reviewer focal points. Also include the false-positive exclusion rules so the reviewer avoids known-safe patterns. Tag the review as `security-sensitive: true` if THREAT_MODEL risk is HIGH+.
+
+### 2. Fan-out parallel reviewers (e45s17)
+
+Beyond the mandatory dual-blind pair (e45s07), optionally dispatch **N dimension-specific subagents in one message** — one check per agent for broader coverage (OpenAI Codex `code-review-*` pattern):
+
+| Agent | Focus |
+|-------|-------|
+| R-correctness | Logic, edge cases, verify command result |
+| R-conventions | CONVENTIONS.md, test quality (F.I.R.S.T) |
+| R-security | Injection, auth, secrets (when `security-sensitive`) |
+| R-design | Simpler alternatives, API shape |
+
+Santa Method still applies: each agent is blind; AND-gate uses Reviewer A + B scores. Fan-out agents feed findings into `respond-review` but do not replace the dual-blind pair.
+
+### 2b. Dispatch both reviewer agents (parallel)
+
+Use the Agent tool twice with completely fresh contexts. Each prompt must be self-contained — no references to "our conversation" or "what we discussed."
+
+```
+You are code reviewer [A|B]. Review the following code changes independently.
+
+Context: [feature description]
+CONVENTIONS.md rules: [paste relevant sections]
+Active epic shard: [paste or summarize from specs/epics/]
+
+Diff: [paste git diff or describe changed files]
+
+Verify command: [runnable command]
+
+Review for:
+1. Correctness — does the code do what was intended?
+2. CONVENTIONS.md compliance — are all rules followed?
+3. Test quality — do tests verify behavior (not implementation)?
+4. Design — are there simpler or more robust approaches?
+5. Edge cases — what inputs or states could cause failures?
+6. Security — any injection, auth, or data exposure risks?
+7. Refactoring smells — explicitly name any detected Fowler smells: Mysterious Name, Duplicated Code, Feature Envy, Data Clumps, Primitive Obsession, Message Chains, Middle Man
+
+For each finding, categorize as: must-fix / should-fix / consider.
+Run the verify command and report the result.
+```
+
+### 3. Collect both reports
+
+When reviewers return:
+- Read every finding from **both** reports before acting on any
+- Note each verify command result
+- Compute quality score per reviewer: `100 × (total_items − must_fix − should_fix) / total_items`
+- **AND-gate check:** both scores ≥ 94% and zero must-fix from **both**?
+
+> **HARD GATE** — If either score < 94% or either has must-fix → FAIL round. Run `respond-review` first. The 94% threshold also applies to `npm run compliance` (scripts/audit-compliance.sh).
+
+### 4. Hand off to respond-review
+
+Pass combined findings to `respond-review` to categorize and apply fixes. Increment iteration counter. Re-dispatch both reviewers until AND-gate passes or iteration 3 exhausted.
+
+Report to user: "Review round [N/3]. Reviewer A: [score], Reviewer B: [score]. AND-gate: [PASS|FAIL]."
+
+## Verify
+
+→ verify: `test -f scripts/lib/parallel-review-worktrees.sh && test -f skills/request-review/SKILL.md`

package/.qwen/skills/research-first/SKILL.md

@@ -0,0 +1,87 @@
+---
+name: research-first
+model: sonnet
+description: "Look-before-build — search registries, repo, existing skills, and web for prior art before implementing. Appends Prior Art to the spec. Use after survey-context and before elaborate-spec, when adding dependencies, or when the task may already be solved."
+---
+
+# story: e03s01
+# story: e21s02
+
+# Research First
+
+> **HARD GATE** — Do NOT implement until prior art is searched. Minimum outcome: adopt, extend, compose, or build — with evidence.
+
+## Process
+
+1. Read `specs/product/SCOPE_LATEST.yaml`, `specs/release-plan.yaml + epic shards`, and the current task statement.
+2. Search in order: this repo → bigpowers skills (`search-skills`) → package registries → web docs.
+3. **Check opensrc cache** — if the task integrates an external library, run `bash scripts/bp-opensrc-check.sh` (or `npx opensrc search <pkg>`) to find locally-cached source. Read the `src/` directory for API shapes before writing any integration code.
+4. For each candidate: note name, URL/path, fit (adopt | extend | compose | build).
+5. Append `## Prior Art` to `requirements/SCOPE_LATEST.yaml` notes or the active epic story.
+
+## opensrc Integration
+
+`opensrc` is a local cache of 200+ open-source repos and npm/PyPI packages. Query it before building any external integration to avoid re-inventing documented API shapes.
+
+```bash
+# Check if a package is cached
+npx opensrc search <package-name>
+
+# Or use the bundled helper (checks all deps from package.json or requirements.txt)
+bash scripts/bp-opensrc-check.sh [package.json|requirements.txt]
+```
+
+If opensrc finds a match, read its `src/` or source directory and append findings to the **Prior Art** section:
+
+```
+opensrc: found <pkg> v<version> — exports <key classes/functions>
+```
+
+If opensrc is not installed or the package is not cached, fall through to web docs normally.
+
+## Outcome matrix
+
+| Verdict | Action |
+|---------|--------|
+| **adopt** | Use as-is; link in plan; no new code |
+| **extend** | Wrap or configure existing solution |
+| **compose** | Chain existing skills/modules |
+| **build** | New implementation — justify why others failed |
+
+## Verify
+
+→ verify: `grep -rq 'Prior Art' specs/product specs/release-plan.yaml specs/epics 2>/dev/null`
+
+See [REFERENCE.md](REFERENCE.md) for search commands and registry checklist.
+
+---
+
+# Research First — Reference
+
+## Search commands
+
+```bash
+# Repo prior art
+rg -l "<keyword>" --glob '!node_modules' .
+find . -maxdepth 3 -name "SKILL.md" | xargs grep -l "<intent>"
+
+# Installed packages (if package.json exists)
+cat package.json | jq '.dependencies,.devDependencies' 2>/dev/null
+```
+
+## Registry checklist
+
+- [ ] npm / PyPI / crates.io (if applicable)
+- [ ] Existing bigpowers skill (`bash scripts/build-skill-index.sh && rg "<intent>" specs/SKILL-SEARCH-INDEX_LATEST.md`)
+- [ ] Project `docs/` and `specs/adr/`
+- [ ] Official library documentation (quote one API detail)
+
+## Prior Art template
+
+```markdown
+## Prior Art
+
+| Candidate | Source | Verdict | Notes |
+|-----------|--------|---------|-------|
+| ... | ... | adopt/extend/compose/build | ... |
+```

package/.qwen/skills/reset-baseline/SKILL.md

@@ -0,0 +1,21 @@
+---
+name: reset-baseline
+model: haiku
+description: "Restore the project to a known clean state between agent runs or experiments. Use between benchmark runs, after a failed spike, or when user wants a clean working tree."
+---
+
+# Reset Baseline
+
+> **HARD GATE** — Confirm with user before any destructive git operation. Never `reset --hard` without explicit approval.
+
+## Process
+
+1. `git status` — list uncommitted and untracked files.
+2. Ask: stash, discard, or keep each category.
+3. Safe defaults: `git stash push -u -m "reset-baseline"` for WIP; never force-push.
+4. Re-run `setup-environment` after reset.
+5. Run test baseline from `kickoff-branch` verify command.
+
+## Verify
+
+→ verify: `git rev-parse --git-dir >/dev/null 2>&1 && test -f skills/reset-baseline/SKILL.md`

package/.qwen/skills/respond-review/SKILL.md

@@ -0,0 +1,71 @@
+---
+name: respond-review
+model: sonnet
+description: "Act on a reviewer agent's feedback systematically — categorize findings, apply fixes, verify tests still pass. Use after request-review returns a report, or when user wants to work through code review findings."
+---
+
+# Respond Review
+> **HARD GATE** — **HARD GATE** — Every reviewer comment must be addressed (fix, disagree + document reason, or ask clarification). Do NOT ignore feedback and merge.
+
+
+Work through reviewer findings systematically. Don't apply changes blindly — categorize first, then decide, then fix, then verify.
+
+## Process
+
+### 1. Read the full review report
+
+Read every finding before acting on any of them. Get the full picture first.
+
+### 2. Categorize findings
+
+For each finding, assign a category:
+
+| Category | Meaning | Action |
+|----------|---------|--------|
+| **must-fix** | Correctness bug, security issue, test failure, CONVENTIONS.md violation | Fix before proceeding |
+| **should-fix** | Code quality issue, naming, clarity — worth fixing but not blocking | Fix if time allows |
+| **consider** | Architectural suggestion, alternative approach — may or may not apply | Discuss with user |
+
+Create a numbered list of all findings with their categories.
+
+### 3. Confirm with user (for consider-category items)
+
+For each "consider" item, briefly describe the trade-off and ask: "Apply, skip, or discuss?"
+
+### 4. Apply must-fix items first
+
+Fix every must-fix item. For each one:
+- Describe what you're changing and why
+- Make the change
+- Run the verify command if one exists for this area
+
+### 5. Apply should-fix items
+
+Apply should-fix items. If any are large enough to warrant their own commit, note them separately.
+
+### 6. Run the full suite
+
+After all changes are applied:
+
+```bash
+<full test command>
+<typecheck command>
+<lint command>
+```
+
+- [ ] All tests pass
+- [ ] No type errors
+- [ ] No lint violations
+
+### 7. Report
+
+Summarize what was applied and what was skipped:
+
+```
+Applied (must-fix): #1, #2, #3
+Applied (should-fix): #4
+Skipped (consider): #5 — agreed with user to defer
+All tests pass.
+```
+
+Suggest next skill: `commit-message`.

package/.qwen/skills/run-benchmark/SKILL.md

@@ -0,0 +1,85 @@
+---
+name: run-benchmark
+model: haiku
+description: "Run skill quality benchmarks from specs/benchmarks/ definitions — N-run with/without-skill delta grading, train/validation split, pass@k + benchmark.json reports. Use before and after evolve-skill to prove quality changes are improvements, not regressions."
+---
+
+# Run Benchmark
+
+> **HARD GATE** — Do NOT use benchmark scores to declare a skill "good" or "bad" in isolation. Benchmarks measure relative quality vs. a baseline — they catch regressions, they do not certify correctness.
+
+Reads benchmark definitions from `specs/benchmarks/`, executes each scenario's grader with and without the skill loaded, and writes a structured `pass@k` report with delta grading that `evolve-skill` consumes.
+
+## With/Without-Skill Delta Grading
+
+Every scenario runs N times (default 3) in two modes: **with** the skill loaded and **without** (bare agent with only CLAUDE.md). The delta `Δ = pass@k_with − pass@k_without` isolates the skill's causal contribution. A negative delta is a regression flag.
+
+## Train/Validation Split
+
+Benchmark definitions partition scenarios into two sets:
+
+| Set | Tag | Purpose |
+|-----|-----|---------|
+| **Train** | `split: train` | Development scenarios — used while iterating. Hitting 100% on train is expected. |
+| **Validation** | `split: validation` | Held-out scenarios — the real quality signal. Overfitting train while validation stagnates is a design smell. |
+
+`pass@k` is reported separately for train and validation. Validation score is authoritative; train score is iteration guidance only.
+
+## Usage
+
+```bash
+bash scripts/run-benchmark.sh <skill-name>           # benchmark single skill
+bash scripts/run-benchmark.sh --all                  # benchmark all with definitions
+bash scripts/run-benchmark.sh <skill-name> --baseline # pin results as baseline
+```
+
+## Process
+
+1. **Locate definition** — Read `specs/benchmarks/<skill>.yaml`. If absent, stop with message.
+
+2. **Partition scenarios** — Split by `split` field (`train` → iteration, `validation` → authoritative, default: `validation`).
+
+3. **Run each scenario (N-run delta)** — For each scenario, run grader N times (default 3, configurable via `runs:`):
+   - **Without skill:** Agent with only CLAUDE.md/CONVENTIONS.md
+   - **With skill:** Agent with the skill under test active
+   - Code grader: `bash -c <command>`, exit 0 → PASS. Timeout: 15s.
+   - Rubric grader: yes/no per criterion, ≥ 80% yes → PASS.
+   - Record: `{scenario_id: {with: [P/F,...], without: [P/F,...]}}`
+
+4. **Calculate scores** — Per split (train, validation) and mode (with, without):
+   - `pass@k = sum(weight × pass_rate) / sum(weights)` where `pass_rate = passes/runs`
+   - `Δ = pass@k_with − pass@k_without` — causal contribution
+   - Round to 2 decimal places
+
+5. **Write benchmark.json** to `specs/benchmarks/reports/benchmark-<skill>.json`:
+   ```json
+   {"skill":"survey-context","run_date":"2026-06-22","runs_per_scenario":3,"train":{"with_skill":0.92,"without_skill":0.67,"delta":0.25,"scenarios":["s01","s02"]},"validation":{"with_skill":0.83,"without_skill":0.60,"delta":0.23,"scenarios":["s03","s04","s05"]}}
+   ```
+
+6. **Write YAML report** to `specs/benchmarks/reports/BENCHMARK-<skill>-<YYYY-MM-DD>.yaml`:
+   ```yaml
+   skill: survey-context
+   run_date: "2026-06-22"
+   runs_per_scenario: 3
+   train:
+     pass_at_k_with: 0.92
+     pass_at_k_without: 0.67
+     delta: 0.25
+   validation:
+     pass_at_k_with: 0.83
+     pass_at_k_without: 0.60
+     delta: 0.23
+   scenarios:
+     - id: s01
+       split: train
+       with_pass_rate: 1.0
+       without_pass_rate: 0.67
+       delta: 0.33
+       weight: 1.0
+   ```
+
+7. **Baseline** (`--baseline`) — Copy to `BASELINE-<skill>.yaml` + `baseline-<skill>.json`.
+
+8. **Compare to baseline** — `IMPROVED: Δ 0.17 → 0.25` / `REGRESSION: Δ 0.25 → 0.17 — do NOT ship` / `STABLE`.
+
+9. **Delta threshold gate** — Validation Δ < 0.0 blocks release. Δ < 0.05 warns (marginal). Min meaningful threshold: 0.05.