shapeup-sdlc 1.6.2

package/skills/spec-evaluator/SKILL.md

@@ -0,0 +1,213 @@
+---
+name: spec-evaluator
+description: "Use this skill whenever the user wants to evaluate, QA, or verify that an implemented task actually matches its spec and acceptance criteria — the judge in a planner→generator→evaluator harness. Trigger on: \"evaluate task TASK-NNN\", \"QA TASK-NNN\", \"verify against spec\", \"check acceptance criteria\", \"does this match the spec\", \"grade this build\", \"run evaluator\", or a tech-lead --order dispatch. Skeptical by default — absence of evidence is a FAIL; probes the running app and files file:line bugs."
+---
+
+# Spec Evaluator (the single judge, pure worker v1.0)
+
+**Assume broken until proven working. Grade evidence, not claims. Return data, not writes.**
+
+The **judge** in a planner → generator → evaluator loop. It reads the committed spec, exercises
+the **running** app, and returns a hard-threshold verdict plus a file:line bug list — as a
+WorkResult envelope the orchestrator ingests. The generator fixes; the evaluator re-runs.
+
+**Core guarantee — skeptical by default.** Out-of-the-box an LLM is a lenient QA: it finds a
+real defect, then talks itself into approving anyway. This skill inverts that posture. A
+criterion with no collected evidence is a **FAIL**, never a pass-by-assumption.
+
+> **Anti-leniency protocol** → `references/anti-leniency.md` — read before printing any verdict.
+> **Verdict ledger (re-probe + confidence + flip detection)** → `references/verdict-ledger.md`.
+> **Dimension contract (injection interface)** → `references/dimension-contract.md`.
+> Where any reference file describes *writing* shared state (task files, `.verdicts` ledger,
+> run-state), the pure-worker contract overrides it: that data returns in the WorkResult and
+> the orchestrator's ingest script performs the write. Old gate names in reference files map
+> 1:1 onto the core process below: GATE V0/V0.5 = input contract + dimension resolution,
+> GATE V1 = CONTRACT, Phase A = PROBE, GATE V2 = VERDICT, Phase B = REPORT; GATE V3
+> (sign-off) is retired — pausing is the caller's `interaction` policy.
+
+---
+
+## Input contract — the WorkOrder
+
+Invoked as `--order <path>`. Fields you may rely on (absent = unknown, never inferred):
+
+| Field | What it is |
+|---|---|
+| `payload.spec_folder` | The committed grading truth: `usecases/` + `domain-model.md` (+ `contracts/`, `scope-summary.md`, `_index.md`). No `usecases/` → HARD STOP, nothing to grade against |
+| `payload.feature` | Feature slug — scopes the probe and names the report |
+| `payload.dimensions[]` | The active dimension set (the caller resolved precedence). Absent → `[spec-conformance]` + the auto-enable rules below |
+| `payload.run_cmd` | How to start the running app. Absent standalone → ask; absent orchestrated → ESCALATE, do not guess |
+| `payload.t0_artifacts[]` | Per-scope T0 verdict paths for this round (scoped specs). An artifact listed but missing/red on disk, or a scoped spec with none listed → the round is NOT gradeable: return `status: failed` naming the scope — a structural precondition, not a criterion |
+| `payload.browser` | `cli` (default, ~4x cheaper) \| `mcp` \| `none` |
+| `payload.tasks[]` | Traceability only (which UCs a task claims): NEVER a grading source — the committed UC text is the criterion, a paraphrase mismatch is a finding |
+| `substrate.allowed` | Your only write surface: `.shapeup/<slug>/evaluation/**` (the report + evidence) |
+
+**Grading source of truth.** `spec-conformance` grades against the committed `usecases/UC-*.md`
+(Steps, Error Cases, Invariants, Test Surface) and `domain-model.md` — never against a task
+file's own AC paraphrase. Task boards are LOCAL, regenerable bookkeeping the judge never touches.
+
+**Dimension resolution (craft, kept).** Base `[spec-conformance]` + always-on `tdd-surface` +
+`integration` (`.be`/`.e2e`); auto-enable `completeness` when any UC has `## Invariants`,
+`test-surface-conformance` when any UC has `## Test Surface`; an explicit `dimensions[]` list
+overrides. Each active dimension's file must satisfy `references/dimension-contract.md` — a
+half-formed dimension is SKIPPED with a warning, never run. Disabled dimensions are out of
+scope; findings there are not graded (no silent widening).
+
+---
+
+## Core process
+
+```
+CONTRACT  extract every criterion from the committed spec; classify each probe
+          [cmd] | [ui] | [data] | [manual]; a [manual]/ambiguous criterion is a spec
+          defect to surface, never a silent pass
+PROBE     exercise the RUNNING app; collect evidence only — no grading yet
+VERDICT   grade each criterion vs its dimension's hard threshold, evidence-only;
+          re-probe every FAIL once; flips force confidence low
+REPORT    write EVAL report (your substrate) + return the WorkResult envelope
+```
+
+**CONTRACT.** Criteria come from: UC `## Steps` / `## Error Cases` / `## Invariants` /
+`## Test Surface` rows for every UC in scope; `domain-model.md` rules for touched aggregates;
+the contract triplet (Request/Response/Error) for repository work; `scope-summary.md`
+Done-when statements; `_index.md` Non-Go list. Which UCs are in scope comes from
+`payload.tasks[]` traceability or, standalone, from the user (max 2 questions).
+
+**PROBE (evidence, not grades)** — `references/probing.md`:
+- `[cmd]`: run it, capture stdout/stderr + exit code.
+- `[ui]`: drive the app (Playwright CLI preferred). **Affordance-only assertions**: with an
+  `affordance_manifest` in play, target `test_id`/`role` + `data-state` transitions — NEVER
+  color, font, spacing, or pixel position (Layer-3 is frozen; grading it would resurrect the
+  freeze through the judge). Ugly-but-correct PASSes; pretty-but-wrong-`data-state` FAILs.
+- `[data]`: query the DB/storage, capture actual state.
+- Contract work: send real requests, compare field-by-field.
+- No evidence collected = recorded "NO EVIDENCE" → FAILs at verdict.
+
+**VERDICT.**
+- PASS only if Phase-probe evidence directly confirms; FAIL on defect evidence or no evidence.
+- Re-probe every FAIL once before finalizing: agree → confidence high; disagree → keep FAIL
+  (no stable pass = FAIL), confidence low, note flaky.
+- Read any existing `.verdicts-*.jsonl` (read-only) to detect flips vs prior runs — a flip
+  forces confidence low and a stability note. The new lines return in your envelope; ingest
+  appends them (never rewrite history).
+- Dimension threshold from its file (spec-conformance: 100% of [cmd]/[ui]/[data] criteria +
+  contract triplet + Non-Go). Overall PASS only if ALL active dimensions pass — the halo
+  effect is banned; a strong dimension never lifts a failing one.
+- **T0 citation (scoped specs).** Recompute each cited artifact's sha256 from disk — never
+  trust a handed hash. A verdict on a scoped spec without a T0 citation is structurally
+  invalid, regardless of how convincing your own probing looked; generator prose ("tests
+  pass", "verified locally") is never admissible evidence.
+
+---
+
+## Anti-rationalization table
+
+| Excuse | Reality |
+|---|---|
+| "The code clearly implements it, no need to run it" | Apps that look right still break when used. Probe the running app. |
+| "It failed, but the feature mostly works" | One FAIL fails the dimension. Thresholds are hard. |
+| "The generator says tests pass" | Generator prose is not evidence. Your probe or the T0 artifact is. |
+| "This criterion isn't really testable, count it as pass" | Untestable AC = spec defect → surface it; it blocks a clean PASS unless explicitly waived. |
+| "The other dimensions are strong, round up" | Halo effect banned. Dimensions never average. |
+| "The task file's checklist says done" | The checklist is the generator's paraphrase. Grade the committed UC text. |
+| "Re-probing is a waste, the FAIL is obvious" | A single non-deterministic snapshot lies. Re-probe; report the flip honestly. |
+
+---
+
+## Output contract — the WorkResult
+
+1. Write the report `.shapeup/<slug>/evaluation/EVAL-FEATURE-<slug>.md` (or
+   `EVAL-<task_id>.md` for a per-task run) per `references/report-schema.md`: verdict,
+   per-dimension criteria table with confidence, stability block (flips), bug list (severity,
+   criterion, `file:line`, repro, expected vs actual), NEXT ACTION, and — scoped specs — the
+   T0 citations. A scoped report with no citation field is malformed; do not write it.
+2. Write `.shapeup/<slug>/results/<order-suffix>.json`:
+
+```json
+{
+  "schema_version": 1,
+  "order_id": "<copied>",
+  "worker": "spec-evaluator",
+  "status": "done",
+  "verdict": {
+    "overall": "PASS | FAIL",
+    "report_path": ".shapeup/<slug>/evaluation/EVAL-FEATURE-<slug>.md",
+    "t0_citations": [ { "scope_id": "cart", "path": "…/t0/verdicts/r2-a3.json", "sha256": "…" } ],
+    "criteria": [ { "criterion": "UC-01 step 3", "dimension": "spec-conformance",
+                    "verdict": "FAIL", "confidence": "high", "reprobed": true,
+                    "evidence": "Pay click throws — apps/web/checkout/Pay.tsx:84" } ],
+    "refuted": [ { "task_id": "TASK-007", "ac": "<the checkbox text your evidence disproves>" } ],
+    "bugs": [ /* report-schema bug entries */ ]
+  }
+}
+```
+
+**Every FAIL criterion's `evidence` MUST carry a `file:line` locator** — schema-enforced, not
+advice: `validate-envelope` rejects the whole result before ingest sees it. A PASS may cite plain
+output. (Measured: a run returned a correct FAIL with `bugs: null` and no locator anywhere, which
+is unactionable without re-investigating. The rule used to be repeated five times in this prompt
+and enforced nowhere; it is now stated once and enforced by `domain.schema.json`.)
+
+The orchestrator's ingest appends the verdict ledger, un-ticks the `refuted` boxes, and sets
+`eval_verdict` frontmatter. You never touch a task file, a board, or run-state — and you
+NEVER set `status: done`: the judge issues verdicts; closure belongs elsewhere. That
+separation is the whole point of the architecture.
+
+---
+
+## Verification checklist
+
+- [ ] Every criterion traces to committed spec text (UC/domain-model/contract/Done-when/Non-Go)
+- [ ] Every PASS cites a confirming probe; every FAIL cites evidence or "NO EVIDENCE"
+- [ ] Every FAIL was re-probed once; confidence assigned per the ledger rule
+- [ ] Scoped spec → T0 citations present with recomputed sha256 (else the run returned `failed`)
+- [ ] Report written inside `evaluation/**` only; no other file touched
+- [ ] `refuted[]` lists exactly the boxes your evidence disproves (un-ticking is ingest's act)
+- [ ] The WorkResult validates against `work-result.schema.json`
+
+---
+
+## Dimension model — how future injection works
+
+The core loops over a **set of dimensions**; each is a self-contained file satisfying
+`references/dimension-contract.md` (id, weight, hard_threshold, applies_to, criteria[],
+bug_template). Adding one (e.g. security) = write `references/dimensions/security.md`, flip
+`enabled: true` in `references/dimensions/_registry.md` (or pass it in `dimensions[]`), re-run
+— zero core edits. Disabled security/performance stubs ship as worked examples.
+
+---
+
+## Invocation
+
+```bash
+# Orchestrated (once per round, after GATE L2) — the canonical form
+/spec-evaluator --order .shapeup/checkout-vnpay/orders/evaluate-r2.json
+
+# Standalone — the preamble shim compiles a minimal order, then the single code path runs:
+#   node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/compile-order.mjs" --operation evaluate --slug <slug> \
+#        --worker spec-evaluator [--payload '{"dimensions": [...], "run_cmd": "..."}']
+/spec-evaluator --spec shapeup/checkout-vnpay/spec/ --task TASK-007
+/spec-evaluator --spec shapeup/checkout-vnpay/spec/ --feature checkout-vnpay --single-pass
+```
+
+Standalone keeps `--task` (per-task check, not round-gated) and `--single-pass` (feature-level)
+— the shim maps them onto the order's payload; missing run command → ask. After writing the
+WorkResult, run `node "${CLAUDE_PLUGIN_ROOT}/skills/tech-lead/scripts/ingest-result.mjs" <result path>` and show its
+summary — standalone has no orchestrator to ingest for you.
+
+---
+
+## Hard Rules (never override without explicit user instruction)
+
+| Rule | Rationale |
+|------|-----------|
+| Absence of evidence = FAIL | Kills pass-by-assumption, the core QA failure mode |
+| Halo effect banned | A strong dimension never lifts a failing one |
+| Disabled dimensions are out of scope | No silent scope creep |
+| Evaluator never sets `status: done`, never edits task files/boards | Judge ≠ doer; refuted boxes return as data, ingest writes |
+| Untestable AC blocks a clean PASS | Forces the spec to be verifiable, not vibes |
+| Probe the RUNNING app, not the source alone | Apps that look right still break when used |
+| Re-probe every FAIL; flip ⇒ confidence low | A single snapshot lies; the ledger makes it visible |
+| Verdict-ledger lines are returned, appended by ingest, never rewritten | Verdict history is how a single-snapshot judge becomes measurable |
+| A verdict on a scoped spec without a T0 citation is structurally invalid | T0 is a machine fact the generator cannot fabricate (DD-7, PA4) |
+| UI assertions target affordances only (test_id/role/data-state) | Layer-3 styling is frozen; grading it resurrects the freeze through the judge |

package/skills/spec-evaluator/references/anti-leniency.md

@@ -0,0 +1,50 @@
+# Anti-Leniency Protocol
+
+Read this immediately before GATE V2. It exists because of a documented failure mode: an
+LLM asked to grade work — even another agent's — finds real defects and then talks itself
+into approving anyway, and tests superficially rather than probing edge cases. This skill
+counteracts that. The evaluator's job is to be the skeptic the generator cannot be about
+its own output.
+
+## The posture
+**Assume broken until proven working.** A criterion is FAIL by default and only earns PASS
+when Phase A produced evidence that directly confirms it. The burden of proof is on the
+build, not on the evaluator to find a reason to fail it.
+
+## Evidence rules
+1. **No evidence → FAIL.** If Phase A collected nothing for a criterion (forgot to probe,
+   app wouldn't start, probe inconclusive), it is a FAIL labeled `NO EVIDENCE` — never a
+   pass-by-assumption. Re-probe if you want a different verdict.
+2. **Every PASS cites the confirming probe.** "PASS — pnpm --filter api test → 12 passed."
+   A PASS with no citation is invalid.
+3. **Every FAIL cites concrete evidence** at a locator: file:line, endpoint, or
+   screen+element, plus the output/console/DB state observed. A FAIL with no evidence is
+   not actionable → re-probe to localize it.
+4. **Probe behavior, not code presence.** A function named `fillRectangle` existing is not
+   evidence the rectangle fill works. Exercise it. Code that looks correct but is wired
+   wrong is the exact defect class this skill targets.
+
+## Banned moves
+- **Halo effect.** A strong dimension or a mostly-working build never lifts a failing
+  criterion. Grade each criterion in isolation, then AND.
+- **Self-negotiation.** Do not reason "this is a minor issue, probably acceptable." If it
+  violates the criterion, it FAILS at the severity it warrants; acceptability is the user's
+  call at GATE V3, not the grader's at V2.
+- **Superficial probing.** Don't stop at the happy path when the criterion implies edge
+  cases (error codes, empty states, invalid input). Under-probing produces false PASSes.
+
+## Forbidden phrases in a verdict
+These are the tells of lenient self-grading. If one appears, the verdict is not done:
+- "looks good" / "looks correct" / "seems to work"
+- "probably works" / "should be fine" / "likely passes"
+- "minor issue, acceptable" / "good enough" / "close enough"
+- "I'll assume" / "presumably" / "in principle this should…"
+
+Replace each with an evidence-anchored statement or a FAIL.
+
+## Calibration toward skepticism
+If across a session the verdicts feel generous, recalibrate by re-reading the FAILs: each
+must survive the test "could the generator act on this without asking a single follow-up
+question?" If not, it is under-specified — tighten the locator and the expected-vs-actual.
+Tuning the grader to be skeptical is more tractable than making the generator self-critical;
+that is the entire reason the judge is a separate agent.

package/skills/spec-evaluator/references/dimension-contract.md

@@ -0,0 +1,126 @@
+# Dimension Contract
+
+The injection interface. Every file in `dimensions/` must satisfy this contract so the
+core can load and run it without modification. This is the *only* thing the core knows
+about a dimension. Adding a new evaluation concern = writing one file that conforms here.
+
+---
+
+## Required frontmatter
+
+```yaml
+---
+id: spec-conformance            # unique, kebab-case. Matches filename.
+title: "Spec conformance"       # human label printed in verdicts
+enabled: true                   # core loads it only if true (or named in --dimensions)
+weight: 1.0                     # informational; thresholds are hard, not weighted averages
+hard_threshold: all-pass        # see "Threshold vocabulary" below
+applies_to:                     # scoping — omit a key to mean "all"
+  lens: [lite, standard]        # which ba-pitch-analyzer lenses
+  package: any                  # apps/api | apps/web | apps/mobile | packages/shared | any
+  variant: any                  # shared | be | web | mobile | e2e | any
+requires_browser: true          # does this dimension need the running app in a browser?
+---
+```
+
+The core (GATE V0.5) validates these fields. A dimension missing `id`, `enabled`,
+`hard_threshold`, or a malformed `criteria` block is **skipped with a warning** — never
+run half-formed.
+
+---
+
+## Required body sections
+
+### `## Criteria`
+A list. Each criterion is the atomic unit the evaluator grades. Schema per criterion:
+
+```yaml
+- id: SC-1                      # unique within the dimension
+  statement: "Every AC checkbox in the task passes by command or UI probe."
+  probe: cmd | ui | data | static   # how Phase A collects evidence for it
+  evidence_required: true       # if true, NO EVIDENCE → automatic FAIL
+  pass_rule: >                  # the objective condition for PASS, evidence-based
+    All "- [ ]" items verified PASS via their probe; zero failures.
+  source: usecases | index | task | contract | scope-summary | code   # where the criterion is read from
+                                   # (usecases/domain-model.md are the grading source of truth
+                                   #  since v0.9 — task is traceability-only, never substituted)
+```
+
+`probe` values map directly to Phase A handlers:
+- `cmd`    → run a shell command, capture output + exit code
+- `ui`     → drive the running app (Playwright CLI by default)
+- `data`   → query DB / inspect storage state
+- `static` → read code/files (use sparingly; prefer probing the running app)
+
+### `## Threshold`
+State the hard threshold in prose + the vocabulary token. The core ANDs all active
+dimensions; within a dimension, the threshold decides PASS/FAIL from its criteria.
+
+### `## Bug template`
+A fenced block the report uses for every FAIL this dimension produces. Must include at
+least: `severity`, `criterion`, `location (file:line or endpoint)`, `repro`,
+`expected`, `actual`. Keeps generator handoff uniform across dimensions.
+
+---
+
+## Threshold vocabulary
+
+| Token | Meaning |
+|-------|---------|
+| `all-pass` | Every criterion must PASS. One FAIL → dimension FAILS. (Use for correctness.) |
+| `no-critical` | FAILs allowed only below `critical` severity. Any `critical` → dimension FAILS. |
+| `count<=N` | At most N FAILs of any severity tolerated. |
+| `score>=N` | For genuinely graded/subjective dimensions: a 0–100 rubric score ≥ N. |
+
+`spec-conformance` uses `all-pass` — correctness is not negotiable and leniency is the
+enemy. A subjective dimension (e.g. `visual`) would use `score>=N` with a calibrated rubric.
+
+---
+
+## What the core promises a dimension
+
+1. It reads `applies_to` and runs the dimension only on matching tasks.
+2. It calls the right Phase A handler for each criterion's `probe`.
+3. It enforces `evidence_required` (no evidence → FAIL) before applying `pass_rule`.
+4. It applies the dimension's own `hard_threshold` independently, then ANDs across dimensions.
+5. It renders FAILs using the dimension's own `## Bug template`.
+
+A dimension author therefore writes *criteria + probes + threshold + bug shape* — never
+touches the gate logic, the loop, or the report assembler.
+
+---
+
+## Minimal valid dimension (copy to start a new one)
+
+```markdown
+---
+id: my-dimension
+title: "My dimension"
+enabled: false
+weight: 1.0
+hard_threshold: all-pass
+applies_to: { variant: web }
+requires_browser: true
+---
+
+## Criteria
+- id: MD-1
+  statement: "<the thing to verify>"
+  probe: ui
+  evidence_required: true
+  pass_rule: "<objective, evidence-based PASS condition>"
+  source: task
+
+## Threshold
+all-pass — every criterion must PASS.
+
+## Bug template
+```
+severity: [critical|major|minor]
+criterion: MD-1
+location: <file:line | endpoint | screen>
+repro: <steps>
+expected: <from criterion>
+actual: <observed evidence>
+```
+```

package/skills/spec-evaluator/references/dimensions/_registry.md

@@ -0,0 +1,40 @@
+# Dimension Registry
+
+The core reads this at GATE V0.5 to decide which dimensions to load. `--dimensions` on the
+command line overrides this table for a single run.
+
+## Active set
+
+| id | enabled | file | applies_to (summary) | notes |
+|----|---------|------|----------------------|-------|
+| `spec-conformance` | ✅ true | `dimensions/spec-conformance.md` | all tasks | baseline correctness: AC + Done-when + contract shapes + non-go |
+| `tdd-surface` | ✅ true | `dimensions/tdd-surface.md` | all tasks | suite green + companion test files for new code; TDD-1/2 critical, TDD-3 advisory |
+| `integration` | ✅ true | `dimensions/integration.md` | `.be` and `.e2e` variants only | full-stack integration test + auth boundary + RLS-JWT pattern; runs only when variant matches |
+| `completeness` | ⚙️ auto | `dimensions/completeness.md` | all tasks (lens lite/standard) | auto-ON when spec has UC `## Invariants` (v2.8+); no-op on older specs |
+| `test-surface-conformance` | ⚙️ auto | `dimensions/test-surface-conformance.md` | all tasks (lens lite/standard) | auto-ON when spec has UC `## Test Surface` (v2.9+ or `--surface-only` retrofit); no-op otherwise. Report must list every TS row probed — qa-edge-hunter's negative-space input |
+| `security` | ⛔ false | `dimensions/security.md` | be / shared | stub — flip on when ready |
+| `performance` | ⛔ false | `dimensions/performance.md` | be / web | stub — flip on when ready |
+
+Default active set = every row with `enabled: true` whose `applies_to` matches the task variant, **plus** each ⚙️ auto dimension when its auto-enable condition holds:
+- All tasks: **`[spec-conformance, tdd-surface]`** (baseline)
+- `.be` / `.e2e` tasks additionally pick up: **`[integration]`**
+- When any UC declares `## Invariants`: also **`[completeness]`**
+- When any UC carries `## Test Surface`: also **`[test-surface-conformance]`**
+
+## Load order
+Dimensions run in table order. `spec-conformance` first so a correctness failure is the
+headline; cosmetic / non-functional dimensions report after it. Order does not change the
+overall verdict (all dimensions are ANDed) — it only affects reading order in the report.
+
+## How to add a dimension (the injection recipe)
+1. Copy the "Minimal valid dimension" block from `dimension-contract.md` to
+   `dimensions/<id>.md` and fill in criteria, probes, threshold, bug template.
+2. Add a row here and set `enabled: true` — or leave it false and turn it on per-run with
+   `--dimensions spec-conformance,<id>`.
+3. Run. No core file changes. GATE V0.5 validates it against the contract; if it doesn't
+   conform, it is skipped with a warning rather than silently mis-run.
+
+## Scoping reminder
+A dimension only grades tasks its `applies_to` matches. Example: `security` with
+`applies_to: { variant: [be, shared] }` is ignored when evaluating `TASK-007.web`, even if
+it is enabled. This keeps the active set honest per task.

package/skills/spec-evaluator/references/dimensions/completeness.md

@@ -0,0 +1,102 @@
+---
+id: completeness
+title: "Spec completeness"
+enabled: false
+weight: 1.0
+hard_threshold: no-critical
+applies_to:
+  lens: [lite, standard]
+  package: any
+  variant: any
+requires_browser: false
+---
+
+# Completeness Dimension
+
+> **Auto-enable rule (read at GATE V0.5):** This dimension stays `enabled: false` by
+> default so it is a no-op on every spec generated before `ba-pitch-analyzer` v2.8.
+> The core flips it ON for a run **only when** the spec's `usecases/` contains at least
+> one `## Invariants` section. A pre-v2.8 spec has no invariants → this dimension never
+> activates → existing audit scores and verdicts are unchanged (non-regression guarantee).
+> An explicit `--dimensions ...,completeness` always wins over the auto rule.
+
+**Why this exists.** `spec-conformance` answers *"does each task pass its acceptance
+criteria?"* — it grades the tasks that exist. It cannot see the task that was never
+written. Shape Up's discovered-task philosophy says the real bulk of work is found while
+building; an invariant declared on a UC but never backed by a regression task is exactly
+that blind spot. This dimension converts "absence of a task" from invisible into a graded
+**GAP** — the judge surfaces it; the planner (a ba-pitch-analyzer reconcile/generate-board order) fills it.
+Judge never fills it itself.
+
+**Distinction held:** conformance = *the tasks that exist are correct*; completeness =
+*the tasks that must exist, do exist*. A spec can be 100% conformant and still incomplete.
+Definition of Done for the harness = conformance PASS **and** completeness no-critical.
+
+---
+
+## Criteria
+
+- id: CMP-1
+  statement: "Every `[INV-NN]` declared in any usecases/UC-*.md ## Invariants section is backed by ≥1 task whose use_case_refs points to that UC AND whose AC references the invariant (by INV id or its assertion)."
+  probe: static
+  evidence_required: true
+  pass_rule: >
+    For each INV-NN found across usecases/: a task backs it only if BOTH hold — (a) the
+    task lists that UC in use_case_refs, and (b) the task names the invariant unambiguously,
+    i.e. cites the INV id (e.g. "INV-02") or links the UC's #Invariants anchor AND its AC
+    restates that specific assertion as a command/observable check. Matching a loose keyword
+    shared by several invariants is NOT sufficient — require the INV id or a 1:1 assertion
+    match to avoid crediting one task for an invariant it does not actually test. Zero
+    uncovered invariants → PASS. Any invariant with no backing task → FAIL at `critical`
+    severity (a missing must-have regression, not a cosmetic gap).
+  source: code
+
+- id: CMP-2
+  statement: "Every use case has ≥1 task referencing it via use_case_refs (no stranded UC with zero tasks pointing at it)."
+  probe: static
+  evidence_required: true
+  pass_rule: >
+    For each usecases/UC-*.md: ≥1 task file lists it in use_case_refs. A UC with zero
+    referencing tasks → FAIL at `critical` (this is the Coverage 🔴 condition, re-checked
+    from the evaluator side as a completeness gap rather than trusting synthesis alone).
+  source: code
+
+- id: CMP-3
+  statement: "Discovery ledger surprise rate is within a healthy band (shaping-quality signal)."
+  probe: static
+  evidence_required: false
+  pass_rule: >
+    If a discovery ledger exists (discovery/ledger.md or a *discovered-tasks*.md whose
+    feature matches run-state.feature), count discovered items marked `[+]` vs imagined
+    `[ ]`. surprise_ratio = discovered / (imagined + discovered). PASS (no defect) when
+    ratio ≤ 0.5. When ratio > 0.5 → emit a `minor` finding "high surprise rate —
+    shaping may have under-specified scope; flag to PO", NOT a critical. evidence_required
+    is false, so a spec with no ledger simply yields no CMP-3 finding (not a FAIL).
+  source: code
+
+---
+
+## Threshold
+
+`no-critical` — `critical` FAILs (CMP-1 uncovered invariant, CMP-2 stranded UC) block the
+dimension. `minor` findings (CMP-3 high surprise) are reported but do not fail the
+dimension. This keeps the gate honest about *missing must-have work* while treating the
+surprise signal as advisory feedback to the PO, not a build blocker.
+
+Rationale for `no-critical` rather than `all-pass`: CMP-3 is a deliberately non-blocking
+signal. Using `all-pass` would let an advisory surprise-rate note sink the verdict, which
+would punish healthy discovery — the opposite of the intent.
+
+---
+
+## Bug template
+
+```
+severity: [critical|minor]
+criterion: [CMP-1|CMP-2|CMP-3]
+location: [usecases/UC-Name.md#Invariants INV-NN | usecases/UC-Name.md | discovery/ledger.md]
+repro: <how the gap was detected — e.g. "grep use_case_refs across tasks/ found 0 backing TASK for INV-02">
+expected: <e.g. "≥1 regression task referencing UC-ConnectObjects covering INV-02">
+actual: <e.g. "no task references INV-02; invariant declared but unverified">
+next: <handoff — e.g. "a ba-pitch-analyzer reconcile order over <ledger> to generate the missing regression task">
+```

package/skills/spec-evaluator/references/dimensions/integration.md

@@ -0,0 +1,129 @@
+---
+id: integration
+title: "System integration"
+enabled: true
+weight: 1.0
+hard_threshold: no-critical
+applies_to:
+  lens: [lite, standard]
+  package: any
+  variant: [be, e2e]
+requires_browser: false
+---
+
+# System Integration Dimension
+
+**Why this exists.** Unit tests verify isolated logic. `spec-conformance` drives the
+running app and confirms AC at the surface. Neither catches the class of bug where each
+layer works in isolation but the *seam* is broken: wrong auth header forwarded, RLS policy
+not invoked because the transaction is missing, Prisma schema migration not applied in the
+test database, or a contract mismatch between web's api-client and the API controller.
+This dimension fills that seam.
+
+**What "system integration" means here** (scoped to the IVS ADR-005 stack):
+the full request path — web api-client → API controller → service → repository (Prisma +
+RLS) → database — exercised by tests that use a real database, not mocked layers.
+An integration test that mocks `PrismaClient` or replaces the repository with an in-memory
+stub is a unit test wearing integration clothing and does not satisfy this dimension.
+
+**Project context (read AGENTS.md Section 2 rules 3 & 4 before probing):**
+- All user-scoped queries must run inside a transaction that injects the JWT claims so RLS
+  fires (`SET LOCAL request.jwt.claims`). An integration test that queries without this
+  transaction bypasses RLS and produces a false-positive PASS.
+- The connection string must point to the **transaction-mode pooler** (port 6543), not the
+  session-mode direct connection (port 5432). Session mode allows `SET LOCAL` to leak.
+- Never trust a `userId` sent in the request body — derive it from the verified JWT.
+
+**`.e2e` variant:** the e2e suite inherently exercises the full stack. INT-1 (integration
+test existence) maps to the e2e test file(s) added by the task. INT-2 and INT-3 apply if
+the e2e task introduces any auth-scoped endpoint.
+
+---
+
+## Criteria
+
+```yaml
+- id: INT-1
+  statement: "At least one integration/e2e test exercises the feature's main flow end-to-end with a real database — no mocked repository or Prisma layer."
+  probe: static + cmd
+  evidence_required: true
+  pass_rule: >
+    Locate integration test files for this task's package (typically `*.integration.spec.ts`,
+    `*.e2e-spec.ts`, or a `test/` directory at the package root). Grep for test cases that
+    touch the happy-path scenario the task implements. Verify the test does NOT mock the
+    data layer: `grep -rE 'jest\.mock|vi\.mock' <integration-test-file>` should return no
+    hits on repository, Prisma, or database imports. Then run `pnpm --filter <pkg> test:e2e`
+    (or the integration-specific script) and capture exit code + test count.
+    Zero integration test files → FAIL at `critical`. Any integration test that mocks the
+    DB layer → FAIL at `critical` (evidence: the jest.mock / vi.mock import path).
+  source: code
+
+- id: INT-2
+  statement: "An unauthorized-access scenario is covered: a request from a different user/org is rejected with the correct error (403 or empty set), exercised by an integration test."
+  probe: cmd + data
+  evidence_required: true
+  pass_rule: >
+    Find the integration test that exercises the unauthorized path: a user from a different
+    org (or an unauthenticated caller) attempts to read/write the resource this task
+    introduces. The test asserts the correct rejection: 403 Forbidden, an empty result set,
+    or an RLS-filtered response. If the resource is read-only to viewers (role_tag check),
+    also verify a `viewer` role cannot mutate it. Evidence = the test case text + the run
+    output showing it passes. No such test → FAIL at `critical` — this is the gap that lets
+    real authorization bugs slip through conformance testing.
+  source: code
+
+- id: INT-3
+  statement: "Integration tests inject JWT claims via the RLS transaction pattern (set_config inside a transaction) and target the transaction-mode pooler (port 6543)."
+  probe: static
+  evidence_required: true
+  pass_rule: >
+    Inspect the integration test setup (beforeAll / test helper): it must execute
+    `SET LOCAL request.jwt.claims` (or the equivalent `$executeRaw\`select set_config(...)\``)
+    inside a Prisma `$transaction` block before any user-scoped query. Also check the
+    DATABASE_URL used in the test config: it must include port 6543 (transaction pooler),
+    not 5432 (session mode). A test that queries without the JWT injection calls the DB as
+    the app role without RLS and will PASS rows it should not see → FAIL at `major`.
+    A test using port 5432 risks SET LOCAL leaking across pooled connections → FAIL at `major`.
+    If both problems are present, report as two separate findings.
+  source: code
+```
+
+---
+
+## Threshold
+
+`no-critical` — INT-1 and INT-2 are `critical`: a task that cannot prove end-to-end
+correctness and cannot prove its authorization boundary is not integration-tested and is
+not done. INT-3 (JWT transaction pattern + pooler port) is `major`: incorrect plumbing in
+the test setup produces false-positive passes (the test appears to work without RLS firing),
+which is a confidence-undermining failure — but it is one step removed from observable
+breakage and is reported as major rather than critical.
+
+---
+
+## Bug template
+
+```
+severity: [critical|major]
+criterion: [INT-1|INT-2|INT-3]
+location: <test file path | package/test/setup.ts | .env.test | file:line>
+repro: <e.g. "grep -rE 'jest.mock.*repository' test/" | "grep DATABASE_URL .env.test" | "pnpm --filter api test:e2e">
+expected: <e.g. "integration test exists targeting BoardRepository with real DB" | "DATABASE_URL ends with :6543/...">
+actual: <e.g. "no *.integration.spec.ts in apps/api/src/boards/" | "DATABASE_URL uses port 5432" | "vi.mock('../../infrastructure/BoardRepository')">
+fix_hint: <e.g. "add boards/boards.integration.spec.ts; use PrismaService with a real test DB; wrap queries in $transaction with set_config">
+```
+
+## Probing notes
+
+- Integration test scripts are often separate from unit tests: `test:e2e` / `test:int` /
+  `test:integration` in package.json. Check all scripts, not just `test`.
+- For mocking detection: `grep -rn "jest\.mock\|vi\.mock" <test-file>` and inspect what
+  is being mocked. Mocking HTTP clients, external APIs, or filesystem utilities is fine;
+  mocking the Prisma client or the repository class defeats integration testing.
+- For port check: the integration DB URL is typically in `.env.test` or `jest.config.ts`
+  (globalSetup). Check `DATABASE_URL` and `DIRECT_URL` separately — some setups split them.
+- For `.e2e` tasks: `supertest` or `axios` against the running NestJS server counts as
+  an integration test if it hits a real DB. Verify the test server's DB config, not just
+  the production server's config.
+- Do not skip INT-3 because the test "seems to work." RLS without `set_config` silently
+  returns all rows as the app role — the test passes but the auth boundary is untested.