pi-dev-team 0.2.0

package/knowledge/test-automation-maturity.md

@@ -0,0 +1,56 @@
+# Test Automation Maturity
+
+Reference file for `test-review` and the `test-health` skill (project-wide audit). It answers one question: **how maintainable is this suite as it grows, and where does it sit on the maturity curve?**
+
+**Scope boundary — three files share "test automation"; keep them disjoint:**
+
+- `test-strategy.md` owns the automation **strategy** axis (Scripted / Data-Driven / Recorded, fixture lifecycle, SUT interaction). Do not restate it here — cross-link.
+- `test-refactoring.md` owns the **goals & principles** (self-checking, isolated, one-condition-per-test, design-for-testability). Do not restate them here — cross-link.
+- **This file** owns only the **maturity diagnostic** and the **abstraction patterns** below.
+
+Source: Meszaros *xUnit Test Patterns*; Crispin & Gregory; Page Object (Fowler/Selenium); Screenplay (Marcano). Stack-agnostic.
+
+---
+
+## Maturity scale (diagnose from evidence, lowest rung first)
+
+| Rung | Signal in the suite | Risk |
+|------|---------------------|------|
+| **0 — Manual / recorded** | record-and-replay scripts, no hand-written assertions | brittle; re-record on every UI change |
+| **1 — Scripted, raw** | hand-written tests, but selectors/URLs/payloads inline in every test | one change touches many files |
+| **2 — Abstracted** | a layer between tests and the SUT (Page Objects / API client / DSL); tests read in domain terms | maintainable; the target for most suites |
+| **3 — Screenplay / actor** | behaviour expressed as actor tasks/questions, reusable across suites | worth it only for large E2E suites; over-engineering below ~50 E2E tests |
+
+Report the rung from observed evidence (grep for duplicated selectors/literals, presence of a `pages/`/`support/` layer), then recommend the **next** rung — never skip ahead.
+
+---
+
+## Single-Point-of-Change test
+
+Pick one volatile detail (a CSS selector, an endpoint path, a field name). Count how many test files would change if it changed once.
+
+- **N files → broken by one rename** is the headline maturity metric. N=1 is healthy; N≫1 is rung 1.
+- Fix: extract that detail behind one abstraction (Page Object method, API client, builder) so the rename lands in **one** place.
+
+## Abstraction patterns
+
+- **Page Object / API client / DSL** — a façade exposing domain actions (`loginAs(user)`, `placeOrder(...)`), hiding selectors/HTTP. The standard rung-2 move.
+- **Screenplay** — actors perform tasks and ask questions; tasks compose. Use for large, cross-cutting E2E suites only.
+
+## Smell: UI-based setup
+
+Driving the **UI** to establish preconditions (clicking through signup just to test checkout) is slow and flaky. Set state through the back door — API, fixture, or seeded DB — and reserve UI steps for the behaviour under test. (Back-door *setup* is legitimate; see `test-strategy.md` → SUT Interaction. It is not the banned "test logic in production" smell.)
+
+## Graduated disclosure (don't over-build)
+
+Scale abstraction to suite size, not aspiration:
+
+| Test count (a given type) | Recommend |
+|---------------------------|-----------|
+| < ~10 | inline is fine; abstracting early is premature |
+| ~10–50 | extract Page Objects / a client / builders (rung 2) |
+| > ~50 E2E | consider Screenplay (rung 3) |
+
+## Boundaries
+
+Maturity is about **maintainability under change**, not coverage or correctness — those are `farley-score` and `test-review`. A small suite at rung 1 is fine; flag low maturity only when suite size makes the single-point-of-change cost real.

package/knowledge/test-automation-principles.md

@@ -0,0 +1,71 @@
+# Test Automation Goals & Principles
+
+Reference file for the `test-design-advisor`, `test-health`, and `cd-test-architecture` skills and the `test-smell-review` agent. This is the **value system** behind every other test knowledge file: the goals a test suite exists to serve and the principles that, when followed, keep it serving them. Use it as the **rubric** when *evaluating* a suite ("is this test good, and why/why not?") and as the **design constraints** when *recommending* a target architecture.
+
+Source: Gerard Meszaros, *xUnit Test Patterns* (xunitpatterns.com) — Ch. 3 *Goals of Test Automation* and Ch. 5 *Principles of Test Automation*. Language- and framework-agnostic.
+
+Core idea: every smell in `test-smells.md` is a **violated principle**, and every pattern in the other files is a **way to honor one**. A finding is more credible when it names the goal it protects and the principle it restores — not just "this is a smell."
+
+---
+
+## Goals — what the suite is *for*
+
+A test is worth its maintenance cost only if it advances these. Grade each goal **served / weak / absent**.
+
+| Goal | What it means | Lost when… |
+|------|---------------|------------|
+| **Tests as Specification** | The test states intended behavior before/while the code is written | Tests are written after the fact to match what the code already does |
+| **Tests as Documentation** | A reader learns the SUT's behavior from the test alone | Obscure Test — intent buried in setup, helpers, magic values |
+| **Defect Localization** | A failure points at *which* behavior broke, narrowly | Eager Test / shared fixtures — many tests fail together, none pinpoints |
+| **Bug Repellent** | Tests catch regressions, so defects don't recur | Coverage gaps; tests that can't fail (see Buggy Tests) |
+| **Fully Automated Test** | Runs with no human steps | Manual Intervention — someone seeds data or flips config |
+| **Self-Checking Test** | The test decides pass/fail itself, no eyeballing output | Asserts missing; result printed but not verified |
+| **Repeatable Test** | Same result every run, any order, anywhere | Erratic Test — clock/RNG/order/shared-resource dependence |
+| **Robust Test** | Survives changes unrelated to the behavior it checks | Fragile Test — bound to internals, signatures, or call sequences |
+| **Simple / Expressive Test** | Minimal, readable, one concern | Conditional logic, duplication, over-mocked interaction checks |
+
+The first three are the *return* on the suite; the rest are the *conditions* that keep that return from leaking away. A suite that is green but **not Repeatable or not Robust** is a liability, not an asset — flag it as such.
+
+---
+
+## The Principles — how to honor the goals
+
+The named principles, each with the test it should pass and the smell it prevents. When evaluating a test, walk this list; when designing one, treat it as a checklist.
+
+| Principle | The rule | Honoring move | Violation smell |
+|-----------|----------|---------------|-----------------|
+| **Write the Test First** | Let the test drive the design; testability falls out for free | TDD red-green-refactor (`test-driven-development` skill) | Hard-to-Test Code retrofitted later |
+| **Design for Testability** | The code must be reachable through a clean seam | Seams from `testability-patterns.md` | Hard-to-Test Code |
+| **Use the Front Door First** | Exercise via the public API; verify via observable state | Round-trip test + State Verification | Overcoupled/Overspecified Software (Back Door overuse) |
+| **Communicate Intent** | The test reads as a statement of behavior (≤ ~10 lines of logic) | Intent-revealing names, Test Utility Methods | Obscure Test |
+| **Don't Modify the SUT** | Test the code you'll actually ship, in a representative config | Double *collaborators*, never the SUT itself | testing a stand-in instead of the SUT |
+| **Keep Tests Independent** | Each test runs alone and in any order | Fresh Fixture per test; no shared writable state | Interacting Tests / Erratic Test |
+| **Isolate the SUT** | Control every input by replacing what you don't test | Injection / lookup + the right double | Context Sensitivity (Fragile Test) |
+| **Minimize Test Overlap** | Each condition covered by exactly one test | Test concerns separately; prune redundant tests | High Test Maintenance Cost |
+| **Minimize Untestable Code** | Move logic out of hard-to-instantiate shells | Humble Object (`testability-patterns.md`) | Untested Code → Production Bugs |
+| **Keep Test Logic Out of Production** | No `if (testing)` paths in shipped code | Seams, not back doors | Test Logic in Production |
+| **Verify One Condition per Test** | One exercise, one logical assertion's worth of verification | Split tests; Custom Assertion to keep one call | Eager Test / Assertion Roulette |
+| **Test Concerns Separately** | One behavior per test method/class | One Testcase Class per concern | Eager Test; poor Defect Localization |
+| **Ensure Commensurate Effort** | Test effort/tooling ≤ feature effort/tooling | Data-Driven Test for config-shaped behavior | tests harder to write than the code |
+
+---
+
+## Using this as a rubric (evaluation)
+
+For a given test or suite:
+
+1. **Name the goals at risk.** Is it Repeatable? Robust? Does a failure localize? If any are *absent*, that is the headline finding — the suite is not yet trustworthy.
+2. **Trace each finding to a principle.** "This is fragile" → *Isolate the SUT* / *Use the Front Door First* violated. The principle names the fix's direction.
+3. **Reach for the pattern that restores it.** The remedy lives in `fixture-construction.md`, `result-verification.md`, `test-organization.md`, `test-doubles.md`, `value-patterns.md`, or `testability-patterns.md`.
+
+A useful order of severity when grading: a suite that **can't fail** or **can't be trusted** (Buggy/Erratic) outranks one that is merely **hard to read or maintain** (Obscure/Fragile), which outranks **stylistic** issues.
+
+---
+
+## How this connects to the rest of the toolkit
+
+- **`test-smells.md`** — each smell is the negative image of a principle here; that file is the detection layer, this is the *why*.
+- **`test-refactoring.md`** — the behavior-preserving moves from a violated principle to an honored one.
+- **`fixture-construction.md` / `result-verification.md` / `test-organization.md` / `value-patterns.md`** — the patterns that honor specific principles.
+- **`testability-patterns.md`** — *Design for Testability*, *Minimize Untestable Code*, *Isolate the SUT* as production-code seams.
+- **`cd-test-architecture.md` / `test-pyramid.md`** — *Keep Tests Independent* and *Repeatable Test* scaled up to pipeline determinism.

package/knowledge/test-cadence-tradeoffs.md

@@ -0,0 +1,68 @@
+# Test-Cadence Tradeoffs: Evidence Bar for Displacing the Standing Cadence
+
+`/build` runs exactly one cadence today — Code-First Small Batches — per
+[ADR 0017](../../../docs/adr/0017-single-build-cadence-remove-classic-tdd-opt-in.md),
+which removed the `--tdd` opt-in. Its stated reopening condition is scoped
+to Classic TDD specifically ("a future, larger-corpus run of Experiment 5
+... shows Classic TDD closing the cost gap"), but the principle behind it —
+"that is a new decision to make with new evidence" — applies by the same
+logic to any other cadence claim. This file is the operational note for
+applying that bar to one such claim — batch-red-per-class (issue #1702) —
+and for any future claim of the same shape.
+
+## The standing default: Code-First Small Batches
+
+**Code-First Small Batches** (IMPLEMENT → TEST → REFACTOR per behavior,
+refactor on every green) is `/build`'s sole cadence (`plugins/dev-team/CLAUDE.md`
+Core Principle 6, decided in ADR 0017). The decision rests on Experiment 5
+(`docs/experiments/05-final-results.md`, n=24 cells/arm): $0.99/cell at
+quality 0.961, versus Classic TDD's $1.59/cell at quality 0.966 —
+statistically indistinguishable on maintainability, though Classic TDD
+posts slightly lower mutation coverage — and Classic TDD is explicitly
+endorsed by `docs/experiments/RECOMMENDATIONS.md` as a "sound second
+choice." The big-batch and split-authorship shapes
+tested in the same experiment line cost 2-4.5x more with worse
+changeability (`docs/experiments/RECOMMENDATIONS.md` § 3) — a materially
+larger gap than Classic TDD's ~60%, and not the comparison a new cadence
+claim needs to clear.
+
+## The external claim: batch-red-per-class
+
+claude-flow's batch-vs-strict-TDD experiment (see the competitive analysis
+in issue #1702) reports that batch-red-per-class testing (the source calls
+it "batch-per-class") — write all tests for a unit, verify they fail as a
+batch, then implement — is 46% cheaper and 61% more efficient (by test-run
+count) than *plain strict row-by-row TDD*, with equivalent quality
+(mutation-testing parity). That baseline is not this repo's Classic TDD
+arm (`tdd-refactor`, which mandates a refactor after every green), and the
+corpus and harness are claude-flow's, not this repo's.
+Nothing in this repo's own experiment line measured
+batch-red-per-class, so the claim doesn't yet compare to either of our two
+validated cadences.
+
+## The decision rule
+
+**Do not adopt batch-red-per-class, and do not remove or deprecate
+`skills/test-driven-development/SKILL.md`, on the strength of the external
+claim alone.** This mirrors a working rule the repo already holds itself to
+(root `CLAUDE.md` → "Deterministic tools over inference" /
+"Verify a runtime property by exercising it at runtime"): an unreplicated
+external report is a hypothesis, not a result, for a question this repo's
+own experiment harness can answer directly.
+
+| Question | Answer |
+| --- | --- |
+| Does an unreplicated external cadence claim, by itself, meet ADR 0017's reopening bar? | **No** — that bar is "a new decision to make with new evidence" from this repo's own harness, not a citation of someone else's. |
+| Should `/build` switch its default cadence to batch-red-per-class? | **Not yet** — no local evidence exists. See `docs/experiments/test-cadence-validation-plan.md` for what a local run would require (monorepo-only — that path is dev-repo process and isn't present in an installed plugin). |
+| Should strict TDD (`skills/test-driven-development`) be removed? | **Not yet** — same reason; it remains available advisory guidance per ADR 0017. |
+| Should a `batch-red-verified` gate be added to `/build`? | **Not yet** — contingent on a local result, including "don't adopt it" as a valid outcome. Tracked as issue #1727. |
+
+## Local measurement, once you have a result
+
+Use `.claude/metrics/` (see `skills/performance-metrics/SKILL.md`) to keep
+measuring cost/efficiency on real `/build` runs after adopting any cadence
+change, the same way the plugin already tracks cost regressions elsewhere
+(`skills/cost-report/SKILL.md`). A cadence change that looked good in a
+local experiment must keep looking good in production use, or it gets
+revisited — and, per ADR 0017, any change to the standing cadence is itself
+a new ADR, not a knowledge-file edit.

package/knowledge/test-doubles.md

@@ -0,0 +1,105 @@
+# Test Doubles
+
+Reference file for `test-smell-review`, `test-review`, and the `test-design-advisor` skill. A test double is any object that stands in for a real collaborator in a test. Choosing the wrong kind of double is the most common cause of fragile, over-specified tests.
+
+Source taxonomy: Gerard Meszaros, *xUnit Test Patterns* (xunitpatterns.com). Language-agnostic — described by role, not by any mocking library's API.
+
+Core principle: **prefer the simplest double that lets the test verify the behavior.** Reach for a Mock only when the interaction *is* the behavior under test. Over-mocking couples tests to implementation and produces the Overspecified Software smell (see `test-smells.md`).
+
+---
+
+## The Five Doubles
+
+| Double | What it does | Verification style | Use when |
+|--------|-------------|-------------------|----------|
+| **Dummy** | Passed but never used; fills a required parameter | none | A constructor/method signature demands an argument the test path never touches |
+| **Stub** | Returns canned answers to calls made during the test | **state** (assert on the SUT's output/state) | The SUT *reads* from a collaborator and you need to control what it reads (config, query result, clock) |
+| **Spy** | A stub that also records how it was called, for later inspection | **behavior** (assert after the act) | You need to confirm an outgoing call happened, but want to assert *after* the action, not pre-program expectations |
+| **Mock** | Pre-programmed with expectations; fails the test if calls don't match | **behavior** (expectations verified) | The interaction itself is the observable behavior (e.g., "an email *is sent*") and there's no state to assert on |
+| **Fake** | A working but lightweight implementation (in-memory DB, in-memory queue) | **state** | A stub/mock would need so much call-by-call setup that a real-ish implementation is simpler and more faithful |
+
+A **Test Spy** and a **Mock** both do behavior verification; the difference is *when* and *how* you specify expectations. Spy = act, then assert on recorded calls. Mock = set expectations up front, library verifies them. Spies usually read better and fail less cryptically.
+
+---
+
+## State Verification vs. Behavior Verification
+
+| | State verification | Behavior verification |
+|---|---|---|
+| **Asserts on** | The SUT's return value or resulting state | The calls the SUT made to collaborators |
+| **Doubles used** | Stub, Fake | Mock, Spy |
+| **Coupling** | Low — survives refactors that preserve outcome | High — breaks when the *how* changes, even if outcome is identical |
+| **Default?** | **Yes — prefer this** | Only when there's no observable state to assert |
+
+Rule of thumb: if you *can* assert on a result or a state change, do that and use a Stub/Fake. Use a Mock/Spy only for genuine side-effect-only boundaries — sending a message, writing to a log/queue, calling a payment gateway — where the call is the whole point.
+
+---
+
+## Choosing a Double (decision flow)
+
+```
+Does the SUT READ from the collaborator (needs a controlled answer)?
+├─ YES → can you then assert on the SUT's output or state?
+│        ├─ YES → Stub (state verification). Done.
+│        └─ NO, the read drives a side effect you must confirm → Spy
+│
+Does the SUT only WRITE to the collaborator (side-effect-only boundary)?
+├─ YES → is the call itself the behavior under test (email sent, event published)?
+│        ├─ YES → Mock or Spy (behavior verification)
+│        └─ NO, it's incidental → Dummy or a no-op Stub; don't assert on it
+│
+Is per-call stub/mock setup so heavy it obscures the test?
+└─ YES → Fake (in-memory implementation), assert on its resulting state
+```
+
+---
+
+## How the double is built: Configurable vs. Hard-Coded
+
+The five doubles above are *roles*; this is the orthogonal question of *how the double gets its canned answers*. Both forms are legitimate — the choice is about reuse vs. transparency.
+
+| Form | What it is | Use when | Watch for |
+|------|-----------|----------|-----------|
+| **Configurable Test Double** | A reusable double the test programs at run time with the values to return / calls to expect (what Mockito/JMock/NSubstitute/Sinon generate) | The same collaborator is doubled across many tests with different canned values; you want one mechanism, configured per test | Configuration becomes so verbose it obscures the test — a sign to extract a builder or downgrade a Mock to a Stub |
+| **Hard-Coded Test Double** | A purpose-built class (or *Inner Test Double* — a nested/anonymous class, or a Test-Specific Subclass) with the canned values **baked into its code** | The behavior is specific to one or a few tests; a framework can't express it; or you want a named, self-documenting stand-in | Proliferation of one-off double classes; if it recurs with varying data, switch to a Configurable form |
+
+Neither is "better." Reach for a **Configurable** double when one collaborator is stubbed many ways across the suite; reach for a **Hard-Coded** double when a specific, named behavior reads more clearly than per-test configuration (or the framework can't produce it).
+
+---
+
+## Test-Specific Subclass
+
+When you can't substitute a collaborator from outside — the dependency is created internally, or the thing you need to control is a *method of the SUT itself* — subclass the real class **in the test** and override just the seam method to inject an indirect input or null out an unwanted side effect.
+
+```
+// Override only the narrow seam; inherit everything else real
+class TestableOrderService extends OrderService:
+  override fetchRate(): return 1.25   // control one indirect input
+```
+
+Use it to: gain control of an indirect input without a full injection refactor; expose a protected hook for the test; or stand up a Hard-Coded double by subclassing a real collaborator. **Constraints** (from *Don't Modify the SUT*, `test-automation-principles.md`): override **only** what the test must control — never a method whose behavior this test is verifying, or you're testing the override, not the code. Treat it as a bridge toward a real seam (`testability-patterns.md`), not a permanent design.
+
+---
+
+## Common Misuses (flag these)
+
+| Misuse | Why it's wrong | Fix |
+|--------|---------------|-----|
+| Mocking a value object or pure function | Nothing to verify; adds coupling for zero benefit | Use the real object |
+| Mocking the type under test | You're testing the mock, not the code | Use the real SUT; double only its *collaborators* |
+| Asserting exact call order/count when order doesn't matter | Overspecified Software smell; fragile | State verification, or assert only the call that matters |
+| Stub returns drive an `if` that's never asserted | Dead control; the test proves nothing about that branch | Assert the branch's effect, or remove the setup |
+| A Mock where a Stub would do (no side-effect being verified) | Tests implementation detail instead of outcome | Downgrade to Stub + state assertion |
+| Mocking a concrete class instead of an interface/port | Couples to the implementation type; see `testability-patterns.md` | Extract an interface/port; double that |
+| Mocking a collaborator internal to / owned by the same component or bounded context as the SUT | This is a boundary-selection error, distinct from the injection-mechanism problem above — the collaborator isn't a true external-boundary system (third-party API, another team's service, infra the team doesn't control), it's part of what should be under test. Applies at or below the component layer regardless of whether the test declares itself solitary or sociable — no unit-test exemption exists | Assemble the real component and exercise the collaborator as real (sociable unit/component test); double only when the collaborator matches a named blocker — see `internal-collaborator-doubling.md` for the full rule and `component-test-patterns.md`'s core principle |
+
+---
+
+## Relationship to testability
+
+If a collaborator can't be substituted at all (new-ed up internally, static singleton, hidden global), that's a **production-code** problem, not a test problem — introduce a seam per `testability-patterns.md` (constructor injection, interface extraction) before choosing a double. The double taxonomy assumes the collaborator is already injectable.
+
+## Boundaries
+
+- Don't flag a Fake (in-memory repo/DB) as "not testing the real thing" — at unit/integration level a Fake is a legitimate, often *better*, choice than a heavily-stubbed real dependency. Real-dependency fidelity is the job of contract and integration tests (see `test-pyramid.md`, `microservice-testing.md`).
+- A single Mock at a true side-effect boundary is correct, not a smell. The smell is *pervasive* mocking that pins down internal call sequences.

package/knowledge/test-file-indicators.md

@@ -0,0 +1,22 @@
+# Test File Indicators
+
+Canonical list of how to recognize a test file by language. Agents and skills
+that need to decide "is this a test?" (for skip logic or for scoping a Farley
+Score) cite this file rather than restating the list, so a newly-supported
+framework annotation is added in one place.
+
+A `.feature` file always counts as a test file — never skip a target that
+contains feature files.
+
+## Indicators by language
+
+| Language | A file is a test when it… |
+| --- | --- |
+| **JS/TS** | matches `*.test.*`, `*.spec.*`, or lives inside `__tests__/` |
+| **C#** | is a `.cs` file containing `[Fact]`, `[Theory]`, `[Test]`, `[TestCase]`, `[TestMethod]`, or `[TestClass]` |
+| **Java** | is a `.java` file containing `@Test`, `@ParameterizedTest`, `@TestFactory`, or a class name ending in `Test`, `Tests`, `TestCase`, or `Spec` |
+| **Python** | matches `test_*.py` or `*_test.py` (pytest/unittest convention) |
+| **BDD/Gherkin** | is a `.feature` file, or a step-definition file (`*.steps.*`, `*StepDefinitions.*`, `*Steps.*`) |
+
+When a target contains none of the above, an agent scoped to test files
+returns its `skip` status.

package/knowledge/test-layer-gates.md

@@ -0,0 +1,35 @@
+# Test Layer Gates
+
+Reference file for the `test-design-advisor` skill. The pyramid heuristic (`test-pyramid.md`) picks the *lowest* layer that verifies a behavior. These **pre-gates** run first and **escalate upward only** — never lower the pick — when a behavior can break in a way the lowest layer misses (Swiss-cheese model).
+
+Layers match `test-pyramid.md` (unit / integration / component / contract / E2E) and map to `cd-test-architecture.md`'s six test types — see its *Terminology Reconciliation* section.
+
+**business-critical** = labelled so in the input, or confirmed when the advisor asks. The redundancy check fires only on a positive determination.
+
+---
+
+## The gates (before pyramid placement)
+
+**Gate A — user-facing dynamic.** The user acts and must *see* a rendered result. → E2E **alongside** lower layers. State the slow/flaky cost; amortize by extending or grouping into a journey test.
+
+**Gate B — bug-fix regression.** → regression at the **layer the bug was discovered** (browser → E2E; unit-debug → unit). Must fail on old code, pass on the fix. Never escalates above the discovery layer.
+
+**Gate C — dynamic-swap delivery chain.** An HTMX / Alpine / Turbo / LiveWire swap. → browser test **REQUIRED** (not "recommended"); integration is complementary, not sufficient. Holds with no server state change too (structural seam: wrong `hx-target`, stale swap, re-init).
+
+**Gate D — visual fidelity.** Output whose layout/appearance matters (PDF, print, email, receipt). With a reference: approval (text) and/or screenshot (CSS/layout), surfacing maintenance cost. No reference: manual review + suggest creating one.
+
+---
+
+## Redundancy check (business-critical only, after placement)
+
+A business-critical behavior at only one layer → flag it, name a second layer with a **different failure mode**, and give a concrete recommendation. Catches/misses:
+
+| Layer | Catches | Misses |
+|-------|---------|--------|
+| Unit | logic, edge cases | wiring, integration, UI |
+| Integration | routing, persistence, security | client behavior, rendering |
+| Component/Frontend | wiring, DOM, attributes | backend logic, real network |
+| Contract | API-shape drift | semantic/business bugs |
+| E2E | full-stack, real browser | slow, flaky, poor localization |
+
+When a gate mandates application-level E2E, **flag the seam (`→ cd-test-architecture`) and defer the harness/pipeline design** to `cd-test-architecture`. This file stays at unit/module altitude.

package/knowledge/test-matrix-examples/django-batch.md

@@ -0,0 +1,24 @@
+# Worked example — Django API + scheduled batch job
+
+Few-shot template for `test-design-advisor`. Adapt the rows. Feature: **a nightly job reads pending invoices, charges a payment gateway through an owned adapter, and emails receipts**. Orchestration/adapter-heavy, little domain logic → **diamond** shape (`test-pyramid.md`).
+
+## Pyramid placement (advisor output shape)
+
+| Behavior | Layer | Gate | Tool (`test-stack-profiles/django`) | Why |
+|----------|-------|------|-------------------------------------|-----|
+| "Invoice is due" predicate | Unit | — | pytest | the one bit of real logic |
+| Job selects due invoices, skips paid, marks charged | Component | — | pytest + Django test DB, gateway/email **adapters doubled** | the orchestration — run pre-merge without real gateway |
+| Payment adapter speaks the gateway's real protocol | Integration | — | pytest + recorded/contract sandbox | the adapter actually works (own your boundaries — `cd-test-architecture.md`) |
+| Receipt PDF/email layout is correct | — | ↑ (Gate D) | approval (text) / screenshot (layout) | visual-fidelity artifact (`test-layer-gates.md`) |
+
+## Quadrants (`testing-quadrants.md`)
+
+Q1 thin (correctly — little logic) · Q2 acceptance example for the "skip already-paid" rule · Q3 N/A · **Q4 important** → resilience: what happens when the gateway times out mid-run? → **chaos**.
+
+## Techniques (`testing-techniques/`, on match)
+
+Gateway timeout/partial-failure resilience → **chaos**. Receipt email body vs a golden file → **approval**. Idempotency of a re-run after a crash → property/scenario coverage.
+
+## Notes
+
+The wide component middle is right for a diamond — the value is in orchestration, not unit logic; don't pad the unit base with tests that just exercise the framework. Double the **owned adapter**, never the gateway SDK directly (`cd-test-architecture.md` → the adapter rule).

package/knowledge/test-matrix-examples/dotnet-grpc-fronting-api.md

@@ -0,0 +1,90 @@
+# Worked example — ASP.NET Core API fronting gRPC microservices
+
+Few-shot template for `test-design-advisor`. Adapt the rows. Feature:
+**an ASP.NET Core 8 Web API fronts several owned gRPC microservices
+(`AccountService`, `PaymentService`, `LedgerService`) behind hand-written
+adapter ports; a SQL store via Dapper persists ledger entries; a
+`POST /payments` endpoint authorizes the caller, debits the account via
+`PaymentService`, writes a ledger entry via Dapper, and returns a result**.
+This is a common payments / banking / e-commerce shape: rich domain logic
+fronts multiple owned remote dependencies; the gate must stay deterministic
+without standing up the back-end services.
+
+Layer names use the MinimumCD six test types from
+`knowledge/cd-test-architecture.md` § *The Six Test Types*.
+
+## Pyramid placement (advisor output shape)
+
+| Behavior | Layer | Gate | Tool (`test-stack-profiles/dotnet`) | Why this layer (not the one above or below) |
+|----------|-------|------|--------------------------------------|---------------------------------------------|
+| Authorization policy denies non-payer caller | Unit | — | xUnit + `AuthorizationHandlerContext` | pure policy logic; a component test would only re-execute the same code through HTTP middleware |
+| `PaymentDomainService` orchestrates rule + debit + ledger (no over-limit) | Unit (sociable) | — | xUnit + stub adapters | wires collaborators; a component test would re-execute the same orchestration through HTTP for no extra signal |
+| `IPaymentServiceAdapter` translates `DebitRequest` → gRPC request and `Ack` → domain reply | Contract | — | xUnit + in-memory gRPC channel (`Grpc.Core.Testing`) or stubbed `CallInvoker` | pin the request/response shape at the boundary; a unit test of the adapter would not exercise gRPC serialization, an integration test would couple the gate to a running provider |
+| `PaymentServiceAdapter` survives a provider timeout / cancellation / malformed reply | Resilience | — | xUnit + simulated `CallInvoker` returning `RpcException`/timeout | failure-mode coverage at the seam; lower than a component test (no HTTP path needed), higher than a unit test (the adapter's failure translation IS the behavior under test) |
+| `ILedgerRepository` (Dapper) writes + reads back a ledger row | Contract | — | xUnit + in-memory SQL fake (e.g. SQLite in-memory) | pins the SQL the adapter emits without the gate depending on a real container; a unit test cannot exercise SQL, an integration test moves it off the gate |
+| `LedgerRepository` against the real SQL flavor (column types, isolation) | Integration (Stage 1 / 2, NOT pre-merge) | off-gate | xUnit + Testcontainers (MS SQL / PG) | the contract test pins shape but cannot prove the real engine accepts the SQL; non-deterministic / requires a container ⇒ never gates the merge per MinimumCD |
+| `POST /payments` end-to-end through middleware, validation, controller, doubled adapters | Component | — | `WebApplicationFactory<Program>` + `HttpClient` + in-process double registrations (Test doubles for `IPaymentServiceAdapter`, `ILedgerRepository`) | exercises the assembled in-process app deterministically; a contract test cannot cover request validation + controller wiring + auth pipeline together; an E2E test would need the back-end services |
+| Cross-cutting filter / exception middleware translates `DomainException` → 422 | Unit | — | xUnit + constructed `HttpContext` | the middleware IS the unit; a component test would only re-prove the same translation through HTTP |
+| Other consumers still accept our `PaymentResult` response shape (and we still accept `PaymentService` reply shape) | Contract | — | Pact / Protobuf schema check against the provider's generated `.proto` (see `microservice-testing.md`) | cross-boundary agreement — never E2E; provider cooperation is not required |
+| Provider adapters' in-memory doubles match the real provider | Scheduled provider verification | out-of-band | nightly job hitting provider test env | proves the in-memory double has not drifted from reality; lower-layer contract tests are insufficient on their own |
+| Critical user journey: a successful payment + ledger write + downstream consumer sees the event (post-deploy) | E2E | post-deploy smoke | one Playwright/HTTP scenario in a deployed environment | (1) contract tests cannot cover the *real* multi-service propagation; (2) component test stubs all adapters; (3) resilience covers failure modes, not the happy path across real services; (4) the payment-confirmed journey is a critical real-component crossing — non-deterministic, NEVER pre-merge per MinimumCD |
+
+## E2E justification (only the last row)
+
+The advisor's E2E justification block for this matrix lists exactly one
+behavior. The other rows show how the four-condition gate forces every
+other behavior into a lower layer.
+
+## Quadrants (`testing-quadrants.md`)
+
+Q1 strong ✓ · Q2 add a BDD acceptance example for "deny over-limit" before
+coding · Q3 minimal (API, no UI) · **Q4** → security-review the auth on
+`/payments`; load-test if it's a hot path. For payments specifically:
+PCI-DSS / SOC 2 coverage of the boundary level (contract pins, resilience
+under provider outage, characterization for any legacy SQL touching
+cardholder data) — flag to security-engineer if a `security-primitives`
+contract is not yet in place.
+
+## Doubles strategy
+
+| Test | Collaborator | Double | Verify by |
+|------|--------------|--------|-----------|
+| `PaymentDomainService` unit | `IPaymentServiceAdapter` | Stub | State (returned `PaymentResult`) |
+| `PaymentDomainService` unit | `ILedgerRepository` | Stub | State (returned `PaymentResult`) |
+| `IPaymentServiceAdapter` contract | gRPC `CallInvoker` | In-memory fake / stubbed invoker | State (returned domain `Ack`) |
+| `IPaymentServiceAdapter` resilience | gRPC `CallInvoker` | Stub that throws `RpcException`/timeout | State (translated domain error) |
+| `ILedgerRepository` contract | SQL connection | In-memory SQL fake (SQLite) | State (round-trip row) |
+| `POST /payments` component | `IPaymentServiceAdapter` | Stub registered in `WebApplicationFactory` | State (HTTP 200 + body) |
+| `POST /payments` component | `ILedgerRepository` | Stub | State (HTTP 200 + body) |
+| Provider contract | the real `PaymentService` provider | None (real provider in scheduled job) | Schema agreement |
+
+Default to state verification and the simplest double per
+`knowledge/test-doubles.md`. Use a mock/spy only when a true side-effect
+boundary requires it (e.g. asserting "an audit event was published" when
+the publisher is fire-and-forget).
+
+## Techniques (`testing-techniques/`, on match)
+
+Money math invariants (sum preserved across a transfer) →
+**property-based**. Resilience claim "degrades if `PaymentService` is
+down" → **chaos** at the adapter seam. Generated wire formats (proto3) →
+**schema-validation** at the contract layer.
+
+## Notes
+
+- Keep the authorization policy a **unit** test, not a component test —
+  driving HTTP middleware to prove the same `requirement.Succeed()` call
+  is "testing through the UI for logic" anti-pattern.
+- The **contract** tests for adapters pin shape and behavior; they are
+  always pre-merge. The **integration** tests against Testcontainers /
+  real providers prove the adapter's understanding of the real engine
+  matches reality, and are always **off the gate**.
+- The single **E2E** test exists because no other layer can prove the
+  payment + ledger + downstream propagation across multiple owned
+  services in a real environment. It runs post-deploy as smoke. Adding
+  a second E2E "to be thorough" would fail the E2E justification gate.
+- For regulated payment paths (card data, account numbers, secrets),
+  confirm coverage at the boundary level — contract pins, resilience
+  tests, and characterization tests for any legacy SQL paths handling
+  cardholder data. Cite `knowledge/security-primitives-contract.md` if
+  present; else flag the gap to `security-engineer`.

package/knowledge/test-matrix-examples/dotnet-http-consumer.md

@@ -0,0 +1,131 @@
+# Worked example — ASP.NET Core service calling a third-party HTTP API
+
+Few-shot template for `test-design-advisor`. Adapt the rows. Feature:
+**an ASP.NET Core 8 service exposes `POST /orders`; the order handler calls
+an upstream third-party Pricing API over HTTPS (JSON), then persists the
+order to its own SQL store via EF Core; the Pricing API is owned by another
+company — no provider cooperation, no CDC tooling on their side — and is
+flakier than the team's own services**. This is the common
+[API Consumer pattern](../component-test-patterns.md#api-consumer); the
+pre-merge gate must not depend on the upstream being reachable, and the
+consumer must survive provider breakage without notice.
+
+Layer names use the MinimumCD six test types from
+`knowledge/cd-test-architecture.md` § *The Six Test Types*. Mechanics for
+the outbound HTTP layer are from
+`knowledge/references/csharp-http-client-testing.md`; tool resolution is from
+`knowledge/test-stack-profiles/dotnet.md`.
+
+## Adapter shape (the seam under test)
+
+`IPricingClient` is the team-owned thin adapter (Adapter Rule, see
+`knowledge/cd-test-architecture.md#the-adapter-rule-own-your-boundaries`):
+
+```csharp
+public interface IPricingClient
+{
+    Task<Money> QuoteAsync(Sku sku, CancellationToken ct);
+}
+
+internal sealed class PricingClient : IPricingClient
+{
+    private readonly HttpClient _http;          // injected; primary handler is swappable
+    private readonly TimeProvider _time;        // determinism (Polly + tests)
+    public PricingClient(HttpClient http, TimeProvider time) { _http = http; _time = time; }
+    // ...
+}
+```
+
+Wired in production with `services.AddHttpClient<IPricingClient, PricingClient>(...)`;
+in tests, `ConfigurePrimaryHttpMessageHandler(() => stub)` swaps only the
+leaf. The adapter is the single place every "the real thing changed" failure
+localizes to.
+
+## Pyramid placement (advisor output shape)
+
+| Behavior | Layer | Gate | Tool (`test-stack-profiles/dotnet`) | Why this layer (not the one above or below) |
+|----------|-------|------|--------------------------------------|---------------------------------------------|
+| `Money` arithmetic invariants (sum preserved, no negative quotes) | Unit | — | xUnit + FluentAssertions; **property-based** via FsCheck on the invariant | pure value-object logic; a component test would only re-execute the same arithmetic through HTTP |
+| `OrderDomainService` orchestrates pricing + persistence (no over-limit) | Unit (sociable) | — | xUnit + stub `IPricingClient`, in-memory repo | wires collaborators; a component test would re-execute the same orchestration through HTTP for no extra signal |
+| `PricingClient` builds the correct request (method, path, query, auth header, `Idempotency-Key`, JSON body) and parses the success response | **Contract** | — | xUnit + `StubHttpMessageHandler` capturing `Requests`; structural JSON compare (`knowledge/references/csharp-http-client-testing.md`) | pins request/response shape at the boundary; a unit test of the adapter would not exercise serialization/headers, an integration test would couple the gate to a running provider we don't control |
+| `PricingClient` maps non-2xx (`401`, `404`, `422`, `500`) to typed domain errors | **Contract** | — | xUnit + `StubHttpMessageHandler` returning each status | failure-mapping IS the contract; a component test would only re-prove the same mapping through HTTP |
+| `PricingClient` survives provider flake: retries `503` per Polly policy, opens circuit-breaker after sustained failure, enforces deadline, fast-fails on half-open | **Component** (resilience) | — | xUnit + `StubHttpMessageHandler` queue `[503,503,200]` etc. + `FakeTimeProvider` + Polly v8 pipeline bound to that `TimeProvider` | the resilience policy is the behavior under test; running against the real provider is non-deterministic and would never be a gate; lower than full HTTP component test because no inbound HTTP is needed to verify the outbound seam |
+| `POST /orders` end-to-end through middleware, validation, controller, doubled pricing adapter, in-memory repo | Component | — | `WebApplicationFactory<Program>` + `HttpClient`; register stub `IPricingClient` in `ConfigureTestServices` | exercises the assembled in-process app deterministically; a contract test cannot cover request validation + controller wiring + auth pipeline together; an E2E test would need the real Pricing API |
+| EF Core `OrderRepository` SQL against the real SQL flavor (column types, isolation, EF-generated SQL) | Integration (Stage 1 / 2, NOT pre-merge) | off-gate | xUnit + Testcontainers (PG / SQL Server) | the in-memory EF provider hides SQL bugs; the real engine is non-deterministic / requires a container ⇒ never gates the merge per MinimumCD |
+| The team's hand-rolled `PricingClient` double has not drifted from the real Pricing API | **Scheduled provider verification** | out-of-band (nightly) | xUnit run vs the provider's real *non-prod* endpoint, decoupled from deploys (`cd-test-architecture.md#double-validation-keeping-doubles-honest`) | proves the double still matches reality; lower-layer contract tests cannot, by construction, observe provider drift; we **do not assume provider cooperation** so consumer-owned detection is the only defense |
+| Critical user journey: a successful order across the real deployed system (pricing + persistence + downstream notification) | E2E | post-deploy smoke | one Playwright/HTTP scenario in a deployed environment | (1) contract tests cannot cover real multi-service propagation; (2) component test stubs the pricing adapter; (3) resilience covers failure modes, not the happy path across real services; (4) the order-confirmed journey is a critical real-component crossing — non-deterministic, NEVER pre-merge per MinimumCD |
+
+## E2E justification (only the last row)
+
+The advisor's E2E justification block for this matrix lists exactly one
+behavior. Every other row above shows how the four-condition gate forces a
+lower layer.
+
+## Doubles strategy
+
+| Test | Collaborator | Double | Verify by |
+|------|--------------|--------|-----------|
+| `OrderDomainService` unit | `IPricingClient` | Stub | State (returned `Order`) |
+| `OrderDomainService` unit | `IOrderRepository` | Stub | State (returned `Order`) |
+| `PricingClient` contract — success | `HttpMessageHandler` | `StubHttpMessageHandler` returning canned 200 | State: captured `Requests[0]` (method/path/headers/JSON body) + returned `Money` |
+| `PricingClient` contract — non-2xx | `HttpMessageHandler` | Stub returning each status | State: typed domain error thrown |
+| `PricingClient` resilience | `HttpMessageHandler` + `TimeProvider` | Stub queue + `FakeTimeProvider`; Polly bound to it | State: final outcome + `Requests.Count` (proves retry count) |
+| `POST /orders` component | `IPricingClient` | Stub registered via `ConfigureTestServices` | State (HTTP 200 + persisted order via in-memory repo) |
+| `POST /orders` component | `IOrderRepository` | In-memory fake | State (persisted row) |
+| Scheduled provider verification | the real Pricing API | None — real provider, real network, non-prod env | Schema + status agreement |
+
+Default to state verification and the simplest double per
+`knowledge/test-doubles.md`. The stub handler's captured `Requests` list IS
+the state surface — never reach for a behavior mock on `HttpMessageHandler`.
+
+## Techniques overlay (`testing-techniques/`)
+
+- Money arithmetic invariants → **property-based** (`property-based.md`).
+- Pricing API responds with JSON governed by an OpenAPI schema →
+  **schema-validation** at the contract layer (`schema-validation.md`).
+- "Service degrades but stays up when Pricing is down for 30s" →
+  **chaos** at the adapter seam (`chaos.md`), realized by the
+  `StubHttpMessageHandler` returning timeouts/errors.
+
+## Smells to refactor away first
+
+Cross-referenced with
+`knowledge/references/csharp-http-client-testing.md` §
+*Common smells in C# HTTP-consumer tests*:
+
+- A `Mock<HttpClient>` in any test → wrap in an adapter and double the
+  `HttpMessageHandler`.
+- A `Mock<IThirdPartyPricingSdk>` directly → own the adapter; double the
+  adapter or its `HttpMessageHandler`.
+- `Thread.Sleep(...)` or `await Task.Delay(...)` waiting for a retry →
+  introduce `TimeProvider` + Polly v8 bound to `FakeTimeProvider`.
+- `Assert.Equal("""{"sku":"X"}""", actualJson)` → deserialize and compare
+  structurally.
+- A test that needs `appsettings.Test.json` with a real Pricing API URL →
+  the test is mis-typed; route it to the scheduled provider verification
+  job, not the pre-merge gate.
+
+## Quadrants (`testing-quadrants.md`)
+
+Q1 strong ✓ · Q2 add a BDD acceptance example for "order rejected when
+pricing fails three times" (Reqnroll; see
+`../test-stack-profiles/dotnet.md` BDD row) · Q3 minimal (API, no UI) ·
+**Q4** → security-review auth/secret handling on the Pricing API
+connection; resilience claims under provider outage cited in any SLO
+documentation.
+
+## Notes
+
+- The contract tests for `PricingClient` are always pre-merge — they're
+  deterministic, in-process, microsecond-fast. The provider-verification
+  job is always **off the gate** — it's the *detection* loop for drift,
+  and it must run on a clock the build doesn't depend on.
+- Resilience component tests with `FakeTimeProvider` complete in
+  microseconds even when the policy says "wait 30s then retry" — the
+  Polly pipeline must be configured with that `TimeProvider` for this to
+  hold. If a resilience test takes longer than the others, the wiring is
+  wrong, not the policy.
+- The single E2E test exists because no lower layer can prove order
+  propagation across the real Pricing API and real downstream notification.
+  Adding a second E2E "to be thorough" would fail the four-condition gate
+  (`cd-test-architecture.md#the-e2e-justification-gate`).