@metabase/cli 0.3.1-alpha.transform-tests.3fbcf8b → 0.3.1-alpha.transform-tests.d9ed765

package/skill-data/transform-test-plan/SKILL.md

@@ -0,0 +1,171 @@
+---
+name: transform-test-plan
+description: Derive a comprehensive test plan for a transform — the fixture cast, the expectations, hand-derived expected rows, and a coverage matrix — from the model's declared design. Covers input partitioning (zero-case, multiplicity, dirty rows), grain / conservation / recomputation / conformance checks, and known-quirk conventions. Load when the user wants tests planned or written for transforms — "write tests for my transforms", "is my model right", "test plan for this pipeline", "add data quality checks" — whether the model is mid-build or already deployed. The `mb transform-test` command and body shapes live in the `transform` skill; this one decides what to test.
+allowed-tools: Read, Write, Edit, Bash, AskUserQuestion
+requires: [transformTests]
+---
+
+# Planning transform tests
+
+Turn a transform into a fixture cast that proves the logic on small known rows, expectations that
+state the model's invariants, and a coverage matrix showing what's checked and what's deliberately
+not. Mechanics — the `inputs`/`expectations` body shape and every `mb transform-test` verb — live in
+the `transform` skill (`mb skills get transform`); load it before authoring, and never restate it
+here.
+
+Every check derives from what the model **declares** — detected from its SQL, confirmed with its
+owner — never from conformance to a modeling doctrine. One plan serves two moments: while the model
+is **built**, checks pin each design decision; once **deployed**, the same SQL screens production
+tables for anomalies.
+
+## Operating rules
+
+- **Detect, then derive.** Classify what the transform is and which conventions it uses (the
+  checklist); derive checks only from that. Star, one-big-table, partial denormalization — all
+  fine; never flag a style.
+- **Judgment calls go through the checklist.** The session's autonomy setting governs which answers
+  you supply yourself and which you bring to the user — it never makes the checklist a formality.
+  Every answer you supply yourself is recorded in the plan as a stated assumption, paired with the
+  named expectation that enforces it — reversing the decision then breaks a test, not a paragraph.
+  And regardless of setting, when genuinely unsure, ask — a wrong-but-confident grain poisons every
+  downstream check.
+- **Expected rows are derived by hand** from the fixture story and business meaning — never captured
+  from the transform's output, which asserts only that the transform equals itself.
+- **Batteries stay off until declared structure switches them on.** No snapshot-density checks
+  without a snapshot, no version-history checks without effective/end/current columns. An empty
+  section beats a speculative one.
+
+## The procedure
+
+1. **Profile the real inputs**: per table, row count; per column, min/max/distinct-count/null
+   incidence; orphan counts across declared links (`mb field summary`, `mb query`). Profiling feeds
+   domains, bounds, null partitions, and key candidates — and every fixture edge cites the real-data
+   condition that warrants it, with its count ("the warehouse has 67 ship-before-order rows").
+2. **Detect.** Read the transform's SQL (`mb transform get <id> --full --json`) for: grain
+   candidates (`GROUP BY` keys, the joins' driving table), join types (orphan handling), correlated
+   aggregates (stored aggregates), `<entity>_<attr>` naming (copies), effective/end/current columns
+   (version history), `now()`/`current_date` (volatile columns). A transform whose grain you cannot
+   state in one phrase is itself a finding — raise it before writing any test.
+3. **Confirm.** Walk [references/checklist.md](references/checklist.md). Three stages: classify the
+   model, per-table declarations, per-column declarations. Each question carries its detection hint;
+   answer autonomously where the hint resolves, ask where it doesn't. In build-along mode these are
+   design questions — treat an undecided answer as a decision to make together, not a blocker.
+4. **Derive.** Route every output column through [references/checks.md](references/checks.md) —
+   declared property → expectation shape, fixture implication, expected-row convention. Read it in
+   full once per plan; it is the plan's content.
+5. **Design the fixture cast**: one small cast per transform (≈5–10 rows per table), human-named
+   rows ("Alice Premium"), every row a named edge — zero-case entities for every outer join and
+   aggregation, ≥2-member groups for every grouping and join, one dirty row per screenable defect,
+   boundary dates. Document it as a table (row → attributes → purpose) in the plan.
+
+   Express the cast as literal rows rather than as a query, so the story stays legible in the test
+   itself; a query is worth it only when the rows are mechanical to generate. A cast is written
+   against one warehouse and does not carry to another.
+
+6. **Hand-derive the expected rows**, arithmetic recorded in the plan (premium: 3 orders / 350.50 /
+   3.0). Every fixture row's fate appears in some expected cell. Pin NULL-vs-0-vs-empty for every
+   zero-case row — that cell is the null policy's only enforcement.
+7. **Author the test.** One test per transform, per coherent story: an `equals` pinning the output,
+   and `empty` expectations stating the invariants that survive a change to the cast.
+
+   Give each expectation a name that states the invariant, because the name is what a
+   failure leads with ("revenue never negative", not "check 3"), and open each `empty` expectation's
+   SQL with a `--` contract comment: the invariant, and the failure modes it catches ("catches both
+   dropped orders and join fan-out").
+
+   Where one invariant applies to several transforms, duplicate the SQL. Nothing is shared between
+   tests, so give each copy its own name and comment rather than one that only makes sense next to
+   its twin.
+
+8. **Emit the coverage matrix** in the plan: rows = output columns with the table's grain; columns =
+   check classes (grain, conservation, recomputation, conformance, domains, referential integrity,
+   temporal, screens); cells name the covering expectation or expected-row cell, or state
+   `gap: <reason>`. Scan shared-attribute columns across transforms for cross-table agreement
+   obligations. Empty cells are honest; silent gaps are not.
+9. **Prove the tests have teeth.** Once per test: corrupt one expected cell, `run`, confirm
+   `cell-mismatches` names exactly that column; revert. Then perturb one input cell and confirm
+   exactly the declared output cells move. Every `empty` expectation passes against an empty output,
+   so a green test can still be vacuous.
+
+## Severity: error, or the tolerated oddity
+
+Every expectation is pass or fail, and one failure fails the run. **error** = forbidden, and it
+becomes an `empty` expectation. There is no warn severity, so **never author an expectation you
+expect to fire**: a permanently red test trains everyone to ignore the result, and a red run gates
+everyone else's work.
+
+A tolerated-but-surfaced oddity — one the owner lives with, like orphan rows or ship-before-order
+dates — gets encoded the two ways that hold: **pin it in the `equals` rows** (an orphan passing
+through with NULLs is a cell in the expected output, so reversing the tolerance breaks the test),
+and **record it in the plan's known-quirks list** with its real-data count, so the tolerance stays a
+conscious choice.
+
+Under budget pressure cut business-rule checks first, then cross-table structure checks; never
+single-column screens (domains, ranges, nulls) — cheapest, and the last line.
+
+## When a check exposes a live bug
+
+Non-negotiable: **never soften the test to green** — expected values state correct behavior; matching
+them to buggy output documents the bug as intended — and **surface the finding with its blast radius
+at both scales**, fixture ("1530.24 of 1600.74 fixture dollars survive") and warehouse ("908 of
+2,050 orders dropped"). What happens next follows the session's terms, not a fixed protocol: propose
+and apply the fix now (when the user wants it or the autonomy setting covers it), or — when the fix
+must wait — hold the correct expectation and record the red in the plan with the minimal fix body.
+A stored test has no red-by-design state, so a deferred
+fix must be visible in the plan or the test reads as broken. Either way, once green the test stays
+as the regression guard.
+
+## When the doctrine doesn't apply
+
+The vocabulary follows Kimball's dimensional modeling (grain, additivity, conformed attributes,
+slowly changing dimensions) — precise, widely understood terms. Real models are Kimball-inspired at
+most; no check may score adherence:
+
+- Full calendar date dimensions are rare. Never demand one; test date _semantics_ — ranges,
+  orderings, volatile derivations.
+- Surrogate keys are doctrine, natural-key joins are practice. Test whichever key the model
+  declares; never flag natural-key joins.
+- "No NULL FKs / no NULL attributes" is doctrine routinely dropped. Null policy is three independent
+  declarations — measures, attributes, FKs — each detected and confirmed, never presumed.
+- One-big-table is legitimate: it still has a grain, its copies still need agreement checks, its
+  functional dependencies still hold.
+- A transform-level `ORDER BY` has no testable effect — output tables carry no row order and the
+  `equals` comparison is a multiset. Flag it as probable dead weight (clustering hints aside); never
+  write an ordering expectation.
+
+## One transform at a time
+
+Each test covers one transform. When the transform under test reads another transform's output,
+that target table is an input like any other: declare it and fake it. Derive those rows from the
+base transform's own expected output, so the two tests tell one story, and note the coupling in both
+plans — changing the base's expected rows means changing this test's input.
+
+## Worked example, condensed
+
+`orders` + `customers` → _enriched_orders_ (order grain; LEFT JOIN attaches `customer_name`,
+`tier`).
+
+Cast, 7 orders: two tiers; Alice and Carol with 2 orders each (multiplicity); two never shipped
+(zero-case for the shipping join); order 106 shipped before ordered (real oddity, count cited);
+order 107's customer_id matches no customer (orphan). Two `rows` inputs, one per source table.
+
+Derived, per catalog: grain uniqueness on `order_id`; row and amount conservation from input to
+output, as scalar subqueries over the target and the seeded input; `tier` domain ⊆ {standard,
+premium}; orphan = keep-with-NULLs → an expected row pinning the NULL pass-through; ship-before-order
+tolerated → the plan's known-quirks list with the warehouse count. The whole output pinned in one
+`equals`, hand-computed, arithmetic in the plan. The inner-vs-LEFT-join bug this cast catches —
+unshipped orders silently dropped from revenue — is what the zero-case rows exist for.
+
+## Don't
+
+- Don't author expectations before loading the `transform` skill — the body shape, the closed
+  create/update contract, and the verb flags live there.
+- Don't capture expected rows from the transform's own output — hand-derive them or they assert
+  nothing.
+- Don't emit checks for structure the model doesn't declare (snapshot density, version history,
+  bridge weights) — an inapplicable battery buries real findings.
+- Don't let a fixture cast go all-clean — no zero-case, no orphan, no dirty row proves the happy
+  path and nothing else; the bugs live in the edges.
+- Don't write an expectation you expect to fail.
+- Don't surface bare check-ids ("per C3…") to the user — name the check in plain words; the ids are
+  for your cross-referencing, not their reading.

package/skill-data/transform-test-plan/references/checklist.md

@@ -0,0 +1,81 @@
+# Confirmation checklist
+
+The judgment calls a plan depends on, in derivation order. Each question carries a **detect** hint —
+how to answer it from the SQL, the metadata, or profiling. Answer autonomously where the hint
+resolves cleanly; ask where it doesn't. Every answer you supply yourself goes into the emitted plan
+as a stated assumption, phrased so the owner can falsify it at a glance ("Assuming `discount` can
+never be negative — correct?").
+
+In build-along mode these are design questions: the owner is deciding, not recalling. An undecided
+answer is a decision to make together — surface the options and the check each implies (the
+`checks.md` entry named in parentheses).
+
+## Stage 1 — classify the model (per transform)
+
+1. **Grain: one row per what?** (G1, G2)
+   Detect: `GROUP BY` keys; otherwise the driving table of the joins. Confirm whenever detection and
+   any written description disagree — that disagreement is itself a finding.
+2. **Fact-table type — and if periodic snapshot, dense or sparse?** (F2)
+   Detect: event-grain with a single event date → transaction; entity×period grain → periodic
+   snapshot (density is the one yes/no that flips its whole invariant set — always confirm); one row
+   per pipeline occurrence with milestone-date columns → accumulating snapshot.
+
+## Stage 2 — per-table declarations
+
+3. **Conservation ties: output rows/sums tie to which input, with which declared exclusions?** (A2)
+   Detect: `WHERE` clauses and join types name the exclusions (filtered rows, dropped duplicates).
+   Exclusions must be declared, or conservation appears to fail.
+4. **Zero-group policy: does a group with no contributing rows appear (with zeros) or stay absent?**
+   (A5)
+   Detect: driving table of the rollup — grouping the detail can't produce empty groups; joining a
+   dimension first can.
+5. **Orphan handling per join: drop / keep-with-NULLs / default row — and is an orphan tolerated or
+   forbidden?** (C3, I5)
+   Detect: join type. INNER = drop, LEFT = keep-with-NULLs, COALESCE to a sentinel = default row.
+   Tolerated-vs-forbidden is the owner's call — profiling says whether orphans exist today, not
+   whether they're acceptable. Forbidden becomes an `empty` expectation; tolerated is pinned in the
+   expected rows and recorded in the known-quirks list.
+6. **Version history: effective/end/current columns anywhere? If not, confirm overwrite-everywhere.**
+   (T1)
+   Detect: column names. Absence means history is silently rewritten in rollups — state that
+   consequence when confirming, not just the mechanism.
+7. **Does this transform read another transform's output?** (A5)
+   Detect: another transform's target table in this one's SQL. It becomes a declared input like any
+   other, faked from that transform's own expected output — confirm which rows, and record the
+   coupling in both plans.
+
+## Stage 3 — per-column declarations
+
+8. **Each measure: additive, semi-additive, or non-additive?** (A1)
+   Detect: sums and counts are additive; balances and levels are semi-additive; ratios, rates, and
+   unit prices are non-additive. Confirm the ambiguous ones (a "score"? a "quantity on hand"?).
+9. **Legal bounds per measure?** (Q1)
+   Detect: profiling min/max suggests, business meaning decides — can `discount` be negative? can
+   `quantity` be zero? Derived bounds are free (a sum of non-negatives is non-negative).
+10. **Each derived measure: the exact recomputation rule?** (A3)
+    Detect: the SELECT expression is the rule — but rounding, business-day adjustments, and NULL
+    handling are conventions to confirm, not read.
+11. **Which columns are stored aggregates, over which detail and filter?** (A4)
+    Detect: correlated subqueries / joined-aggregate CTEs in the SQL.
+12. **Which categorical columns have closed domains — enumerate; is NULL a member?** (C2)
+    Detect: profiling distinct values gives today's set; the owner confirms it's closed rather than
+    merely small so far.
+13. **Which columns are denormalized copies, of which owning attribute?** (C1)
+    Detect: `<entity>_<attr>` naming; any column functionally dependent on a non-grain key. Every hit
+    must be classified: copy (C1), stored aggregate (A4), or smuggled coarser-grain fact — a
+    finding.
+14. **Which many-to-one edges hold within one output table?** (C4)
+    Detect: hierarchy-shaped column pairs (product/category, zip/state). Applies where no owning
+    table is in scope; otherwise C1 covers it.
+15. **Which date orderings are business-guaranteed vs. known-violated and tolerated (with real-data
+    counts)?** (T2)
+    Detect: profiling counts the violations that exist; the owner decides tolerated vs. bug.
+16. **Null policy per column — measures, attributes, FKs separately.** (Q2)
+    Detect: the SQL shows what's produced (COALESCE, CASE); the owner confirms intent — "count of
+    nothing": 0 or NULL? empty date: NULL or sentinel?
+17. **Which columns are volatile across runs?** (T3)
+    Detect: `now()` / `current_date` / run-metadata expressions in the SQL. A volatile column is left
+    out of the `equals` columns; its form can still be asserted with an `empty`.
+18. **Per tolerated oddity: does it stay tolerated, and what is its current count?** (Q3)
+    Detect: can't — this is the conscious-choice question, and the plan's known-quirks list is where
+    it survives.

package/skill-data/transform-test-plan/references/checks.md

@@ -0,0 +1,259 @@
+# Check catalog
+
+Every check the plan can derive, grouped by theme. Per entry: **when it applies** (the declared
+property that switches it on — blank means always), **assert** (the expectation shape), and
+**fixtures** (what the input cast must contain for the check to have teeth).
+
+Expectation SQL names the transform's **target table** and its **declared input tables** under their
+real names; both are rewritten to the run's temp tables, so per-row recomputation joins against
+inputs are writable. Anything else you name is left exactly as written and reads the real table.
+
+Ids (G1, A2, …) are for cross-referencing within the plan documents only — never surface them to the
+user bare.
+
+Throughout, `<target>` stands for the transform's target table as written in the transform's own
+definition, and `<input>` for a declared input table.
+
+## Grain & keys
+
+**G1 — Declared grain.** Every output table states "one row per X"; the declaration anchors every
+other check.
+
+- Applies: always. Elicit or detect (GROUP BY keys; the join's driving table).
+- Assert: nothing directly — G1 is the plan's opening move. A table with no statable grain, or a
+  grain stated inconsistently between docs and SQL, is a finding before any SQL runs.
+- Fixtures: the grain declaration decides the cast's row structure.
+
+**G2 — Grain-key uniqueness.** The grain key stays a key; doubles as the fan-out guard for every
+enriching join.
+
+- Applies: always — one-big-table, rollups, and stars all have a grain.
+- Assert (`empty`): `SELECT <grain cols>, COUNT(*) FROM <target> GROUP BY <grain cols> HAVING
+COUNT(*) > 1`. NULL grain values form their own bucket.
+- Fixtures: a parent with ≥2 children on every join (I4) is what makes this check able to fail.
+
+## Additivity & reconciliation
+
+**A1 — Measure additivity classification.** Additive / semi-additive (balances) / non-additive
+(ratios, rates, unit prices) decides which aggregations are valid tests.
+
+- Applies: every numeric measure; per-column declaration.
+- Assert: routes the measure — additive → A2; semi-additive → sum across non-time slices only (a
+  test summing a balance over time is a bug in the plan); non-additive → A3 recomputation, never
+  reconciled by summing.
+- Fixtures: none directly; one coverage-matrix axis (measure × valid aggregation set).
+
+**A2 — Conservation reconciliation.** Row counts and additive-measure sums tie from input to output;
+catches dropped rows and join double-counting at once.
+
+- Applies: per declared tie (which input, which declared exclusions).
+- Assert (`empty`): independent scalar subqueries compared with `IS DISTINCT FROM`, returning the
+  mismatched pair:
+  `SELECT (SELECT SUM(t.m) FROM <target> t) AS output_sum, (SELECT SUM(s.m) FROM <input> s) AS
+input_sum WHERE (…) IS DISTINCT FROM (…)`.
+  Never reconcile via a fact-to-fact join — cardinality is uncontrollable and wrong results are
+  silent.
+- Fixtures: amounts chosen so partial survival is visible (distinct values, odd cents).
+
+**A3 — Derived-measure recomputation.** Averages, lags, ratios, rounded presentations recompute from
+their components per row.
+
+- Applies: every derived column, with its declared rule (rounding, business-day adjustment, NULL
+  handling).
+- Assert (`empty`): `SELECT <key> FROM <target> t WHERE t.<derived> IS DISTINCT FROM <recomputation
+from inputs or sibling columns>`.
+- Fixtures: component values whose derivation is non-trivial (a NULL in the AVG, a negative lag).
+
+**A4 — Stored aggregates on entity tables.** Lifetime/rollup stats carried on an entity-grain table
+(`lifetime_orders`, `review_count`, `first_order_date`) equal recomputation from detail — per row
+_and_ in total.
+
+- Applies: columns detected as correlated aggregates over a detail table.
+- Assert (`empty`), two per column: per-row — `WHERE t.<agg> IS DISTINCT FROM (SELECT
+COUNT(*)/SUM(…)/MIN(…) FROM <detail> d WHERE d.<fk> = t.<key>)`; total — the A2 scalar-pair shape.
+  Totals alone cancel offsetting per-row errors; the per-row form is the one that catches them.
+- Fixtures: an entity with several detail rows and an entity with none (I3/I4).
+
+**A5 — Rollup consistency.** A layered rollup always agrees with its base.
+
+- Applies: any transform reading another transform's output. The base is a declared input like any
+  other, faked with rows taken from the base transform's own expected output.
+- Assert (`empty`): every rollup measure equals the corresponding aggregation over the base (A2
+  shape); group-set equality both directions — `SELECT <group> FROM <target> EXCEPT SELECT DISTINCT
+<group col> FROM <base>` and the reverse, filtered by the declared zero-group policy.
+- Fixtures: a group with no contributing rows pins the zero-group policy.
+
+## Fact-table type
+
+**F2 — Fact-table type bundle.** Transaction / periodic snapshot / accumulating snapshot each carry
+a distinct invariant set.
+
+- Applies: per fact-shaped output, always classified; **transaction** needs nothing beyond G2 + A2
+  (sparsity is legitimate).
+- Periodic snapshot, if declared _dense_: `COUNT(*) = |entities| × |periods|` (or the declared
+  subset); per-entity gap detection in the period series; inactive-period representation (zero vs
+  NULL) pinned in the expected rows. If sparse, density checks off — the one yes/no flips the whole
+  set.
+- Accumulating snapshot: milestone dates monotone in pipeline order where set (`WHERE <later> <
+<earlier>` per adjacent pair); unset-milestone default (NULL vs sentinel) pinned in the expected
+  rows; completion flags ∈ {0,1} and consistent with their date's set-ness; lags via A3.
+- Fixtures (accumulating): occurrences at every completion stage — none, some, all milestones.
+
+## Conformance & domains
+
+**C1 — Denormalized-copy agreement.** Every copied attribute agrees with the owning table's value
+for that key.
+
+- Applies: columns declared as copies (detect: `<entity>_<attr>` naming; any column functionally
+  dependent on a non-grain key). Each such column must be a declared copy (this check), a stored
+  aggregate (A4), or it's a smuggled coarser-grain fact — a finding: it double-counts under
+  summation.
+- Assert (`empty`): `SELECT t.<key>, t.<copy>, d.<attr> FROM <target> t JOIN <owning input> d ON
+t.<key> = d.<key> WHERE t.<copy> IS DISTINCT FROM d.<attr>`.
+- Scope: in a test, the copies are produced by the very join under test, so this join-form is near-
+  tautological — the C4 functional-dependency form plus expected-row cell pinning carries the test.
+  The join-form against an independently materialized owning table is the _drift_ check, which
+  belongs to whatever screens the deployed tables.
+- Fixtures: copies with distinct values per entity so a crossed join is visible.
+
+**C2 — Closed-domain screen.** A categorical column's values stay inside the declared enumeration.
+
+- Applies: per column declared closed (detect from profiling; confirm the set and whether NULL is a
+  member).
+- Assert (`empty`): `SELECT <key>, <col> FROM <target> WHERE <col> IS NOT NULL AND <col> NOT IN
+(<domain>)`.
+- Fixtures: every domain value represented where practical; one out-of-domain input row if the
+  source can produce one (I5).
+
+**C3 — Referential integrity / orphan policy.** Every FK resolves, or the declared orphan handling is
+pinned.
+
+- Applies: per join, conditioned on the declared response — drop / keep-with-NULLs / default row;
+  tolerated or forbidden.
+- Assert, forbidden (`empty`): `SELECT t.<fk> FROM <target> t LEFT JOIN <dim input> d ON t.<fk> =
+d.<key> WHERE t.<fk> IS NOT NULL AND d.<key> IS NULL`.
+  Tolerated: not an expectation — pin the orphan's pass-through as a row in the `equals` (NULL in
+  the copied attributes), and record the tolerance with its warehouse count in the plan.
+  Default-row convention adds: distinct unknown keys must not collapse into one output row.
+- Fixtures: one orphan row (I5). Without it the join direction is untested — an inner join silently
+  dropping unmatched rows is the classic bug this catches.
+
+**C4 — Many-to-one consistency.** Each declared many-to-one edge holds within the output (product →
+one category; zip → one state).
+
+- Applies: per declared edge; where an owning table is in scope, C1 subsumes it — this is the
+  one-big-table variant with nothing to join against.
+- Assert (`empty`): `SELECT t.<many>, COUNT(DISTINCT t.<one>) FROM <target> t GROUP BY t.<many>
+HAVING COUNT(DISTINCT t.<one>) > 1`.
+- Fixtures: a violating input row if the source can produce one (I5), pinning the transform's
+  behavior on dirty input.
+
+## Temporal & version history
+
+**T1 — Change handling per attribute.** How the model treats a changed source attribute decides the
+temporal fixtures.
+
+- Applies: detect effective/end/current housekeeping columns.
+- Present → version-history battery (`empty` each): per durable key exactly one current row;
+  `effective < end` per row; intervals contiguous and non-overlapping; current row's end = the
+  declared far-future default; fact rows join the version whose interval contains the fact date.
+- Absent → confirm overwrite-everywhere as a stated assumption (history is silently rewritten in
+  rollups), and derive the propagation probe: change an attribute in an input, run, assert the
+  output regrouped.
+- Fixtures: a before/after change pair; for version history, an entity with ≥2 versions and a fact
+  row dated inside each interval.
+
+**T2 — Date-pair ordering.** Business-guaranteed orderings asserted; known violations recorded.
+
+- Applies: per declared date pair (ordered ≤ shipped ≤ delivered; signup ≤ first order), each
+  classified guaranteed vs. tolerated-violated.
+- Assert: guaranteed (`empty`) — `SELECT <key>, <earlier>, <later> FROM <target> WHERE <later> <
+<earlier>`. Tolerated — pin the violating row's downstream arithmetic (a negative lag inside an
+  average) in the expected rows, and record the real-data count in the plan.
+- Fixtures: one violating row for every tolerated ordering.
+
+**T3 — Volatile columns.** Values that change across runs can't be pinned.
+
+- Applies: columns derived from `now()`/`current_date` (age), run metadata (load timestamps, batch
+  ids).
+- Assert: leave the column out of the `equals` `columns` list — an undeclared column never enters the
+  comparison. Separately assert its form where warranted (`empty`): `WHERE age NOT BETWEEN 0 AND
+120`. The stable source column (`birth_date`) stays exact in the expected rows.
+- Fixtures: none special; the split is the point — omit the volatile, pin the stable.
+
+## Screens & severity
+
+**Q1 — Range / sign screens.** Each measure's declared bounds hold.
+
+- Applies: per bounded measure; bounds from business meaning plus profiling (derived bounds are
+  free: a sum of non-negatives is non-negative).
+- Assert (`empty`): `SELECT <key>, <col> FROM <target> WHERE <col> < <lo> OR <col> > <hi>` (one-sided
+  where only one bound exists).
+- Fixtures: boundary values where the bound is business-set.
+
+**Q2 — Null policy, three ways.** Measures, descriptive attributes, and FKs carry independent null
+policies; never presume one from another.
+
+- Applies: per column, asked separately — "count of nothing": 0 or NULL? empty date: NULL or
+  sentinel? FK: see C3.
+- Assert: declared non-null columns get `WHERE <col> IS NULL` (`empty`). Otherwise the policy is
+  enforced by the expected-row cell of a zero-case fixture row — NULL vs 0 vs empty is invisible
+  until a fixture forces the choice into a cell.
+- Fixtures: the zero-case row (I3) is the enforcement mechanism.
+
+**Q3 — Naming & the tolerated oddity.** Every expectation declares its meaning.
+
+- Applies: always, every expectation.
+- Convention: the expectation's `name` states the invariant in plain words, because a failure leads
+  with it. Each `empty` SQL opens with 1–3 comment lines naming the invariant and the failure modes
+  it catches. Budget cuts drop business-rule checks first, then cross-table structure checks, never
+  single-column screens.
+- A forbidden condition is an `empty` expectation. A tolerated one is pinned in the expected rows
+  and recorded in the plan's known-quirks list with its warehouse count. Never author an expectation
+  designed to fire.
+
+## Input modeling (fixture-design rules)
+
+**I1 — Profiling-derived partitions.** Fixture edges and tolerated non-ties cite profiled reality
+with counts; the plan's known-quirks section carries each tolerated oddity with its reason and
+count.
+
+**I2 — Shared fixture cast.** One small human-named cast per transform, every row a named edge,
+documented as a story table (row → attributes → purpose) in the plan. Where one transform's output
+is another's input, derive the faked rows from the first's expected output so the two tests tell one
+story.
+
+**I3 — Zero-case partitions.** For every outer join and aggregation relation: one entity with zero
+matches. Its expected row pins the absence representation (Q2) and the join direction (C3) — the
+all-clean cast is the single most common cause of vacuous tests. When the schema distinguishes
+states the profiled data never exhibits (a literal zero in a source that only has NULLs and
+positives), fixture the missing state — nothing else forces it onto an expected cell.
+
+**I4 — Multiplicity partitions.** Every aggregation gets a >1-member group and an exactly-1 group
+(the 0 case is I3); every join gets a parent with ≥2 children. Expected values then differ from any
+single row's, so copy-through bugs can't pass.
+
+**I5 — Dirty-input pinning.** Per screenable defect the source can carry (orphan FK, out-of-domain
+value, duplicate natural key, hierarchy violation, ordering violation): one fixture row exhibiting
+it, an expected row pinning the transform's response (drop / pass-through / default), and — where
+the defect is forbidden — the matching `empty` expectation. Which duplicate survives a dedupe is
+invisible until a conflicting-duplicate fixture pins it.
+
+**I6 — Mutation probes.** Run at plan-validation time, not on every run: corrupt one expected cell →
+`cell-mismatches` must name that exact column; perturb one input cell → exactly the declared output
+cells move. Both guard against vacuous tests: every `empty` expectation passes against an empty
+output.
+
+## Plan-level artifacts
+
+**P1 — Coverage matrix.** Output columns (with the table's grain) × check classes; cells name the
+covering expectation or expected-row cell, or state `gap: <reason>`; shared-attribute columns scanned
+across transforms for agreement obligations (C1).
+
+**P2 — Hand-derived expected rows.** From the fixture story and business meaning, arithmetic recorded
+in the plan; never captured from output; every fixture row's fate appears in some expected cell.
+
+**P3 — Live-bug handling.** A check failing against a deployed transform is a real finding: never
+soften the test to green; quantify the damage at fixture and warehouse scale. Then fix now, or record
+the red in the plan with the minimal fix body, per the session's terms. A stored test has no
+red-by-design state, so a deferred fix must be visible in the plan or the test reads as broken.

package/skill-data/visualization/SKILL.md

@@ -27,7 +27,10 @@ Decide which relationship in the data matters most, then pick the chart. The sha
 - **Compare a measure across categories** → `bar` (vertical). Use `row` (horizontal bar) when labels are long or there are many categories. Sort by value unless the dimension has a natural order.
 - **Change over time / trend** → `line` for a continuous series; `bar`/`area` for a few discrete periods. Two measures on unlike scales → `combo` (line + bar, dual-axis) — only when the metrics are genuinely related.
 - **Part-to-whole, one snapshot** → `pie`, but only for a meaningful whole with **≤5 slices**; beyond that use a sorted `bar`/`row`. Composition over time → stacked `area`/`bar`.
-- **Distribution / spread / outliers** → a `bar` histogram (bin the measure — see `mbql` binning); on **v59+** servers `boxplot` compares several groups' spread directly.
+- **Distribution / spread / outliers** → a `bar` histogram (bin the measure — see `mbql` binning).
+<!-- requires: boxplotDisplay -->
+- **Spread across several groups** → `boxplot` compares the groups' distributions directly.
+<!-- /requires -->
 - **Correlation between two measures** → `scatter` (a third measure → bubble size).
 - **Sequential additive contributions** (start → +/− steps → total) → `waterfall`.
 - **Stage drop-off in an ordered, cumulative funnel** → `funnel`.
@@ -35,7 +38,7 @@ Decide which relationship in the data matters most, then pick the chart. The sha
 - **Geographic** → `map`: region/choropleth (a region dimension + a measure), pin (lat + long), or grid/heat (coordinates + measure).
 - **Precise values, many columns, mixed types, or no chart fits** → `table`; `pivot` for a cross-tab of two dimensions; `object` for a single record's detail.
 
-Valid `display` values — the registered visualizations: `table`, `bar`, `line`, `area`, `row`, `pie`, `scalar`, `smartscalar`, `combo`, `pivot`, `funnel`, `map`, `scatter`, `waterfall`, `progress`, `gauge`, `object`, `sankey`. The API types `display` as a plain string and accepts any value — it renders an unknown one as nothing. (`boxplot` is registered only on **v59+** servers — older ones render it blank; use a `bar` histogram for distributions instead. `scalar` **is** the "Number" viz — `display: number` is a legacy serialization alias, not a registered visualization; use `scalar`. `list` exists but is hidden — don't pick it. `heading`/`text`/`link`/`iframe`/`action` are dashcard virtuals, not standalone cards — see references.) A typo like `bargraph`/`linechart` is accepted and renders blank — the most common "why is my chart blank" cause.
+Valid `display` values — the registered visualizations: `table`, `bar`, `line`, `area`, `row`, `pie`, `scalar`, `smartscalar`, `combo`, `pivot`, `funnel`, `map`, `scatter`, `waterfall`, `progress`, `gauge`, `object`, `sankey`. The API types `display` as a plain string and accepts any value — it renders an unknown one as nothing. (`scalar` **is** the "Number" viz — `display: number` is a legacy serialization alias, not a registered visualization; use `scalar`. `list` exists but is hidden — don't pick it. `heading`/`text`/`link`/`iframe`/`action` are dashcard virtuals, not standalone cards — see references.) A typo like `bargraph`/`linechart` is accepted and renders blank — the most common "why is my chart blank" cause.
 
 ## Step 2 — bind data columns and set options
 
@@ -53,7 +56,6 @@ Valid `display` values — the registered visualizations: `table`, `bar`, `line`
 | `row`                       | as bar; prefer for long/many category labels              | `graph.dimensions`, `graph.metrics`                        |
 | `scatter`                   | two numeric measures (correlation)                        | `graph.dimensions`, `graph.metrics` (`scatter.bubble` opt) |
 | `waterfall`                 | exactly 1 dimension + ≥1 measure; sequential              | `graph.dimensions` (1), `graph.metrics` (1)                |
-| `boxplot` _(v59+)_          | ≥3 cols, ≥2 dimensions, ≥1 measure                        | `graph.dimensions`, `graph.metrics`                        |
 | `pie`                       | ≥2 rows, ≥2 cols, ≥1 dimension + ≥1 measure; ≤~5 slices   | `pie.dimension`, `pie.metric`                              |
 | `funnel`                    | 2 columns (stage + value); ordered stages                 | `funnel.dimension`, `funnel.metric`                        |
 | `map` (region)              | a string/region dimension + a measure                     | `map.region`, `map.dimension`, `map.metric`                |
@@ -62,6 +64,12 @@ Valid `display` values — the registered visualizations: `table`, `bar`, `line`
 | `pivot`                     | ≥2 cols, all aggregated/breakout                          | — (`pivot_table.column_split` auto)                        |
 | `table` `object`            | anything (table is the universal fallback)                | — (always renders)                                         |
 
+<!-- requires: boxplotDisplay -->
+
+`boxplot` is registered too: ≥3 cols, ≥2 dimensions, ≥1 measure, on **unaggregated** rows; required keys `graph.dimensions`, `graph.metrics`.
+
+<!-- /requires -->
+
 ### The rule that trips everyone: settings name **output columns**, by name
 
 `graph.dimensions`, `graph.metrics`, `pie.dimension`, `pie.metric`, `scalar.field`, `funnel.metric`, `map.latitude_column`, `sankey.source`, … all take **output column-name strings** — the names the query _produces_, not field ids. A `count` aggregation outputs the column `count`; a breakout on a field outputs that field's name; a named aggregation outputs its `name`. These strings are **identical in the API form and the portable (git-sync) form** — no numeric-vs-name footgun here.

package/skill-data/visualization/references/settings.md

@@ -2,7 +2,7 @@
 
 ## Contents
 
-- [Cartesian — `bar`, `line`, `area`, `combo`, `scatter`, `waterfall`, `row`, `boxplot`](#cartesian--bar-line-area-combo-scatter-waterfall-row-boxplot) — shared keys (binding, stacking, goal/trend, data labels, axes, tooltip) plus `scatter`, `waterfall`, `row`, `boxplot` extras
+- [Cartesian — `bar`, `line`, `area`, `combo`, `scatter`, `waterfall`, `row`](#cartesian--bar-line-area-combo-scatter-waterfall-row) — shared keys (binding, stacking, goal/trend, data labels, axes, tooltip) plus `scatter`, `waterfall`, `row` extras
 - [Part-to-whole & single value — `pie`, `funnel`, `gauge`, `progress`, `scalar`, `smartscalar`](#part-to-whole--single-value--pie-funnel-gauge-progress-scalar-smartscalar)
 - [Tabular, geographic & flow — `table`, `pivot`, `object`, `map`, `sankey`](#tabular-geographic--flow--table-pivot-object-map-sankey) — includes `table.column_formatting` conditional formatting
 - [`column_settings` — per-column formatting](#column_settings--per-column-formatting) — number/date/currency, `view_as`, alignment, mini bars
@@ -18,7 +18,7 @@ All column-naming keys (`graph.dimensions`, `pie.dimension`, `table.columns[].na
 
 ---
 
-# Cartesian — `bar`, `line`, `area`, `combo`, `scatter`, `waterfall`, `row`, `boxplot`
+# Cartesian — `bar`, `line`, `area`, `combo`, `scatter`, `waterfall`, `row`
 
 `bar`/`line`/`area`/`combo` share an identical key set (combo just defaults the first series to a line, the rest to bars). `scatter`/`waterfall`/`row`/`boxplot` use a subset plus their own extras. Most allow up to 2 dimensions and unlimited metrics; **waterfall is 1 dimension + 1 metric**, boxplot allows 2 dimensions.
 
@@ -102,9 +102,9 @@ Use for sequential additive contributions (start → +/− steps → total). **U
 
 Horizontal bars — use when category labels are long or numerous. Here `graph.dimensions` is the y-axis (categories) and `graph.metrics` the x-axis (values). X-scale is `"ordinal"` only; no trend line, tooltip columns, or split panels. **Required:** `graph.dimensions`, `graph.metrics`.
 
-## boxplot
+<!-- requires: boxplotDisplay -->
 
-_Registered only on **v59+** servers — older ones render `display: boxplot` blank; use a `bar` histogram there._
+## boxplot
 
 Use for distribution/spread/outliers, especially across several groups. Needs **unaggregated** rows. **Use for:** ≥3 columns, ≥2 dimensions, ≥1 measure. **Required:** `graph.dimensions`, `graph.metrics`. X-scale is `"ordinal"`.
 
@@ -115,6 +115,8 @@ Use for distribution/spread/outliers, especially across several groups. Needs **
 | `boxplot.show_mean`        | boolean |                                                | `true`       |
 | `boxplot.show_values_mode` | string  | `"median"`, `"all"` (when `graph.show_values`) | `"median"`   |
 
+<!-- /requires -->
+
 ---
 
 # Part-to-whole & single value — `pie`, `funnel`, `gauge`, `progress`, `scalar`, `smartscalar`

package/skills/metabase-cli/SKILL.md

@@ -17,7 +17,7 @@ Before running any `mb` command, load the workflow content from the CLI:
 
 ```bash
 mb skills get core    # auth, flag conventions, every command group
-mb skills list        # everything available on the installed version
+mb skills list        # every bundled skill the profile's server can use (--unfiltered for all of them)
 ```
 
 **Doing a whole job, not one command?** If the user wants an outcome — "make sense of my data", "build a data model", "go from raw data to a dashboard", "answer questions about my data", "be my data analyst", "set up analytics for X" — load the guided end-to-end skill instead and let it drive:

package/dist/card-p2r_szeS.mjs

@@ -1,24 +0,0 @@
-import { t as defineCommandGroup } from "./group-DBM87IgC.mjs";
-//#region src/commands/card/index.ts
-var card_default = defineCommandGroup({
-	name: "card",
-	description: "Manage Metabase cards (questions, models, metrics)",
-	skills: [{
-		skill: "mbql",
-		purpose: "author the dataset_query"
-	}, {
-		skill: "visualization",
-		purpose: "choose display and visualization_settings"
-	}],
-	subCommands: {
-		list: () => import("./list-C2qxVTu7.mjs").then((mod) => mod.default),
-		get: () => import("./get-N5EUCDL0.mjs").then((mod) => mod.default),
-		query: () => import("./query-CpD46hL6.mjs").then((mod) => mod.default),
-		alerts: () => import("./alerts-Bp2vByJf.mjs").then((mod) => mod.default),
-		create: () => import("./create-BGy4L_yh.mjs").then((mod) => mod.default),
-		update: () => import("./update-MWaDj2VX.mjs").then((mod) => mod.default),
-		archive: () => import("./archive-lDnTIypd.mjs").then((mod) => mod.default)
-	}
-});
-//#endregion
-export { card_default as default };