@malloy-publisher/skills 0.1.0

package/skills/malloy-getting-started/SKILL.md

@@ -0,0 +1,57 @@
+---
+name: malloy-getting-started
+description: First steps for using a Malloy Publisher deployment through its MCP tools. Use when connecting to Publisher for the first time, when you do not yet know the available environments, packages, or models, or when a user asks what data they can explore. Covers verifying the server, discovering data with malloy_getContext, and running a first grounded query.
+---
+
+# Getting started with Malloy Publisher
+
+Goal: go from "connected" to a correct, grounded answer without guessing any names.
+
+## 0. Confirm the tools are reachable
+
+At minimum you need `malloy_getContext`, `malloy_executeQuery`, and `malloy_searchDocs`. Authoring a model also needs `malloy_compile` and `malloy_reloadPackage` (see section 4); an older Publisher may not serve those two.
+
+If none of the tools are there, either the server is not running or your client connected before it was. Start the server (`npx @malloy-publisher/server --port 4000`, or `bun run build && bun run start` from a clone) and wait until `curl -s http://localhost:4000/api/v0/status` reports `operationalState: serving`.
+
+If you started the server yourself in this session, the tools still will not appear: your tool list was fixed when you connected, and you cannot reconnect yourself. Tell the user the tools are missing for that reason and ask them to run `/mcp`, select `malloy`, and choose Reconnect. The panel offers `Authenticate` first and reports `Auth: not authenticated`; that is a red herring, the endpoint has no auth. Restarting Claude Code also works. Continue once the tools are there.
+
+When a user is present, do not route around it by calling the REST API with curl. It appears to work, so the user never learns their session is missing the tools, and you lose what they are for: grounded discovery instead of guessed names, `malloy_compile` instead of throwaway queries, and `malloy_reloadPackage` instead of a restart. Say the tools are missing and let the user fix it in five seconds. Running unattended, with nobody who can reconnect you, is different: there the REST API is the supported interface, not a workaround. Discovery, query, compile, and reload all have REST equivalents (`malloy_searchDocs` and `malloy_getContext`'s plain-English ranking do not; read the bundled skills for syntax and ground from model metadata instead); the running server serves the full spec at `http://localhost:4000/api-doc.yaml`, and AGENTS.md carries the endpoint map.
+
+## 1. Discover what exists (never guess names)
+
+`malloy_getContext` is progressive. Call it with as much as you know:
+
+- No arguments: the available environments, each with its package names.
+- `environmentName` only: the packages in that environment.
+- `environmentName` + `packageName`: that package's sources.
+- `environmentName` + `packageName` + `query` (plain English): the sources, views, named queries, and dimension/measure fields most relevant to the question.
+
+Use the names it returns exactly. Do not invent environments, packages, sources, or fields.
+
+## 2. Run the query
+
+Call `malloy_executeQuery` with the `environmentName`, `packageName`, and `modelPath` from the context results, plus either:
+
+- a named view or query: pass its `name` as `queryName` (with `sourceName` for a view), or
+- an ad-hoc query: pass Malloy code as `query`.
+
+The result is JSON. Charts and dashboards defined in the model render in the Publisher UI at http://localhost:4000.
+
+## 3. When you need Malloy syntax
+
+Use `malloy_searchDocs` for language questions (filters, aggregates, joins, nesting, renderers).
+
+## 4. What else you can do here
+
+Answering questions is the start, not the whole surface. When the user asks what is possible, say so rather than offering queries alone. Switch skills for the deeper work:
+
+- `malloy-modeling`: build or change a model. Validate the edit with `malloy_compile`, save it, then `malloy_reloadPackage` so the new sources and views run by name without restarting the server.
+- `malloy-analysis`: explore a package and answer data questions.
+- `malloy-html-data-apps`: build a data app, a hand-authored HTML page in the package's `public/` directory that Publisher serves, backed by the package's models and needing no build step.
+- `malloy-review`: check Malloy for correctness.
+
+## Contract
+
+- Ground every query in `malloy_getContext` results. If a name is not in the results, do not use it.
+- Start broad and narrow down: environments, then packages, then sources, then query.
+- Confirm the environment and package before running a query.

package/skills/malloy-gotchas-modeling/SKILL.md

@@ -0,0 +1,277 @@
+---
+name: malloy-gotchas-modeling
+description: Common Malloy modeling mistakes and how to avoid them. Read BEFORE writing source definitions, dimensions, measures, or joins. Covers reserved words, NULL checks, date functions, type casts, field management (extend except/accept/rename vs include public/internal/private), and query-based source gotchas.
+---
+
+# Modeling Gotchas
+
+> **Read this before writing Malloy code.** These patterns cause most modeling errors.
+
+> **Tool names** are written bare here - `get_context`, `execute_query`, `search_malloy_docs`. The exact prefixed name depends on the host surface; match each against the tools you actually have.
+
+## Reserved Words: Backtick Them
+
+**When in doubt, backtick it.** Unquoted reserved words cause cascading errors on unrelated lines.
+
+```malloy
+// WRONG                        // RIGHT
+dimension: d is Date::date      dimension: d is `Date`::date
+```
+
+Words most likely to appear as column names:
+```
+date, time, day, month, year, quarter, week, hour, minute, second,
+number, string, boolean, type, table, source, index, count, sum, avg, min, max,
+true, false, null, is, on, with, all, from, by, in, to, for, select, order_by,
+top, bottom, desc, asc, row, range, current, window, rank
+```
+
+- `number`: only the bare word needs backticking; `account_number` is fine
+- `source`: reserved; use a different alias like `traffic_source`
+
+## NULL Checks: `is not null`, NOT `!= null`
+
+```malloy
+// WRONG                             // RIGHT
+dimension: is_sold is sold_at != null   dimension: is_sold is sold_at is not null
+```
+
+## Date Functions vs Properties
+
+```malloy
+// WRONG: day_of_week is a function        // RIGHT
+dimension: dow is created_at.day_of_week   dimension: dow is day_of_week(created_at)
+```
+
+**Property access:** `.month`, `.year`, `.quarter`, `.day`, `::date`
+**Function call required:** `day_of_week()`, `week()`, `hour()`, `minute()`, `second()`
+
+## `.date` Is a Cast, Not a Truncation
+
+Calendar truncations are `.day`, `.week`, `.month`, `.quarter`, `.year` (plus `.hour`, `.minute`, `.second` for timestamps). `.date` is **not** among them: it's a **cast** (`::date`), not a truncation, so `created_at.date` does not compile. This bites twice: once at compile time, and again as a latent bad `#(doc)` comment that only a review pass catches ("truncated to date" is a doc smell; it should say "to day").
+
+```malloy
+// WRONG                          // RIGHT
+created_at.date                   created_at.day     // truncate to day
+                                   created_at::date   // cast to a date
+```
+
+## Interval Functions: Only `seconds` / `minutes` / `hours` / `days`
+
+`weeks()`, `months()`, `quarters()`, `years()` are **documented but don't work** in this build; only `seconds`, `minutes`, `hours`, `days` actually function. Compute in days and derive the larger unit: a *units conversion*, not a calendar-floored duration:
+
+```malloy
+// WRONG: weeks()/months() don't compile
+dimension: weeks_open is weeks(opened_at to closed_at)
+
+// RIGHT: measure in days, convert (documents that it's approximate)
+dimension: days_open  is days(opened_at to closed_at)
+dimension: weeks_open is days(opened_at to closed_at) / 7      // ≈ weeks
+dimension: months_open is days(opened_at to closed_at) / 30.44 // ≈ months
+```
+
+(Contrast: `search_malloy_docs` gets this right when asked narrowly; trust the docs on the supported units, not on the missing ones.)
+
+## Safe Division: Always `nullif`
+
+```malloy
+// WRONG              // RIGHT
+a / b                 a / nullif(b, 0)
+```
+
+## String Columns Need Casts for Aggregates
+
+```malloy
+// WRONG: "Can't use type string"     // RIGHT
+measure: avg_score is avg(score)      measure: avg_score is avg(score::number)
+```
+
+**Dirty columns: null the sentinel before casting.** `::number` is a strict cast, so a column that carries non-numeric sentinels (`'NA'`, `'N/A'`, `''`, `'-'`, `'null'`) compiles fine but fails at query time with `Could not convert string 'NA' to DOUBLE`. Strip the sentinel with `nullif` first, then cast (aggregates skip nulls):
+
+```malloy
+// WRONG: throws on 'NA' at query time   // RIGHT: nulls 'NA', then casts
+measure: s is avg(score::number)         measure: s is avg(nullif(score, 'NA')::number)
+```
+
+Chain `nullif` for multiple sentinels: `nullif(nullif(score, 'NA'), '')::number`. Sample the column's values first (`run: source -> { group_by: score; limit: 20 }`) to see which sentinels it uses.
+
+## Boolean Columns: No Quotes
+
+```malloy
+// WRONG                                     // RIGHT
+count() { where: complaint = 'true' }        count() { where: complaint = true }
+```
+
+Check schema: if `BOOL`, use `true`/`false`. If `STRING`, use `'true'`/`'false'`.
+
+## `greatest()` / `least()` Are Null-Poisoning
+
+Malloy's `greatest()` / `least()` return **NULL if *any* argument is null**, unlike Postgres `GREATEST`/`LEAST`, which ignore nulls. Porting a LookML/SQL expression verbatim is a silent parity bug: the number just goes null for any row with a missing input. Coalesce the result back to a non-null argument:
+
+```malloy
+// WRONG: one null input nulls the whole thing
+dimension: last_touch is greatest(email_at, call_at)
+
+// RIGHT: fall back so a null arg can't poison the result
+dimension: last_touch is greatest(email_at, call_at) ?? email_at ?? call_at
+```
+
+## No Scalar Median; Raw-SQL Aggregates Don't Compile
+
+**There is no scalar `median`, and `PERCENTILE_CONT` cannot be expressed as a measure in this build.** Every documented form for a custom SQL aggregate - `percentile_cont!(x, 0.5)`, `sql_number(...)`, `sql_number(...) { is_aggregate: true }`, and the `# is_aggregate` annotation - resolves as a **scalar** and fails with *"Cannot use a scalar field in a measure declaration."* The docs' own `avg_dist` example fails the same way. This is a deployed-runtime limitation, not a syntax error you can fix: **do not** burn cycles trying `!`, `sql_number`, or `is_aggregate` variations to get a median.
+
+```malloy
+// DOES NOT COMPILE in this build (all forms resolve as scalar):
+measure: median_x is percentile_cont!(x, 0.5)
+measure: median_x is sql_number("PERCENTILE_CONT(...) ...") { is_aggregate: true }
+```
+
+**Ship `avg` instead, or defer median with a documented gap** ("median deferred: no scalar median / runtime rejects raw-SQL aggregates"). Tell the user; don't silently substitute `avg` for a metric that was specified as median.
+
+## Field Management: `extend {}` vs `include {}` Don't Compose
+
+Malloy has two field-management mechanisms for base sources. **`include {}` is the curated default; `extend { except / accept / rename }` is the fallback when a `rename:` is unavoidable.** They have different capabilities and **do not combine**.
+
+| Mechanism | Where it lives | Keywords | Compatible with `rename:`? | Experimental flag? |
+|---|---|---|---|---|
+| Access modifiers (default) | `include {}` | `public:` / `internal:` / `private:` | **No** | Yes (`##! experimental.access_modifiers`) |
+| Field management (fallback) | `extend {}` | `accept:` / `except:` / `rename:` | Yes (same block) | No |
+
+### Default: `include {}` for documented, curated base sources
+
+Use `include {}` whenever the source doesn't need a `rename:`. It's the only way to attach `#(doc)` tags to raw columns, and it's the canonical way to hide empty/garbage/duplicate columns (`internal:`) and sensitive ones (`private:`). See `skill:malloy-model` § Access Modifiers.
+
+```malloy
+##! experimental.access_modifiers
+source: orders is conn.table('orders') include {
+  public:
+    #(doc) Order identifier
+    order_id
+
+    #(doc) Customer who placed the order
+    user_id
+
+  internal:
+    raw_payload_json    // empty after JSON extraction
+    legacy_status_code  // superseded by status_code
+}
+```
+
+### When `rename:` is unavoidable: fall back to `extend {}`
+
+`include {}` does not compose with `rename:`. The combination errors with `Can't find field 'X' to set access modifier` because `rename:` runs first and leaves no `X` for `include` to attach a modifier to. There's also a collision inside `include {}` itself: a measure cannot share a name with a raw column, even one tagged `internal:` (`Cannot redefine 'X'`), and the natural fix for that is `rename:`, which then triggers the first error.
+
+When a rename is genuinely required (most often during `conn.sql()` to `conn.table()` migration where a SQL alias matches a measure name that's already in heavy use downstream), drop `include {}` and curate the source with `extend { except: ... }` + `rename:` instead. You forfeit `#(doc)` on raw columns and the `public/internal/private` tiers, but keep column gating and the rename.
+
+```malloy
+// RIGHT: rename is required to free `revenue` for the measure
+extend {
+  except: legacy_status_code   // hide garbage column without include {}
+  rename: raw_revenue is revenue
+  measure: revenue is raw_revenue.sum()
+}
+```
+
+If you can rename the measure or split the source instead, prefer that: it preserves `include {}` and the curated surface.
+
+### `extend {}` clauses (reference)
+
+- **`accept:`**: allow-list, keep only the named columns
+- **`except:`**: deny-list, drop the named columns; keep everything else (mutually exclusive with `accept:`)
+- **`rename:`**: alias a raw column to free up its original name for a measure or dimension
+
+### Migrating `conn.sql()` to `conn.table()` + Malloy clauses
+
+The biggest reason teams reach for `conn.sql()` is column gating, aliasing, and per-row derivation in one place. All three have native equivalents:
+
+1. **Verify the schema**: `run: <source> -> { select: *; limit: 1 }` to discover all columns. Anything in the table but not in the SQL's `SELECT` was being intentionally hidden, so preserve that gating.
+2. Switch to `conn.table('…')`.
+3. Hidden columns: preferably `include { internal: ... }` (lets you also `#(doc)` the public columns). If a `rename:` is also needed in the same source, fall back to `extend { except: ... }`.
+4. SQL aliases: `extend { rename: ... }` (forces the fallback path, since `rename:` and `include {}` don't compose). If the alias was to free up a name for a measure, use `rename: raw_X is X`, then `measure: X is raw_X.sum()`.
+5. SQL derivations: `dimension:` definitions in `extend {}`.
+6. SQL `WHERE`: source-level `where:`.
+
+## Cannot Redefine Query-Based Source Columns
+
+Columns from `table -> { group_by, aggregate }` or `conn.sql()` already exist. You cannot re-declare them.
+
+```malloy
+// WRONG: "Cannot redefine 'user_id'"
+source: facts is conn.table('t') -> { group_by: user_id, aggregate: total is sum(amt) }
+  extend { dimension: user_id is user_id }
+// RIGHT: add only NEW derived dimensions
+source: facts is conn.table('t') -> { group_by: user_id, aggregate: total is sum(amt) }
+  extend { dimension: is_high_value is total > 1000 }
+```
+
+To add `#(doc)` tags to existing query columns, use `include {}` between the query and extend.
+
+## Never Use `conn.sql()` When Malloy Has a Native Pattern
+
+```malloy
+// WRONG: raw SQL for pre-aggregation
+source: facts is conn.sql("""SELECT user_id, SUM(amount) AS total FROM orders GROUP BY user_id""")
+// RIGHT: Malloy query-based source
+source: facts is conn.table('orders') -> { group_by: user_id, aggregate: total is sum(amount) }
+```
+
+**Mandatory: call `search_malloy_docs` before reaching for `conn.sql()`.** Don't argue from intuition. Most patterns that look SQL-only have a Malloy equivalent, including the ones reviewers historically said couldn't be expressed.
+
+| Looks like it needs SQL | Malloy equivalent |
+|---|---|
+| Multi-CTE pipeline | Stacked query-based sources: `source: a is t -> {...}`; `source: b is a -> {...}`; `source: c is b -> {...}` |
+| UNNEST / array column access | `array_column.each.field`: arrays auto-join as nested tables ([data types docs](https://docs.malloydata.dev/documentation/language/datatypes#array-access)) |
+| PIVOT (conditional aggregation) | Filtered aggregates: `aggregate: a is x.sum() { where: cat = 'a' }, b is x.sum() { where: cat = 'b' }` |
+| Window functions (any frame, including custom) | `calculate:` with `sum_cumulative`, `lag`, `lead`, `rank`, `row_number`, `avg_moving`, `first_value`, `last_value`: supports `partition_by:` and `order_by:` ([window functions docs](https://docs.malloydata.dev/documentation/language/functions#window-functions)) |
+| `ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING` | `sum_cumulative(x) - x` (cumulative-including-current minus current = cumulative-excluding-current) |
+| `WHERE date = (SELECT max(date) FROM …)` (latest snapshot) | `join_cross` to a one-row aggregate source, then filter on the joined `max_date` field |
+| Multi-key joins | `join_one: x is target on a = x.a and b = x.b and c = x.c` |
+| `greatest()` / `least()` / `CASE` chains | All native: `greatest(a, b, c)`, `least(a, b)`, `pick 'x' when cond else 'y'` |
+| Dialect-specific scalar functions | `function_name!return_type(args)`: Malloy's raw-SQL function escape (no `conn.sql()` block needed) |
+
+**Genuinely valid `conn.sql()` candidates (rare):**
+
+- SQL features Malloy explicitly doesn't model (e.g., DML/DDL, specific `MERGE` patterns)
+- Multi-stage transformations where every CTE has 3+ joins to different tables AND the result is consumed by multiple downstream sources, but in this case an intermediate table in the data warehouse is usually still better than `conn.sql()`
+
+**Never use `conn.sql()` for:** simple column selection or renaming, `WHERE` filters, two-table joins, column type casts, latest-snapshot patterns, conditional aggregation, or window functions of any kind.
+
+If a project's standards file specifies a stricter policy (e.g., a `search_malloy_docs` rationale comment requirement above every `conn.sql()` block), defer to that.
+
+## Duplicate Rows: Check Before Building Measures
+
+```malloy
+run: source -> { group_by: pk_field, aggregate: n is count(), having: n > 1, limit: 10 }
+```
+
+Symptoms: `sum()` returns astronomical values. Causes: event tables, batch retries, merged sources.
+
+## `except:` Removes Fields From Namespace Entirely
+
+`except:` in `include {}` completely removes fields: dimensions and measures cannot reference excluded fields. Use `internal:` instead when derived dimensions need the raw column.
+
+```malloy
+// WRONG: dimension references excluded field
+source: x is conn.table('t')
+include { except: raw_date }
+extend { dimension: order_date is raw_date::date }  // ERROR! raw_date is gone
+
+// RIGHT: internal fields are still available in extend
+source: x is conn.table('t')
+include { internal: raw_date }
+extend { dimension: order_date is raw_date::date }  // Works
+```
+
+## Source Order: Define Joined Tables First
+
+Malloy compiles top-to-bottom. Define lookup/dimension tables before the source that joins them, or use `import` statements in multi-file projects.
+
+## MUST Search Docs Before Using Unfamiliar Patterns
+
+Call `search_malloy_docs` BEFORE first use of any of these. Don't guess the syntax:
+- `pick` expressions
+- Window functions (`calculate`)
+- `percentile` or statistical functions: but see the hard limit above, raw-SQL aggregates (`sql_number` / `is_aggregate` / `percentile_cont!`) do **not** compile as measures in this build; there is no scalar median
+- Time interval functions (`days()`, `seconds()`): only `seconds`/`minutes`/`hours`/`days` exist (see above)
+- Query-based sources (`from()`)
+- `!` operator / `sql_number()`

package/skills/malloy-gotchas-queries/SKILL.md

@@ -0,0 +1,132 @@
+---
+name: malloy-gotchas-queries
+description: Common Malloy query and view mistakes. Read BEFORE writing views, queries, or notebooks. Covers chart constraints, aggregate filters, joined field aliasing, method syntax, and time truncation vs extraction.
+---
+
+# Query & View Gotchas
+
+> **Read this before writing views or queries.** These patterns cause most query errors.
+
+## Charts: ONE Aggregate Per View
+
+Charts render only the **first** aggregate. Use exactly one aggregate per `# bar_chart` / `# line_chart` view.
+
+```malloy
+// WRONG: revenue is ignored
+# bar_chart
+view: x is { group_by: status, aggregate: order_count, revenue }
+// RIGHT: single aggregate
+# bar_chart
+view: x is { group_by: status, aggregate: revenue }
+```
+
+For multiple metrics: nest separate chart views in a `# dashboard`, or use `y=['revenue','cost']` for multi-measure series.
+
+## Joined Fields in `order_by`: Must Alias First
+
+```malloy
+// WRONG: compile error
+view: x is { group_by: races.season_year, aggregate: pts, order_by: races.season_year }
+// RIGHT: alias then reference
+view: x is { group_by: yr is races.season_year, aggregate: pts, order_by: yr }
+```
+
+Any time you `group_by` a joined field, create an alias and use it in `order_by`.
+
+## `having:` vs `where:`: Aggregate Filters
+
+```malloy
+// WRONG: "Aggregate expressions not allowed in where"
+view: x is { group_by: cat, aggregate: n is count(), where: n > 10 }
+// RIGHT
+view: x is { group_by: cat, aggregate: n is count(), having: n > 10 }
+```
+
+- `where:` filters rows BEFORE aggregation (dimensions/raw columns)
+- `having:` filters AFTER aggregation (measures)
+
+## Aggregating Joined Fields: Method Syntax
+
+```malloy
+// WRONG: compile error: "Join path is required for this calculation; use 'inventory_items.item_cost.sum()'"
+measure: cogs is sum(inventory_items.item_cost)
+// RIGHT: method syntax
+measure: cogs is inventory_items.item_cost.sum()
+```
+
+`sum`, `avg`, `min`, and `max` over a dotted joined path all produce that compile error; the diagnostic message even tells you the exact fix. Don't worry about catching this in code review; the compiler does it for you.
+
+**Exception: `count(joined.field)` is correct, not a bug.** `count(joined.field)` is the **canonical Malloy idiom** for distinct-count through a join. Keep it as-is even when nearby `sum`/`avg`/`min`/`max` calls have to use method syntax. The closest method-syntax form `joined.count()` counts *rows* in the joined source (different semantics, differs from the distinct count when the joined field has duplicates within the joined table). The Malloy docs example `joined.count(field)` does NOT compile against current Malloy (error: `Expression illegal inside path.count()`); it only works for double-nested paths like `aircraft.count(aircraft_models.code)`.
+
+## Chart Annotation Placement
+
+Place `# bar_chart` / `# line_chart` on the **nested view definition**, not on `nest:` itself. Putting it on `nest:` causes "not a repeated record" errors.
+
+## DRY: Define in Source, Reference in View
+
+```malloy
+// WRONG: inline in view
+view: summary is { aggregate: revenue is sum(total) }
+// RIGHT: reference existing measure
+view: summary is { aggregate: revenue }
+```
+
+## Time Truncation vs Extraction
+
+| Syntax | What it does | Returns |
+|--------|--------------|---------|
+| `ts.month` | Truncates to start of month | Timestamp (`@2024-03-01`) |
+| `month(ts)` | Extracts month number | Integer (1-12) |
+| `ts.year` | Truncates to start of year | Timestamp (`@2024-01-01`) |
+| `year(ts)` | Extracts year number | Integer (2024) |
+
+Use `.month` for time series charts (proper date ordering). Use `month()` for cross-year comparison.
+
+**Year integers render with commas.** `year(ts)` displays as `2,018`. Tag with `# number=id` to suppress commas. Same for zip codes, IDs.
+
+## `?` Alternation: Use Commas to Combine Filters
+
+The `?` operator is Malloy's **alternation operator**: a shorthand for "match any of these values." `party ? 'Democrat' | 'Republican'` means `party = 'Democrat' OR party = 'Republican'`. The `|` separates the alternatives.
+
+When combining an alternation filter with other filters, **use a comma**:
+
+```malloy
+// CANONICAL: commas separate independent filter conditions
+where: is_us = true, party ? 'Democrat' | 'Republican'
+```
+
+`and` works in some arrangements (when the alternation is the second operand) but produces a confusing `'logical operator' Can't use type string` compile error when the alternation comes first. The comma form is unambiguous in every position, so just use it.
+
+## Query Clauses Are Newline-Separated
+
+Do not use trailing commas between query clauses. Each clause goes on its own line.
+
+```malloy
+// WRONG: trailing comma before limit
+run: source -> { group_by: status, aggregate: n is count(), limit: 10 }
+// RIGHT: newline-separated
+run: source -> {
+  group_by: status
+  aggregate: n is count()
+  limit: 10
+}
+```
+
+Clauses: `group_by:`, `aggregate:`, `nest:`, `order_by:`, `limit:`, `where:`, `having:`, `select:`, `calculate:`
+
+## Fields Within a Clause: Commas or Newlines, Never Semicolons
+
+Semicolons are not a separator anywhere in Malloy. Multiple fields under one `aggregate:` / `group_by:` are separated by commas (inline) or newlines (one per line); a `;` fails with `no viable alternative at input '<next-field>'` pointing at the field right after it.
+
+```malloy
+// WRONG: semicolons between fields
+run: schools -> { aggregate: total is count(); charters is count() { where: is_charter } }
+// RIGHT: commas inline...
+run: schools -> { aggregate: total is count(), charters is count() { where: is_charter } }
+// ...or newlines
+run: schools -> {
+  aggregate:
+    total is count()
+    charters is count() { where: is_charter }
+}
+```

package/skills/malloy-gotchas-rendering/SKILL.md

@@ -0,0 +1,124 @@
+---
+name: malloy-gotchas-rendering
+description: Common Malloy renderer annotation mistakes. Read BEFORE adding chart annotations, formatting tags, or building dashboards. Covers tag syntax, scale rules, sparkline setup, and big_value patterns.
+---
+
+# Rendering Gotchas
+
+> **Read this before adding renderer annotations.** These patterns cause most rendering issues.
+
+## One Tag Per Line
+
+Each `#` annotation must be on its own line directly above the field. Never combine tags on one line.
+
+```malloy
+// WRONG, will not work
+# label="Revenue" # currency
+revenue
+
+// RIGHT
+# label="Revenue"
+# currency
+revenue
+```
+
+## No Fixed Scale on Measures
+
+Use `# currency` (no scale) on measure definitions. The same measure renders at many granularities: `usd0m` turns $500 into `$0.0M`.
+
+```malloy
+// WRONG on a measure definition
+# currency=usd0m
+measure: revenue is sum(total)
+
+// RIGHT, no scale on measure
+# currency
+measure: revenue is sum(total)
+```
+
+Add scale (e.g., `# currency=usd0m`) only in views after confirming value ranges with queries.
+
+## `# big_value` Needs `# label` on Each Measure
+
+```malloy
+# big_value
+view: summary is {
+  aggregate:
+    # label="Revenue"
+    # currency
+    revenue
+
+    # label="Orders"
+    # number=auto
+    order_count
+}
+```
+
+Without `# label`, big_value cards show raw field names which are often unclear.
+
+## Sparkline Setup
+
+Sparklines in `# big_value` require TWO things: a `# hidden` nested view AND a `.sparkline=` reference.
+
+```malloy
+# big_value { sparkline=trend }
+view: revenue_kpi is {
+  aggregate:
+    # label="Revenue"
+    # currency
+    revenue
+  nest:
+    # line_chart { size=spark }
+    # hidden
+    trend is { group_by: order_date, aggregate: revenue, order_by: order_date }
+}
+```
+
+If the sparkline doesn't show: check that `# hidden` is on the nested view AND the view name matches `.sparkline=`.
+
+## Comparison Deltas
+
+```malloy
+# big_value { comparison_field=prior_month comparison_label="vs Last Month" }
+view: rev_delta is {
+  aggregate:
+    # label="Revenue"
+    # currency
+    revenue
+    # hidden
+    prior_month
+}
+```
+
+Use `down_is_good=true` for metrics where decrease is positive (churn, defects).
+
+## `# dashboard` Layout
+
+In a `# dashboard` view, fields render by role: `group_by` -> a repeating row header, `aggregate` measures -> KPI cards, each `nest:` -> a tile. Put each tile tag on its own line above the nested view. A lone renderer tag also works on the `nest:` line, but a tile usually carries several tags (`# break`, `# colspan`, `# subtitle`, then `# bar_chart`), and only the own-line form keeps each tag on its own line.
+
+```malloy
+// RIGHT: tag above the nested view; the measure auto-renders as a card
+# dashboard
+view: overview is {
+  group_by: category
+  # currency
+  aggregate: avg_retail is retail_price.avg()
+  nest:
+    # bar_chart
+    by_brand is { group_by: brand, aggregate: avg_retail is retail_price.avg(), limit: 10 }
+}
+```
+
+- **`# colspan` only works in columns mode.** Set `# dashboard { columns=N }` first; in flex mode `# colspan` is ignored. `# break` (new row) works in both modes.
+- **`gap` is spacing, not a mode.** `columns=N` enters columns mode; `# dashboard { gap=24 }` only changes tile spacing.
+- **`# dashboard` needs a row-producing view (no effect on a scalar).** A flat view of top-level `aggregate:` measures still renders KPI cards; add `nest:` for chart and table tiles.
+- **Use `# colspan`, not the old `# span`.** Tile styling comes from the instance theme.
+
+## Theming
+
+- **Per-chart theme keys are nested, not flat.** In Publisher, `# theme.palette.tableHeader.dark = "#94a3b8"` works; a flat `# theme.tableHeaderColor = "#94a3b8"` is silently dropped. Publisher's annotation reader only understands the structured `palette.*` / `font.*` form (the same vocabulary as the config), not the renderer's flat `MalloyExplicitTheme` key names.
+- **A per-chart annotation OVERRIDES the instance theme.** Precedence, highest to lowest, per key: `# theme.*` (view), then `## theme.*` (model), then the instance theme (config / Settings, then Theme editor), then built-in defaults. A `# theme.*` view tag beats a `## theme.*` model default, and both beat the instance theme for the keys they set. This is the opposite of a bare `@malloydata/render` embed, where the embedder wins.
+- **Only seven palette keys take light/dark.** `background`, `tableHeader`, `tableHeaderBackground`, `tableBody`, `tile`, `tileTitle`, and `mapColor` each accept a `.light` and/or `.dark` variant. `palette.series`, `font.family`, and `font.size` are single values shared across modes; a `.light`/`.dark` on them does nothing.
+- **`# theme.palette.mapColor.{light,dark}` recolors choropleths only.** It sets the saturated end of the `# shape_map` / `# segment_map` gradient (per mode). Rect-mark heatmaps keep their built-in scheme.
+- **`defaultMode` and `allowUserToggle` are instance-only.** No per-chart annotation controls the light/dark default or the toggle lock; set them in the config `theme` block or the editor.
+- **Environment-level theming is not applied yet.** Only the instance theme and the `# theme.*` / `## theme.*` per-chart annotations take effect today.

package/skills/malloy-html-data-app-embedding/SKILL.md

@@ -0,0 +1,39 @@
+---
+name: malloy-html-data-app-embedding
+description: Embed an in-package HTML data app into a host page or another application, including auto-sizing and auth. Read when embedding a Publisher page via Publisher.embed.
+---
+
+# Embedding an HTML Data App
+
+> `Publisher.embed(selector, { src })` drops a package page into a host page as a sandboxed, auto-resizing iframe. Same-origin embeds authenticate with the browser's cookies; cross-origin embeds need a signed token.
+
+## The host-page pattern
+
+```html
+<script src="https://your-publisher/sdk/publisher.js"></script>
+<div id="dashboard"></div>
+<script>
+  const handle = Publisher.embed("#dashboard", {
+    src: "https://your-publisher/environments/demo/packages/sales/index.html",
+  });
+  // handle.destroy() removes the iframe and detaches its listeners.
+</script>
+```
+
+`embed(selector, options)` returns `{ iframe, destroy() }`. Options: `src` (required), `token` (a signed token for cross-origin auth, appended as `embed_token`), `height` (omit to auto-size; a number is treated as pixels), and `allow` (the iframe permissions policy).
+
+## Sizing and the resize contract
+
+Omit `height` and the frame auto-sizes. The embedded page measures its real content height and posts a `publisher:resize` message to the host, which resizes the iframe and accepts that message only from the iframe it created. You write none of this; it ships in `/sdk/publisher.js`, so the embedded page only has to load that script.
+
+Do not rely on `body { min-height: 100vh }` to drive the frame height. The runtime deliberately measures the content's bottom edge, not the viewport, to avoid a grow-forever loop.
+
+## Auth
+
+- Same-origin or same-tenant: pass no token. The browser's cookies authenticate the iframe.
+- Cross-origin: mint a short-lived signed token server-side and pass it as `options.token`. The runtime appends it to the iframe URL as `embed_token`; the embedded page must read it (from `location.search`) and call `Publisher.setToken(token)`. Because it rides in the URL, it can land in browser history, Referer headers, and server logs, so keep it short-lived and scoped to that one embed, and never put a long-lived or admin token in client HTML.
+
+## Guardrails (v1)
+
+- The iframe is sandboxed (`allow-scripts allow-same-origin allow-forms`). Design for that: no top-level navigation, no popups.
+- Embedded author JavaScript runs with the viewing user's data authority, so treat everything under `public/` as strictly first-party code: do not load untrusted third-party scripts, and do not move query results off to other hosts. Tighter per-embed isolation is planned.