@malloy-publisher/skills 0.1.2 → 0.1.3

package/package.json

@@ -1,7 +1,7 @@
 {
    "name": "@malloy-publisher/skills",
    "description": "Agent skills for Malloy and the Malloy Publisher",
-   "version": "0.1.2",
+   "version": "0.1.3",
    "license": "MIT",
    "type": "module",
    "main": "dist/index.js",

package/skills/malloy-analysis/SKILL.md

@@ -36,6 +36,12 @@ A name is a pointer, not confirmation. A field, source, or view name you saw in
 
 Write Malloy using only the model's names. Load `skill:malloy-queries` for syntax (aggregates vs dimensions, joins and field paths, dates, `where:` vs `having:`, counting) and `skill:malloy-gotchas-queries` to avoid the common compile errors. If a model `view:` already matches, run it directly rather than rewriting it.
 
+**Check these three before your first `execute_query`** - they account for most first-attempt compile failures, and they are the ones a SQL habit gets wrong:
+
+- **Counting.** `count(field)` is already the *distinct* count of that field. Malloy has no `count(distinct field)`; it is a parse error, not a deprecation.
+- **Separators.** Within a clause, fields are separated by commas or newlines, never `;`. A semicolon fails with `no viable alternative at input '<next-field>'`.
+- **Join paths.** A dotted path like `carriers.name` resolves only if the source declares that join. Confirm the join name and the field under it in a `get_context` result instead of inferring either from a table name.
+
 If you define a calculated field that is not already in the model, treat it carefully: ad-hoc definitions are a common source of subtle errors.
 
 - Announce it: tell the user you are adding an ad-hoc field, what it computes, and why the model does not already provide it.
@@ -54,7 +60,7 @@ Your first result is a draft, not an answer. The difference between a useful ana
 - **Ground it.** Before interpreting any result, query and state the dataset scope: the time range (`min`/`max` of the primary date dimension) and the row or entity count. Every number is meaningless without it.
 - **Ask "what would make this wrong?"** then run the query that would expose that problem. A plausible-looking wrong answer is the most dangerous kind.
 - **Check the common failure modes:**
-  - Fan-out / double-counting: if you joined across grain, compare `count()` to `count(distinct key)`. A large gap means duplication is inflating the aggregates.
+  - Fan-out / double-counting: if you joined across grain, compare `count()` to `count(key)` - in Malloy `count(field)` is already the distinct count. A large gap means duplication is inflating the aggregates.
   - Broken filters: a quick count confirms a filter narrowed the data as expected. Watch case, spelling, and date-format mismatches; a filter that matches nothing still returns a result, just the wrong one.
   - Null-driven loss: `count() - count(the_field)` shows how many rows a key field drops.
   - Parts that do not sum to the whole: if you split a total into categories, confirm they add up.

package/skills/malloy-gotchas-queries/SKILL.md

@@ -79,6 +79,40 @@ Two separate rules produce those errors:
 
 **Exception: `count(joined.field)` is correct, not a bug.** `count(joined.field)` is the **canonical Malloy idiom** for distinct-count through a join. Keep it as-is even when nearby `sum`/`avg`/`min`/`max` calls have to use method syntax. The closest method-syntax form `joined.count()` counts *rows* in the joined source (different semantics, differs from the distinct count when the joined field has duplicates within the joined table). The Malloy docs example `joined.count(field)` does NOT compile against current Malloy (error: `Expression illegal inside path.count()`); it only works for double-nested paths like `aircraft.count(aircraft_models.code)`.
 
+## `sum`/`avg` Need a Numeric Field
+
+```malloy
+// WRONG: "Can't use type string" - status is a string column
+aggregate: avg_status is avg(status)
+// RIGHT: aggregate a numeric field; count a string one
+aggregate: avg_price is avg(price), statuses is count(status)
+```
+
+Check the field's type in the `get_context` result before aggregating it. A name that reads numeric (`order_number`, `zip`, `account_id`) is very often typed string.
+
+## Dotted Paths Must Name a Declared Join
+
+```malloy
+// WRONG: the source declares the join as `carrier`, so this fails with
+//   "'carriers.name' is not a source or join"
+run: flights -> { group_by: carriers.name }
+// RIGHT: use the join name the source actually declares
+run: flights -> { group_by: carrier.nickname }
+```
+
+A dotted path resolves only against a join declared on the source you are running. Confirm both the join name and the field under it in a `get_context` result; do not infer either from a table name or a plural/singular guess.
+
+## `order_by:` Can Only Name an Output Column
+
+```malloy
+// WRONG: "Unknown field total in output space" - total is never emitted
+run: orders -> { group_by: state, aggregate: revenue is sum(total), order_by: total }
+// RIGHT: order by a column the query actually outputs
+run: orders -> { group_by: state, aggregate: revenue is sum(total), order_by: revenue }
+```
+
+`order_by:` resolves against the query's *output* columns, not the source's fields. To order by something, `group_by` or `aggregate` it first - and if it comes through a join, alias it (see above).
+
 ## Chart Annotation Placement
 
 Place `# bar_chart` / `# line_chart` on the **nested view definition**, not on `nest:` itself. Putting it on `nest:` causes "not a repeated record" errors.

package/skills/malloy-materialization/SKILL.md

@@ -30,13 +30,17 @@ Materialize an expensive source once so queries read a **pre-built warehouse tab
    ```jsonc
    {
      "name": "my-package",
-     "scope": "package",  // default; "version" = each published version owns its own tables
-     "materialization": { "freshness": { "window": "24h", "fallback": "live" } }
+     "materialization": {
+       "scope": "package",  // default; "version" = each published version owns its own tables
+       "freshness": { "window": "24h", "fallback": "live" },
+       "queryMetadata": { "team": "finance" }  // tags the build's backend statements
+     }
    }
    ```
    Enforced at publish (strict), on edits (strict), at load (warn, still serves), and by the scheduler (an offending package is skipped):
-   - **`scope`**: `package` (default; artifacts reused across published versions) or `version` (each artifact owned by one version). Package-level only; there is no per-source scope.
+   - **`scope`**: `package` (default; artifacts reused across published versions) or `version` (each artifact owned by one version). Package-level only; there is no per-source scope. A root-level `scope` is the deprecated home and still works, with a warning; declaring both homes with different values is rejected.
    - **`materialization.freshness`** (`window` + `fallback` of `live`/`stale_ok`/`fail`) is the objective a **hosted control plane** enforces by refreshing the table to meet it (`fallback: "live"` serves live compute while stale/absent). A **standalone** Publisher does **not** act on `freshness` for refresh - see **Building and refreshing**.
+   - **`materialization.queryMetadata`** is a bag of string properties attached to every statement the build issues, for the backend's own cost attribution (Snowflake `QUERY_TAG`, BigQuery job labels, a leading SQL comment elsewhere). Overridable per source with `#@ persist queryMetadata.<name>="<value>"`. Observability only: it never changes what gets built. See `docs/query-metadata.md`.
    - **`materialization.schedule`** is a 5-field UTC cron (`min hour dom mon dow`; `L`/`W`/`#`/`?` rejected). It **requires `scope: "version"`** and is **mutually exclusive with `freshness`**. This is how a standalone Publisher refreshes on a cadence.
 
 4. **Reads vs writes.** The persist source can *read* any dataset the connection can read; the persist *target* (`name=`'s dataset) must be a dataset the connection can **write** (typically a scratch dataset).

package/skills/malloy-materialization-tuning/SKILL.md

@@ -93,7 +93,7 @@ Frame each recommendation with the evidence from Step 2 (the run IDs, timings, b
 
   `set` also sets `scope: version` (a schedule requires it); the server rejects an invalid cron or an illegal scope/freshness combination, so a rejection means the change was unsafe.
 
-- **Change the scope:** `scope` is declared at the root of the package's `publisher.json` (`"scope": "package"` or `"version"`). Edit it there, then reload/republish the package. Remember a schedule pins scope to `version`, so clear the schedule first if moving to `package`.
+- **Change the scope:** `scope` is declared in the package's `publisher.json` under `materialization` (`"scope": "package"` or `"version"`; the root-level form is deprecated but still read). Edit it there, then reload/republish the package. Remember a schedule pins scope to `version`, so clear the schedule first if moving to `package`.
 
 After applying, re-run Step 2 on the next few builds to confirm the change did what you predicted (more reuse, shorter builds, or a table that is actually read).