@malloy-publisher/skills 0.1.0

package/skills/malloy-lookml-review/reference/_concepts.md

@@ -0,0 +1,42 @@
+# LookML → Malloy Concept Mapping
+
+Reference table for translating LookML constructs to Malloy. Referenced by multiple reference files.
+
+| LookML | Malloy | Notes |
+|--------|--------|-------|
+| `view:` | `source:` (base source file) | One source per physical table |
+| `explore:` | `source:` (source file with joins) | One source per analytical domain |
+| `dimension:` | `dimension:` | Direct mapping |
+| `dimension_group: { type: time }` | `.month`, `.year`, `::date` (native) | Malloy handles time natively; no explicit timeframe list needed |
+| `dimension: { type: yesno }` | `dimension: x is condition` | Boolean expression |
+| `measure: { type: count }` | `count()` | Always distinct in Malloy |
+| `measure: { type: count_distinct }` | `count(field)` | Direct mapping |
+| `measure: { type: sum }` | `sum(field)` | Direct mapping |
+| `measure: { type: average }` | `avg(field)` | Direct mapping |
+| `measure: { type: number }` | Derived measure expression | Usually a ratio; use `nullif()` for division |
+| `measure: { filters: [...] }` | `measure { where: condition }` | Filtered aggregate |
+| `primary_key: yes` | `primary_key: field_name` | Direct mapping |
+| `hidden: yes` | `# hidden` tag (cosmetic) | Classify reason first; see `curate-visibility.md` |
+| `fields` exclusion (explore/join) | `internal:` (with access modifiers) | Structurally excluded; `internal:` candidate |
+| `required_access_grants` | `private:` (with access modifiers) | Security-restricted; `private:` candidate |
+| `description:` | `#(doc)` tag | Direct mapping |
+| `label:` (simple rename) | `internal:` old + `dimension: new_name is old_name` | Never use `rename:` |
+| `label:` (complex) | `# label="Display Name"` | When name differs from identifier |
+| `sql_table_name:` | `conn.table('schema.table')` | Use the connection name from the model definition if available |
+| `join: { relationship: many_to_one }` | `join_one:` | Direct mapping |
+| `join: { relationship: one_to_one }` | `join_one:` | Direct mapping |
+| `join: { relationship: one_to_many }` | `join_many:` | Direct mapping |
+| `join: { relationship: many_to_many }` | `join_cross:` | Direct mapping |
+| `sql_on: ${a.field} = ${b.field}` | `on a_field = b.b_field` | Translate `${}` references |
+| `CASE WHEN ... END` (in SQL) | `pick ... when ... else` | Direct syntax translation |
+| `COALESCE(a, b)` | `a ?? b` | Direct mapping |
+| `IFNULL(a, b)` | `a ?? b` | Direct mapping |
+| `${TABLE}.field` | `field` (direct column reference) | Malloy references columns directly |
+| `${view_name.field}` | `view_name.field` (join path) | In join conditions and cross-source refs |
+| `+view:` (refinement) | User decides: consolidate or `extend` | Malloy `extend` serves the same purpose |
+| `derived_table: { sql: ... }` (perf-only) | Use base table directly | PDT optimization is Looker-specific |
+| `derived_table: { sql: ... }` (transformation) | Flag for user | Recommend base table + dims or upstream dbt |
+| `derived_table: { explore_source: ... }` (NDT) | `from(source -> { group_by:, aggregate: }) extend { }` | Computed source pattern |
+| `value_format: "$#,##0.00"` | `# currency` | Map to Malloy render tags |
+| `value_format: "0.00%"` | `# percent` | Map to Malloy render tags |
+| `value_format_name: decimal_2` | `# number="0.00"` | Map to Malloy render tags |

package/skills/malloy-lookml-review/reference/build-derived-tables.md

@@ -0,0 +1,110 @@
+# LookML Derived Table Conversion (Step 5)
+
+> Classify LookML derived tables and convert them to Malloy patterns. Reference `_concepts.md` for syntax translation. This runs during Step 5 (BUILD) when the prior-art notes flag derived tables.
+
+## Decision Tree
+
+```
+derived_table:
+├── explore_source: → NDT path
+│   ├── Simple aggregation → Malloy from() extend {}
+│   ├── With derived_column: (window functions) → Malloy window function patterns
+│   ├── Chained NDTs → dependency ordering, multi-stage computed source
+│   └── With bind_filters → flag, no direct Malloy equivalent
+└── sql: → PDT path
+    ├── Performance-only → strip to base table
+    └── Transformation → recommend base table + Malloy dims or upstream dbt
+```
+
+## NDT Path (`explore_source:`)
+
+### Simple Aggregation NDT
+
+Express the aggregation as a Malloy query, then build a source from it:
+
+```malloy
+source: source_name is from(
+  base_source -> {
+    group_by: group_field
+    aggregate:
+      metric_one is count()
+      metric_two is sum(amount_field)
+  }
+) extend {
+  primary_key: group_field
+  dimension: derived_dim is metric_one > 1
+}
+```
+
+### NDT with `derived_column:` (Window Functions)
+
+LookML `derived_column:` adds window functions on top of the NDT result. Map to Malloy window function patterns. Call `search_malloy_docs("window functions")` for current syntax.
+
+### Chained NDTs
+
+When one NDT references another's explore_source, determine the dependency order. Build the upstream computed source first, then reference it in the downstream one. Each becomes its own `.malloy` file.
+
+### NDTs with `bind_filters`
+
+No direct Malloy equivalent. Flag for user and propose alternatives:
+- Parameterized source (if Malloy supports it; check `search_malloy_docs`)
+- Pre-filtered views
+- Document the intent and let the user decide
+
+## PDT Path (`sql:`)
+
+### Performance-Only PDT
+
+**Signal:** SQL is essentially `SELECT *, generate_uuid() as pk FROM table WHERE increment_condition` with partitioning/clustering/incremental keys.
+
+**Action:** Use the base table directly. The PDT optimization is Looker-specific. Strip to the actual `FROM` table, resolve any manifest constants (`@{TABLE_NAME}`), and use `conn.table('resolved_table')`.
+
+**Synthetic PK investigation:** If the PDT generates a synthetic PK (`generate_uuid()`, `concat(field_a, field_b)`), the actual grain is unknown. Run queries to determine it:
+
+```malloy
+// Check if candidate columns form a unique grain
+run: source -> {
+  group_by: candidate_pk_field
+  aggregate: row_count is count()
+  having: row_count > 1
+  order_by: row_count desc
+  limit: 10
+}
+```
+
+### Transformation PDT
+
+**Signal:** SQL has JOINs, CTEs, complex WHERE clauses, or computed columns.
+
+**Action:** Flag for user. Recommend:
+1. Use the base table + Malloy dimensions for the computed columns
+2. Push the transformation to upstream dbt/SQL
+3. Only carry forward verbatim if user insists (last resort)
+
+**CRITICAL: Never use `conn.sql()` when Malloy has a native pattern.** For aggregation, window functions (`calculate`), and filtering, use Malloy query-based sources (`table -> { group_by, aggregate } extend {}`). `conn.sql()` is a last resort for patterns with NO Malloy equivalent (UNNEST, PIVOT, dialect-specific functions). Call `search_malloy_docs` before writing any SQL block.
+
+## Long→wide entity-values pivot (custom fields)
+
+A common LookML shape: an entity-attribute-value (EAV) table (custom fields / properties) that LookML widened by joining the same table **N times, once per attribute**. Don't port those N joins. Pivot with **filtered aggregates** in a single query-based source — one aggregate per attribute value, no repeated joins:
+
+```malloy
+// EAV table: (entity_id, field_name, field_value) — one row per attribute.
+// Wide result: one row per entity, one column per attribute of interest.
+source: entity_custom_fields is conn.table('custom_field_values') -> {
+  group_by: entity_id
+  aggregate:
+    industry      is field_value.max() { where: field_name = 'industry' }
+    tier          is field_value.max() { where: field_name = 'tier' }
+    account_owner is field_value.max() { where: field_name = 'account_owner' }
+} extend {
+  primary_key: entity_id
+}
+```
+
+`field_value.max()` (the `expr.aggregate() { where: … }` filtered-aggregate form — same shape as `x.sum() { where: … }` in `skill:malloy-gotchas-modeling`) collapses the one matching row per attribute to a scalar; `max` is a native Malloy aggregate that works on strings. Do **not** reach for `any_value`/`ANY_VALUE` — that's a warehouse SQL function, not a Malloy aggregate, and would force a raw-SQL escape that (per the median gotcha in `skill:malloy-gotchas-modeling`) doesn't compile in aggregate position anyway.
+
+Then `join_one` this once onto the entity source. This replaces LookML's N self-joins with one grouped scan — fewer joins, one pass, and new attributes are one more `aggregate:` line. (If an attribute can legitimately repeat per entity, that's not a wide column — model it as a nested/joined detail instead.)
+
+## Examples
+
+Grounded in real patterns from `lookml/block-google-cloud-billing/` and `lookml/bq_thelook/`.

package/skills/malloy-lookml-review/reference/build-unnest.md

@@ -0,0 +1,47 @@
+# LookML UNNEST & Struct Conversion (Step 5)
+
+> Convert LookML UNNEST joins and struct field access patterns to Malloy. This runs during Step 5 (BUILD) when the prior-art notes flag UNNEST joins or struct access.
+
+## UNNEST Join Patterns
+
+LookML joins containing `UNNEST()` in the `sql:` parameter are unnesting nested/repeated fields (common in BigQuery). Example from `gcp_billing_export.explore.lkml`:
+
+```
+# LookML: unnests a repeated field
+join: gcp_billing_export__labels {
+  sql: LEFT JOIN UNNEST(${gcp_billing_export.labels}) as gcp_billing_export__labels ;;
+  relationship: one_to_many
+}
+```
+
+Malloy handles nested structures natively. The conversion depends on whether you want to:
+1. **Keep nested**: access nested fields directly via the parent source
+2. **Flatten**: create a separate source from the unnested data
+
+Call `search_malloy_docs("nested repeated")` for current Malloy syntax for nested/repeated field handling.
+
+## Struct Field Access
+
+LookML accesses struct fields via `${TABLE}.struct.field` syntax:
+
+```
+# LookML
+dimension: project_id { sql: ${TABLE}.project.id ;; }
+dimension: adjustment_description { sql: ${TABLE}.adjustment_info.description ;; }
+```
+
+In Malloy, struct field access uses dot notation directly. Check `search_malloy_docs("struct")` for the current syntax.
+
+## Multiple Views from One Table
+
+LookML often creates separate views for each unnested array (e.g., `gcp_billing_export__labels`, `gcp_billing_export__credits`, `gcp_billing_export__system_labels`). These all derive from the same base table.
+
+**Options:**
+1. **Consolidate**: model as nested sources within the parent source
+2. **Keep separate**: create individual sources if they have distinct analytical value
+
+Present options to the user with the tradeoffs.
+
+## Flattened Naming Conventions
+
+LookML flattens nested field names with double underscores: `project__id`, `labels__key`, `labels__value`. In Malloy, use cleaner names that reflect the actual structure. Propose renames during Step 4 field proposals.

package/skills/malloy-lookml-review/reference/curate-visibility.md

@@ -0,0 +1,66 @@
+# LookML Visibility → Malloy Access Modifiers (Step 8)
+
+> Classify LookML visibility mechanisms and map them to Malloy access modifiers. This runs during Step 8 (CURATE) when the prior-art notes have a Visibility Seeds section.
+
+## LookML Visibility Mechanisms
+
+| LookML Mechanism | Semantics | Strength |
+|---|---|---|
+| `hidden: yes` | Hidden from Explore field picker, but still queryable via URL/API | Cosmetic: UI decluttering |
+| `fields` (explore-level) | Excluded from an explore's available field pool. Not queryable. Can use `ALL_FIELDS*` with `-field` exclusions. | Structural: field genuinely inaccessible |
+| `fields` (join-level) | Restricts which fields from a joined view enter the explore. Include-list only. | Structural: field never enters the pool |
+| `fields_hidden_by_default: yes` | All view fields hidden unless individually opted in with `hidden: no` | Cosmetic: bulk hide |
+| `required_access_grants` | User-attribute-based restriction. True security mechanism. | Security: access controlled |
+
+## Classification → Malloy Treatment
+
+For each hidden/excluded field, determine the **reason** and map to the correct Malloy treatment:
+
+| Classification | LookML Signal | Malloy Treatment |
+|---|---|---|
+| **Intermediate calculation** | `hidden: yes` + referenced by other fields via `${field}` | `# hidden` tag: keep accessible, hide from display |
+| **Join key / FK** | `hidden: yes` + used in `sql_on:` conditions | Keep public, no `#(doc)`: needed for joins |
+| **UI clutter** | `hidden: yes` + not referenced elsewhere | Assess: if truly unused, `internal:` candidate; if just verbose, `# hidden` |
+| **Explore-scoped exclusion** | Excluded via `fields` parameter | `internal:` candidate: field was structurally inaccessible |
+| **Bulk hidden** | `fields_hidden_by_default: yes` on view | Treat each field individually based on its role |
+| **Access-restricted** | `required_access_grants` | `private:` candidate: flag for user decision |
+| **Not hidden** | No visibility restriction | Public + `#(doc)` candidate |
+
+## Key Distinction
+
+LookML `hidden: yes` ≈ Malloy `# hidden` (cosmetic). LookML `fields` exclusion ≈ Malloy `internal:` (structural).
+
+**Do NOT map `hidden: yes` directly to `internal:`.** That over-restricts fields that may be needed as intermediate calculations or join keys.
+
+## Pattern: Restricted-field masking (`include { private: } + pick`)
+
+When a sensitive field should be *present in a redacted/bucketed form* rather than dropped, don't reach for an access modifier alone: modifiers hide or expose, they don't transform. Keep the raw column **`private:`** and derive a masked dimension from it in the source's immediate `extend {}`: a `private:` field is referenceable from the immediate extension, so the raw value stays out of the queryable surface while the mask is public. The mask must key on **data the model can see**, a row-level column or a source `parameter`, not on the viewer:
+
+```malloy
+##! experimental.access_modifiers
+source: revenue is conn.table('…') include {
+  private: raw_amount              // raw stays out of the queryable surface…
+} extend {
+  dimension: amount is             // …but is referenceable here, in the immediate extension
+    pick null when is_confidential          // is_confidential is a ROW column, not the viewer
+    else raw_amount
+  // or bucket instead of null-out:
+  //   pick 'under 50k' when raw_amount < 50000
+  //   pick '50k–100k'  when raw_amount < 100000
+  //   else '100k+'
+}
+```
+
+Give the mask a **different name** from the private raw column (`amount` vs `raw_amount`), reusing the name is a redefinition. If the mask must carry the raw column's *exact* original name, that forces a `rename:`, which pushes you onto the `extend { rename }` path that does **not** compose with `include {}` (see `skill:malloy-gotchas-modeling`); there the raw column can only be `# hidden` (dropped from display but still queryable), so prefer a distinct mask name and keep the raw `private:`.
+
+**This is NOT per-viewer access control.** LookML `required_access_grants` gates on the *viewing user's* attributes; Malloy models have no per-viewer context, so a model-layer `pick` can only redact based on row data or a parameter. To gate a field by *caller identity/role* (the real `required_access_grants` equivalent), use `#(authorize)` on the source, driven by trusted attributes in Malloy Publisher (see `skill:malloy-model` § Access Control). Masking and `#(authorize)` are independent layers; use the mask for "everyone sees a coarsened value," `#(authorize)` for "only some callers see the field at all."
+
+## Pattern: Long→wide custom-field pivot
+
+See `build-derived-tables.md` → "Long→wide entity-values pivot" for turning an entity-attribute-value (EAV) custom-field table into wide columns with **filtered aggregates**, the right move when LookML modeled N per-attribute joins.
+
+## Process
+
+1. Read the Visibility Seeds notes captured during discovery
+2. For each field, apply the classification rules above
+3. Present the classified list to the user for confirmation before applying access modifiers

package/skills/malloy-lookml-review/reference/discover.md

@@ -0,0 +1,102 @@
+# LookML Discovery (Step 1)
+
+> Inventory a LookML project, classify its contents, extract architecture-level candidates, and capture prior-art notes in the conversation. Does NOT extract individual field definitions; that's deferred to `propose-fields.md`.
+
+## 1. Locate `.lkml` Files
+
+Scan the project directory and immediate subdirectories (especially `lookml/`, `lkml/`, `looker/`, or any directory containing `.lkml` files). Ask the user to confirm: "I found LookML files. Use as prior art?"
+
+## 2. Categorize by File Type
+
+| File Pattern | Type | Priority |
+|-------------|------|----------|
+| `manifest.lkml` | Manifest | Read FIRST: contains constants |
+| `*.model.lkml` | Model | Connection name, includes, datagroups |
+| `*.view.lkml` | View | Source-level knowledge (dimensions, measures) |
+| `*.explore.lkml` | Explore | Relationship-level knowledge (joins) |
+| `*.dashboard.lookml` | Dashboard | **SKIP entirely**: analysis is a separate workflow |
+
+## 3. Resolve Manifest Constants
+
+Read `manifest.lkml` and build a lookup table. LookML references like `@{BILLING_TABLE}` must be resolved to actual table names before analysis.
+
+```
+constant: BILLING_TABLE {
+  value: "instance.billing.gcp_billing_export_public"
+}
+# → @{BILLING_TABLE} resolves to "instance.billing.gcp_billing_export_public"
+```
+
+## 4. Extract Connection Info
+
+From model files, note the `connection:` value. This is the **LookML connection name**; it may differ from the connection name used in the Malloy model. In LookML-only mode, use as fallback and flag as unverified.
+
+## 5. Extract Source Candidates from Views
+
+For each `.view.lkml` file, extract architecture-level info only:
+
+- **Table reference:** `sql_table_name:` → direct table, `derived_table:` → classify (see `build-derived-tables.md`)
+- **Primary key:** field with `primary_key: yes`
+- **Grain:** inferred from PK and table name
+- **Role:** Fact (has measures, transactional) or Dimension (lookup, one row per base source)
+
+### Identify Refinements
+
+- **Base view:** `view: view_name { ... }`
+- **Refinement:** `view: +view_name { ... }`: extends an existing view
+
+Present the refinement structure to the user for a consolidate/extend decision.
+
+## 6. Extract Source Candidates from Explores
+
+For each explore, extract:
+
+- **Base view** and **joined views** with cardinality (`relationship:` → `join_one:` / `join_many:` / `join_cross:`)
+- **Join conditions** (`sql_on:`): translate `${view.field}` references
+- **UNNEST joins**: flag any `sql:` containing `UNNEST()` (BigQuery nested fields)
+- **`sql_always_where:`**: document as context, do NOT bake into Malloy
+- **`hidden: yes`** on explores: flag for scope decisions
+
+## 7. Quality Evaluation: Flag Situations
+
+Classify patterns into KEEP, SKIP, and FLAG categories:
+
+**KEEP:** dimension/measure names, aggregation formulas, PKs, join relationships, simple SQL refs, CASE/WHEN logic, filtered aggregates, value_format patterns, yesno dimensions.
+
+**SKIP:** `link:`, `drill_fields:`, `html:`, `action:`, `parameter:` (note intent), Liquid templates (strip, note intent), `datagroup_trigger:`, PDT optimization keys, `convert_tz:`, dashboards, `view_label:`, `group_item_label:`.
+
+**FLAG for user decision:**
+
+| Pattern | Default Recommendation |
+|---------|----------------------|
+| Complex SQL dimensions (50+ lines) | Simplify or push upstream |
+| `derived_table { sql: ... }` with complex SQL | Classify as performance-only vs transformation |
+| `sql_always_where:` constraints | Document as context, don't bake in |
+| Synthetic primary keys (`generate_uuid()`, `concat()`) | Ask about actual grain |
+| Refinement structure (`+view` across files) | Consolidate or preserve layering? |
+
+## 8. Extract Visibility Seeds
+
+Scan for fields with non-default visibility. Capture a compact summary (not full field extraction):
+
+- `hidden: yes` fields: note whether they're join keys, intermediate calcs, or UI clutter
+- `fields` parameter exclusions (explore/join level)
+- `required_access_grants`: flag for user decision
+
+## 9. Extract Documentation Seeds
+
+Scan for fields with high-quality `description:` values worth preserving as `#(doc)` tags. Capture source, field name, and the description text.
+
+## 10. Capture Prior-Art Notes
+
+Hold a lean routing summary in the conversation with these sections:
+
+- **Source**: Type, Location, Mode, Confidence
+- **Source Candidates**: table of sources with table, grain, PK, role
+- **Source Candidates**: table of sources with base source, joins, notes
+- **Flags**: numbered list of situations requiring attention
+- **Visibility Seeds**: compact table of non-default visibility fields
+- **Documentation Seeds**: compact table of fields with good descriptions
+- **Decisions Made During Discovery**: refinement choice, connection name resolution, etc.
+
+No field-level detail beyond seeds. Architecture + flags + decisions only.

package/skills/malloy-lookml-review/reference/document.md

@@ -0,0 +1,28 @@
+# LookML Documentation Seeds (Step 9)
+
+> Extract LookML descriptions and formatting hints as starting material for Malloy `#(doc)` tags. This runs during Step 9 (DOCUMENT) when the prior-art notes have a Documentation Seeds section.
+
+## Process
+
+1. **Start from seeds**: read the Documentation Seeds captured during discovery for fields with high-quality LookML `description:` values.
+
+2. **Read `.lkml` view files** for the full `description:` text on dimensions and measures not captured in seeds.
+
+3. **Rewrite for retrieval**: LookML descriptions are written for Looker's field picker. Malloy `#(doc)` strings power AI search. Rewrite to match how analysts search:
+   - Use business meaning, not Looker jargon
+   - Include units (USD, count, percentage) and valid values for categorical fields
+   - Avoid "filterable", "groupable", "dimension", "measure"
+
+4. **Map formatting to render tags:**
+
+   | LookML | Malloy |
+   |--------|--------|
+   | `value_format: "$#,##0.00"` | `# currency` |
+   | `value_format: "0.00%"` | `# percent` |
+   | `value_format_name: decimal_2` | `# number="0.00"` |
+   | `value_format_name: usd` | `# currency` |
+   | `value_format_name: percent_2` | `# percent` |
+
+5. **Map labels:**
+   - `label:` that simply renames → `internal:` + `dimension:` candidate
+   - `label:` that differs from a clean snake_case name → `# label="Display Name"` candidate

package/skills/malloy-lookml-review/reference/propose-fields.md

@@ -0,0 +1,75 @@
+# LookML Field Extraction (Step 4)
+
+> Read `.lkml` view files and extract field proposals for the Malloy model. Reference `_concepts.md` for type mapping. This runs during Step 4 (PROPOSE DEFINITIONS) to provide LookML-sourced field proposals alongside schema-derived proposals.
+
+## Prerequisites
+
+- Prior-art notes exist with Type: lookml and a Location path
+- Read `reference/_concepts.md` for the LookML → Malloy type mapping table
+
+## Extract Fields from Views
+
+Read every `.view.lkml` file at the location from the prior-art notes. For each view in scope (including refinements merged onto their base view):
+
+### For Each Field, Capture:
+
+| Attribute | What to Capture | Maps to |
+|-----------|----------------|---------|
+| Field name | The LookML field name | Dimension/measure name candidate |
+| `type:` | Data type (string, number, date, yesno, etc.) | Malloy type hint |
+| `sql:` | The SQL expression | Dimension/measure logic |
+| `description:` | Business description | `#(doc)` tag content |
+| `label:` | Display name (if different from field name) | `internal:` + `dimension:` or `# label=` candidate |
+| `hidden: yes` | Hidden from Explore picker | Classify per `curate-visibility.md` |
+| `group_label:` | Logical grouping | Source organization hint |
+| `primary_key: yes` | Primary key | `primary_key:` |
+| `value_format:` / `value_format_name:` | Display formatting | `# currency`, `# percent`, `# number` |
+| `filters:` (on measures) | Filtered aggregate | `{ where: ... }` syntax |
+
+### Handle `dimension_group` (Time Dimensions)
+
+LookML `dimension_group` with `type: time` generates multiple dimensions from one timestamp column. Malloy handles time natively. Extract:
+- The **base column** from `sql:` (e.g., `${TABLE}.created_at`)
+- Note declared timeframes (signals which granularities were used)
+- No explicit dimension needed for each timeframe in Malloy
+
+### Handle `type: yesno`
+
+Boolean flags derived from a SQL condition:
+```
+# LookML
+dimension: is_complete { type: yesno  sql: ${status} = 'complete' ;; }
+
+# Malloy
+dimension: is_complete is status = 'complete'
+```
+
+### Handle Measure Types
+
+| LookML Measure Type | Malloy Equivalent | Notes |
+|---------------------|-------------------|-------|
+| `type: count` | `count()` | Malloy `count()` is always distinct |
+| `type: count_distinct` | `count(field)` | Direct mapping |
+| `type: sum` | `sum(field)` | Direct mapping |
+| `type: average` | `avg(field)` | Direct mapping |
+| `type: min` / `max` | `min(field)` / `max(field)` | Direct mapping |
+| `type: number` | Derived measure expression | Usually a ratio; use `nullif()` for division |
+| `type: list` | No direct equivalent | Flag for alternative approach |
+| `type: percentile` | `field.percentile(n)` | Direct mapping |
+
+**Filtered measures:** `filters:` → Malloy `{ where: }` syntax:
+```
+# LookML
+measure: completed_revenue { type: sum  sql: ${total} ;;  filters: [status: "completed"] }
+
+# Malloy
+measure: completed_revenue is sum(total) { where: status = 'completed' }
+```
+
+## Field-Level Visibility Classification
+
+For each field with visibility restrictions, classify per `curate-visibility.md` rules and include the classification in the proposal's `Malloy Treatment` column.
+
+## Present Proposals
+
+Present field proposals with a `Provenance: lookml` column alongside schema-derived proposals. For fields flagged with Liquid/parameter usage in the prior-art notes, note the business intent without trying to convert the mechanism.

package/skills/malloy-lookml-review/reference/review-coverage.md

@@ -0,0 +1,102 @@
+# LookML Coverage Review (Step 7)
+
+> Compare the built Malloy model against the original LookML project. Show the user what was modeled, what was skipped, and why. This runs during Step 7 (REVIEW) when prior-art notes exist.
+
+## Data Sources
+
+Read these before building the coverage report:
+
+| Source | What it tells you |
+|--------|------------------|
+| Original `.lkml` files (location from the prior-art notes) | Full inventory of views, explores, dimensions, measures |
+| Prior-art notes | Source candidates, flags, visibility seeds |
+| Definition notes | What was proposed, confirmed, and deferred |
+| Built `.malloy` files in package root | What actually shipped |
+
+## 1. Source Coverage
+
+Compare LookML views against Malloy base source files. Every LookML view should appear in this table.
+
+| LookML View | Malloy Source | Status | Rationale |
+|-------------|--------------|--------|-----------|
+| orders | orders.malloy | modeled | (none) |
+| users | customers.malloy | modeled (renamed) | Renamed to match business terminology |
+| order_items | (none) | deferred | Bridge table: defer until line-item analysis needed |
+| admin_audit | (none) | skipped:not-analytical | Operational/ETL table |
+| order_facts_pdt | user_order_facts.malloy | modeled (rearchitected) | Performance-only PDT stripped; rebuilt as computed source |
+
+**Status values:**
+- `modeled`: directly represented in a Malloy source
+- `modeled (renamed)`: represented under a different name
+- `modeled (rearchitected)`: LookML pattern was restructured (e.g., PDT → computed source)
+- `deferred`: valid source, postponed to a later iteration
+- `skipped:not-analytical`: operational, staging, or ETL table
+- `skipped:pre-aggregated`: snapshot/summary table; compute fresh in Malloy
+- `skipped:looker-specific`: view exists only for Looker UI purposes
+
+## 2. Field Coverage (Per Modeled Source)
+
+For each modeled source, walk the original LookML view's dimensions and measures. Show what mapped and what didn't.
+
+| LookML Field | Type | Malloy Field | Status |
+|-------------|------|-------------|--------|
+| order_id | dimension (PK) | order_id | modeled |
+| total_price | measure (sum) | revenue | modeled (renamed) |
+| status | dimension | order_status | modeled (renamed) |
+| created_month | dimension_group | order_month (.month) | modeled (native) |
+| status_link | dimension | (none) | skipped:looker-ui |
+| period_comparison | measure | (none) | skipped:liquid |
+| _pk | dimension | (none) | skipped:synthetic-key |
+
+**Status values:**
+- `modeled`: direct mapping
+- `modeled (renamed)`: mapped under a clearer name
+- `modeled (native)`: LookML pattern replaced by native Malloy feature (e.g., dimension_group → `.month`)
+- `deferred`: valid field, not included in this iteration
+- `skipped:looker-ui`: `link:`, `drill_fields:`, `html:`, `action:` patterns
+- `skipped:liquid`: Liquid templating with no direct equivalent (intent documented)
+- `skipped:synthetic-key`: generated PK, not meaningful as a dimension
+- `skipped:duplicate`: redundant field, another field covers the same data
+- `skipped:upstream`: complex SQL that belongs in dbt/ETL, not the semantic layer
+
+**Show field counts per base source** to give a quick coverage ratio:
+
+> **orders:** 18 of 24 LookML fields modeled (75%). 6 skipped: 3 Looker UI, 2 Liquid, 1 synthetic key.
+
+## 3. Source/Explore Coverage
+
+Compare LookML explores against Malloy source files.
+
+| LookML Explore | Malloy Source | Status | Rationale |
+|----------------|--------------|--------|-----------|
+| order_analysis | order_analysis.malloy | modeled | (none) |
+| customer_health | customer_health.malloy | modeled | (none) |
+| admin_overview | (none) | skipped:not-analytical | Operational dashboard explore |
+
+Include join coverage within each modeled source: did all the LookML joins carry over?
+
+## 4. Skipped Patterns Summary
+
+Group all skipped items by reason with counts. This gives the user a quick sense of what categories of things were left out and whether any warrant reconsideration.
+
+| Category | Count | Examples |
+|----------|-------|---------|
+| Looker UI patterns (link, drill, html, action) | 12 | link on order_id, drill on customer_name |
+| Liquid templating | 3 | period_over_period parameter, dynamic_timeframe |
+| Filter-only fields | 2 | status_filter, date_filter |
+| Synthetic keys | 1 | _pk (generate_uuid) |
+| Dashboard files | 1 | overview.dashboard.lookml |
+| Upstream SQL | 2 | complex_attribution, geo_enrichment |
+
+## 5. Presentation
+
+Present coverage in this order:
+
+1. **Overall summary**: "Modeled X of Y sources, covering Z% of LookML fields. N items deferred, M skipped."
+2. **Source coverage table** (Section 1)
+3. **Per-source field counts** with ratios (Section 2 summaries)
+4. **Skipped patterns summary** (Section 4)
+5. **Deferred items**: list everything marked `deferred` across sources and fields, with rationale, so the user knows what's available for future iterations
+6. **Full field coverage tables** (Section 2 detail): present per base source if the user wants to drill in
+
+**Ask the user:** "Does this coverage look right? Anything deferred that you'd like to add now, or anything modeled that should be removed?"