@malloydata/cli 0.0.53 → 0.0.54-next.2026.5.1.cb0227a

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/dist/index.js CHANGED
@@ -1,3 +1,3 @@
1
1
  #!/usr/bin/env node
2
- process.MALLOY_CLI_VERSION="0.0.53";
3
- require('./cli.js')
2
+ process.MALLOY_CLI_VERSION="0.0.54-next.2026.5.1.cb0227a";
3
+ require('./cli.js')
@@ -0,0 +1,98 @@
1
+ ---
2
+ description: How to append a Malloy query and its interpretation to a .malloynb notebook file. Follow this when the user says "let's log this", "save this", "add this to the notebook", or similar.
3
+ ---
4
+ # Logging an Analysis to a .malloynb Notebook
5
+
6
+ ## When to use this skill
7
+
8
+ Trigger phrases: "let's log this", "save this to the notebook", "add this to the file", "log that", "add that to the analysis", or any request to persist a query and its results to a notebook.
9
+
10
+ ## Step 1 — Identify the target file
11
+
12
+ If the user has not specified a `.malloynb` file in this conversation, ask:
13
+
14
+ > "Which `.malloynb` file should I append this to?"
15
+
16
+ Do not proceed until you have a file path.
17
+
18
+ ## Step 2 — Read the file to understand its current state
19
+
20
+ Before appending, read the file so you know:
21
+ - Whether the first cell already contains the experimental flags and import (it should — don't add them again).
22
+ - The existing section structure, so your new section follows the same style.
23
+
24
+ ## Step 3 — Append the new section
25
+
26
+ **Always append** to the end of the file. Never rewrite existing content.
27
+
28
+ Each logged analysis consists of these cells in order:
29
+
30
+ ### 1. User question cell
31
+ The user's question, verbatim, as a blockquote. Use a backslash before `>` so it renders as a blockquote, not a Malloy cell marker:
32
+
33
+ ```
34
+ >>>markdown
35
+ \> What are the top directors by total ratings?
36
+ ```
37
+
38
+ ### 2. Section header + description cell
39
+ A `##` heading and one sentence describing what the query does:
40
+
41
+ ```
42
+ >>>markdown
43
+ ## Top Directors by Total Ratings
44
+ Ranked by sum of vote counts (in thousands) across all their films.
45
+ ```
46
+
47
+ ### 3. Query cell
48
+ The Malloy query **without** an import statement (the first cell in the file already imports the model):
49
+
50
+ ```
51
+ >>>malloy
52
+ run: movies -> top_directors
53
+ ```
54
+
55
+ ### 4. Interpretation cell
56
+ Bullet-pointed markdown analysis of the results. Lead with the most interesting finding. Use `**bold**` for names and key numbers:
57
+
58
+ ```
59
+ >>>markdown
60
+ - **Christopher Nolan** leads with 18M total votes across 12 films.
61
+ - **Spielberg / Williams** is the most loyal partnership at 85% of films.
62
+ ```
63
+
64
+ ## Cell format rules
65
+
66
+ - Cell delimiters are `>>>malloy` and `>>>markdown` on their own line — no trailing spaces.
67
+ - No blank lines between the delimiter and the cell content.
68
+ - No blank lines between cells (the next `>>>` starts immediately).
69
+ - Do **not** add `>>>malloy` cells for imports or flags — only for executable queries.
70
+ - Lists and bold in markdown cells render correctly; use them freely for analysis.
71
+
72
+ ## Example — what a complete logged section looks like
73
+
74
+ ```
75
+ >>>markdown
76
+ \> Can you find composers that work with the top directors?
77
+ >>>markdown
78
+ ## Directors and Their Composer Collaborators
79
+ For each top director, shows the composers they work with most frequently.
80
+ >>>malloy
81
+ run: movies -> {
82
+ where: job = 'director'
83
+ group_by: director is name
84
+ aggregate: total_ratings, title_count
85
+ nest: composers is {
86
+ group_by: principals2.people.primaryName
87
+ where: principals2.category = 'composer'
88
+ aggregate: title_count, percent_of_titles
89
+ limit: 5
90
+ }
91
+ order_by: total_ratings desc
92
+ limit: 20
93
+ }
94
+ >>>markdown
95
+ - **Most loyal partnerships** by % of films: Zemeckis/Silvestri (91%), Coen/Burwell (89%), Spielberg/Williams (85%).
96
+ - **Hans Zimmer** is the most in-demand, appearing across six top directors.
97
+ - **Martin Scorsese** is the most eclectic — 31 films, top composer at only 16%.
98
+ ```
@@ -0,0 +1,528 @@
1
+ ---
2
+ description: Malloy language reference — concepts, syntax, compilation model. Load this before writing or reviewing Malloy code.
3
+ ---
4
+ # Malloy Language Reference
5
+
6
+ Malloy is a semantic data modeling and query language. It compiles to SQL and runs against existing database engines (DuckDB, BigQuery, Snowflake, PostgreSQL, MySQL, Trino, Presto). It is not a SQL wrapper or abstraction layer — it has its own type system, scoping rules, expression semantics, and compilation pipeline.
7
+
8
+ Malloy is designed around how humans think about data, not how data computations are mechanically accomplished. SQL is oriented around the machine — you specify joins, group-by columns, subqueries, and window functions in terms of what the database needs to do. Malloy is oriented around the analyst — you describe relationships, name computations, and compose questions in terms of what the data means. Malloy bridges the gap between these two by compiling the human-oriented description into correct, efficient SQL.
9
+
10
+ A core design principle is that **most queries are themselves designing a new semantic model.** Formulating a question about data — choosing what to group by, what to aggregate, what to nest — is inherently an act of defining a new way to look at that data. Malloy is built around this idea: the output of every query is not just a result set but a new source with its own schema, and data comprehension is an ongoing iterative process where later stages want not only the data from a previous stage but how that data came into being. This is why query output carries metadata, why queries can be used as sources, and why views and pipelines compose naturally.
11
+
12
+ ## Documents and Statements
13
+
14
+ A Malloy file (`.malloy`) is a sequence of statements, optionally separated by semicolons. There are four statement types:
15
+
16
+ - **`import`** — import sources and queries from another `.malloy` file
17
+ - **`source:`** — define a named, reusable data source with its schema and extensions
18
+ - **`query:`** — define a named query (source + view) for reuse
19
+ - **`run:`** — execute a query (the "do it now" statement)
20
+
21
+ ```malloy
22
+ import "shared_model.malloy"
23
+
24
+ source: flights is duckdb.table('flights.parquet') extend {
25
+ measure: flight_count is count()
26
+ }
27
+
28
+ query: carrier_summary is flights -> {
29
+ group_by: carrier
30
+ aggregate: flight_count
31
+ }
32
+
33
+ run: carrier_summary
34
+ ```
35
+
36
+ Comments use `//` or `--` (both are line comments).
37
+
38
+ ## Sources
39
+
40
+ A **source** is anything you can hand a SQL database and get a schema back — a table name, a SQL SELECT, or the output of another Malloy query. The columns in that schema become the source's initial fields (all dimensions).
41
+
42
+ ```malloy
43
+ source: flights is duckdb.table('flights.parquet')
44
+ source: limited is duckdb.sql("""SELECT * FROM flights LIMIT 100""")
45
+ source: carrier_facts is carrier_summary -- a query used as a source
46
+ ```
47
+
48
+ What makes sources central to Malloy is **extension**. The `extend` block lets you layer on dimensions, measures, views, joins, filters, primary keys, field restrictions, and renames. These extensions travel with the source — any query against it gets them for free.
49
+
50
+ ```malloy
51
+ source: flights is duckdb.table('flights.parquet') extend {
52
+ primary_key: id
53
+
54
+ dimension: distance_km is distance * 1.609344
55
+
56
+ measure:
57
+ flight_count is count()
58
+ total_distance is sum(distance)
59
+
60
+ join_one: carriers with carrier
61
+ join_one: origin_airport is airports on origin_airport.code = origin
62
+
63
+ where: dep_time > @2001
64
+
65
+ view: by_carrier is {
66
+ group_by: carrier
67
+ aggregate: flight_count, total_distance
68
+ }
69
+ }
70
+ ```
71
+
72
+ Sources can extend other sources, creating a refinement chain:
73
+
74
+ ```malloy
75
+ source: ca_flights is flights extend {
76
+ where: origin.state = 'CA'
77
+ }
78
+ ```
79
+
80
+ Field access control uses `accept:` (allowlist) or `except:` (denylist) to restrict which inherited columns are visible. Fields can be renamed with `rename: new_name is old_name`.
81
+
82
+ ## Joins
83
+
84
+ Joins are declared in the source, not reconstructed in every query. This is a fundamental design difference from SQL: the graph structure of your data is a property of the model.
85
+
86
+ ```malloy
87
+ join_one: carriers with carrier -- FK → PK shorthand
88
+ join_one: origin_airport is airports on origin_airport.code = origin -- explicit ON
89
+ join_many: line_items on line_items.order_id = id -- one-to-many
90
+ join_cross: other_table on other_table.key = key -- cross join
91
+ ```
92
+
93
+ - `join_one` — the joined source has at most one row per source row (many-to-one or one-to-one)
94
+ - `join_many` — the joined source has potentially many rows per source row
95
+ - `join_cross` — a full cross product
96
+
97
+ The `with` shorthand requires the joined source to have a declared `primary_key`. All joins are left outer by default. There is no right join — Malloy's graph model doesn't need one.
98
+
99
+ When you reference a joined source's fields, you use dot notation: `carriers.nickname`, `origin_airport.state`. This is one of Malloy's most important abstractions: **the access path to nested data is identical regardless of how the nesting is physically stored.** An array of records embedded in a column, a `join_many` to a separate table, a record-typed column — all are navigated with the same dot notation. The SQL required to traverse these different physical arrangements varies wildly (unnesting arrays, LEFT JOINs, correlated subqueries, ARRAY_AGG), but Malloy hides all of that. You think about the logical shape of your data — "flights have carriers, carriers have a nickname" — and write `carriers.nickname`. The compiler figures out what SQL is needed to get there. This means you can restructure your physical schema (normalize a nested array into a separate table, or denormalize a joined table into a record column) without changing any of the Malloy that references that data.
100
+
101
+ ## Fields
102
+
103
+ Malloy has four kinds of fields: **dimensions**, **measures**, **views**, and **calculations**.
104
+
105
+ ### Dimensions
106
+
107
+ Scalar expressions — they compute a value per row. All columns inherited from a table are dimensions. Computed dimensions reference other dimensions or columns:
108
+
109
+ ```malloy
110
+ dimension: full_name is concat(first_name, ' ', last_name)
111
+ dimension: is_long_haul is distance > 1000
112
+ ```
113
+
114
+ ### Measures
115
+
116
+ Aggregate expressions — they compute a value across a set of rows. A field is a measure when its defining expression contains an aggregate function (`count`, `sum`, `avg`, `min`, `max`):
117
+
118
+ ```malloy
119
+ measure:
120
+ flight_count is count()
121
+ total_distance is sum(distance)
122
+ avg_distance is avg(distance)
123
+ pct_delayed is count() { where: dep_delay > 30 } / count()
124
+ ```
125
+
126
+ **`count(expr)` counts distinct values.** Unlike SQL's `COUNT(DISTINCT expr)`, Malloy uses `count(expr)` for distinct counting. The `count(distinct expr)` form is a deprecated syntax that will produce an error. Use `count()` for total row count, `count(field)` for distinct values of that field:
127
+
128
+ ```malloy
129
+ aggregate:
130
+ total_rows is count() -- all rows
131
+ unique_carriers is count(carrier) -- distinct carriers
132
+ ```
133
+
134
+ Measures can be filtered inline with `{ where: ... }`, which is how you build things like "percent of flights delayed" without subqueries.
135
+
136
+ ### Views
137
+
138
+ A view is a query saved into the source — a reusable transformation:
139
+
140
+ ```malloy
141
+ view: by_carrier is {
142
+ group_by: carrier
143
+ aggregate: flight_count, total_distance
144
+ limit: 10
145
+ }
146
+ ```
147
+
148
+ Views can reference other views from the same source as a starting point, and can be extended with `+`.
149
+
150
+ ### Calculations
151
+
152
+ Window functions over the grouped result. Calculations can only be defined in a query stage with `calculate:`, never in a source definition, because they depend on the output columns of the query:
153
+
154
+ ```malloy
155
+ run: flights -> {
156
+ group_by: carrier
157
+ aggregate: flight_count
158
+ calculate: carrier_rank is rank()
159
+ }
160
+ ```
161
+
162
+ ## Queries and Views
163
+
164
+ A query pairs a source with a view (the transformation). Everything after the first `->` is the view.
165
+
166
+ ```malloy
167
+ run: flights -> {
168
+ group_by: carrier
169
+ aggregate: flight_count
170
+ }
171
+ ```
172
+
173
+ ### Reduction vs. Projection
174
+
175
+ Each stage of a view performs exactly one of:
176
+
177
+ - **Reduction** — uses `group_by:` and/or `aggregate:` to reduce grain. Analogous to `SELECT ... GROUP BY` in SQL.
178
+ - **Projection** — uses `select:` to pick fields without aggregation. Analogous to `SELECT` without `GROUP BY`.
179
+
180
+ These cannot be mixed in a single stage. A stage with `group_by:` cannot have `select:`, and vice versa.
181
+
182
+ ### Source-level definitions vs. query-level operations
183
+
184
+ The same `name is expression` syntax defines fields in both sources and queries:
185
+
186
+ ```malloy
187
+ -- In a source (reusable):
188
+ source: flights is ... extend {
189
+ measure: flight_count is count() -- defines a measure in the model
190
+ }
191
+
192
+ -- In a query (ad hoc):
193
+ run: flights -> {
194
+ aggregate: flight_count is count() -- defines the same measure inline
195
+ }
196
+ ```
197
+
198
+ When used in a source, `measure:` and `dimension:` are **definition statements** — they add named fields to the source's schema. When used in a query, `group_by:`, `aggregate:`, `select:`, `nest:`, and `calculate:` are **query operations** — they specify what the query does. The field definitions are syntactically identical in both contexts, but the enclosing keyword determines the role:
199
+
200
+ | Source keyword | Query keyword | What it holds |
201
+ |---|---|---|
202
+ | `dimension:` | `group_by:` or `select:` | scalar expressions |
203
+ | `measure:` | `aggregate:` | aggregate expressions |
204
+ | `view:` | `nest:` | sub-queries |
205
+ | _(n/a)_ | `calculate:` | window functions |
206
+
207
+ This is why `measure` and `aggregate` are separate keywords. `measure:` is a *modeling* statement — "this source has a reusable aggregate computation called X." `aggregate:` is a *query* statement — "in this query, include these aggregate values in the output." A query's `aggregate:` can reference a previously defined measure by name, or define one inline. The distinction parallels the separation between defining a dimension in a source and using it via `group_by:` in a query.
208
+
209
+ ### Multi-stage Pipelines
210
+
211
+ Stages chain with `->`. Each stage's output becomes the next stage's source:
212
+
213
+ ```malloy
214
+ run: flights -> {
215
+ group_by: carrier
216
+ aggregate: flight_count is count()
217
+ } -> {
218
+ where: flight_count > 1000
219
+ select: *
220
+ }
221
+ ```
222
+
223
+ ### Refinement with `+`
224
+
225
+ The refinement operator `+` merges query operations together. It works both within a view and at the top level on a named query:
226
+
227
+ ```malloy
228
+ -- Refining a view within a query:
229
+ run: flights -> by_carrier + { limit: 5 } + { nest: by_destination }
230
+
231
+ -- Refining a named query at the top level:
232
+ run: carrier_summary + { group_by: origin } -- add origin grouping to existing query
233
+ ```
234
+
235
+ When a dimension name appears as a bare reference, it expands to `{ group_by: name }`. A measure name expands to `{ aggregate: name }`:
236
+
237
+ ```malloy
238
+ run: flights -> carrier + flight_count + { limit: 10 }
239
+ -- equivalent to: flights -> { group_by: carrier; aggregate: flight_count; limit: 10 }
240
+ ```
241
+
242
+ For multi-stage queries, refinement semantics get more complex — but for single-stage queries, `+` straightforwardly merges operations into the stage.
243
+
244
+ ### Nesting
245
+
246
+ `nest:` embeds an aggregating subquery inside a reduction. Each row of the outer query gets a subtable from the nested query. Nests can nest arbitrarily deep:
247
+
248
+ ```malloy
249
+ run: flights -> {
250
+ group_by: carrier
251
+ aggregate: flight_count
252
+ nest: top_routes is {
253
+ group_by: origin, destination
254
+ aggregate: flight_count
255
+ limit: 3
256
+ }
257
+ }
258
+ ```
259
+
260
+ ### Other query operations
261
+
262
+ - **`where:`** — filter rows (pre-aggregation). Comma-separated filters are ANDed.
263
+ - **`having:`** — filter groups (post-aggregation), like SQL's HAVING.
264
+ - **`limit:`** / **`order_by:`** — limit and sort output.
265
+ - **`extend`** — add fields or joins to a source inline within a query expression.
266
+
267
+ ## Aggregate Locality (Symmetric Aggregates)
268
+
269
+ This is one of Malloy's most important features. In SQL, when you join tables and aggregate, you risk double-counting (the "fan trap"). Malloy solves this with **aggregate locality** — you specify *where in the join graph* an aggregation should be computed.
270
+
271
+ ```malloy
272
+ run: flights -> {
273
+ aggregate:
274
+ -- avg seats weighted by number of flights (locality: source, i.e. flights)
275
+ avg_seats_per_flight is source.avg(aircraft.aircraft_models.seats)
276
+ -- avg seats per aircraft model (locality: aircraft_models)
277
+ avg_seats_per_model is aircraft.aircraft_models.seats.avg()
278
+ }
279
+ ```
280
+
281
+ Three syntactic forms:
282
+
283
+ - `avg(expr)` — aggregate at the current source (implicit locality)
284
+ - `joined_source.avg(expr)` — aggregate at the specified join point (explicit locality)
285
+ - `joined_source.field.avg()` — shorthand for aggregating the field at its parent source
286
+
287
+ For `sum` and `avg` (asymmetric aggregates), when the expression crosses a join boundary, Malloy *requires* explicit locality — it won't silently give you a wrong answer. For `min`, `max`, and `count` (symmetric), locality doesn't change the result, so implicit is always fine.
288
+
289
+ Malloy implements this with a technique called **symmetric aggregates** — it internally de-duplicates rows based on primary keys at the appropriate join level, so aggregations are always mathematically correct regardless of join fan-out.
290
+
291
+ ## Ungrouped Aggregates
292
+
293
+ `all()` and `exclude()` allow computing aggregates at different grouping levels within a single query:
294
+
295
+ ```malloy
296
+ run: airports -> {
297
+ group_by: state, faa_region
298
+ aggregate:
299
+ airport_count is count()
300
+ total_airports is all(count()) -- ungrouped: total across all rows
301
+ region_airports is all(count(), faa_region) -- grouped only by faa_region
302
+ pct_of_total is count() / all(count())
303
+ }
304
+ ```
305
+
306
+ `all(expr)` removes all grouping. `all(expr, dim1, dim2)` keeps only the specified grouping dimensions. `exclude(expr, dim)` removes the specified dimension from grouping.
307
+
308
+ **Important:** `all(expr, dim)` takes the **local alias name** as defined in the query's `group_by:`, not a dotted path. If you want to partition by a joined field, alias it first:
309
+
310
+ ```malloy
311
+ -- WRONG: all(count(), director.primaryName) -- dot paths don't work here
312
+ -- RIGHT:
313
+ run: movies -> {
314
+ group_by: director is director.primaryName -- alias it
315
+ aggregate:
316
+ movies is count()
317
+ director_total is all(count(), director) -- reference the alias
318
+ pct is count() / all(count(), director)
319
+ }
320
+ ```
321
+
322
+ ## Expressions
323
+
324
+ Malloy expressions include arithmetic, comparison, logical operators, function calls, type casts, and several Malloy-specific forms.
325
+
326
+ ### Evaluation Spaces
327
+
328
+ Every expression has an evaluation space: **literal**, **constant**, **input**, or **output**. Input expressions reference source columns/dimensions. Output expressions reference the results of the current query stage (used in `calculate:`). Some functions constrain their arguments — e.g., `lag(expr)` requires an output expression, `avg(expr)` requires an input expression.
329
+
330
+ ### Application and Partial Comparison
331
+
332
+ The `?` operator applies a condition to a value. Partial comparisons are conditions without a left-hand side:
333
+
334
+ ```malloy
335
+ where: state ? 'CA' | 'NY' -- state is 'CA' or 'NY'
336
+ where: distance ? > 500 & < 2000 -- distance between 500 and 2000
337
+ ```
338
+
339
+ `|` is alternation (OR), `&` is conjunction (AND) within partials.
340
+
341
+ ### Pick Expressions
342
+
343
+ Malloy's equivalent of CASE:
344
+
345
+ ```malloy
346
+ dimension: size_bucket is
347
+ pick 'short' when distance < 500
348
+ pick 'medium' when distance < 1500
349
+ else 'long'
350
+ ```
351
+
352
+ ### Filtered Aggregate Expressions
353
+
354
+ Any aggregate can be filtered inline:
355
+
356
+ ```malloy
357
+ measure: ca_flights is count() { where: origin.state = 'CA' }
358
+ ```
359
+
360
+ ### Type Casting
361
+
362
+ ```malloy
363
+ total_distance::string -- Malloy type cast
364
+ name::"VARCHAR(32)" -- database-native type cast
365
+ ```
366
+
367
+ ### Time Literals and Ranges
368
+
369
+ ```malloy
370
+ @2003 -- the year 2003
371
+ @2003-Q2 -- second quarter of 2003
372
+ @2024-01-15 10:30:00 -- timestamp literal
373
+ dep_time ? @2003 to @2005 -- range comparison
374
+ now -- current timestamp
375
+ ```
376
+
377
+ ## Data Types
378
+
379
+ Malloy's type system: `string`, `number`, `boolean`, `date`, `timestamp`, `timestamptz`, `json`, and `sql native` (for unsupported database types). Compound types: `type[]` for arrays, `{ name :: type, ... }` for records, nesting arbitrarily: `{ x :: number, tags :: string[] }[]`.
380
+
381
+ ## Annotations and Tags
382
+
383
+ These are related but distinct concepts.
384
+
385
+ ### Annotations
386
+
387
+ Annotations are **text strings** attached to objects during compilation. They are metadata — they never affect query execution or SQL generation. An annotation starts with `#` and continues to end of line:
388
+
389
+ ```malloy
390
+ # bar_chart
391
+ view: by_carrier is { ... }
392
+ ```
393
+
394
+ - `#` annotations attach to the next object defined below them
395
+ - `##` annotations attach to the model (the file)
396
+ - Block annotations use `#|` ... `|#` for multi-line content (closing delimiter must match the column position of the opener)
397
+
398
+ Annotations distribute over definition lists:
399
+
400
+ ```malloy
401
+ # currency
402
+ measure: -- all three measures get the # currency annotation
403
+ revenue is sum(amount)
404
+ # percent -- this measure also gets # percent
405
+ margin is revenue / cost
406
+ cost is sum(amount)
407
+ ```
408
+
409
+ ### Tags (a use of annotations)
410
+
411
+ Tags are the primary *consumer* of annotation strings. They interpret annotation text using a structured property language (MOTLY). The key distinction: **annotations are the transport mechanism (raw strings attached to objects), tags are the interpretation layer (parsed key-value properties).**
412
+
413
+ Not all annotations are tags. An annotation is just text. Tags are annotations that happen to be written in the tag property language and parsed by an application.
414
+
415
+ ### Annotation prefixes (routing)
416
+
417
+ The character(s) immediately after `#` route the annotation to different consumers:
418
+
419
+ - `# ` (hash-space) — renderer tags, parsed by the Malloy VS Code extension for visualization
420
+ - `##!` — compiler directives (e.g., `##! experimental.parameters`)
421
+ - `#"` — reserved for documentation strings
422
+ - `#(appName)` — application-specific tags (e.g., `#(docs) hidden`)
423
+
424
+ ```malloy
425
+ # bar_chart size=large -- renderer tag: tells VS Code how to render
426
+ ##! experimental.parameters -- compiler tag: enables a feature flag
427
+ #(myApp) priority=high -- custom app tag: ignored by renderer/compiler
428
+ ```
429
+
430
+ ### Tag property syntax
431
+
432
+ ```
433
+ tName -- boolean flag (exists = true)
434
+ tName=value -- set property value
435
+ tName=[a, b, c] -- array value
436
+ tName: { p1=v1 p2=v2 } -- nested properties (replaces)
437
+ tName { p1=v1 } -- nested properties (merges)
438
+ -tName -- delete a property
439
+ tName.sub.path=value -- deep path assignment
440
+ ```
441
+
442
+ Values can be unquoted identifiers, quoted strings, numbers, or typed values prefixed with `@` (`@true`, `@false`, `@2024-01-15`).
443
+
444
+ ## How a Malloy Query Becomes SQL
445
+
446
+ The compilation pipeline has two phases:
447
+
448
+ ### Phase 1: Translation (source code → IR)
449
+
450
+ ```
451
+ Malloy source → ANTLR lexer/parser → parse tree → AST builder → AST → IR generator → IR
452
+ ```
453
+
454
+ The **Intermediate Representation (IR)** is a plain, serializable data structure (JSON-compatible) that fully describes the semantic model and query. Note that IR is *not* dialect-agnostic — the same Malloy source compiled against different databases can produce different IR, because schema information, type mappings, and available functions vary by backend. It can be cached, transmitted, and reused. Key IR types:
455
+
456
+ - **`SourceDef`** — a source's complete definition: schema, fields, joins, filters
457
+ - **`Query`** — a source paired with a pipeline of operations
458
+ - **`FieldDef`** — definition of any field (dimension, measure, join, calculation)
459
+ - **`Expr`** — expression tree (arithmetic, comparisons, aggregates, function calls, field references)
460
+
461
+ The translator handles all language-level semantics: scoping, name resolution, type checking, evaluation space validation.
462
+
463
+ ### Phase 2: Compilation (IR → SQL)
464
+
465
+ ```
466
+ IR → query compiler → expression compiler → dialect-specific SQL generator → SQL + metadata
467
+ ```
468
+
469
+ The compiler walks the IR query pipeline, translating each stage into SQL constructs (CTEs, subqueries, GROUP BY, window functions). A **Dialect** layer handles database-specific SQL generation.
470
+
471
+ The compiler also produces **metadata** alongside the SQL — structural information needed to interpret the result set (column types, nesting structure, annotation data). This metadata is what allows Malloy renderers to reconstruct nested/hierarchical results from the flat SQL result set and apply visualization tags.
472
+
473
+ ### Key architectural consequences
474
+
475
+ - Because the IR is serializable, it can be cached and reused across compilations (though IR is database-specific — the same source compiled against different backends may produce different IR).
476
+ - Because joins are declared in the source (not the query), the compiler knows the full join graph and can compute symmetric aggregates correctly.
477
+ - Because nested queries are first-class, the compiler generates the appropriate SQL (correlated subqueries or ARRAY_AGG patterns depending on dialect) automatically.
478
+ - Because measures are typed as aggregates in the IR, the compiler can validate that they only appear in aggregate context and enforce locality rules.
479
+
480
+ ## Where to Go Deeper
481
+
482
+ This document is a conceptual reference — enough to reason about the language and its design, but not exhaustive. Here's where to find more detail.
483
+
484
+ ### Language Documentation
485
+
486
+ The full docs live at [https://docs.malloydata.dev](https://docs.malloydata.dev). Key pages by topic:
487
+
488
+ | Topic | URL |
489
+ |---|---|
490
+ | Sources, extensions, joins, primary keys | [documentation/language/source](https://docs.malloydata.dev/documentation/language/source) |
491
+ | Queries, views, reduction vs projection | [documentation/language/query](https://docs.malloydata.dev/documentation/language/query), [views](https://docs.malloydata.dev/documentation/language/views) |
492
+ | Fields: dimensions, measures, views, calculations | [documentation/language/fields](https://docs.malloydata.dev/documentation/language/fields) |
493
+ | Aggregate functions and aggregate locality | [documentation/language/aggregates](https://docs.malloydata.dev/documentation/language/aggregates) |
494
+ | Ungrouped aggregates (`all`, `exclude`) | [documentation/language/ungrouped-aggregates](https://docs.malloydata.dev/documentation/language/ungrouped-aggregates) |
495
+ | Nested views / aggregating subqueries | [documentation/language/nesting](https://docs.malloydata.dev/documentation/language/nesting) |
496
+ | Joins | [documentation/language/join](https://docs.malloydata.dev/documentation/language/join) |
497
+ | Expressions, operators, pick, apply | [documentation/language/expressions](https://docs.malloydata.dev/documentation/language/expressions) |
498
+ | Evaluation spaces (literal, constant, input, output) | [documentation/language/eval_space](https://docs.malloydata.dev/documentation/language/eval_space) |
499
+ | Filters and filter placement | [documentation/language/filters](https://docs.malloydata.dev/documentation/language/filters) |
500
+ | Annotations and tags | [documentation/language/tags](https://docs.malloydata.dev/documentation/language/tags) |
501
+ | Calculations and window functions | [documentation/language/calculations_windows](https://docs.malloydata.dev/documentation/language/calculations_windows) |
502
+ | Data types | [documentation/language/datatypes](https://docs.malloydata.dev/documentation/language/datatypes) |
503
+ | Time operations, ranges, timezones | [documentation/language/timestamp-operations](https://docs.malloydata.dev/documentation/language/timestamp-operations), [time-ranges](https://docs.malloydata.dev/documentation/language/time-ranges), [timezones](https://docs.malloydata.dev/documentation/language/timezones) |
504
+ | Imports | [documentation/language/imports](https://docs.malloydata.dev/documentation/language/imports) |
505
+ | Top-level statements and model structure | [documentation/language/statement](https://docs.malloydata.dev/documentation/language/statement) |
506
+ | Functions reference | [documentation/language/functions](https://docs.malloydata.dev/documentation/language/functions) |
507
+
508
+ ### Examples and Patterns
509
+
510
+ The docs site includes worked examples of common analytical patterns at [documentation/patterns](https://docs.malloydata.dev/documentation/patterns/): percent-of-total, year-over-year, cohort analysis, sessionization, moving averages, nested subtotals, and more.
511
+
512
+ End-to-end guides are at [documentation/user_guides](https://docs.malloydata.dev/documentation/user_guides/), including [Malloy by Example](https://docs.malloydata.dev/documentation/user_guides/malloy_by_example) (a comprehensive walkthrough) and a three-part series for SQL users ([part 1](https://docs.malloydata.dev/documentation/user_guides/sql_experts1), [part 2](https://docs.malloydata.dev/documentation/user_guides/sql_experts2), [part 3](https://docs.malloydata.dev/documentation/user_guides/sql_experts3)).
513
+
514
+ ### Source Code
515
+
516
+ The Malloy implementation lives at [github.com/malloydata/malloy](https://github.com/malloydata/malloy). Key entry points:
517
+
518
+ | What | Where |
519
+ |---|---|
520
+ | ANTLR grammar (lexer + parser) | `packages/malloy/src/lang/grammar/` |
521
+ | AST node hierarchy | `packages/malloy/src/lang/ast/` |
522
+ | Parse tree → AST builder | `packages/malloy/src/lang/malloy-to-ast.ts` |
523
+ | IR type definitions | `packages/malloy/src/model/malloy_types.ts` |
524
+ | IR → SQL compiler | `packages/malloy/src/model/` |
525
+ | Dialect-specific SQL generation | `packages/malloy/src/dialect/` |
526
+ | Tag/annotation parsing (MOTLY) | `packages/malloy-tag/` |
527
+ | Renderer | `packages/malloy-render/` |
528
+ | Architecture overview | `CONTEXT.md` (root and in each package) |