@microsoft/rayfin-semantic-model-pack 0.0.0-stage → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,461 @@
1
+ ---
2
+ name: analytics
3
+ description: >
4
+ Use when the app reads an existing Power BI semantic model — a dashboard,
5
+ report, KPI, or any view over data the app does not own. Turns on the Fabric
6
+ data client and the semantic-model query hook, then covers the whole workflow:
7
+ pick a model, discover its schema, write and test DAX, and render it. A chart
8
+ over data the app already has needs the visuals skill instead, not this one.
9
+ Triggers: semantic model, dataset, Power BI, DAX, measure, "our sales data",
10
+ "from the model", "the existing report".
11
+ ---
12
+
13
+ # Analytics — reading a semantic model
14
+
15
+ ## Turn it on first
16
+
17
+ ```sh
18
+ npm run pack:add -- connectors
19
+ npm run pack:add -- analytics
20
+ ```
21
+
22
+ The connectors pack installs the composable client in
23
+ `packages/frontend/src/lib/rayfin-client.ts`.
24
+ The analytics pack installs the semantic-model connector packages and the
25
+ `useSemanticModelQuery` hook.
26
+ It also turns the `connectors` service on in `rayfin/rayfin.yml`, which unlocks
27
+ the `rayfin connector` commands.
28
+ Run both before writing any code.
29
+
30
+ To chart what the queries return, add the visuals pack as well:
31
+
32
+ ```sh
33
+ npm run pack:add -- visuals
34
+ ```
35
+
36
+ That brings `VegaVisual`, `DataGrid`, `toDataTable`, and `validate:visual`.
37
+ Analytics does not pull it in, because reading a model and drawing one are
38
+ separate jobs — an app that shows a handful of figures needs no chart library.
39
+ Most dashboards want both.
40
+
41
+ Re-applying either is safe: existing files are preserved, not overwritten.
42
+ The connectors pack and `rayfin connector add` own `connectors.ts`; analytics
43
+ does not install a competing copy.
44
+
45
+ The analytics pack can also install a local semantic-model catalog. It is a
46
+ preview capability and ships off, so `schema:sql` exists only in an app that
47
+ set `RAYFIN_FEATURE_FLAGS=schema-sqlite` in `rayfin/.env` before applying the
48
+ pack — see [`schema-discovery`](../schema-discovery/SKILL.md). When it is
49
+ absent, discover the model with read-only `INFO.VIEW.*` queries instead.
50
+
51
+ - `npm run --silent schema:sql -- "<SQL>"` queries the catalog.
52
+ - `npm run --silent schema:sql -- --search "<terms>"` searches tables, columns, measures, relationships, descriptions, expressions, and display folders.
53
+ - `--search`, `--sql`, and `--file` are repeatable and answer in one process, so a plan of up to eight questions costs one round trip instead of eight. The whole batch shares one byte budget.
54
+ - `npm run schema:refresh` explicitly rebuilds it.
55
+
56
+ The first catalog query builds the database automatically.
57
+ It rebuilds after the 15-minute freshness window expires.
58
+ One refresh runs five schema-only source queries from a hardcoded list, plus retries if the model throttles, then later discovery is local SQLite.
59
+ Do not replace this with progressive `INFO.VIEW.*` connector invocations.
60
+
61
+ The database is written under `.schema/models/<itemId>/model.sqlite`.
62
+ Its `catalog.md` companion describes the tables and marks the catalog **INCOMPLETE** if any source view failed.
63
+ The `catalog_status` table carries the same persistent marker.
64
+ Always check that marker before treating absence from an untruncated SQL result as evidence that an object does not exist.
65
+
66
+ The catalog stores the untouched outputs of `INFO.VIEW.TABLES()`, `INFO.VIEW.COLUMNS()`, `INFO.VIEW.MEASURES()`, and `INFO.VIEW.RELATIONSHIPS()`.
67
+ It also stores `COLUMNSTATISTICS()` output, including actual Min/Max values derived from model rows, cardinality, and maximum text length.
68
+ The catalog is therefore customer data, not metadata-only content, and `.schema/` is gitignored.
69
+ Applying the analytics pack adds the catalog and interrupted-query patterns to
70
+ the app's root `.gitignore`, including for apps created from older templates.
71
+ Do not copy, commit, or expose it.
72
+
73
+ > If `npm run dev` re-optimizes dependencies on first load, add the visual
74
+ > packages to `optimizeDeps.include` in `packages/frontend/vite.config.ts`. They are left out of
75
+ > the base because a bare app does not install them.
76
+
77
+ Read [`schema-discovery`](../schema-discovery/SKILL.md) next.
78
+ It routes ordinary table, column, measure, relationship, and statistics discovery through the local catalog.
79
+ After that, pull in what the job actually needs rather than printing whole catalog tables into the conversation.
80
+ Each row is a sibling skill:
81
+
82
+ | Read | When |
83
+ |---|---|
84
+ | [`dax-authoring`](../dax-authoring/SKILL.md) | You are writing a measure or a non-trivial DAX expression |
85
+ | [`query-design`](../query-design/SKILL.md) | The dashboard needs more than one grain, or you are shaping a query for several visuals |
86
+ | [`visuals`](../visuals/SKILL.md) | You are building any chart, or configuring `DataGrid` |
87
+ | [`app-design`](../app-design/SKILL.md) | You are deciding layout and theming |
88
+
89
+ Each of those has a `references/` folder that is deliberately not required
90
+ reading; open a reference when the SKILL.md points you at it for the specific
91
+ thing you are doing.
92
+
93
+ ## Recommended workflow
94
+
95
+ Use these steps unless the user asks for a different sequence.
96
+
97
+ The source contract and consuming UI have one hard ordering rule: run each real query before writing code that depends on its columns or values.
98
+ That does not require leaving the starter/welcome view untouched during all schema discovery.
99
+
100
+ - **Foundation:** Apply the packs, identify and attach the intended model, and preserve the authentication gate.
101
+ - **Runnable shell:** Establish the app's layout, final content regions, and honest loading states while independent schema and query discovery continues.
102
+ - **Data integration:** Validate each query through `npx rayfin connector invoke`, then bind the exact returned shape into the corresponding component.
103
+ - **Completion:** Add interactions and polish, then run the final validation sequence once.
104
+
105
+ Do not put invented business values into the shell.
106
+ Loading content should reserve the real layout without pretending the source has returned.
107
+
108
+ ### 1. Ask the user for a semantic model
109
+
110
+ **Local `.pbix` files are not supported.** This app connects to semantic models published to the Power BI Service (cloud), not to local `.pbix` files on disk. If the user provides a local file path (e.g., `C:\...\Model.pbix`), **do not** attempt to open, upload, or search for it. Instead:
111
+ 1. Inform the user that local `.pbix` files are not supported — only models published to the Power BI Service can be used.
112
+ 2. Ask the user whether they would like to:
113
+ - **Search the Power BI Service** for a semantic model by name — run `npx rayfin connector search` on their behalf, or
114
+ - **Provide a specific online model** directly (workspace ID + dataset ID, or a Power BI / Fabric URL).
115
+
116
+ Once the user confirms and connects a published semantic model, read the [schema-discovery](../schema-discovery/SKILL.md) skill.
117
+ Let the fixed catalog builder materialize the five schema sources once, then progressively retrieve only relevant rows and columns with local SQL.
118
+
119
+ Once the model is identified, attach it as a connector. Choose the path that matches the input the user gave you:
120
+
121
+ ```bash
122
+ # Path A — user wants you to search by name.
123
+ # --all-workspaces is required here: this step runs before the app is
124
+ # deployed, so there is no deployment to infer a workspace scope from and
125
+ # the command would otherwise exit with "workspace scope is required".
126
+ # --json is recommended so you can reason over displayName, workspaceName
127
+ # and ids without parsing tables. If more than one could match, confirm the
128
+ # right one with the user (e.g. two same-named models in different workspaces).
129
+ npx rayfin connector search "<name>" --type fabric-semanticmodel --all-workspaces --json
130
+
131
+ # Path B / C — user gave a URL or explicit IDs. Take workspaceId + itemId from
132
+ # whichever they provided and declare the connector:
133
+ npx rayfin connector add --type fabric-semanticmodel --workspace-id <workspaceId> --item-id <itemId> --name <alias>
134
+ ```
135
+
136
+ `connector add` writes the entry into `rayfin/rayfin.yml`, scaffolds
137
+ `rayfin/connectors/<alias>/`, and prints version-pinned package specifications.
138
+ Install those exact versions into the frontend workspace from the app root:
139
+
140
+ ```sh
141
+ npm install -w @rayfin-app/frontend <printed-packages-and-versions>
142
+ ```
143
+
144
+ Do not install them at the workspace root or create a nested lockfile.
145
+
146
+ `connector add` also regenerates
147
+ `packages/frontend/src/lib/connectors.ts` with the connector config, schema, and
148
+ runtime.
149
+ Do not hand-write those maps or replace the client installed by the connectors
150
+ pack.
151
+ That client composes connectors through `ExtendableRayfinClient`, and the pack
152
+ selects the functions-aware variant automatically when functions are present.
153
+ If the CLI reports that the wiring file is app-owned, apply the exact manual
154
+ snippet it prints instead of inventing a second client recipe.
155
+
156
+ ### 2. Prove the connector returns data before wiring source values
157
+
158
+ `fabric-semanticmodel` is the one connector type whose `invoke` runs under your
159
+ own identity, so it works **without** a deploy. That makes this check nearly
160
+ free, and it is the difference between finding a dead connector now or after
161
+ building a dashboard on top of it:
162
+
163
+ ```bash
164
+ npx rayfin connector invoke <alias> executeQuery --file ./inspection/smoke.query.json
165
+ ```
166
+
167
+ Where `smoke.query.json` is the operation's input envelope, not the bare query:
168
+
169
+ ```json
170
+ { "query": "EVALUATE TOPN(1, Sales)" }
171
+ ```
172
+
173
+ Do not wire source-backed values if this fails.
174
+ A connector, permission, or enablement failure must remain an error rather than an endless loading state.
175
+ A standalone-capable layout shell may already exist, but no amount of presentation work fixes the source contract.
176
+
177
+ **Pass DAX in a `--file` payload, not with `--input`.** A DAX query is full of
178
+ quotes and brackets, and an inline `--input` string goes through a shell that
179
+ treats them as syntax — `EVALUATE ROW("it's", 1)` breaks on the apostrophe
180
+ alone. `--file` is `JSON.parse`d, so it must be JSON: a raw `.dax` file fails
181
+ before the connector is reached. The path resolves against the project root and
182
+ must stay inside it.
183
+
184
+ ### 3. Write and test DAX queries
185
+
186
+ Follow the [dax-authoring](../dax-authoring/SKILL.md) skill to write and test DAX queries.
187
+ The skill covers DAX syntax rules, query patterns, time intelligence, and an iterative test workflow.
188
+ Use the [schema-discovery](../schema-discovery/SKILL.md) skill to query model metadata from the local SQLite catalog.
189
+ Do not hand-run the catalog's `INFO.VIEW.*` or `COLUMNSTATISTICS()` source queries.
190
+
191
+ Use `npx rayfin connector invoke <alias> executeQuery --file <query>.query.json` to test queries. This runs the query through the same connector the app uses at runtime, so the results (column names, data types, row structure) are identical to what the app will produce.
192
+
193
+ Iterate on queries until they return the expected columns and data shape.
194
+
195
+ Delegate query exploration only when it is a substantial independent body of work with a clear result contract.
196
+ For a small dashboard or one tightly coupled query flow, keep discovery and component integration together.
197
+
198
+ Once queries are validated, **promote them to the app** following the [Query and spec organization](#query-and-spec-organization) conventions above.
199
+
200
+ **Does this app draw a chart or a grid?** The answer decides what to build, and
201
+ it is worth settling before writing files rather than after. `ColumnMetadataMap`,
202
+ Vega-Lite specs and `vegaLiteSpec` all come from the `visuals` pack: in an app
203
+ that never applied it those imports do not resolve and `typecheck` fails.
204
+
205
+ *If it charts* — the usual case, and the rest of this skill assumes it:
206
+
207
+ 1. Save each query as a `.dax` file in `packages/frontend/src/queries/` following the naming and grouping conventions.
208
+ 2. **Capture column metadata** — copy the exact column names from the query output as dictionary keys in a `columnMetadata: ColumnMetadataMap` constant. See the conventions above for the full format and rules.
209
+ 3. Create the corresponding `.json` Vega-Lite spec — refer to [visuals](../visuals/SKILL.md). Use the cleaned `name` values from the metadata for Vega-Lite field encodings (they are already free of characters that require escaping). Use `displayName` values for axis titles, legend labels, and tooltip headers. Use `format` values for axis/tooltip formatting.
210
+ 4. Create the barrel `.ts` file with a **factory function** that returns `{ connection, query, columnMetadata, vegaLiteSpec }`.
211
+
212
+ *If it does not* — figures in custom components, a summary, an export:
213
+
214
+ 1. Save each query as a `.dax` file exactly as above.
215
+ 2. Create the barrel `.ts` file with a factory returning `{ connection, query }`.
216
+ Nothing else: there is no spec and no metadata constant.
217
+
218
+ Read the result off `data.table` — `columns` and `rows` — and render it however
219
+ the app renders anything else.
220
+
221
+ ### 4. UX Design for the app
222
+
223
+ Principles for overall aesthetics, theming, layout, and accessibility requirements are outlined in [app-design](../app-design/SKILL.md) skill - refer to it before creating or modifying any UI component, layout, page, or style.
224
+
225
+ ### 5. Build components with data (app code phase)
226
+
227
+ Use the `useSemanticModelQuery` hook from `packages/frontend/src/hooks/use-semantic-model-query.ts`. Components call the factory functions from `packages/frontend/src/queries/` and destructure what those factories return.
228
+
229
+ **Without the `visuals` pack**, that is `{ connection, query }`, and `data.table`
230
+ is the whole rendering API:
231
+
232
+ ```tsx
233
+ const { data } = useSemanticModelQuery(revenueByRegion());
234
+ if (data?.status === "success") {
235
+ const [row] = data.table.rows; // columns and rows, nothing else
236
+ return <p>Revenue this quarter: {String(row?.[0])}</p>;
237
+ }
238
+ ```
239
+
240
+ Everything below is the charting path and needs `visuals` applied.
241
+
242
+ Factories there return `{ connection, query, columnMetadata, vegaLiteSpec }`.
243
+
244
+ Use `toDataTable()` from `packages/frontend/src/lib/to-data-table.ts` to convert the SDK's `QueryTable` (from `data.table`) into a `DataTable` by merging it with the `columnMetadata` from the factory function result. It ships with the `visuals` pack. This applies everywhere the data is consumed like rendering in `VegaVisual` or `DataGrid` (pass the `DataTable` via their `data` prop), displaying values in custom components, or any other usage.
245
+
246
+ `VegaVisual` and `DataGrid` should consume factory-produced query, spec, and column metadata together so static specs remain checkable. Runtime-built specs are allowed but require browser validation. Partial static coverage exits normally; use the preview command only when coverage is zero. Follow the [visuals](../visuals/SKILL.md) validation guidance before completion.
247
+
248
+ `App.tsx` ships with the starter/welcome view behind the
249
+ `packages/frontend/src/EmptyStatePreview.tsx` module.
250
+ Replace what the app renders for the real dashboard, but preserve that module and its existing named `EmptyStatePreview` export so the starter view can evolve independently.
251
+ Do not couple application guidance to the view's artwork, animation, or other internal implementation.
252
+
253
+ #### Example
254
+
255
+ Charting path — this one needs `visuals`:
256
+
257
+ ```tsx
258
+ import { revenueByRegion } from "@/queries/sales/revenue-by-region";
259
+ import { useSemanticModelQuery } from "@/hooks/use-semantic-model-query";
260
+ import { toDataTable } from "@/lib/to-data-table";
261
+ import { VegaVisual, useCssTheme } from "@microsoft/fabric-visuals";
262
+ import { DataGrid } from "@microsoft/fabric-datagrid";
263
+
264
+ function RevenueByRegionChart() {
265
+ const theme = useCssTheme();
266
+ const { connection, query, columnMetadata, vegaLiteSpec } = revenueByRegion({
267
+ categories: ["Category A"],
268
+ });
269
+
270
+ const { data, isLoading, error } = useSemanticModelQuery({
271
+ connection,
272
+ query,
273
+ });
274
+
275
+ if (isLoading) return <LoadingSpinner />;
276
+
277
+ // Two disjoint channels, so both are checked. `error` is what the connector
278
+ // threw — a wiring mistake, transport, or auth — and it clears `data`, so
279
+ // skipping it renders nothing at all for those failures.
280
+ if (error) return <ErrorMessage message={error.message} />;
281
+
282
+ // `data.status === "error"` is a failure Power BI reported inside a successful
283
+ // response: invalid DAX, a permission refusal, an overflow.
284
+ if (data?.status === "error") {
285
+ return <ErrorMessage message={data.error.message} />;
286
+ }
287
+
288
+ if (data?.status !== "success") return null;
289
+
290
+ // Convert the SDK's QueryTable into a DataTable enriched with column metadata.
291
+ // toDataTable merges data.table with columnMetadata so that display names,
292
+ // format strings, and cleaned field names are available to visuals.
293
+ const dataTable = toDataTable(data.table, columnMetadata);
294
+
295
+ // Pass the DataTable to VegaVisual (chart) or DataGrid (table) via the data prop.
296
+ return (
297
+ <div>
298
+ <VegaVisual spec={vegaLiteSpec} data={dataTable} theme={theme} />
299
+ <DataGrid data={dataTable} theme={theme} />
300
+ </div>
301
+ );
302
+ }
303
+ ```
304
+
305
+ **Runtime client:** `getRayfinClient()` is asynchronous in this template.
306
+ Always await it before reading `.connectors`.
307
+
308
+ **Caching:** There is none. Every call to `useSemanticModelQuery` reaches Power BI.
309
+ The hook's `refetch()` re-runs the current query.
310
+ If a screen re-queries more than you want, hold the result in state or memoize at
311
+ the call site — do not reintroduce a shared cache here without a per-principal key
312
+ and a TTL, because a process-global one serves another user's rows after a sign-in
313
+ change and stale rows after a model refresh.
314
+
315
+ **Error handling:** Two channels, and they are **disjoint** — a component that
316
+ reads only one renders nothing for half its failures.
317
+
318
+ - `error` carries what the connector **threw** — a wiring mistake, transport, or
319
+ auth. It is the original error, so its status and service code survive. `data`
320
+ is cleared when this is set.
321
+ - `data.status === "error"` carries what Power BI **returned**, already
322
+ categorised by the connector as `"query"` (bad DAX), `"overflow"` (a numeric
323
+ value too large for a JavaScript number), `"api"` (dataset-level or auth), or `"network"`.
324
+ `error` stays `undefined` for these, so the category is always reachable.
325
+
326
+ Check both, in either order; they never both apply. Then check `data.status`
327
+ before reading `data.table`.
328
+
329
+ Retry only a known transient network or service-starting condition, and cap the attempts.
330
+ Authentication, permission, connector configuration, and query errors are permanent until something changes, so surface them immediately.
331
+
332
+ ### 6. Final validation
333
+
334
+ Follow the [app-validation](../app-validation/SKILL.md) skill to
335
+ run targeted checks during implementation and the required final sequence once.
336
+ Rerun only a failed gate and later checks invalidated by its fix.
337
+ Complete the browser check through the Fabric portal embed.
338
+
339
+ ---
340
+
341
+ ## Query and spec organization
342
+
343
+ This section describes an app that charts its results, so it assumes the
344
+ `visuals` pack. Without it there are no specs: keep the `.dax` files and the
345
+ factory, and drop the `.json` and `vegaLiteSpec` from both.
346
+
347
+ DAX queries and Vega-Lite specs live in `packages/frontend/src/queries/`, grouped by dashboard page or domain. Each visualization gets files sharing the same kebab-case base name: one or more `.dax` files for queries, a `.json` file for the Vega-Lite spec, and a `.ts` factory file that imports them and exports `{ connection, query, columnMetadata, vegaLiteSpec }`. The factory function accepts optional parameters to select between query variants or modify the spec:
348
+
349
+ ```text
350
+ packages/frontend/src/queries/
351
+ ├── index.ts # Re-exports all query modules
352
+ ├── {page-or-domain}/ # Group by dashboard page or domain
353
+ │ ├── {visualization-name}.dax # DAX query (plain text)
354
+ │ ├── {visualization-name}-{variant}.dax # Additional query variants (optional)
355
+ │ ├── {visualization-name}.json # Vega-Lite spec (JSON)
356
+ │ ├── {visualization-name}.ts # Factory function: imports .dax + .json, exports { connection, query, vegaLiteSpec, columnMetadata }
357
+ │ └── index.ts # Re-exports all visualizations in this group
358
+ ```
359
+
360
+ ### Example TS File
361
+
362
+ **`revenue-by-region.ts`** — the factory function accepts use-case-specific parameters and uses them to modify the DAX query and/or Vega-Lite spec as appropriate:
363
+
364
+ ```ts
365
+ import type { VisualizationSpec } from "@microsoft/fabric-visuals";
366
+ import type { ColumnMetadataMap } from "@/lib/to-data-table";
367
+ import baseQuery from "./revenue-by-region.dax?raw";
368
+ import spec from "./revenue-by-region.json";
369
+
370
+ const connection = "{connection-alias}"; // from rayfin/rayfin.yml
371
+
372
+ /** Column metadata keyed by original DAX column name. */
373
+ const columnMetadata: ColumnMetadataMap = {
374
+ "Products[Region]": { name: "ProductsRegion", displayName: "Region" },
375
+ "[Total Revenue]": { name: "Total Revenue", displayName: "Total Revenue", format: "$#,0.00" },
376
+ };
377
+
378
+ interface RevenueByRegionParams {
379
+ /** Filter to specific product categories (modifies the DAX query). */
380
+ categories?: string[];
381
+ /** Only show regions with revenue above this threshold (modifies the Vega-Lite spec). */
382
+ minRevenue?: number;
383
+ }
384
+
385
+ export function revenueByRegion(params?: RevenueByRegionParams) {
386
+ let query = baseQuery;
387
+ let vegaLiteSpec = spec; // Should clone if modifying
388
+
389
+ if (params?.categories?.length) {
390
+ // make changes to the DAX query to filter by the specified categories and update the query variable
391
+ }
392
+
393
+ if (params?.minRevenue != null) {
394
+ // Clone spec and append a client-side filter transform to the Vega-Lite spec
395
+ }
396
+
397
+ return { connection, query, columnMetadata, vegaLiteSpec };
398
+ }
399
+ ```
400
+
401
+ The parameters, their types, and how they modify the query or spec are **entirely use-case-specific**. Some visualizations may need no parameters at all; others may accept date ranges, top-N limits, grouping dimensions, or string search terms. The factory function is the single place that translates caller intent into DAX and/or Vega-Lite modifications.
402
+
403
+ #### Query variants with multiple `.dax` files
404
+
405
+ When a parameter changes the **structure** of the query (e.g. different GROUP BY columns, different aggregations), use separate `.dax` files for each variant. The factory function selects the right one:
406
+
407
+ ```text
408
+ revenue-trend/
409
+ revenue-trend-yearly.dax
410
+ revenue-trend-quarterly.dax
411
+ revenue-trend-monthly.dax
412
+ revenue-trend.json
413
+ revenue-trend.ts
414
+ ```
415
+
416
+ ```ts
417
+ import yearlyQuery from "./revenue-trend-yearly.dax?raw";
418
+ import quarterlyQuery from "./revenue-trend-quarterly.dax?raw";
419
+ import monthlyQuery from "./revenue-trend-monthly.dax?raw";
420
+
421
+ type Granularity = "yearly" | "quarterly" | "monthly";
422
+
423
+ const queryByGranularity: Record<Granularity, string> = {
424
+ yearly: yearlyQuery,
425
+ quarterly: quarterlyQuery,
426
+ monthly: monthlyQuery,
427
+ };
428
+
429
+ export function revenueTrend(params?: { granularity?: Granularity }) {
430
+ const query = queryByGranularity[params?.granularity ?? "monthly"];
431
+ return { connection, query, columnMetadata, vegaLiteSpec };
432
+ }
433
+ ```
434
+
435
+ #### Column Metadata
436
+
437
+ Metadata is part of the charting path — it exists to feed `toDataTable`, and both `ColumnMetadataMap` and `ColumnDef` come from the `visuals` pack. An app with no chart in it skips this section.
438
+
439
+ Capture column metadata in the barrel `.ts` file as a `columnMetadata: ColumnMetadataMap` constant (import `ColumnMetadataMap` from `@/lib/to-data-table`). The dictionary **must be keyed by the exact column name from the CLI query output** (the `name` field in `table.columns`). Do not guess or clean these keys — copy them verbatim from the query result. Each value is a `ColumnDef` (from `@microsoft/fabric-visuals-core`) containing:
440
+ - `name` — a cleaned-up identifier derived from the original column name by removing `.`, `[`, `]`, `\`, `"`, and `'` characters. E.g., `"Products[Region]"` → `"ProductsRegion"`, `"[Total Revenue]"` → `"Total Revenue"` — to be used when building visual specs.
441
+ - `displayName` — a human-readable label sourced from the semantic model schema (e.g., `"Region"`, `"Total Revenue"`). Used for axis titles, grid headers, and tooltips.
442
+ - `format` — a VBA/ECMA-376 format string for number/date formatting (e.g., `#,##0.00`, `0.00%`, and `mm/dd/yyyy`). Omit for text type columns.
443
+
444
+ **Example workflow:** Run `npx rayfin connector invoke myModel executeQuery --file ./inspection/probe.query.json`, observe the output columns are `[SalesPersonID]` and `[Name]`, then use those exact strings as metadata keys:
445
+
446
+ ```ts
447
+ export const columnMetadata: ColumnMetadataMap = {
448
+ "[SalesPersonID]": { name: "SalesPersonID", displayName: "Sales Person ID" },
449
+ "[Name]": { name: "Name", displayName: "Name" },
450
+ };
451
+ ```
452
+
453
+ #### Key rules
454
+
455
+ - **All DAX lives in `.dax` files.** Never inline full DAX query strings in `.ts` factory files. If a parameter changes the query structure, create a separate `.dax` file for each variant and select the right one in the factory function. Small modifications — such as replacing filter value placeholders, wrapping the query with `CALCULATETABLE` to apply filters, or substituting a column reference — are acceptable in `.ts`, but the base query must always come from a `.dax` import.
456
+ - **Name files after the visualization they drive.** Use kebab-case base names (e.g., `revenue-by-region.dax`, `revenue-by-region.json`, `revenue-by-region.ts`). Variant `.dax` files append a suffix (e.g., `revenue-trend-yearly.dax`, `revenue-trend-quarterly.dax`).
457
+ - **Use `.dax` for queries.** Plain-text DAX files keep queries readable and diff-friendly. Import them with Vite's `?raw` suffix.
458
+ - **Use `.json` for specs.** JSON files get free schema validation in editors and are importable as modules by default in Vite.
459
+ - **Barrel `.ts` exports a factory function.** The function name is the camelCase version of the kebab-case file name (e.g., `revenueByRegion` for `revenue-by-region.ts`). It accepts optional parameters (typed per use case) and returns `{ connection, query, columnMetadata, vegaLiteSpec }` — or just `{ connection, query }` when the app draws no charts.
460
+ - **Group by page/domain.** Use subfolders when the dashboard has multiple pages or logical sections. For simple single-page dashboards, a flat structure under `packages/frontend/src/queries/` is fine.
461
+ - **Re-export via `index.ts`.** Each subfolder and the root `packages/frontend/src/queries/index.ts` should re-export all modules for clean imports.
@@ -0,0 +1,170 @@
1
+ ---
2
+ name: dax-authoring
3
+ description: >
4
+ Write and test DAX queries against Power BI semantic models.
5
+ Covers DAX syntax rules, query patterns, time intelligence,
6
+ and an iterative test workflow using `npx rayfin connector invoke`.
7
+ ---
8
+
9
+ # DAX Authoring
10
+
11
+ ## Table of Contents
12
+
13
+ | Task | Reference | Notes |
14
+ |---|---|---|
15
+ | Must/Prefer/Avoid | [SKILL.md: Must/Prefer/Avoid](#must--prefer--avoid) | Guardrails for DAX query generation |
16
+ | Generating DAX Queries | [SKILL.md: Generating DAX Queries](#generating-dax-queries) | Core rules, inline examples, EVALUATE/DEFINE patterns |
17
+ | DAX Query Structure & Syntax | [dax-query-patterns.md: Query Structure](./references/dax-query-patterns.md#query-structure) | DEFINE / EVALUATE / ORDER BY / START AT |
18
+ | DAX Query Key Components | [dax-query-patterns.md: Key Components](./references/dax-query-patterns.md#key-components) | DEFINE VAR, DEFINE MEASURE, table expressions |
19
+ | DAX Query Worked Examples | [dax-query-patterns.md: Common Patterns](./references/dax-query-patterns.md#common-patterns) | 11 annotated examples from simple aggregation to cross-table joins |
20
+ | DAX Query Anti-Patterns | [dax-query-patterns.md: Anti-Patterns](./references/dax-query-patterns.md#anti-patterns) | What to avoid in DAX queries |
21
+ | CALCULATE & CALCULATETABLE | [dax-core-reference.md: CALCULATE & CALCULATETABLE](./references/dax-core-reference.md#calculate--calculatetable) | Context transition, filter types, boolean restrictions, common patterns |
22
+ | SUMMARIZECOLUMNS | [dax-core-reference.md: SUMMARIZECOLUMNS](./references/dax-core-reference.md#summarizecolumns) | Argument order, auto-blank elimination, filter args, vs SUMMARIZE |
23
+ | ALL & ALLEXCEPT | [dax-core-reference.md: ALL & ALLEXCEPT](./references/dax-core-reference.md#all--allexcept) | CALCULATE modifier vs table function; percentage patterns |
24
+ | TREATAS | [dax-core-reference.md: TREATAS](./references/dax-core-reference.md#treatas) | Virtual relationships, multi-column filtering |
25
+ | DAX Syntax Rules | [dax-core-reference.md: DAX Syntax Rules](./references/dax-core-reference.md#dax-syntax-rules) | EVALUATE, CALCULATE, naming, SQL keywords, DEFINE rules |
26
+ | Common Mistakes | [dax-core-reference.md: Common Mistakes](./references/dax-core-reference.md#common-mistakes) | Variable naming, quoting, escaping, scalar EVALUATE, multi-table SUMMARIZECOLUMNS |
27
+ | BLANK Semantics | [dax-core-reference.md: BLANK Semantics](./references/dax-core-reference.md#blank-semantics) | BLANK vs NULL, propagation, equality, ISBLANK, DIVIDE, non-empty semantics |
28
+ | Time Intelligence Patterns | [SKILL.md: Time Intelligence](#time-intelligence) | When to consult TI reference |
29
+ | Date Table Prerequisites | [dax-time-intelligence.md: Prerequisites](./references/dax-time-intelligence.md#prerequisites) | Date table requirements, mark as date table |
30
+ | YTD / QTD / MTD | [dax-time-intelligence.md: Period-to-Date](./references/dax-time-intelligence.md#period-to-date) | TOTALYTD, DATESYTD, DATESINPERIOD patterns |
31
+ | Year-over-Year / Period Comparisons | [dax-time-intelligence.md: Period Comparisons](./references/dax-time-intelligence.md#period-comparisons) | SAMEPERIODLASTYEAR, DATEADD, PARALLELPERIOD |
32
+ | Rolling Windows | [dax-time-intelligence.md: Rolling Windows](./references/dax-time-intelligence.md#rolling-windows) | DATESINPERIOD rolling 12-month patterns |
33
+ | Opening/Closing Balances | [dax-time-intelligence.md: Balances](./references/dax-time-intelligence.md#balances) | Semi-additive measures, LASTDATE, LASTNONBLANK |
34
+ | TI in DAX Queries (Critical Rules) | [dax-time-intelligence.md: Critical Rules for TI in Queries](./references/dax-time-intelligence.md#critical-rules-for-ti-in-dax-queries) | CALCULATETABLE + TREATAS pattern for query context |
35
+ | TI Common Mistakes | [dax-time-intelligence.md: Common Mistakes](./references/dax-time-intelligence.md#common-mistakes) | Missing date table, wrong granularity, fiscal calendar |
36
+ | Testing & Iteration | [SKILL.md: Testing & Iteration](#testing--iteration) | Execute → inspect → fix → re-test workflow |
37
+
38
+ ## Must / Prefer / Avoid
39
+
40
+ ### Must
41
+
42
+ - Always test generated DAX via `npx rayfin connector invoke <alias> executeQuery --file <query>.query.json` before using in app code
43
+ - Use fully-qualified `'Table'[Column]` for column references
44
+ - Use simple `[Measure]` for measure references
45
+ - Use DEFINE for VAR and local MEASURE declarations (single DEFINE block, no commas)
46
+ - Prefer existing model measures over re-aggregating raw data
47
+ - **Never reshape a stored measure to work around a decoding bug.** Wrapping it
48
+ in `CONVERT(...)` changes the value the app reads, and `CONVERT([Measure],
49
+ DOUBLE)` silently drops every row whose branch returned text. If the CLI and
50
+ the app disagree by a power of ten, check that both carry a connector build
51
+ with the Arrow union decoder fix — `@microsoft/rayfin-cli` bundles its own
52
+ copy, and the app installs `@microsoft/rayfin-connector-fabric-semanticmodel`
53
+ separately under `packages/frontend`, so upgrading one does not update the
54
+ other. Fix the build, keep the measure.
55
+
56
+ ### Prefer
57
+
58
+ - SUMMARIZECOLUMNS as the primary grouping function for queries
59
+ - TREATAS for filter arguments in SUMMARIZECOLUMNS
60
+ - Variables (VAR) to improve readability and avoid repeated calculations
61
+
62
+ ### Avoid
63
+
64
+ - SQL keywords (SELECT, WHERE, HAVING, etc.) within DAX expressions
65
+ - EVALUATE with scalar functions directly (wrap in ROW or table function)
66
+
67
+ ## Generating DAX Queries
68
+
69
+ ### DAX Syntax Rules
70
+
71
+ - Measures are named objects in the semantic model. They specify how to aggregate data. **Measures should be used first** to answer user requests before a new DAX formula is used to aggregate data.
72
+ - **EVALUATE Statement**: Not a function but a statement. It must always precede a table expression. Avoid pairing EVALUATE with scalar functions like CONCATENATEX, SUMX, DISTINCTCOUNT, etc.
73
+ - Use the fully-qualified name `'Table'[Column]` for column references, and the simple name `[Measure]` for measure references.
74
+ - CALCULATE takes a scalar expression as its first argument. CALCULATETABLE takes a table expression as its first argument.
75
+ - SUMMARIZECOLUMNS requires a specific order: groupby columns, then filters, then aggregations/measures.
76
+ - Do not use SUMMARIZECOLUMNS when there is no aggregation and the groupby columns belong to more than one table; use VALUES, SUMMARIZE, or SELECTCOLUMNS instead.
77
+ - When using SELECTCOLUMNS or CALCULATETABLE, include any columns needed downstream (ORDER BY, FILTER).
78
+ - Filters propagate across relationships based on unidirectional or bidirectional settings.
79
+ - INTERSECT, UNION, EXCEPT require identical column counts in both inputs.
80
+ - For current date/time, use TODAY() or NOW().
81
+
82
+ ### Inline Examples
83
+
84
+ **Simple filtered aggregation:**
85
+
86
+ ```dax
87
+ // Total sales for red products
88
+ EVALUATE
89
+ ROW("Total Sales Amount", CALCULATE([Total Amount], 'Product'[Color] == "Red"))
90
+ ```
91
+
92
+ **Multi-filter grouping with SUMMARIZECOLUMNS:**
93
+
94
+ ```dax
95
+ DEFINE
96
+ VAR _Filter1 = TREATAS({"Consumer Electronics"}, 'Product'[Category])
97
+ VAR _Filter2 = FILTER(ALL('Calendar'[Year]), 'Calendar'[Year] >= 2022 && 'Calendar'[Year] <= 2023)
98
+
99
+ EVALUATE
100
+ SUMMARIZECOLUMNS(
101
+ 'Calendar'[Year],
102
+ 'Calendar'[Month],
103
+ _Filter1,
104
+ _Filter2,
105
+ "Total Quantity", SUM('Sales'[Order Quantity]),
106
+ "Discount", [Total Discount]
107
+ )
108
+ ```
109
+
110
+ **TopN with filtering:**
111
+
112
+ ```dax
113
+ DEFINE
114
+ VAR _Filter = TREATAS({"Red", "Black"}, 'Product'[Color])
115
+ VAR _Core = SUMMARIZECOLUMNS('Product'[Name], _Filter, "Total Sales", [Total Amount])
116
+
117
+ EVALUATE
118
+ TOPN(10, _Core, [Total Sales], DESC)
119
+ ```
120
+
121
+ For full syntax reference, worked examples, and anti-patterns, see [dax-query-patterns.md](./references/dax-query-patterns.md).
122
+ For function details, see [dax-core-reference.md](./references/dax-core-reference.md).
123
+
124
+ ## Time Intelligence
125
+
126
+ Time intelligence functions enable period-based analysis (YTD, YoY, rolling windows, etc.). They require a properly configured Date table.
127
+
128
+ Consult [dax-time-intelligence.md](./references/dax-time-intelligence.md) whenever the user's request involves:
129
+ - Period-to-date calculations (YTD, QTD, MTD)
130
+ - Period comparisons (Year-over-Year, Month-over-Month)
131
+ - Rolling windows (last 12 months, last 30 days)
132
+ - Opening/closing balances
133
+ - Custom date ranges
134
+
135
+ ## Testing & Iteration
136
+
137
+ 1. Generate the DAX query expression
138
+ 2. Execute via `npx rayfin connector invoke <alias> executeQuery --file <query>.query.json`
139
+ 3. Inspect results: check column names, data types, row counts, and actual data values
140
+ 4. If error: consult [dax-core-reference.md](./references/dax-core-reference.md), fix, and re-test
141
+ 5. Iterate until the query returns expected results
142
+
143
+ ## Query Execution
144
+
145
+ Use `npx rayfin connector invoke <alias> executeQuery --file <query>.query.json` to run queries. This uses the same SDK pipeline as the running app, so results are identical to what the app produces at runtime. `--file` takes the operation's JSON input envelope — `{"query":"EVALUATE …"}` — not a bare `.dax` file, which is `JSON.parse`d and fails. Prefer it over `--input` for anything non-trivial: DAX is full of quotes and brackets that a shell reads as syntax. For full CLI options, see `npx rayfin connector invoke --help`.
146
+
147
+ **There is no row cap.** A query with no `resultSetRowCountLimit` returns every
148
+ row it matches, on the CLI and in the app alike, so an unbounded `EVALUATE` over
149
+ a large table is a real hazard rather than a truncated preview. Bound it
150
+ yourself — either in the DAX with an aggregation or `TOPN(...)`, or by adding the
151
+ field to the input envelope:
152
+
153
+ ```json
154
+ { "query": "EVALUATE Sales", "resultSetRowCountLimit": 500 }
155
+ ```
156
+
157
+ `resultSetRowCountLimit` silently truncates, so a result with exactly that many rows may be incomplete. `withValidatedRowLimit` silently drops values that are not positive integers, leaving no explicit `resultSetRowCountLimit` on the request.
158
+
159
+ An `overflow` error is separate: `toQueryResult` uses this category for per-table errors, including the observed case of a numeric value too large for a JavaScript number. It means a failed query result, never that `resultSetRowCountLimit` was hit.
160
+
161
+ ## Troubleshooting
162
+
163
+ | Problem | Solution |
164
+ |---------|----------|
165
+ | `connector invoke` fails with "not signed in" | Run `npx rayfin login` |
166
+ | `connector invoke` fails with "alias not found" | Run `npx rayfin connector list` to check available aliases, then `npx rayfin connector add --type fabric-semanticmodel` to register |
167
+ | `connector invoke` fails with "Do not know how to serialize a BigInt" | A measure whose branches return different types decodes as an Arrow union; connector builds without the union decoder fix leave some of its cells unconverted. Update the connector build — do not reshape the stored measure (see Must) |
168
+ | DAX syntax errors | Consult [dax-core-reference.md](./references/dax-core-reference.md) — check reserved keywords, quoting rules, EVALUATE/scalar mistakes |
169
+ | Unexpected query results | Check filter context, relationship direction, BLANK handling in [dax-core-reference.md](./references/dax-core-reference.md#blank-semantics) |
170
+ | Time intelligence returns wrong values | Check date table prerequisites and critical rules in [dax-time-intelligence.md](./references/dax-time-intelligence.md) |