@microsoft/rayfin-semantic-model-pack 1.0.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/CHANGELOG +8 -0
  2. package/README.md +34 -10
  3. package/kit/hooks/use-query-factory-target.spec.ts +191 -0
  4. package/kit/hooks/use-query-factory-target.ts +123 -0
  5. package/{skills/analytics/kit → kit}/hooks/use-semantic-model-query.spec.ts +59 -23
  6. package/{skills/analytics/kit → kit}/hooks/use-semantic-model-query.ts +30 -54
  7. package/kit/lib/query-factory.spec.ts +97 -0
  8. package/kit/lib/query-factory.ts +68 -0
  9. package/kit/lib/semantic-model-errors.spec.ts +131 -0
  10. package/kit/lib/semantic-model-errors.ts +110 -0
  11. package/kit/lib/to-data-table.spec.ts +444 -0
  12. package/kit/lib/to-data-table.ts +221 -0
  13. package/pack.json +141 -0
  14. package/package.json +3 -1
  15. package/skills/analytics/SKILL.md +126 -421
  16. package/skills/dax-authoring/SKILL.md +99 -34
  17. package/skills/query-design/SKILL.md +22 -29
  18. package/skills/query-design/references/anti-patterns.md +13 -11
  19. package/skills/query-design/references/format-strings.md +6 -12
  20. package/skills/query-design/references/multi-grain-patterns.md +121 -108
  21. package/skills/schema-discovery/SKILL.md +197 -368
  22. package/skills/schema-discovery/references/discovery-queries.md +6 -8
  23. package/skills/visual-interactivity/SKILL.md +68 -0
  24. package/skills/visual-interactivity/references/advanced-query-shapes.md +155 -0
  25. package/skills/visual-interactivity/references/filter-aware-grid-totals.md +101 -0
  26. package/skills/visual-interactivity/references/highlight-queries.md +88 -0
  27. package/skills/visual-interactivity/references/interaction-manager.md +127 -0
  28. package/skills/visual-interactivity/references/native-selections.md +20 -0
  29. package/skills/visual-interactivity/references/query-factories.md +169 -0
  30. package/skills/analytics/kit/lib/connectors.ts +0 -94
  31. package/skills/analytics/kit/scripts/rayfin-connector.mjs +0 -265
  32. package/skills/analytics/kit/scripts/rayfin-connector.test.mjs +0 -243
  33. package/skills/analytics/kit/scripts/schema-sql-batch.test.mjs +0 -2097
  34. package/skills/analytics/kit/scripts/schema-sql-cli.test.mjs +0 -488
  35. package/skills/analytics/kit/scripts/schema-sql.mjs +0 -2650
  36. package/skills/analytics/kit/scripts/schema-sql.test.mjs +0 -1387
  37. package/skills/analytics/kit/scripts/schema.mjs +0 -166
  38. package/skills/analytics/kit/scripts/schema.test.mjs +0 -271
  39. package/skills/analytics/pack.json +0 -49
  40. package/skills/query-design/references/filter-strategy.md +0 -68
  41. package/skills/query-design/references/highlight-queries.md +0 -122
@@ -1,460 +1,165 @@
1
1
  ---
2
2
  name: analytics
3
3
  description: >
4
- Use when the app reads an existing Power BI semantic model — a dashboard,
5
- report, KPI, or any view over data the app does not own. Turns on the Fabric
6
- data client and the semantic-model query hook, then covers the whole workflow:
7
- pick a model, discover its schema, write and test DAX, and render it. A chart
8
- over data the app already has needs the visuals skill instead, not this one.
9
- Triggers: semantic model, dataset, Power BI, DAX, measure, "our sales data",
10
- "from the model", "the existing report".
4
+ Read Power BI semantic-model data through Rayfin connectors. Use the
5
+ semantic-model query hook with normalized errors, validated DAX and exact
6
+ output columns, then shape results for charts and grids.
11
7
  ---
12
8
 
13
9
  # Analytics — reading a semantic model
14
10
 
15
- ## Turn it on first
11
+ ## Connector and discovery
16
12
 
17
- ```sh
18
- npm run pack:add -- connectors
19
- npm run pack:add -- analytics
20
- ```
21
-
22
- The connectors pack installs the composable client in
23
- `packages/frontend/src/lib/rayfin-client.ts`.
24
- The analytics pack installs the semantic-model connector packages and the
25
- `useSemanticModelQuery` hook.
26
- It also turns the `connectors` service on in `rayfin/rayfin.yml`, which unlocks
27
- the `rayfin connector` commands.
28
- Run both before writing any code.
29
-
30
- To chart what the queries return, add the visuals pack as well:
13
+ Connect only to a published Power BI semantic model; local `.pbix` files are
14
+ not supported. Never guess ids or add a source without the user's consent.
15
+ Use ids from a Fabric URL or search results. Registration requires sign-in:
31
16
 
32
17
  ```sh
33
- npm run pack:add -- visuals
34
- ```
35
-
36
- That brings `VegaVisual`, `DataGrid`, `toDataTable`, and `validate:visual`.
37
- Analytics does not pull it in, because reading a model and drawing one are
38
- separate jobs — an app that shows a handful of figures needs no chart library.
39
- Most dashboards want both.
40
-
41
- Re-applying either is safe: an existing file is preserved, not overwritten, so a
42
- `connectors.ts` you have registered models in survives.
43
-
44
- The analytics pack can also install a local semantic-model catalog. It is a
45
- preview capability and ships off, so `schema:sql` exists only in an app that
46
- set `RAYFIN_FEATURE_FLAGS=schema-sqlite` in `rayfin/.env` before applying the
47
- pack — see [`schema-discovery`](../schema-discovery/SKILL.md). When it is
48
- absent, discover the model with read-only `INFO.VIEW.*` queries instead.
49
-
50
- - `npm run --silent schema:sql -- "<SQL>"` queries the catalog.
51
- - `npm run --silent schema:sql -- --search "<terms>"` searches tables, columns, measures, relationships, descriptions, expressions, and display folders.
52
- - `--search`, `--sql`, and `--file` are repeatable and answer in one process, so a plan of up to eight questions costs one round trip instead of eight. The whole batch shares one byte budget.
53
- - `npm run schema:refresh` explicitly rebuilds it.
54
-
55
- The first catalog query builds the database automatically.
56
- It rebuilds after the 15-minute freshness window expires.
57
- One refresh runs five schema-only source queries from a hardcoded list, plus retries if the model throttles, then later discovery is local SQLite.
58
- Do not replace this with progressive `INFO.VIEW.*` connector invocations.
59
-
60
- The database is written under `.schema/models/<itemId>/model.sqlite`.
61
- Its `catalog.md` companion describes the tables and marks the catalog **INCOMPLETE** if any source view failed.
62
- The `catalog_status` table carries the same persistent marker.
63
- Always check that marker before treating absence from an untruncated SQL result as evidence that an object does not exist.
64
-
65
- The catalog stores the untouched outputs of `INFO.VIEW.TABLES()`, `INFO.VIEW.COLUMNS()`, `INFO.VIEW.MEASURES()`, and `INFO.VIEW.RELATIONSHIPS()`.
66
- It also stores `COLUMNSTATISTICS()` output, including actual Min/Max values derived from model rows, cardinality, and maximum text length.
67
- The catalog is therefore customer data, not metadata-only content, and `.schema/` is gitignored.
68
- Applying the analytics pack adds the catalog and interrupted-query patterns to
69
- the app's root `.gitignore`, including for apps created from older templates.
70
- Do not copy, commit, or expose it.
71
-
72
- > If `npm run dev` re-optimizes dependencies on first load, add the visual
73
- > packages to `optimizeDeps.include` in `packages/frontend/vite.config.ts`. They are left out of
74
- > the base because a bare app does not install them.
75
-
76
- Read [`schema-discovery`](../schema-discovery/SKILL.md) next.
77
- It routes ordinary table, column, measure, relationship, and statistics discovery through the local catalog.
78
- After that, pull in what the job actually needs rather than printing whole catalog tables into the conversation.
79
- Each row is a sibling skill:
80
-
81
- | Read | When |
82
- |---|---|
83
- | [`dax-authoring`](../dax-authoring/SKILL.md) | You are writing a measure or a non-trivial DAX expression |
84
- | [`query-design`](../query-design/SKILL.md) | The dashboard needs more than one grain, or you are shaping a query for several visuals |
85
- | [`visuals`](../visuals/SKILL.md) | You are building any chart, or configuring `DataGrid` |
86
- | [`app-design`](../app-design/SKILL.md) | You are deciding layout and theming |
87
- | [`fabric-sdk`](../fabric-sdk/SKILL.md) | Reference material for the CLI and the runtime client |
88
-
89
- Each of those has a `references/` folder that is deliberately not required
90
- reading; open a reference when the SKILL.md points you at it for the specific
91
- thing you are doing.
92
-
93
- ## Recommended workflow
94
-
95
- Use these steps unless the user asks for a different sequence.
96
-
97
- The source contract and consuming UI have one hard ordering rule: run each real query before writing code that depends on its columns or values.
98
- That does not require leaving the starter/welcome view untouched during all schema discovery.
99
-
100
- - **Foundation:** Apply the packs, identify and attach the intended model, and preserve the authentication gate.
101
- - **Runnable shell:** Establish the app's layout, final content regions, and honest loading states while independent schema and query discovery continues.
102
- - **Data integration:** Validate each query through `npx rayfin connector invoke`, then bind the exact returned shape into the corresponding component.
103
- - **Completion:** Add interactions and polish, then run the final validation sequence once.
104
-
105
- Do not put invented business values into the shell.
106
- Loading content should reserve the real layout without pretending the source has returned.
107
-
108
- ### 1. Ask the user for a semantic model
109
-
110
- **Local `.pbix` files are not supported.** This app connects to semantic models published to the Power BI Service (cloud), not to local `.pbix` files on disk. If the user provides a local file path (e.g., `C:\...\Model.pbix`), **do not** attempt to open, upload, or search for it. Instead:
111
- 1. Inform the user that local `.pbix` files are not supported — only models published to the Power BI Service can be used.
112
- 2. Ask the user whether they would like to:
113
- - **Search the Power BI Service** for a semantic model by name — run `npx rayfin connector search` on their behalf, or
114
- - **Provide a specific online model** directly (workspace ID + dataset ID, or a Power BI / Fabric URL).
115
-
116
- Once the user confirms and connects a published semantic model, read the [schema-discovery](../schema-discovery/SKILL.md) skill.
117
- Let the fixed catalog builder materialize the five schema sources once, then progressively retrieve only relevant rows and columns with local SQL.
118
-
119
- Once the model is identified, attach it as a connector. Choose the path that matches the input the user gave you:
120
-
121
- ```bash
122
- # Path A — user wants you to search by name.
123
- # --all-workspaces is required here: this step runs before the app is
124
- # deployed, so there is no deployment to infer a workspace scope from and
125
- # the command would otherwise exit with "workspace scope is required".
126
- # --json is recommended so you can reason over displayName, workspaceName
127
- # and ids without parsing tables. If more than one could match, confirm the
128
- # right one with the user (e.g. two same-named models in different workspaces).
129
18
  npx rayfin connector search "<name>" --type fabric-semanticmodel --all-workspaces --json
130
-
131
- # Path B / C — user gave a URL or explicit IDs. Take workspaceId + itemId from
132
- # whichever they provided and declare the connector:
133
- npx rayfin connector add --type fabric-semanticmodel --workspace-id <workspaceId> --item-id <itemId> --name <alias>
134
- ```
135
-
136
- `connector add` writes the entry into `rayfin/rayfin.yml`, scaffolds
137
- `rayfin/connectors/<alias>/`, and prints version-pinned package specifications.
138
- Install those exact versions into the frontend workspace from the app root:
139
-
140
- ```sh
141
- npm install -w @rayfin-app/frontend <printed-packages-and-versions>
142
- ```
143
-
144
- Do not install them at the workspace root or create a nested lockfile.
145
-
146
- `connector add` also regenerates
147
- `packages/frontend/src/lib/connectors.ts` with the connector config, schema, and
148
- runtime.
149
- Do not hand-write those maps or replace the client installed by the connectors
150
- pack.
151
- That client composes connectors through `ExtendableRayfinClient`, and the pack
152
- selects the functions-aware variant automatically when functions are present.
153
- If the CLI reports that the wiring file is app-owned, apply the exact manual
154
- snippet it prints instead of inventing a second client recipe.
155
-
156
- ### 2. Prove the connector returns data before wiring source values
157
-
158
- `fabric-semanticmodel` is the one connector type whose `invoke` runs under your
159
- own identity, so it works **without** a deploy. That makes this check nearly
160
- free, and it is the difference between finding a dead connector now or after
161
- building a dashboard on top of it:
162
-
163
- ```bash
164
- npx rayfin connector invoke <alias> executeQuery --file ./inspection/smoke.query.json
165
- ```
166
-
167
- Where `smoke.query.json` is the operation's input envelope, not the bare query:
168
-
169
- ```json
170
- { "query": "EVALUATE TOPN(1, Sales)" }
171
- ```
172
-
173
- Do not wire source-backed values if this fails.
174
- A connector, permission, or enablement failure must remain an error rather than an endless loading state.
175
- A standalone-capable layout shell may already exist, but no amount of presentation work fixes the source contract.
176
-
177
- **Pass DAX in a `--file` payload, not with `--input`.** A DAX query is full of
178
- quotes and brackets, and an inline `--input` string goes through a shell that
179
- treats them as syntax — `EVALUATE ROW("it's", 1)` breaks on the apostrophe
180
- alone. `--file` is `JSON.parse`d, so it must be JSON: a raw `.dax` file fails
181
- before the connector is reached. The path resolves against the project root and
182
- must stay inside it.
183
-
184
- ### 3. Write and test DAX queries
185
-
186
- Follow the [dax-authoring](../dax-authoring/SKILL.md) skill to write and test DAX queries.
187
- The skill covers DAX syntax rules, query patterns, time intelligence, and an iterative test workflow.
188
- Use the [schema-discovery](../schema-discovery/SKILL.md) skill to query model metadata from the local SQLite catalog.
189
- Do not hand-run the catalog's `INFO.VIEW.*` or `COLUMNSTATISTICS()` source queries.
190
-
191
- Use `npx rayfin connector invoke <alias> executeQuery --file <query>.query.json` to test queries. This runs the query through the same connector the app uses at runtime, so the results (column names, data types, row structure) are identical to what the app will produce.
192
-
193
- Iterate on queries until they return the expected columns and data shape.
194
-
195
- Delegate query exploration only when it is a substantial independent body of work with a clear result contract.
196
- For a small dashboard or one tightly coupled query flow, keep discovery and component integration together.
197
-
198
- Once queries are validated, **promote them to the app** following the [Query and spec organization](#query-and-spec-organization) conventions above.
199
-
200
- **Does this app draw a chart or a grid?** The answer decides what to build, and
201
- it is worth settling before writing files rather than after. `ColumnMetadataMap`,
202
- Vega-Lite specs and `vegaLiteSpec` all come from the `visuals` pack: in an app
203
- that never applied it those imports do not resolve and `typecheck` fails.
204
-
205
- *If it charts* — the usual case, and the rest of this skill assumes it:
206
-
207
- 1. Save each query as a `.dax` file in `packages/frontend/src/queries/` following the naming and grouping conventions.
208
- 2. **Capture column metadata** — copy the exact column names from the query output as dictionary keys in a `columnMetadata: ColumnMetadataMap` constant. See the conventions above for the full format and rules.
209
- 3. Create the corresponding `.json` Vega-Lite spec — refer to [visuals](../visuals/SKILL.md). Use the cleaned `name` values from the metadata for Vega-Lite field encodings (they are already free of characters that require escaping). Use `displayName` values for axis titles, legend labels, and tooltip headers. Use `format` values for axis/tooltip formatting.
210
- 4. Create the barrel `.ts` file with a **factory function** that returns `{ connection, query, columnMetadata, vegaLiteSpec }`.
211
-
212
- *If it does not* — figures in custom components, a summary, an export:
213
-
214
- 1. Save each query as a `.dax` file exactly as above.
215
- 2. Create the barrel `.ts` file with a factory returning `{ connection, query }`.
216
- Nothing else: there is no spec and no metadata constant.
217
-
218
- Read the result off `data.table` — `columns` and `rows` — and render it however
219
- the app renders anything else.
220
-
221
- ### 4. UX Design for the app
222
-
223
- Principles for overall aesthetics, theming, layout, and accessibility requirements are outlined in [app-design](../app-design/SKILL.md) skill - refer to it before creating or modifying any UI component, layout, page, or style.
224
-
225
- ### 5. Build components with data (app code phase)
226
-
227
- Use the `useSemanticModelQuery` hook from `packages/frontend/src/hooks/use-semantic-model-query.ts`. Components call the factory functions from `packages/frontend/src/queries/` and destructure what those factories return.
228
-
229
- **Without the `visuals` pack**, that is `{ connection, query }`, and `data.table`
230
- is the whole rendering API:
231
-
232
- ```tsx
233
- const { data } = useSemanticModelQuery(revenueByRegion());
234
- if (data?.status === "success") {
235
- const [row] = data.table.rows; // columns and rows, nothing else
236
- return <p>Revenue this quarter: {String(row?.[0])}</p>;
237
- }
19
+ npx rayfin connector add --type fabric-semanticmodel --name <alias> --workspace-id <workspaceId> --item-id <itemId>
238
20
  ```
239
21
 
240
- Everything below is the charting path and needs `visuals` applied.
22
+ For My workspace use `me`. A `401 GroupNotAccessible` after successful search
23
+ means missing query access, not necessarily an expired session: check
24
+ `npx rayfin login status` before signing in again.
241
25
 
242
- Factories there return `{ connection, query, columnMetadata, vegaLiteSpec }`.
26
+ The CLI owns connector registration and generated runtime maps. Do not hand-wire
27
+ them or replace the template's auth/client bootstrap. The browser sends the
28
+ query and connector name, never workspace/item ids.
243
29
 
244
- Use `toDataTable()` from `packages/frontend/src/lib/to-data-table.ts` to convert the SDK's `QueryTable` (from `data.table`) into a `DataTable` by merging it with the `columnMetadata` from the factory function result. It ships with the `visuals` pack. This applies everywhere the data is consumed like rendering in `VegaVisual` or `DataGrid` (pass the `DataTable` via their `data` prop), displaying values in custom components, or any other usage.
30
+ The first [schema-discovery](../schema-discovery/SKILL.md) catalog query is the
31
+ connectivity preflight; do not run a separate connector probe. Discovery is
32
+ always available through `fabric-app-semantic-model-tools query-schema`.
33
+ Use [dax-authoring](../dax-authoring/SKILL.md) and
34
+ [query-design](../query-design/SKILL.md) together for query work.
35
+ Follow the [DAX validation gate](../dax-authoring/SKILL.md#must--prefer--avoid)
36
+ before implementing UI that depends on a query's results. Execute final queries through
37
+ `fabric-app-semantic-model-tools run-dax-query`.
38
+ Follow the consuming app's instructions for discovery/source-reading order,
39
+ UI sequencing, runtime client setup and cache policy.
245
40
 
246
- `VegaVisual` and `DataGrid` should consume factory-produced query, spec, and column metadata together so static specs remain checkable. Runtime-built specs are allowed but require browser validation. Partial static coverage exits normally; use the preview command only when coverage is zero. Follow the [visuals](../visuals/SKILL.md) validation guidance before completion.
41
+ ## Runtime contract
247
42
 
248
- `App.tsx` ships with the starter/welcome view behind the
249
- `packages/frontend/src/EmptyStatePreview.tsx` module.
250
- Replace what the app renders for the real dashboard, but preserve that module and its existing named `EmptyStatePreview` export so the starter view can evolve independently.
251
- Do not couple application guidance to the view's artwork, animation, or other internal implementation.
252
-
253
- #### Example
43
+ Use the shipped `useSemanticModelQuery` hook. Call hooks unconditionally before
44
+ loading/error returns. Check `error`, then require `data.status === "success"`
45
+ before accessing tables. Query and transport failures are normalized into
46
+ `data.status === "error"` and mirrored onto `error`; unexpected internal
47
+ failures may set only `error`.
254
48
 
255
- Charting path — this one needs `visuals`:
49
+ For coordinated semantic-model visuals, use `useQueryFactoryTarget` instead;
50
+ see [Managed semantic-model interactions](#managed-semantic-model-interactions).
256
51
 
257
52
  ```tsx
258
- import { revenueByRegion } from "@/queries/sales/revenue-by-region";
259
- import { useSemanticModelQuery } from "@/hooks/use-semantic-model-query";
260
- import { toDataTable } from "@/lib/to-data-table";
261
- import { VegaVisual, useCssTheme } from "@microsoft/fabric-visuals";
262
- import { DataGrid } from "@microsoft/fabric-datagrid";
263
-
264
- function RevenueByRegionChart() {
265
- const theme = useCssTheme();
266
- const { connection, query, columnMetadata, vegaLiteSpec } = revenueByRegion({
267
- categories: ["Category A"],
268
- });
269
-
270
- const { data, isLoading, error } = useSemanticModelQuery({
271
- connection,
53
+ const { data, isLoading, error, refetch } = useSemanticModelQuery({
54
+ connection: "sales",
272
55
  query,
273
- });
274
-
275
- if (isLoading) return <LoadingSpinner />;
276
-
277
- // Two disjoint channels, so both are checked. `error` is what the connector
278
- // threw — a wiring mistake, transport, or auth — and it clears `data`, so
279
- // skipping it renders nothing at all for those failures.
280
- if (error) return <ErrorMessage message={error.message} />;
281
-
282
- // `data.status === "error"` is a failure Power BI reported inside a successful
283
- // response: invalid DAX, a permission refusal, an overflow.
284
- if (data?.status === "error") {
285
- return <ErrorMessage message={data.error.message} />;
286
- }
287
-
288
- if (data?.status !== "success") return null;
289
-
290
- // Convert the SDK's QueryTable into a DataTable enriched with column metadata.
291
- // toDataTable merges data.table with columnMetadata so that display names,
292
- // format strings, and cleaned field names are available to visuals.
293
- const dataTable = toDataTable(data.table, columnMetadata);
294
-
295
- // Pass the DataTable to VegaVisual (chart) or DataGrid (table) via the data prop.
296
- return (
297
- <div>
298
- <VegaVisual spec={vegaLiteSpec} data={dataTable} theme={theme} />
299
- <DataGrid data={dataTable} theme={theme} />
300
- </div>
301
- );
302
- }
303
- ```
304
-
305
- **Caching:** There is none. Every call to `useSemanticModelQuery` reaches Power BI.
306
- If a screen re-queries more than you want, hold the result in state or memoize at
307
- the call site — do not reintroduce a shared cache here without a per-principal key
308
- and a TTL, because a process-global one serves another user's rows after a sign-in
309
- change and stale rows after a model refresh.
310
-
311
- **Error handling:** Two channels, and they are **disjoint** — a component that
312
- reads only one renders nothing for half its failures.
313
-
314
- - `error` carries what the connector **threw** — a wiring mistake, transport, or
315
- auth. It is the original error, so its status and service code survive. `data`
316
- is cleared when this is set.
317
- - `data.status === "error"` carries what Power BI **returned**, already
318
- categorised by the connector as `"query"` (bad DAX), `"overflow"` (row/byte cap
319
- — the rows are truncated), `"api"` (dataset-level or auth), or `"network"`.
320
- `error` stays `undefined` for these, so the category is always reachable.
321
-
322
- Check both, in either order; they never both apply. Then check `data.status`
323
- before reading `data.table`.
324
-
325
- Retry only a known transient network or service-starting condition, and cap the attempts.
326
- Authentication, permission, connector configuration, and query errors are permanent until something changes, so surface them immediately.
327
-
328
- For the connector client, the response shapes, and advanced query options, refer to
329
- the [fabric-sdk](../fabric-sdk/SKILL.md) skill.
330
-
331
- ### 6. Final validation
332
-
333
- Follow the [app-validation](../app-validation/SKILL.md) skill to
334
- run targeted checks during implementation and the required final sequence once.
335
- Rerun only a failed gate and later checks invalidated by its fix.
336
- Complete the browser check through the Fabric portal embed.
337
-
338
- ---
339
-
340
- ## Query and spec organization
341
-
342
- This section describes an app that charts its results, so it assumes the
343
- `visuals` pack. Without it there are no specs: keep the `.dax` files and the
344
- factory, and drop the `.json` and `vegaLiteSpec` from both.
345
-
346
- DAX queries and Vega-Lite specs live in `packages/frontend/src/queries/`, grouped by dashboard page or domain. Each visualization gets files sharing the same kebab-case base name: one or more `.dax` files for queries, a `.json` file for the Vega-Lite spec, and a `.ts` factory file that imports them and exports `{ connection, query, columnMetadata, vegaLiteSpec }`. The factory function accepts optional parameters to select between query variants or modify the spec:
347
-
348
- ```text
349
- packages/frontend/src/queries/
350
- ├── index.ts # Re-exports all query modules
351
- ├── {page-or-domain}/ # Group by dashboard page or domain
352
- │ ├── {visualization-name}.dax # DAX query (plain text)
353
- │ ├── {visualization-name}-{variant}.dax # Additional query variants (optional)
354
- │ ├── {visualization-name}.json # Vega-Lite spec (JSON)
355
- │ ├── {visualization-name}.ts # Factory function: imports .dax + .json, exports { connection, query, vegaLiteSpec, columnMetadata }
356
- │ └── index.ts # Re-exports all visualizations in this group
56
+ });
57
+ if (error) return <p role="alert">{error.message}</p>;
58
+ if (isLoading && !data) return <p>Loading…</p>;
59
+ if (data?.status !== "success") return null;
60
+ const table = toDataTable(data.table, columnMetadata);
357
61
  ```
358
62
 
359
- ### Example TS File
63
+ | Category | Meaning | Response |
64
+ |---|---|---|
65
+ | `query` | Power BI rejected DAX | Fix the query |
66
+ | `overflow` | Numeric value cannot fit a JS number | Inspect precision, not row count |
67
+ | `network` | No response (offline, DNS, CORS) | Retry |
68
+ | `api` | Service answered with failure | Check auth/access |
69
+ | `unknown` | Other failures | Read the diagnostics |
360
70
 
361
- **`revenue-by-region.ts`** — the factory function accepts use-case-specific parameters and uses them to modify the DAX query and/or Vega-Lite spec as appropriate:
71
+ Preserve `category`, `code`, Fabric `workload`, and original `cause`.
72
+ No error result contains a partial table.
73
+ `refetch()` runs the query again. Empty connection/query inputs leave the hook
74
+ idle. `getRayfinClient()` may be synchronous or asynchronous depending on the
75
+ template: await it before using `.connectors` outside the hook.
362
76
 
363
- ```ts
364
- import type { VisualizationSpec } from "@microsoft/fabric-visuals";
365
- import type { ColumnMetadataMap } from "@/lib/to-data-table";
366
- import baseQuery from "./revenue-by-region.dax?raw";
367
- import spec from "./revenue-by-region.json";
77
+ ## Managed semantic-model interactions
368
78
 
369
- const connection = "{connection-alias}"; // from rayfin/rayfin.yml
79
+ Follow the shared [visual-interactivity](../visual-interactivity/SKILL.md) skill
80
+ for default semantic-model coordination, manager configuration, source binding,
81
+ query-factory targets, cross-highlighting and retained-data error handling.
82
+ That skill owns interaction policy; the consuming template owns runtime caching.
370
83
 
371
- /** Column metadata keyed by original DAX column name. */
372
- const columnMetadata: ColumnMetadataMap = {
373
- "Products[Region]": { name: "ProductsRegion", displayName: "Region" },
374
- "[Total Revenue]": { name: "Total Revenue", displayName: "Total Revenue", format: "$#,0.00" },
375
- };
84
+ ## Rows and field names
376
85
 
377
- interface RevenueByRegionParams {
378
- /** Filter to specific product categories (modifies the DAX query). */
379
- categories?: string[];
380
- /** Only show regions with revenue above this threshold (modifies the Vega-Lite spec). */
381
- minRevenue?: number;
382
- }
86
+ Use one `EVALUATE` per query and aggregate or `TOPN` to bound result size.
87
+ `resultSetRowCountLimit` silently truncates without a completeness flag; a result
88
+ exactly at the cap is possibly incomplete. The hook does not forward that option.
89
+ Too many rows is not numeric `overflow`.
383
90
 
384
- export function revenueByRegion(params?: RevenueByRegionParams) {
385
- let query = baseQuery;
386
- let vegaLiteSpec = spec; // Should clone if modifying
91
+ Overflow means an Int64 beyond ±9,007,199,254,740,991 or a fixed-decimal value
92
+ whose unscaled digits exceed that limit (roughly ±900 billion at four decimals).
93
+ Only for genuine numeric overflow, use text, scale the value, or convert to
94
+ DOUBLE if the precision change is acceptable. Never reshape a stored measure
95
+ to work around a decoder bug; a conversion can drop mixed text/numeric branches.
387
96
 
388
- if (params?.categories?.length) {
389
- // make changes to the DAX query to filter by the specified categories and update the query variable
390
- }
97
+ Use exact raw returned column names as `columnMetadata` keys (`Calendar[Year]`,
98
+ `[Sales]`). Supply bracket-free `name` aliases, display names and model formats.
99
+ Do not branch on wire `dataType`, which may be `"unknown"`, to choose display
100
+ formatting. Missing first-row fields can yield missing inferred columns.
101
+ Never mutate a result; copy before sorting.
391
102
 
392
- if (params?.minRevenue != null) {
393
- // Clone spec and append a client-side filter transform to the Vega-Lite spec
394
- }
395
-
396
- return { connection, query, columnMetadata, vegaLiteSpec };
397
- }
398
- ```
103
+ Keep query factories and `.dax`/`.json` files together under frontend-relative
104
+ `src/queries/<page>/`, with kebab-case names and group/root barrel exports.
105
+ Import DAX using `?raw`; do not inline it in components.
106
+ Read [visuals](../visuals/SKILL.md) for rendering and
107
+ [app-design](../app-design/SKILL.md) for template-owned presentation policies.
399
108
 
400
- The parameters, their types, and how they modify the query or spec are **entirely use-case-specific**. Some visualizations may need no parameters at all; others may accept date ranges, top-N limits, grouping dimensions, or string search terms. The factory function is the single place that translates caller intent into DAX and/or Vega-Lite modifications.
109
+ ## Universal App installation and sequencing
401
110
 
402
- #### Query variants with multiple `.dax` files
403
-
404
- When a parameter changes the **structure** of the query (e.g. different GROUP BY columns, different aggregations), use separate `.dax` files for each variant. The factory function selects the right one:
405
-
406
- ```text
407
- revenue-trend/
408
- revenue-trend-yearly.dax
409
- revenue-trend-quarterly.dax
410
- revenue-trend-monthly.dax
411
- revenue-trend.json
412
- revenue-trend.ts
413
- ```
111
+ Run these before writing code that imports the hook:
414
112
 
415
- ```ts
416
- import yearlyQuery from "./revenue-trend-yearly.dax?raw";
417
- import quarterlyQuery from "./revenue-trend-quarterly.dax?raw";
418
- import monthlyQuery from "./revenue-trend-monthly.dax?raw";
419
-
420
- type Granularity = "yearly" | "quarterly" | "monthly";
421
-
422
- const queryByGranularity: Record<Granularity, string> = {
423
- yearly: yearlyQuery,
424
- quarterly: quarterlyQuery,
425
- monthly: monthlyQuery,
426
- };
427
-
428
- export function revenueTrend(params?: { granularity?: Granularity }) {
429
- const query = queryByGranularity[params?.granularity ?? "monthly"];
430
- return { connection, query, columnMetadata, vegaLiteSpec };
431
- }
113
+ ```sh
114
+ npm run pack:add -- connectors
115
+ npm run pack:add -- analytics
432
116
  ```
433
117
 
434
- #### Column Metadata
435
-
436
- Metadata is part of the charting path — it exists to feed `toDataTable`, and both `ColumnMetadataMap` and `ColumnDef` come from the `visuals` pack. An app with no chart in it skips this section.
118
+ Add `visuals` separately to chart results. Analytics does not install chart
119
+ rendering components or live visual validation. It installs visuals-core
120
+ and Interaction Manager for managed query factories; add the visuals pack
121
+ for chart, grid and slicer components. The analytics pack installs the schema
122
+ tools. There is no discovery feature flag. Use the client installed by the connectors
123
+ pack (`RayfinClient.create` with `connectorsService` registered under
124
+ `services.connectors`), its functions and storage variants, and the CLI-managed
125
+ registration maps. Preserve its async runtime-config resolution
126
+ (`rayfin.config.json` first), authentication gate and deployment posture.
127
+ Install exact connector package versions printed by the CLI in the frontend
128
+ workspace, not at the root, and do not create a nested lockfile:
437
129
 
438
- Capture column metadata in the barrel `.ts` file as a `columnMetadata: ColumnMetadataMap` constant (import `ColumnMetadataMap` from `@/lib/to-data-table`). The dictionary **must be keyed by the exact column name from the CLI query output** (the `name` field in `table.columns`). Do not guess or clean these keys — copy them verbatim from the query result. Each value is a `ColumnDef` (from `@microsoft/fabric-visuals-core`) containing:
439
- - `name` — a cleaned-up identifier derived from the original column name by removing `.`, `[`, `]`, `\`, `"`, and `'` characters. E.g., `"Products[Region]"` → `"ProductsRegion"`, `"[Total Revenue]"` → `"Total Revenue"` — to be used when building visual specs.
440
- - `displayName` — a human-readable label sourced from the semantic model schema (e.g., `"Region"`, `"Total Revenue"`). Used for axis titles, grid headers, and tooltips.
441
- - `format` — a VBA/ECMA-376 format string for number/date formatting (e.g., `#,##0.00`, `0.00%`, and `mm/dd/yyyy`). Omit for text type columns.
442
-
443
- **Example workflow:** Run `npx rayfin connector invoke myModel executeQuery --file ./inspection/probe.query.json`, observe the output columns are `[SalesPersonID]` and `[Name]`, then use those exact strings as metadata keys:
444
-
445
- ```ts
446
- export const columnMetadata: ColumnMetadataMap = {
447
- "[SalesPersonID]": { name: "SalesPersonID", displayName: "Sales Person ID" },
448
- "[Name]": { name: "Name", displayName: "Name" },
449
- };
130
+ ```sh
131
+ npm install -w @rayfin-app/frontend <printed-packages-and-versions>
450
132
  ```
451
133
 
452
- #### Key rules
453
-
454
- - **All DAX lives in `.dax` files.** Never inline full DAX query strings in `.ts` factory files. If a parameter changes the query structure, create a separate `.dax` file for each variant and select the right one in the factory function. Small modifications — such as replacing filter value placeholders, wrapping the query with `CALCULATETABLE` to apply filters, or substituting a column reference — are acceptable in `.ts`, but the base query must always come from a `.dax` import.
455
- - **Name files after the visualization they drive.** Use kebab-case base names (e.g., `revenue-by-region.dax`, `revenue-by-region.json`, `revenue-by-region.ts`). Variant `.dax` files append a suffix (e.g., `revenue-trend-yearly.dax`, `revenue-trend-quarterly.dax`).
456
- - **Use `.dax` for queries.** Plain-text DAX files keep queries readable and diff-friendly. Import them with Vite's `?raw` suffix.
457
- - **Use `.json` for specs.** JSON files get free schema validation in editors and are importable as modules by default in Vite.
458
- - **Barrel `.ts` exports a factory function.** The function name is the camelCase version of the kebab-case file name (e.g., `revenueByRegion` for `revenue-by-region.ts`). It accepts optional parameters (typed per use case) and returns `{ connection, query, columnMetadata, vegaLiteSpec }` — or just `{ connection, query }` when the app draws no charts.
459
- - **Group by page/domain.** Use subfolders when the dashboard has multiple pages or logical sections. For simple single-page dashboards, a flat structure under `packages/frontend/src/queries/` is fine.
460
- - **Re-export via `index.ts`.** Each subfolder and the root `packages/frontend/src/queries/index.ts` should re-export all modules for clean imports.
134
+ Frontend-relative paths in the shared guidance are under `packages/frontend/`.
135
+ Follow the shared [DAX validation gate](../dax-authoring/SKILL.md#must--prefer--avoid):
136
+ execute and validate each final DAX query before implementing UI that depends on its results.
137
+ Follow the app's `AGENTS.md` for app shells, loading regions and other UI that does
138
+ not depend on DAX results. Package installation and connector setup can precede this gate.
139
+ Never show invented business values as if the source returned them.
140
+ Keep this template's data-source policies and use its existing records and
141
+ other capabilities as appropriate; analytics is only for semantic-model reads.
142
+ `App.tsx` ships with the starter/welcome view behind the
143
+ `packages/frontend/src/EmptyStatePreview.tsx` module. Replace what the app renders
144
+ for the real dashboard, but preserve that module and its
145
+ existing named `EmptyStatePreview` export so the starter view can evolve independently.
146
+
147
+ This hook is uncached. Do not import data-app session-cache policies. Do not add
148
+ a cache without principal isolation and an explicit lifetime policy.
149
+ The installed `querySemanticModel` and `executeQueryFactory` are also uncached;
150
+ the managed target hook shares the same typed loader/selection contract as the
151
+ data-app without copying its synchronous client or cache bootstrap.
152
+ Run `npm run typecheck` separately from the build (`--noCheck`), and follow
153
+ [app-validation](../app-validation/SKILL.md), including browser checks.
154
+
155
+ ## Catalog migration
156
+
157
+ Schema discovery uses `npx fabric-app-semantic-model-tools query-schema` and DAX
158
+ uses `npx fabric-app-semantic-model-tools run-dax-query`. Do not run legacy
159
+ catalog helper scripts alongside these. The `schema:sql`/`schema:refresh`
160
+ aliases installed by this pack invoke the tools package, not the legacy kit.
161
+ For an existing app, remove only pristine old helper scripts and replace old
162
+ aliases after reviewing app customizations; remove `.schema/` to rebuild the
163
+ different catalog format. Never overwrite customized scripts.
164
+ The new cache and interrupted-query payloads remain ignored and are customer
165
+ data, not public metadata. Do not commit or expose them.