@microsoft/rayfin-semantic-model-pack 0.0.0-stage → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG +10 -0
- package/LICENSE +21 -0
- package/README.md +65 -2
- package/package.json +10 -3
- package/skills/analytics/SKILL.md +460 -0
- package/skills/analytics/kit/hooks/use-semantic-model-query.spec.ts +342 -0
- package/skills/analytics/kit/hooks/use-semantic-model-query.ts +229 -0
- package/skills/analytics/kit/lib/connectors.ts +94 -0
- package/skills/analytics/kit/scripts/rayfin-connector.mjs +265 -0
- package/skills/analytics/kit/scripts/rayfin-connector.test.mjs +243 -0
- package/skills/analytics/kit/scripts/schema-sql-batch.test.mjs +2097 -0
- package/skills/analytics/kit/scripts/schema-sql-cli.test.mjs +488 -0
- package/skills/analytics/kit/scripts/schema-sql.mjs +2650 -0
- package/skills/analytics/kit/scripts/schema-sql.test.mjs +1387 -0
- package/skills/analytics/kit/scripts/schema.mjs +166 -0
- package/skills/analytics/kit/scripts/schema.test.mjs +271 -0
- package/skills/analytics/pack.json +49 -0
- package/skills/dax-authoring/SKILL.md +172 -0
- package/skills/dax-authoring/references/dax-core-reference.md +862 -0
- package/skills/dax-authoring/references/dax-query-patterns.md +396 -0
- package/skills/dax-authoring/references/dax-time-intelligence.md +347 -0
- package/skills/query-design/SKILL.md +114 -0
- package/skills/query-design/references/anti-patterns.md +285 -0
- package/skills/query-design/references/filter-strategy.md +68 -0
- package/skills/query-design/references/format-strings.md +62 -0
- package/skills/query-design/references/highlight-queries.md +122 -0
- package/skills/query-design/references/multi-grain-patterns.md +159 -0
- package/skills/schema-discovery/SKILL.md +382 -0
- package/skills/schema-discovery/references/discovery-queries.md +153 -0
package/CHANGELOG
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `@microsoft/rayfin-semantic-model-pack` will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## 1.0.0 - Initial release
|
|
9
|
+
|
|
10
|
+
- Add the `analytics` capability pack and the `dax-authoring`, `query-design`, and `schema-discovery` skills.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
Copyright (c) Microsoft Corporation.
|
|
2
|
+
|
|
3
|
+
MIT License
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,66 @@
|
|
|
1
|
-
#
|
|
1
|
+
# @microsoft/rayfin-semantic-model-pack
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Agent skills and starter code for Power BI semantic-model analytics in Rayfin apps.
|
|
4
|
+
|
|
5
|
+
## Quick Reference
|
|
6
|
+
|
|
7
|
+
> **Package**: `@microsoft/rayfin-semantic-model-pack`
|
|
8
|
+
> **Purpose**: Capability pack that teaches a coding agent to query a Power BI semantic model from a Rayfin app: connect the model, discover its schema, write DAX, and shape queries for visuals.
|
|
9
|
+
> **Use when**: A Rayfin app template adds semantic-model analytics to an app.
|
|
10
|
+
> **Do NOT use when**: You need a runtime library (use `@microsoft/rayfin-connector-fabric-semanticmodel`), or charts and tables (use `@microsoft/rayfin-visuals-pack`).
|
|
11
|
+
> **Contents**: `skills/analytics`, `skills/dax-authoring`, `skills/query-design`, `skills/schema-discovery`
|
|
12
|
+
> **Install**: `npm install @microsoft/rayfin-semantic-model-pack`
|
|
13
|
+
|
|
14
|
+
## Contents
|
|
15
|
+
|
|
16
|
+
The package has no entry point. Each folder under `skills/` is one agent skill
|
|
17
|
+
and maps 1:1 to `.agents/skills/<name>/` in an app.
|
|
18
|
+
|
|
19
|
+
| Skill | Contents |
|
|
20
|
+
| --- | --- |
|
|
21
|
+
| `analytics` | Capability pack: `SKILL.md`, `pack.json`, and `kit/` (semantic-model query hook, connector wiring, schema scripts) |
|
|
22
|
+
| `dax-authoring` | Writing and testing DAX queries |
|
|
23
|
+
| `query-design` | Splitting work between DAX and TypeScript, including multi-grain and cross-highlight queries |
|
|
24
|
+
| `schema-discovery` | Exploring a model's tables, columns, measures, and relationships |
|
|
25
|
+
|
|
26
|
+
`pack.json` tells a template how to apply the pack, such as which dependencies
|
|
27
|
+
to add and which kit files to copy where. The Rayfin universal template applies
|
|
28
|
+
it with `npm run pack:add -- analytics`, and the copy targets assume that
|
|
29
|
+
template's layout.
|
|
30
|
+
|
|
31
|
+
## Usage
|
|
32
|
+
|
|
33
|
+
Resolve `@microsoft/rayfin-semantic-model-pack/package.json`, then copy each
|
|
34
|
+
`skills/<name>/` folder next to it into the app's `.agents/skills/<name>/`.
|
|
35
|
+
|
|
36
|
+
## Key Constraints & Gotchas
|
|
37
|
+
|
|
38
|
+
- Apply the template's `connectors` pack first. It installs the
|
|
39
|
+
`@/lib/rayfin-client` that the kit imports.
|
|
40
|
+
- The skills link to `visuals` from `@microsoft/rayfin-visuals-pack`, and to
|
|
41
|
+
`app-design`, `fabric-sdk`, and `app-validation`, which the app template
|
|
42
|
+
provides.
|
|
43
|
+
- The schema scripts need Node.js 22.13 or later.
|
|
44
|
+
|
|
45
|
+
## Trademarks
|
|
46
|
+
|
|
47
|
+
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.
|
|
48
|
+
|
|
49
|
+
## Security
|
|
50
|
+
|
|
51
|
+
Microsoft takes the security of our software products and services seriously, which
|
|
52
|
+
includes all source code repositories in our GitHub organizations.
|
|
53
|
+
|
|
54
|
+
**Please do not report security vulnerabilities through public GitHub issues.**
|
|
55
|
+
|
|
56
|
+
For security reporting information, locations, contact information, and policies,
|
|
57
|
+
please review the latest guidance for Microsoft repositories at
|
|
58
|
+
[https://aka.ms/SECURITY.md](https://aka.ms/SECURITY.md).
|
|
59
|
+
|
|
60
|
+
## Code of conduct
|
|
61
|
+
|
|
62
|
+
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
|
|
63
|
+
|
|
64
|
+
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
|
|
65
|
+
|
|
66
|
+
See full [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/)
|
package/package.json
CHANGED
|
@@ -1,6 +1,13 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@microsoft/rayfin-semantic-model-pack",
|
|
3
|
-
"
|
|
4
|
-
"
|
|
5
|
-
"
|
|
3
|
+
"description": "Agent skills and starter code for Power BI semantic-model analytics in Rayfin apps",
|
|
4
|
+
"version": "1.0.0",
|
|
5
|
+
"author": "Microsoft",
|
|
6
|
+
"license": "MIT",
|
|
7
|
+
"type": "module",
|
|
8
|
+
"files": [
|
|
9
|
+
"skills",
|
|
10
|
+
"CHANGELOG",
|
|
11
|
+
"LICENSE"
|
|
12
|
+
]
|
|
6
13
|
}
|
|
@@ -0,0 +1,460 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: analytics
|
|
3
|
+
description: >
|
|
4
|
+
Use when the app reads an existing Power BI semantic model — a dashboard,
|
|
5
|
+
report, KPI, or any view over data the app does not own. Turns on the Fabric
|
|
6
|
+
data client and the semantic-model query hook, then covers the whole workflow:
|
|
7
|
+
pick a model, discover its schema, write and test DAX, and render it. A chart
|
|
8
|
+
over data the app already has needs the visuals skill instead, not this one.
|
|
9
|
+
Triggers: semantic model, dataset, Power BI, DAX, measure, "our sales data",
|
|
10
|
+
"from the model", "the existing report".
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Analytics — reading a semantic model
|
|
14
|
+
|
|
15
|
+
## Turn it on first
|
|
16
|
+
|
|
17
|
+
```sh
|
|
18
|
+
npm run pack:add -- connectors
|
|
19
|
+
npm run pack:add -- analytics
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
The connectors pack installs the composable client in
|
|
23
|
+
`packages/frontend/src/lib/rayfin-client.ts`.
|
|
24
|
+
The analytics pack installs the semantic-model connector packages and the
|
|
25
|
+
`useSemanticModelQuery` hook.
|
|
26
|
+
It also turns the `connectors` service on in `rayfin/rayfin.yml`, which unlocks
|
|
27
|
+
the `rayfin connector` commands.
|
|
28
|
+
Run both before writing any code.
|
|
29
|
+
|
|
30
|
+
To chart what the queries return, add the visuals pack as well:
|
|
31
|
+
|
|
32
|
+
```sh
|
|
33
|
+
npm run pack:add -- visuals
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
That brings `VegaVisual`, `DataGrid`, `toDataTable`, and `validate:visual`.
|
|
37
|
+
Analytics does not pull it in, because reading a model and drawing one are
|
|
38
|
+
separate jobs — an app that shows a handful of figures needs no chart library.
|
|
39
|
+
Most dashboards want both.
|
|
40
|
+
|
|
41
|
+
Re-applying either is safe: an existing file is preserved, not overwritten, so a
|
|
42
|
+
`connectors.ts` you have registered models in survives.
|
|
43
|
+
|
|
44
|
+
The analytics pack can also install a local semantic-model catalog. It is a
|
|
45
|
+
preview capability and ships off, so `schema:sql` exists only in an app that
|
|
46
|
+
set `RAYFIN_FEATURE_FLAGS=schema-sqlite` in `rayfin/.env` before applying the
|
|
47
|
+
pack — see [`schema-discovery`](../schema-discovery/SKILL.md). When it is
|
|
48
|
+
absent, discover the model with read-only `INFO.VIEW.*` queries instead.
|
|
49
|
+
|
|
50
|
+
- `npm run --silent schema:sql -- "<SQL>"` queries the catalog.
|
|
51
|
+
- `npm run --silent schema:sql -- --search "<terms>"` searches tables, columns, measures, relationships, descriptions, expressions, and display folders.
|
|
52
|
+
- `--search`, `--sql`, and `--file` are repeatable and answer in one process, so a plan of up to eight questions costs one round trip instead of eight. The whole batch shares one byte budget.
|
|
53
|
+
- `npm run schema:refresh` explicitly rebuilds it.
|
|
54
|
+
|
|
55
|
+
The first catalog query builds the database automatically.
|
|
56
|
+
It rebuilds after the 15-minute freshness window expires.
|
|
57
|
+
One refresh runs five schema-only source queries from a hardcoded list, plus retries if the model throttles, then later discovery is local SQLite.
|
|
58
|
+
Do not replace this with progressive `INFO.VIEW.*` connector invocations.
|
|
59
|
+
|
|
60
|
+
The database is written under `.schema/models/<itemId>/model.sqlite`.
|
|
61
|
+
Its `catalog.md` companion describes the tables and marks the catalog **INCOMPLETE** if any source view failed.
|
|
62
|
+
The `catalog_status` table carries the same persistent marker.
|
|
63
|
+
Always check that marker before treating absence from an untruncated SQL result as evidence that an object does not exist.
|
|
64
|
+
|
|
65
|
+
The catalog stores the untouched outputs of `INFO.VIEW.TABLES()`, `INFO.VIEW.COLUMNS()`, `INFO.VIEW.MEASURES()`, and `INFO.VIEW.RELATIONSHIPS()`.
|
|
66
|
+
It also stores `COLUMNSTATISTICS()` output, including actual Min/Max values derived from model rows, cardinality, and maximum text length.
|
|
67
|
+
The catalog is therefore customer data, not metadata-only content, and `.schema/` is gitignored.
|
|
68
|
+
Applying the analytics pack adds the catalog and interrupted-query patterns to
|
|
69
|
+
the app's root `.gitignore`, including for apps created from older templates.
|
|
70
|
+
Do not copy, commit, or expose it.
|
|
71
|
+
|
|
72
|
+
> If `npm run dev` re-optimizes dependencies on first load, add the visual
|
|
73
|
+
> packages to `optimizeDeps.include` in `packages/frontend/vite.config.ts`. They are left out of
|
|
74
|
+
> the base because a bare app does not install them.
|
|
75
|
+
|
|
76
|
+
Read [`schema-discovery`](../schema-discovery/SKILL.md) next.
|
|
77
|
+
It routes ordinary table, column, measure, relationship, and statistics discovery through the local catalog.
|
|
78
|
+
After that, pull in what the job actually needs rather than printing whole catalog tables into the conversation.
|
|
79
|
+
Each row is a sibling skill:
|
|
80
|
+
|
|
81
|
+
| Read | When |
|
|
82
|
+
|---|---|
|
|
83
|
+
| [`dax-authoring`](../dax-authoring/SKILL.md) | You are writing a measure or a non-trivial DAX expression |
|
|
84
|
+
| [`query-design`](../query-design/SKILL.md) | The dashboard needs more than one grain, or you are shaping a query for several visuals |
|
|
85
|
+
| [`visuals`](../visuals/SKILL.md) | You are building any chart, or configuring `DataGrid` |
|
|
86
|
+
| [`app-design`](../app-design/SKILL.md) | You are deciding layout and theming |
|
|
87
|
+
| [`fabric-sdk`](../fabric-sdk/SKILL.md) | Reference material for the CLI and the runtime client |
|
|
88
|
+
|
|
89
|
+
Each of those has a `references/` folder that is deliberately not required
|
|
90
|
+
reading; open a reference when the SKILL.md points you at it for the specific
|
|
91
|
+
thing you are doing.
|
|
92
|
+
|
|
93
|
+
## Recommended workflow
|
|
94
|
+
|
|
95
|
+
Use these steps unless the user asks for a different sequence.
|
|
96
|
+
|
|
97
|
+
The source contract and consuming UI have one hard ordering rule: run each real query before writing code that depends on its columns or values.
|
|
98
|
+
That does not require leaving the starter/welcome view untouched during all schema discovery.
|
|
99
|
+
|
|
100
|
+
- **Foundation:** Apply the packs, identify and attach the intended model, and preserve the authentication gate.
|
|
101
|
+
- **Runnable shell:** Establish the app's layout, final content regions, and honest loading states while independent schema and query discovery continues.
|
|
102
|
+
- **Data integration:** Validate each query through `npx rayfin connector invoke`, then bind the exact returned shape into the corresponding component.
|
|
103
|
+
- **Completion:** Add interactions and polish, then run the final validation sequence once.
|
|
104
|
+
|
|
105
|
+
Do not put invented business values into the shell.
|
|
106
|
+
Loading content should reserve the real layout without pretending the source has returned.
|
|
107
|
+
|
|
108
|
+
### 1. Ask the user for a semantic model
|
|
109
|
+
|
|
110
|
+
**Local `.pbix` files are not supported.** This app connects to semantic models published to the Power BI Service (cloud), not to local `.pbix` files on disk. If the user provides a local file path (e.g., `C:\...\Model.pbix`), **do not** attempt to open, upload, or search for it. Instead:
|
|
111
|
+
1. Inform the user that local `.pbix` files are not supported — only models published to the Power BI Service can be used.
|
|
112
|
+
2. Ask the user whether they would like to:
|
|
113
|
+
- **Search the Power BI Service** for a semantic model by name — run `npx rayfin connector search` on their behalf, or
|
|
114
|
+
- **Provide a specific online model** directly (workspace ID + dataset ID, or a Power BI / Fabric URL).
|
|
115
|
+
|
|
116
|
+
Once the user confirms and connects a published semantic model, read the [schema-discovery](../schema-discovery/SKILL.md) skill.
|
|
117
|
+
Let the fixed catalog builder materialize the five schema sources once, then progressively retrieve only relevant rows and columns with local SQL.
|
|
118
|
+
|
|
119
|
+
Once the model is identified, attach it as a connector. Choose the path that matches the input the user gave you:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
# Path A — user wants you to search by name.
|
|
123
|
+
# --all-workspaces is required here: this step runs before the app is
|
|
124
|
+
# deployed, so there is no deployment to infer a workspace scope from and
|
|
125
|
+
# the command would otherwise exit with "workspace scope is required".
|
|
126
|
+
# --json is recommended so you can reason over displayName, workspaceName
|
|
127
|
+
# and ids without parsing tables. If more than one could match, confirm the
|
|
128
|
+
# right one with the user (e.g. two same-named models in different workspaces).
|
|
129
|
+
npx rayfin connector search "<name>" --type fabric-semanticmodel --all-workspaces --json
|
|
130
|
+
|
|
131
|
+
# Path B / C — user gave a URL or explicit IDs. Take workspaceId + itemId from
|
|
132
|
+
# whichever they provided and declare the connector:
|
|
133
|
+
npx rayfin connector add --type fabric-semanticmodel --workspace-id <workspaceId> --item-id <itemId> --name <alias>
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
`connector add` writes the entry into `rayfin/rayfin.yml`, scaffolds
|
|
137
|
+
`rayfin/connectors/<alias>/`, and prints version-pinned package specifications.
|
|
138
|
+
Install those exact versions into the frontend workspace from the app root:
|
|
139
|
+
|
|
140
|
+
```sh
|
|
141
|
+
npm install -w @rayfin-app/frontend <printed-packages-and-versions>
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Do not install them at the workspace root or create a nested lockfile.
|
|
145
|
+
|
|
146
|
+
`connector add` also regenerates
|
|
147
|
+
`packages/frontend/src/lib/connectors.ts` with the connector config, schema, and
|
|
148
|
+
runtime.
|
|
149
|
+
Do not hand-write those maps or replace the client installed by the connectors
|
|
150
|
+
pack.
|
|
151
|
+
That client composes connectors through `ExtendableRayfinClient`, and the pack
|
|
152
|
+
selects the functions-aware variant automatically when functions are present.
|
|
153
|
+
If the CLI reports that the wiring file is app-owned, apply the exact manual
|
|
154
|
+
snippet it prints instead of inventing a second client recipe.
|
|
155
|
+
|
|
156
|
+
### 2. Prove the connector returns data before wiring source values
|
|
157
|
+
|
|
158
|
+
`fabric-semanticmodel` is the one connector type whose `invoke` runs under your
|
|
159
|
+
own identity, so it works **without** a deploy. That makes this check nearly
|
|
160
|
+
free, and it is the difference between finding a dead connector now or after
|
|
161
|
+
building a dashboard on top of it:
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
npx rayfin connector invoke <alias> executeQuery --file ./inspection/smoke.query.json
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
Where `smoke.query.json` is the operation's input envelope, not the bare query:
|
|
168
|
+
|
|
169
|
+
```json
|
|
170
|
+
{ "query": "EVALUATE TOPN(1, Sales)" }
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Do not wire source-backed values if this fails.
|
|
174
|
+
A connector, permission, or enablement failure must remain an error rather than an endless loading state.
|
|
175
|
+
A standalone-capable layout shell may already exist, but no amount of presentation work fixes the source contract.
|
|
176
|
+
|
|
177
|
+
**Pass DAX in a `--file` payload, not with `--input`.** A DAX query is full of
|
|
178
|
+
quotes and brackets, and an inline `--input` string goes through a shell that
|
|
179
|
+
treats them as syntax — `EVALUATE ROW("it's", 1)` breaks on the apostrophe
|
|
180
|
+
alone. `--file` is `JSON.parse`d, so it must be JSON: a raw `.dax` file fails
|
|
181
|
+
before the connector is reached. The path resolves against the project root and
|
|
182
|
+
must stay inside it.
|
|
183
|
+
|
|
184
|
+
### 3. Write and test DAX queries
|
|
185
|
+
|
|
186
|
+
Follow the [dax-authoring](../dax-authoring/SKILL.md) skill to write and test DAX queries.
|
|
187
|
+
The skill covers DAX syntax rules, query patterns, time intelligence, and an iterative test workflow.
|
|
188
|
+
Use the [schema-discovery](../schema-discovery/SKILL.md) skill to query model metadata from the local SQLite catalog.
|
|
189
|
+
Do not hand-run the catalog's `INFO.VIEW.*` or `COLUMNSTATISTICS()` source queries.
|
|
190
|
+
|
|
191
|
+
Use `npx rayfin connector invoke <alias> executeQuery --file <query>.query.json` to test queries. This runs the query through the same connector the app uses at runtime, so the results (column names, data types, row structure) are identical to what the app will produce.
|
|
192
|
+
|
|
193
|
+
Iterate on queries until they return the expected columns and data shape.
|
|
194
|
+
|
|
195
|
+
Delegate query exploration only when it is a substantial independent body of work with a clear result contract.
|
|
196
|
+
For a small dashboard or one tightly coupled query flow, keep discovery and component integration together.
|
|
197
|
+
|
|
198
|
+
Once queries are validated, **promote them to the app** following the [Query and spec organization](#query-and-spec-organization) conventions above.
|
|
199
|
+
|
|
200
|
+
**Does this app draw a chart or a grid?** The answer decides what to build, and
|
|
201
|
+
it is worth settling before writing files rather than after. `ColumnMetadataMap`,
|
|
202
|
+
Vega-Lite specs and `vegaLiteSpec` all come from the `visuals` pack: in an app
|
|
203
|
+
that never applied it those imports do not resolve and `typecheck` fails.
|
|
204
|
+
|
|
205
|
+
*If it charts* — the usual case, and the rest of this skill assumes it:
|
|
206
|
+
|
|
207
|
+
1. Save each query as a `.dax` file in `packages/frontend/src/queries/` following the naming and grouping conventions.
|
|
208
|
+
2. **Capture column metadata** — copy the exact column names from the query output as dictionary keys in a `columnMetadata: ColumnMetadataMap` constant. See the conventions above for the full format and rules.
|
|
209
|
+
3. Create the corresponding `.json` Vega-Lite spec — refer to [visuals](../visuals/SKILL.md). Use the cleaned `name` values from the metadata for Vega-Lite field encodings (they are already free of characters that require escaping). Use `displayName` values for axis titles, legend labels, and tooltip headers. Use `format` values for axis/tooltip formatting.
|
|
210
|
+
4. Create the barrel `.ts` file with a **factory function** that returns `{ connection, query, columnMetadata, vegaLiteSpec }`.
|
|
211
|
+
|
|
212
|
+
*If it does not* — figures in custom components, a summary, an export:
|
|
213
|
+
|
|
214
|
+
1. Save each query as a `.dax` file exactly as above.
|
|
215
|
+
2. Create the barrel `.ts` file with a factory returning `{ connection, query }`.
|
|
216
|
+
Nothing else: there is no spec and no metadata constant.
|
|
217
|
+
|
|
218
|
+
Read the result off `data.table` — `columns` and `rows` — and render it however
|
|
219
|
+
the app renders anything else.
|
|
220
|
+
|
|
221
|
+
### 4. UX Design for the app
|
|
222
|
+
|
|
223
|
+
Principles for overall aesthetics, theming, layout, and accessibility requirements are outlined in [app-design](../app-design/SKILL.md) skill - refer to it before creating or modifying any UI component, layout, page, or style.
|
|
224
|
+
|
|
225
|
+
### 5. Build components with data (app code phase)
|
|
226
|
+
|
|
227
|
+
Use the `useSemanticModelQuery` hook from `packages/frontend/src/hooks/use-semantic-model-query.ts`. Components call the factory functions from `packages/frontend/src/queries/` and destructure what those factories return.
|
|
228
|
+
|
|
229
|
+
**Without the `visuals` pack**, that is `{ connection, query }`, and `data.table`
|
|
230
|
+
is the whole rendering API:
|
|
231
|
+
|
|
232
|
+
```tsx
|
|
233
|
+
const { data } = useSemanticModelQuery(revenueByRegion());
|
|
234
|
+
if (data?.status === "success") {
|
|
235
|
+
const [row] = data.table.rows; // columns and rows, nothing else
|
|
236
|
+
return <p>Revenue this quarter: {String(row?.[0])}</p>;
|
|
237
|
+
}
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
Everything below is the charting path and needs `visuals` applied.
|
|
241
|
+
|
|
242
|
+
Factories there return `{ connection, query, columnMetadata, vegaLiteSpec }`.
|
|
243
|
+
|
|
244
|
+
Use `toDataTable()` from `packages/frontend/src/lib/to-data-table.ts` to convert the SDK's `QueryTable` (from `data.table`) into a `DataTable` by merging it with the `columnMetadata` from the factory function result. It ships with the `visuals` pack. This applies everywhere the data is consumed like rendering in `VegaVisual` or `DataGrid` (pass the `DataTable` via their `data` prop), displaying values in custom components, or any other usage.
|
|
245
|
+
|
|
246
|
+
`VegaVisual` and `DataGrid` should consume factory-produced query, spec, and column metadata together so static specs remain checkable. Runtime-built specs are allowed but require browser validation. Partial static coverage exits normally; use the preview command only when coverage is zero. Follow the [visuals](../visuals/SKILL.md) validation guidance before completion.
|
|
247
|
+
|
|
248
|
+
`App.tsx` ships with the starter/welcome view behind the
|
|
249
|
+
`packages/frontend/src/EmptyStatePreview.tsx` module.
|
|
250
|
+
Replace what the app renders for the real dashboard, but preserve that module and its existing named `EmptyStatePreview` export so the starter view can evolve independently.
|
|
251
|
+
Do not couple application guidance to the view's artwork, animation, or other internal implementation.
|
|
252
|
+
|
|
253
|
+
#### Example
|
|
254
|
+
|
|
255
|
+
Charting path — this one needs `visuals`:
|
|
256
|
+
|
|
257
|
+
```tsx
|
|
258
|
+
import { revenueByRegion } from "@/queries/sales/revenue-by-region";
|
|
259
|
+
import { useSemanticModelQuery } from "@/hooks/use-semantic-model-query";
|
|
260
|
+
import { toDataTable } from "@/lib/to-data-table";
|
|
261
|
+
import { VegaVisual, useCssTheme } from "@microsoft/fabric-visuals";
|
|
262
|
+
import { DataGrid } from "@microsoft/fabric-datagrid";
|
|
263
|
+
|
|
264
|
+
function RevenueByRegionChart() {
|
|
265
|
+
const theme = useCssTheme();
|
|
266
|
+
const { connection, query, columnMetadata, vegaLiteSpec } = revenueByRegion({
|
|
267
|
+
categories: ["Category A"],
|
|
268
|
+
});
|
|
269
|
+
|
|
270
|
+
const { data, isLoading, error } = useSemanticModelQuery({
|
|
271
|
+
connection,
|
|
272
|
+
query,
|
|
273
|
+
});
|
|
274
|
+
|
|
275
|
+
if (isLoading) return <LoadingSpinner />;
|
|
276
|
+
|
|
277
|
+
// Two disjoint channels, so both are checked. `error` is what the connector
|
|
278
|
+
// threw — a wiring mistake, transport, or auth — and it clears `data`, so
|
|
279
|
+
// skipping it renders nothing at all for those failures.
|
|
280
|
+
if (error) return <ErrorMessage message={error.message} />;
|
|
281
|
+
|
|
282
|
+
// `data.status === "error"` is a failure Power BI reported inside a successful
|
|
283
|
+
// response: invalid DAX, a permission refusal, an overflow.
|
|
284
|
+
if (data?.status === "error") {
|
|
285
|
+
return <ErrorMessage message={data.error.message} />;
|
|
286
|
+
}
|
|
287
|
+
|
|
288
|
+
if (data?.status !== "success") return null;
|
|
289
|
+
|
|
290
|
+
// Convert the SDK's QueryTable into a DataTable enriched with column metadata.
|
|
291
|
+
// toDataTable merges data.table with columnMetadata so that display names,
|
|
292
|
+
// format strings, and cleaned field names are available to visuals.
|
|
293
|
+
const dataTable = toDataTable(data.table, columnMetadata);
|
|
294
|
+
|
|
295
|
+
// Pass the DataTable to VegaVisual (chart) or DataGrid (table) via the data prop.
|
|
296
|
+
return (
|
|
297
|
+
<div>
|
|
298
|
+
<VegaVisual spec={vegaLiteSpec} data={dataTable} theme={theme} />
|
|
299
|
+
<DataGrid data={dataTable} theme={theme} />
|
|
300
|
+
</div>
|
|
301
|
+
);
|
|
302
|
+
}
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
**Caching:** There is none. Every call to `useSemanticModelQuery` reaches Power BI.
|
|
306
|
+
If a screen re-queries more than you want, hold the result in state or memoize at
|
|
307
|
+
the call site — do not reintroduce a shared cache here without a per-principal key
|
|
308
|
+
and a TTL, because a process-global one serves another user's rows after a sign-in
|
|
309
|
+
change and stale rows after a model refresh.
|
|
310
|
+
|
|
311
|
+
**Error handling:** Two channels, and they are **disjoint** — a component that
|
|
312
|
+
reads only one renders nothing for half its failures.
|
|
313
|
+
|
|
314
|
+
- `error` carries what the connector **threw** — a wiring mistake, transport, or
|
|
315
|
+
auth. It is the original error, so its status and service code survive. `data`
|
|
316
|
+
is cleared when this is set.
|
|
317
|
+
- `data.status === "error"` carries what Power BI **returned**, already
|
|
318
|
+
categorised by the connector as `"query"` (bad DAX), `"overflow"` (row/byte cap
|
|
319
|
+
— the rows are truncated), `"api"` (dataset-level or auth), or `"network"`.
|
|
320
|
+
`error` stays `undefined` for these, so the category is always reachable.
|
|
321
|
+
|
|
322
|
+
Check both, in either order; they never both apply. Then check `data.status`
|
|
323
|
+
before reading `data.table`.
|
|
324
|
+
|
|
325
|
+
Retry only a known transient network or service-starting condition, and cap the attempts.
|
|
326
|
+
Authentication, permission, connector configuration, and query errors are permanent until something changes, so surface them immediately.
|
|
327
|
+
|
|
328
|
+
For the connector client, the response shapes, and advanced query options, refer to
|
|
329
|
+
the [fabric-sdk](../fabric-sdk/SKILL.md) skill.
|
|
330
|
+
|
|
331
|
+
### 6. Final validation
|
|
332
|
+
|
|
333
|
+
Follow the [app-validation](../app-validation/SKILL.md) skill to
|
|
334
|
+
run targeted checks during implementation and the required final sequence once.
|
|
335
|
+
Rerun only a failed gate and later checks invalidated by its fix.
|
|
336
|
+
Complete the browser check through the Fabric portal embed.
|
|
337
|
+
|
|
338
|
+
---
|
|
339
|
+
|
|
340
|
+
## Query and spec organization
|
|
341
|
+
|
|
342
|
+
This section describes an app that charts its results, so it assumes the
|
|
343
|
+
`visuals` pack. Without it there are no specs: keep the `.dax` files and the
|
|
344
|
+
factory, and drop the `.json` and `vegaLiteSpec` from both.
|
|
345
|
+
|
|
346
|
+
DAX queries and Vega-Lite specs live in `packages/frontend/src/queries/`, grouped by dashboard page or domain. Each visualization gets files sharing the same kebab-case base name: one or more `.dax` files for queries, a `.json` file for the Vega-Lite spec, and a `.ts` factory file that imports them and exports `{ connection, query, columnMetadata, vegaLiteSpec }`. The factory function accepts optional parameters to select between query variants or modify the spec:
|
|
347
|
+
|
|
348
|
+
```text
|
|
349
|
+
packages/frontend/src/queries/
|
|
350
|
+
├── index.ts # Re-exports all query modules
|
|
351
|
+
├── {page-or-domain}/ # Group by dashboard page or domain
|
|
352
|
+
│ ├── {visualization-name}.dax # DAX query (plain text)
|
|
353
|
+
│ ├── {visualization-name}-{variant}.dax # Additional query variants (optional)
|
|
354
|
+
│ ├── {visualization-name}.json # Vega-Lite spec (JSON)
|
|
355
|
+
│ ├── {visualization-name}.ts # Factory function: imports .dax + .json, exports { connection, query, vegaLiteSpec, columnMetadata }
|
|
356
|
+
│ └── index.ts # Re-exports all visualizations in this group
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
### Example TS File
|
|
360
|
+
|
|
361
|
+
**`revenue-by-region.ts`** — the factory function accepts use-case-specific parameters and uses them to modify the DAX query and/or Vega-Lite spec as appropriate:
|
|
362
|
+
|
|
363
|
+
```ts
|
|
364
|
+
import type { VisualizationSpec } from "@microsoft/fabric-visuals";
|
|
365
|
+
import type { ColumnMetadataMap } from "@/lib/to-data-table";
|
|
366
|
+
import baseQuery from "./revenue-by-region.dax?raw";
|
|
367
|
+
import spec from "./revenue-by-region.json";
|
|
368
|
+
|
|
369
|
+
const connection = "{connection-alias}"; // from rayfin/rayfin.yml
|
|
370
|
+
|
|
371
|
+
/** Column metadata keyed by original DAX column name. */
|
|
372
|
+
const columnMetadata: ColumnMetadataMap = {
|
|
373
|
+
"Products[Region]": { name: "ProductsRegion", displayName: "Region" },
|
|
374
|
+
"[Total Revenue]": { name: "Total Revenue", displayName: "Total Revenue", format: "$#,0.00" },
|
|
375
|
+
};
|
|
376
|
+
|
|
377
|
+
interface RevenueByRegionParams {
|
|
378
|
+
/** Filter to specific product categories (modifies the DAX query). */
|
|
379
|
+
categories?: string[];
|
|
380
|
+
/** Only show regions with revenue above this threshold (modifies the Vega-Lite spec). */
|
|
381
|
+
minRevenue?: number;
|
|
382
|
+
}
|
|
383
|
+
|
|
384
|
+
export function revenueByRegion(params?: RevenueByRegionParams) {
|
|
385
|
+
let query = baseQuery;
|
|
386
|
+
let vegaLiteSpec = spec; // Should clone if modifying
|
|
387
|
+
|
|
388
|
+
if (params?.categories?.length) {
|
|
389
|
+
// make changes to the DAX query to filter by the specified categories and update the query variable
|
|
390
|
+
}
|
|
391
|
+
|
|
392
|
+
if (params?.minRevenue != null) {
|
|
393
|
+
// Clone spec and append a client-side filter transform to the Vega-Lite spec
|
|
394
|
+
}
|
|
395
|
+
|
|
396
|
+
return { connection, query, columnMetadata, vegaLiteSpec };
|
|
397
|
+
}
|
|
398
|
+
```
|
|
399
|
+
|
|
400
|
+
The parameters, their types, and how they modify the query or spec are **entirely use-case-specific**. Some visualizations may need no parameters at all; others may accept date ranges, top-N limits, grouping dimensions, or string search terms. The factory function is the single place that translates caller intent into DAX and/or Vega-Lite modifications.
|
|
401
|
+
|
|
402
|
+
#### Query variants with multiple `.dax` files
|
|
403
|
+
|
|
404
|
+
When a parameter changes the **structure** of the query (e.g. different GROUP BY columns, different aggregations), use separate `.dax` files for each variant. The factory function selects the right one:
|
|
405
|
+
|
|
406
|
+
```text
|
|
407
|
+
revenue-trend/
|
|
408
|
+
revenue-trend-yearly.dax
|
|
409
|
+
revenue-trend-quarterly.dax
|
|
410
|
+
revenue-trend-monthly.dax
|
|
411
|
+
revenue-trend.json
|
|
412
|
+
revenue-trend.ts
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
```ts
|
|
416
|
+
import yearlyQuery from "./revenue-trend-yearly.dax?raw";
|
|
417
|
+
import quarterlyQuery from "./revenue-trend-quarterly.dax?raw";
|
|
418
|
+
import monthlyQuery from "./revenue-trend-monthly.dax?raw";
|
|
419
|
+
|
|
420
|
+
type Granularity = "yearly" | "quarterly" | "monthly";
|
|
421
|
+
|
|
422
|
+
const queryByGranularity: Record<Granularity, string> = {
|
|
423
|
+
yearly: yearlyQuery,
|
|
424
|
+
quarterly: quarterlyQuery,
|
|
425
|
+
monthly: monthlyQuery,
|
|
426
|
+
};
|
|
427
|
+
|
|
428
|
+
export function revenueTrend(params?: { granularity?: Granularity }) {
|
|
429
|
+
const query = queryByGranularity[params?.granularity ?? "monthly"];
|
|
430
|
+
return { connection, query, columnMetadata, vegaLiteSpec };
|
|
431
|
+
}
|
|
432
|
+
```
|
|
433
|
+
|
|
434
|
+
#### Column Metadata
|
|
435
|
+
|
|
436
|
+
Metadata is part of the charting path — it exists to feed `toDataTable`, and both `ColumnMetadataMap` and `ColumnDef` come from the `visuals` pack. An app with no chart in it skips this section.
|
|
437
|
+
|
|
438
|
+
Capture column metadata in the barrel `.ts` file as a `columnMetadata: ColumnMetadataMap` constant (import `ColumnMetadataMap` from `@/lib/to-data-table`). The dictionary **must be keyed by the exact column name from the CLI query output** (the `name` field in `table.columns`). Do not guess or clean these keys — copy them verbatim from the query result. Each value is a `ColumnDef` (from `@microsoft/fabric-visuals-core`) containing:
|
|
439
|
+
- `name` — a cleaned-up identifier derived from the original column name by removing `.`, `[`, `]`, `\`, `"`, and `'` characters. E.g., `"Products[Region]"` → `"ProductsRegion"`, `"[Total Revenue]"` → `"Total Revenue"` — to be used when building visual specs.
|
|
440
|
+
- `displayName` — a human-readable label sourced from the semantic model schema (e.g., `"Region"`, `"Total Revenue"`). Used for axis titles, grid headers, and tooltips.
|
|
441
|
+
- `format` — a VBA/ECMA-376 format string for number/date formatting (e.g., `#,##0.00`, `0.00%`, and `mm/dd/yyyy`). Omit for text type columns.
|
|
442
|
+
|
|
443
|
+
**Example workflow:** Run `npx rayfin connector invoke myModel executeQuery --file ./inspection/probe.query.json`, observe the output columns are `[SalesPersonID]` and `[Name]`, then use those exact strings as metadata keys:
|
|
444
|
+
|
|
445
|
+
```ts
|
|
446
|
+
export const columnMetadata: ColumnMetadataMap = {
|
|
447
|
+
"[SalesPersonID]": { name: "SalesPersonID", displayName: "Sales Person ID" },
|
|
448
|
+
"[Name]": { name: "Name", displayName: "Name" },
|
|
449
|
+
};
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
#### Key rules
|
|
453
|
+
|
|
454
|
+
- **All DAX lives in `.dax` files.** Never inline full DAX query strings in `.ts` factory files. If a parameter changes the query structure, create a separate `.dax` file for each variant and select the right one in the factory function. Small modifications — such as replacing filter value placeholders, wrapping the query with `CALCULATETABLE` to apply filters, or substituting a column reference — are acceptable in `.ts`, but the base query must always come from a `.dax` import.
|
|
455
|
+
- **Name files after the visualization they drive.** Use kebab-case base names (e.g., `revenue-by-region.dax`, `revenue-by-region.json`, `revenue-by-region.ts`). Variant `.dax` files append a suffix (e.g., `revenue-trend-yearly.dax`, `revenue-trend-quarterly.dax`).
|
|
456
|
+
- **Use `.dax` for queries.** Plain-text DAX files keep queries readable and diff-friendly. Import them with Vite's `?raw` suffix.
|
|
457
|
+
- **Use `.json` for specs.** JSON files get free schema validation in editors and are importable as modules by default in Vite.
|
|
458
|
+
- **Barrel `.ts` exports a factory function.** The function name is the camelCase version of the kebab-case file name (e.g., `revenueByRegion` for `revenue-by-region.ts`). It accepts optional parameters (typed per use case) and returns `{ connection, query, columnMetadata, vegaLiteSpec }` — or just `{ connection, query }` when the app draws no charts.
|
|
459
|
+
- **Group by page/domain.** Use subfolders when the dashboard has multiple pages or logical sections. For simple single-page dashboards, a flat structure under `packages/frontend/src/queries/` is fine.
|
|
460
|
+
- **Re-export via `index.ts`.** Each subfolder and the root `packages/frontend/src/queries/index.ts` should re-export all modules for clean imports.
|