@microsoft/rayfin-semantic-model-pack 1.1.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG +4 -0
- package/README.md +16 -5
- package/kit/hooks/use-query-factory-target.spec.ts +191 -0
- package/kit/hooks/use-query-factory-target.ts +123 -0
- package/kit/hooks/use-semantic-model-query.spec.ts +51 -34
- package/kit/hooks/use-semantic-model-query.ts +30 -49
- package/kit/lib/query-factory.spec.ts +97 -0
- package/kit/lib/query-factory.ts +68 -0
- package/kit/lib/semantic-model-errors.spec.ts +131 -0
- package/kit/lib/semantic-model-errors.ts +110 -0
- package/kit/lib/to-data-table.spec.ts +444 -0
- package/kit/lib/to-data-table.ts +221 -0
- package/pack.json +122 -25
- package/package.json +1 -1
- package/skills/analytics/SKILL.md +126 -422
- package/skills/dax-authoring/SKILL.md +98 -31
- package/skills/query-design/SKILL.md +22 -29
- package/skills/query-design/references/anti-patterns.md +13 -11
- package/skills/query-design/references/format-strings.md +6 -12
- package/skills/query-design/references/multi-grain-patterns.md +121 -108
- package/skills/schema-discovery/SKILL.md +197 -368
- package/skills/schema-discovery/references/discovery-queries.md +6 -8
- package/skills/visual-interactivity/SKILL.md +68 -0
- package/skills/visual-interactivity/references/advanced-query-shapes.md +155 -0
- package/skills/visual-interactivity/references/filter-aware-grid-totals.md +101 -0
- package/skills/visual-interactivity/references/highlight-queries.md +88 -0
- package/skills/visual-interactivity/references/interaction-manager.md +127 -0
- package/skills/visual-interactivity/references/native-selections.md +20 -0
- package/skills/visual-interactivity/references/query-factories.md +169 -0
- package/kit/scripts/rayfin-connector.mjs +0 -265
- package/kit/scripts/rayfin-connector.test.mjs +0 -243
- package/kit/scripts/schema-sql-batch.test.mjs +0 -2097
- package/kit/scripts/schema-sql-cli.test.mjs +0 -488
- package/kit/scripts/schema-sql.mjs +0 -2650
- package/kit/scripts/schema-sql.test.mjs +0 -1387
- package/kit/scripts/schema.mjs +0 -166
- package/kit/scripts/schema.test.mjs +0 -271
- package/skills/query-design/references/filter-strategy.md +0 -68
- package/skills/query-design/references/highlight-queries.md +0 -122
|
@@ -1,159 +1,172 @@
|
|
|
1
1
|
# Multi-Grain Patterns
|
|
2
2
|
|
|
3
|
-
When a component needs data at multiple grains
|
|
3
|
+
When a component needs data at multiple grains, return each metric at the grain where DAX must evaluate it. DataGrid body rows and a grand total can share one flagged result table; unrelated datasets such as a monthly trend and YTD annotation should use separate queries.
|
|
4
4
|
|
|
5
5
|
## Contents
|
|
6
6
|
|
|
7
|
-
- [File
|
|
8
|
-
- [
|
|
9
|
-
- [Rendering in
|
|
10
|
-
- [
|
|
11
|
-
- [
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
├── revenue-total.dax # Summary grain (single-row total)
|
|
21
|
-
├── revenue-total.ts # Factory: returns totalQuery (no spec needed)
|
|
22
|
-
└── index.ts
|
|
23
|
-
```
|
|
7
|
+
- [File organization](#file-organization) — keep rollups together and independent result tables in separate query files
|
|
8
|
+
- [DataGrid grand-total query](#datagrid-grand-total-query) — return flagged body and optional grand-total rows in one table
|
|
9
|
+
- [Rendering in DataGrid](#rendering-in-datagrid) — split the result into body and grand-total `DataTable`s
|
|
10
|
+
- [Independent query organization and wiring](#independent-query-organization-and-wiring) — connect multiple factories and hook calls
|
|
11
|
+
- [Rendering in VegaVisual](#rendering-in-vegavisual) — use separate queries for independently consumed datasets
|
|
12
|
+
- [Consistency rules](#consistency-rules) — keep measures, filters, metadata, and row classification aligned
|
|
13
|
+
|
|
14
|
+
## File organization
|
|
15
|
+
|
|
16
|
+
Follow the frontend-relative `src/queries/<page>/` factory organization described in analytics.
|
|
17
|
+
|
|
18
|
+
- **DataGrid body and grand total:** Use one `.dax` file and one factory. The query returns both as flagged rows.
|
|
19
|
+
- **Independent datasets:** Use one `.dax` file and factory per result table. Keep the shared Vega-Lite spec with the primary visualization factory and re-export every factory through the appropriate barrel.
|
|
24
20
|
|
|
25
|
-
##
|
|
21
|
+
## DataGrid grand-total query
|
|
26
22
|
|
|
27
|
-
|
|
23
|
+
Use `ROLLUPADDISSUBTOTAL` to evaluate every metric at both the body grain and grand-total grain in one DAX query. Wrap all current body grouping columns in `ROLLUPGROUP` so they roll up as one unit and do not produce intermediate subtotal rows.
|
|
28
24
|
|
|
29
|
-
|
|
25
|
+
### Basic flagged rollup
|
|
26
|
+
|
|
27
|
+
This starter emits body rows and, when at least one total expression is non-BLANK, one flagged grand-total row:
|
|
30
28
|
|
|
31
29
|
```dax
|
|
32
30
|
EVALUATE
|
|
33
31
|
SUMMARIZECOLUMNS(
|
|
34
|
-
|
|
32
|
+
ROLLUPADDISSUBTOTAL(
|
|
33
|
+
ROLLUPGROUP(
|
|
34
|
+
'Region'[Name],
|
|
35
|
+
'Product'[Category]
|
|
36
|
+
),
|
|
37
|
+
"IsGrandTotal"
|
|
38
|
+
),
|
|
35
39
|
"Revenue", [Total Revenue],
|
|
36
40
|
"Customer Count", DISTINCTCOUNT('Sales'[Customer Key])
|
|
37
41
|
)
|
|
38
|
-
ORDER BY 'Region'[Name]
|
|
42
|
+
ORDER BY 'Region'[Name], 'Product'[Category]
|
|
39
43
|
```
|
|
40
44
|
|
|
41
|
-
|
|
45
|
+
### Flagged-rollup scaffold
|
|
46
|
+
|
|
47
|
+
Replace each `<placeholder>` before running this query:
|
|
42
48
|
|
|
43
49
|
```dax
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
50
|
+
DEFINE
|
|
51
|
+
// Application/page filter: constrains body and grand-total evaluation.
|
|
52
|
+
VAR _BodyFilter =
|
|
53
|
+
TREATAS({<body-filter-values>}, '<FilterDimension>'[<FilterColumn>])
|
|
54
|
+
|
|
55
|
+
// Maximum number of body rows to return.
|
|
56
|
+
VAR _MaxBodyRows = 500
|
|
57
|
+
|
|
58
|
+
VAR _RollupResult =
|
|
59
|
+
SUMMARIZECOLUMNS(
|
|
60
|
+
ROLLUPADDISSUBTOTAL(
|
|
61
|
+
ROLLUPGROUP(
|
|
62
|
+
'<Dimension1>'[<Column1>],
|
|
63
|
+
'<Dimension2>'[<Column2>]
|
|
64
|
+
),
|
|
65
|
+
"IsGrandTotal"
|
|
66
|
+
),
|
|
67
|
+
_BodyFilter,
|
|
68
|
+
"<MetricAlias>", [<Measure>]
|
|
69
|
+
)
|
|
70
|
+
|
|
71
|
+
VAR _ResultLimit =
|
|
72
|
+
_MaxBodyRows
|
|
73
|
+
+ COUNTROWS(FILTER(_RollupResult, [IsGrandTotal] = TRUE()))
|
|
74
|
+
|
|
75
|
+
// Prioritize the total inside TOPN so payload limiting cannot discard it.
|
|
76
|
+
VAR _LimitedResult =
|
|
77
|
+
TOPN(
|
|
78
|
+
_ResultLimit,
|
|
79
|
+
_RollupResult,
|
|
80
|
+
[IsGrandTotal], DESC,
|
|
81
|
+
[<MetricAlias>], DESC,
|
|
82
|
+
'<Dimension1>'[<Column1>], ASC,
|
|
83
|
+
'<Dimension2>'[<Column2>], ASC
|
|
84
|
+
)
|
|
85
|
+
|
|
86
|
+
EVALUATE _LimitedResult
|
|
87
|
+
ORDER BY
|
|
88
|
+
[<MetricAlias>] DESC,
|
|
89
|
+
'<Dimension1>'[<Column1>] ASC,
|
|
90
|
+
'<Dimension2>'[<Column2>] ASC
|
|
49
91
|
```
|
|
50
92
|
|
|
51
|
-
|
|
93
|
+
`_BodyFilter` is a regular `SUMMARIZECOLUMNS` filter, so it constrains both body and grand-total evaluation. Add one variable per application or page filter and pass each one as a `SUMMARIZECOLUMNS` filter argument.
|
|
94
|
+
|
|
95
|
+
DataGrid filtering is enabled by default. Extend this scaffold with [filter-aware grid totals](../../visual-interactivity/references/filter-aware-grid-totals.md) so local grid filters update the provided total without removing body rows needed for client-side filtering.
|
|
96
|
+
|
|
97
|
+
`_ResultLimit` adds one slot only when the grand-total row exists. `TOPN` ranks that row first internally so it survives the limit while returning no more than `_MaxBodyRows` body rows. Because `TOPN` does not guarantee output order, the final `ORDER BY` makes body-row order deterministic for testing and debugging; DataGrid still owns user-facing sorting. Include every body dimension after the metric in both sort lists to break ties deterministically.
|
|
52
98
|
|
|
53
|
-
|
|
99
|
+
When present, the row where `IsGrandTotal` is `TRUE` contains all grand-total column values. Each expression is evaluated again without the body grouping context; DAX does not add the body-row results. For example, the grand-total `Customer Count` is a distinct count across the regions in the current filter scope rather than a sum of their per-region distinct counts. If every grand-total expression evaluates to `BLANK` under the current filters, `SUMMARIZECOLUMNS` omits the row.
|
|
100
|
+
|
|
101
|
+
Copy the semantic model's format strings into the query factory's `columnMetadata`. Do not add metadata for the rollup flag because `toRollupDataTables()` removes it:
|
|
54
102
|
|
|
55
103
|
```typescript
|
|
56
104
|
const columnMetadata: ColumnMetadataMap = {
|
|
105
|
+
"Region[Name]": { name: "RegionName", displayName: "Region" },
|
|
57
106
|
"[Revenue]": { name: "Revenue", displayName: "Revenue", format: "$#,##0.00" },
|
|
58
107
|
"[Customer Count]": { name: "Customer Count", displayName: "Customer Count", format: "#,##0" },
|
|
59
108
|
};
|
|
60
109
|
```
|
|
61
110
|
|
|
62
|
-
|
|
111
|
+
Test the query with `npx fabric-app-semantic-model-tools run-dax-query --dax <file> --alias <name>`. Verify that the flag values are booleans, and copy the exact returned rollup flag name into the conversion options rather than assuming its serialized form.
|
|
63
112
|
|
|
64
|
-
|
|
113
|
+
## Rendering in DataGrid
|
|
65
114
|
|
|
66
|
-
|
|
67
|
-
// Two hook calls, two DataTables. Both factories target the same model,
|
|
68
|
-
// so the two connection objects are equivalent — we destructure both
|
|
69
|
-
// (factory signature requires it) but use one for both hooks.
|
|
70
|
-
const { connection, query: detailQuery, columnMetadata: detailMeta, vegaLiteSpec } = revenueByRegion();
|
|
71
|
-
const { connection: _summaryConn, query: totalQuery, columnMetadata: totalMeta } = revenueTotal();
|
|
72
|
-
|
|
73
|
-
const detail = useSemanticModelQuery({ connection, query: detailQuery });
|
|
74
|
-
const summary = useSemanticModelQuery({ connection, query: totalQuery });
|
|
75
|
-
|
|
76
|
-
// After handling both hooks' loading and error states:
|
|
77
|
-
const detailTable = toDataTable(detail.data.table, detailMeta);
|
|
78
|
-
const summaryTable = toDataTable(summary.data.table, totalMeta);
|
|
79
|
-
```
|
|
115
|
+
DataGrid filtering is enabled by default. Follow [component wiring](../../visual-interactivity/references/filter-aware-grid-totals.md#component-wiring) to derive the rollup query from grid filter state, split its result with `toRollupDataTables()`, and connect `onFilterChange`.
|
|
80
116
|
|
|
81
|
-
|
|
82
|
-
<VegaVisual
|
|
83
|
-
spec={vegaLiteSpec}
|
|
84
|
-
data={{ detail: detailTable, summary: summaryTable }}
|
|
85
|
-
theme={theme}
|
|
86
|
-
/>
|
|
87
|
-
```
|
|
117
|
+
`toRollupDataTables()` treats rows where all configured rollup flags are `FALSE` as body rows and rows where all flags are `TRUE` as the provided grand total. It rejects multiple grand-total rows and intermediate subtotal rows until DataGrid has an explicit subtotal API. The array-based flag contract leaves room to classify additional rollup levels when that support is added.
|
|
88
118
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
```json
|
|
92
|
-
{
|
|
93
|
-
"layer": [
|
|
94
|
-
{
|
|
95
|
-
"data": { "name": "detail" },
|
|
96
|
-
"mark": "bar",
|
|
97
|
-
"encoding": {
|
|
98
|
-
"x": { "field": "RegionName", "type": "nominal" },
|
|
99
|
-
"y": { "field": "Revenue", "type": "quantitative" }
|
|
100
|
-
}
|
|
101
|
-
},
|
|
102
|
-
{
|
|
103
|
-
"data": { "name": "summary" },
|
|
104
|
-
"mark": { "type": "rule", "color": "firebrick", "strokeDash": [4, 4] },
|
|
105
|
-
"encoding": {
|
|
106
|
-
"y": { "field": "Revenue", "type": "quantitative" }
|
|
107
|
-
}
|
|
108
|
-
}
|
|
109
|
-
]
|
|
110
|
-
}
|
|
111
|
-
```
|
|
119
|
+
Always pass `grandTotalTable` through `grandTotals.data` so DataGrid treats it as authoritative; omitting `data` selects client-computed mode. Conversion failures are query-authoring errors and intentionally surface for malformed totals.
|
|
112
120
|
|
|
113
|
-
|
|
121
|
+
DataGrid owns grand-total rendering; do not append the total to body rows or use `cellRenderer` for it.
|
|
114
122
|
|
|
115
|
-
##
|
|
123
|
+
## Independent query organization and wiring
|
|
116
124
|
|
|
117
|
-
|
|
125
|
+
Use separate queries when a consumer needs independently shaped result tables rather than classified rows from one shared result. Give each result its own `.dax` file and factory output (`connection`, `query`, and `columnMetadata`), and keep the shared Vega-Lite spec with the visualization factory. Use matching kebab-case file names, structural variants and barrel exports under frontend-relative `src/queries/<page>/`.
|
|
118
126
|
|
|
119
|
-
|
|
127
|
+
Call each factory with the same shared inputs, then execute each returned query with its own hook call:
|
|
120
128
|
|
|
121
|
-
```
|
|
122
|
-
|
|
129
|
+
```typescript
|
|
130
|
+
const detailConfig = revenueByRegion(params);
|
|
131
|
+
const summaryConfig = revenueSummary(params);
|
|
132
|
+
|
|
133
|
+
const detail = useSemanticModelQuery({
|
|
134
|
+
connection: detailConfig.connection,
|
|
135
|
+
query: detailConfig.query,
|
|
136
|
+
});
|
|
137
|
+
const summary = useSemanticModelQuery({
|
|
138
|
+
connection: summaryConfig.connection,
|
|
139
|
+
query: summaryConfig.query,
|
|
140
|
+
});
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
Each `useSemanticModelQuery` invocation has its own loading and error state. Caching and request coalescing are template-specific; do not assume each hook owns a separate cache or causes a separate SDK round trip. When the tables form a matched set, replace the rendered set only after every request has succeeded and none is still loading; surface a failure from any request instead of combining fresh and stale results.
|
|
123
144
|
|
|
124
|
-
|
|
125
|
-
const { query: totalQuery, columnMetadata: totalMeta } = revenueTotal({ filters: gridFilters });
|
|
126
|
-
const summary = useSemanticModelQuery({ connection, query: totalQuery });
|
|
145
|
+
## Rendering in VegaVisual
|
|
127
146
|
|
|
128
|
-
|
|
147
|
+
Convert each successful result with its own metadata, then pass the `DataTable`s as named datasets:
|
|
129
148
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
? toDataTable(summary.data.table, totalMeta)
|
|
134
|
-
: undefined;
|
|
149
|
+
```typescript
|
|
150
|
+
const detailTable = toDataTable(detail.data.table, detailConfig.columnMetadata);
|
|
151
|
+
const summaryTable = toDataTable(summary.data.table, summaryConfig.columnMetadata);
|
|
135
152
|
```
|
|
136
153
|
|
|
137
154
|
```tsx
|
|
138
|
-
<
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
onFilterChange={(columnId, selectedValues) =>
|
|
142
|
-
setGridFilters((current) => ({ ...current, [columnId]: selectedValues }))
|
|
143
|
-
}
|
|
155
|
+
<VegaVisual
|
|
156
|
+
spec={detailConfig.vegaLiteSpec}
|
|
157
|
+
data={{ detail: detailTable, summary: summaryTable }}
|
|
144
158
|
theme={theme}
|
|
145
159
|
/>
|
|
146
160
|
```
|
|
147
161
|
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
## Consistency Rule
|
|
151
|
-
|
|
152
|
-
When splitting a visualization across multiple queries, all queries must share the same semantic contract:
|
|
162
|
+
See the visuals skill's [multi-data input](../../visuals/references/multi-data-input.md) reference for named dataset binding, overlays, reference lines, axis spines, and compatible field contracts. For a baseline plus selected-subset overlay, follow [Highlight queries](../../visual-interactivity/references/highlight-queries.md) for query generation and synchronization.
|
|
153
163
|
|
|
154
|
-
|
|
155
|
-
- Local DataGrid filters remain enabled; pass `onFilterChange` selections to the summary factory so the provided total matches the visible rows
|
|
156
|
-
- Summary metadata names match the detail table and include formats
|
|
157
|
-
- Use `ROW(...)` for the summary so it returns exactly one row and preserves BLANK values; `SUMMARIZECOLUMNS` can return zero rows when all measures are BLANK
|
|
164
|
+
## Consistency rules
|
|
158
165
|
|
|
159
|
-
|
|
166
|
+
- Use one `EVALUATE` per DAX query; the Execute Queries transport supports one result table per query.
|
|
167
|
+
- For DataGrid grand totals, keep body and total expressions in one `SUMMARIZECOLUMNS` and split its flagged rows in TypeScript.
|
|
168
|
+
- Apply application filters to every row and local DataGrid filters through `ROLLUPADDISSUBTOTAL`'s `grandtotalFilter`; DataGrid continues filtering the complete body table locally.
|
|
169
|
+
- Keep output aliases and metadata names aligned; the body and grand-total `DataTable`s use the same format metadata.
|
|
170
|
+
- Validate that the query never returns multiple flagged grand-total rows.
|
|
171
|
+
- Pass the same model connection and shared filter parameters to every related query factory so tables do not drift.
|
|
172
|
+
- Keep shared field aliases, types, and encoded keys compatible across independently queried tables. Cross-highlight baseline and subset queries must also share grouping columns, measures, and row keys as described in [Highlight queries](../../visual-interactivity/references/highlight-queries.md).
|