@microsoft/rayfin-guide 1.35.0-alpha.1412 → 1.35.0-alpha.1541
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/assets/docs/app-backend/deployment-pipelines.md +101 -0
- package/assets/docs/cli/connectors/add.md +27 -3
- package/assets/docs/cli/connectors/category-a-entities.md +675 -0
- package/assets/docs/cli/connectors/category-b-function-bridge.md +269 -0
- package/assets/docs/cli/connectors/index.md +19 -2
- package/assets/docs/data/overview.md +1 -1
- package/package.json +1 -1
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
---
|
|
2
|
+
sidebar_position: 46
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Fabric Deployment Pipelines
|
|
6
|
+
|
|
7
|
+
Microsoft Fabric's **Deployment Pipelines** feature promotes an AppBackend between dev/test/prod-style workspace stages without rebuilding it — the same pre-built artifact moves forward as-is.
|
|
8
|
+
This page covers what that means for a Rayfin app and how to check whether yours is ready for it.
|
|
9
|
+
|
|
10
|
+
## What actually moves
|
|
11
|
+
|
|
12
|
+
An AppBackend's exported definition is a portable package of pre-built artifacts, not source code:
|
|
13
|
+
|
|
14
|
+
```text
|
|
15
|
+
MyAppBackend.AppBackend/
|
|
16
|
+
├── .platform Fabric-managed item metadata
|
|
17
|
+
├── rayfin.yml Services config (auth, data, storage, staticHosting) — promotes as-is
|
|
18
|
+
├── dab-config.json Pre-generated DAB entity schema — promotes as-is (dialect-locked)
|
|
19
|
+
└── static-app.zip Compiled SPA bundle — promotes as-is (byte-identical across stages)
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### `rayfin.config.json` — generated per stage, not promoted
|
|
23
|
+
|
|
24
|
+
When you deploy with `rayfin up`, the CLI writes a `rayfin.config.json` file into `StaticAssets/` alongside the compiled SPA, containing that stage's API URL, publishable key, and Fabric item metadata.
|
|
25
|
+
`resolveRayfinConfig()` reads this file automatically at runtime when it's present, and falls back to whatever `apiUrl`/`publishableKey` defaults you pass in — typically your `VITE_RAYFIN_*` env vars — when it's not, which is the local-dev case before you've ever run `rayfin up`.
|
|
26
|
+
|
|
27
|
+
This is exactly why `rayfin.config.json` is **deliberately excluded** from the exported definition above: Fabric regenerates it fresh in the target workspace on every Import, using that stage's own API URL, publishable key, and item metadata, so a stale Dev config can never leak into Prod as your app moves through the pipeline.
|
|
28
|
+
|
|
29
|
+
## Read your backend config with `resolveRayfinConfig()`
|
|
30
|
+
|
|
31
|
+
Because the compiled SPA bundle moves unchanged between stages, resolve your backend URL and key through `resolveRayfinConfig()` before constructing `RayfinClient` / `RayfinServerClient`, rather than constructing the client from your env vars directly.
|
|
32
|
+
Your `VITE_RAYFIN_API_URL` / `VITE_RAYFIN_PUBLISHABLE_KEY` env vars still belong in your app — `rayfin dev` depends on them — just pass them into `resolveRayfinConfig()` as defaults instead of using them to build the client yourself:
|
|
33
|
+
|
|
34
|
+
```ts
|
|
35
|
+
import { RayfinClient, resolveRayfinConfig } from '@microsoft/rayfin-client';
|
|
36
|
+
|
|
37
|
+
const resolved = await resolveRayfinConfig({
|
|
38
|
+
apiUrl: import.meta.env.VITE_RAYFIN_API_URL,
|
|
39
|
+
publishableKey: import.meta.env.VITE_RAYFIN_PUBLISHABLE_KEY,
|
|
40
|
+
});
|
|
41
|
+
const client = new RayfinClient<Schema>({ ...resolved, authStorage: true });
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
`resolveRayfinConfig()` prefers `rayfin.config.json` when it's present — every deployed stage — and falls back to the `apiUrl`/`publishableKey` defaults you pass in only when the file is absent, which is the local-dev case. Constructing the client directly from build-time values, skipping `resolveRayfinConfig()`, bakes in whatever you pass and never checks for `rayfin.config.json`, so a promoted bundle would keep pointing at the source environment's backend.
|
|
45
|
+
|
|
46
|
+
For your backend URL and key alone, call `resolveRayfinConfig()`, not `loadRayfinConfig()` directly — `loadRayfinConfig()` is the lower-level primitive `resolveRayfinConfig()` calls internally to fetch `rayfin.config.json`, and `resolveRayfinConfig()` is the one that overlays the result over your defaults and returns what you should construct the client with.
|
|
47
|
+
If your app also needs Fabric coordinates for its auth broker, pass them alongside `apiUrl`/`publishableKey` in `resolveRayfinConfig()`'s defaults — see [Fabric auth coordinates](#fabric-auth-coordinates-need-clientruntimeconfig) below.
|
|
48
|
+
|
|
49
|
+
If you're not sure whether your app already resolves config before constructing the client, check for direct construction:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
grep -rn "new RayfinClient(\|new RayfinServerClient(" src/
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Any match there should be preceded by a `resolveRayfinConfig()` call, with its result spread into the constructor as shown above — your existing `VITE_RAYFIN_*` reads can stay, just pass them in as `resolveRayfinConfig()`'s defaults instead of using them to construct the client directly.
|
|
56
|
+
|
|
57
|
+
## Fabric auth coordinates need `client.runtimeConfig`
|
|
58
|
+
|
|
59
|
+
If your app embeds in Fabric and drives its own auth flow against the Fabric secure-embed broker, `resolveRayfinConfig()`'s `apiUrl`/`publishableKey` alone aren't enough — your auth wiring also needs the Fabric coordinates (`workspaceId`, `itemId`, `portalUrl`) from the same `rayfin.config.json`.
|
|
60
|
+
|
|
61
|
+
`client.runtimeConfig` is a single `RayfinRuntimeConfig` bag holding every value `rayfin.config.json` can supply (`apiUrl`, `publishableKey`, `workspaceId`, `itemId`, `portalUrl`, `tenantId`) — all fields optional, since neither the remote config nor your defaults are guaranteed to provide every one. Pass your build-time `VITE_FABRIC_*` values into `resolveRayfinConfig()`'s defaults alongside `apiUrl`/`publishableKey`, and read the resolved coordinates off `client.runtimeConfig` after construction:
|
|
62
|
+
|
|
63
|
+
```ts
|
|
64
|
+
const resolved = await resolveRayfinConfig({
|
|
65
|
+
apiUrl: import.meta.env.VITE_RAYFIN_API_URL,
|
|
66
|
+
publishableKey: import.meta.env.VITE_RAYFIN_PUBLISHABLE_KEY,
|
|
67
|
+
workspaceId: import.meta.env.VITE_FABRIC_WORKSPACE_ID,
|
|
68
|
+
itemId: import.meta.env.VITE_FABRIC_ITEM_ID,
|
|
69
|
+
portalUrl: import.meta.env.VITE_FABRIC_PORTAL_URL,
|
|
70
|
+
});
|
|
71
|
+
const client = new RayfinClient<Schema>({ ...resolved, authStorage: true });
|
|
72
|
+
|
|
73
|
+
const { workspaceId, itemId, portalUrl } = client.runtimeConfig ?? {};
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
`resolveRayfinConfig()` resolves `runtimeConfig` from the same `rayfin.config.json` fetch it already makes for `apiUrl`/`publishableKey`, overlaying remote values over your defaults per-field — no extra request. `resolveFabricConfig()`, which made its own independent fetch for just the Fabric fields, has been removed as redundant — always resolve through `client.runtimeConfig` instead.
|
|
77
|
+
|
|
78
|
+
Reading `workspaceId`/`itemId`/`portalUrl` directly from `VITE_FABRIC_*` env vars — without going through `client.runtimeConfig` — bakes in whatever stage the bundle was *built* in.
|
|
79
|
+
Since deployment pipelines promote the same compiled bundle across stages without rebuilding, that mismatch only shows up after a promotion: your API calls correctly hit the new stage's backend (via `resolveRayfinConfig()`'s `rayfin.config.json` read), but your auth broker still points at the *old* stage's workspace/item, because it never read from the runtime config at all.
|
|
80
|
+
|
|
81
|
+
If you're not sure whether your app resolves Fabric coordinates this way, check for direct reads of `VITE_FABRIC_*` that don't go through `runtimeConfig`/`client.runtimeConfig`:
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
grep -rn "VITE_FABRIC_" src/
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Any match not passed through `runtimeConfig`/`client.runtimeConfig` should be updated as shown above.
|
|
88
|
+
|
|
89
|
+
## Out of scope today
|
|
90
|
+
|
|
91
|
+
- **Functions** (`rayfin/functions/`) — not covered by this pattern yet.
|
|
92
|
+
- **Secrets** — a separate concern from config-file promotion.
|
|
93
|
+
- **Source-code / Git integration** — Deployment Pipelines move built artifacts, not source. [Git integration](https://learn.microsoft.com/en-us/fabric/cicd/git-integration/intro-to-git-integration) is a related but distinct Fabric feature for source-controlling item definitions.
|
|
94
|
+
|
|
95
|
+
## Related Fabric platform concepts
|
|
96
|
+
|
|
97
|
+
These are Fabric platform features, not something the Rayfin CLI/SDK controls — see Fabric's own [CI/CD documentation](https://learn.microsoft.com/en-us/fabric/cicd/) for depth:
|
|
98
|
+
|
|
99
|
+
- **Git integration** — links a workspace to a Git branch for source control of Fabric item definitions.
|
|
100
|
+
- **Deployment pipelines** — promotes items between dev/test/prod-style workspace stages; the feature this page covers.
|
|
101
|
+
- **Variable libraries** — Fabric-managed, stage-scoped values usable across items; not currently wired to `rayfin.config.json` generation, but conceptually related.
|
|
@@ -10,7 +10,7 @@ npx rayfin connector add --type <type> --workspace-id <ws-id> --item-id <item-id
|
|
|
10
10
|
|
|
11
11
|
`connector add` declares a connector in `rayfin/rayfin.yml` and scaffolds its supporting files.
|
|
12
12
|
|
|
13
|
-
The CLI verifies the Fabric item, derives a connector `name` from the item's display name (override with `--name`), writes the entry to `rayfin.yml`, scaffolds `rayfin/connectors/<name>/schema.ts
|
|
13
|
+
The CLI verifies the Fabric item, derives a connector `name` from the item's display name (override with `--name`), writes the entry to `rayfin.yml`, and scaffolds `rayfin/connectors/<name>/schema.ts`. For Category A connectors it also runs schema discovery and writes `rayfin/connectors/<name>/metadata.json` so a subset of entities can be generated later.
|
|
14
14
|
|
|
15
15
|
If you do not already know the workspace and item IDs, run [`connector search`](./search.md) first — its `--json` output includes a ready-to-run `addCommand` for each result.
|
|
16
16
|
|
|
@@ -58,15 +58,39 @@ Rules:
|
|
|
58
58
|
|
|
59
59
|
## Category B connectors
|
|
60
60
|
|
|
61
|
-
For `kusto` and `fabric-semanticmodel`, `connector add` writes the `rayfin.yml` entry but there are no GraphQL entities to discover, so no entity files are generated and no row-level security applies:
|
|
61
|
+
For `kusto` and `fabric-semanticmodel`, `connector add` writes the `rayfin.yml` entry and generates a complete typed `schema.ts`, but there are no GraphQL entities to discover, so no entity files are generated and no row-level security applies:
|
|
62
62
|
|
|
63
|
-
- `executeQuery`
|
|
63
|
+
- `fabric-semanticmodel` allows `executeQuery`; `kusto` allows `executeQuery` and `executeCommand`. Check `rayfin connector types --json` for the current set.
|
|
64
64
|
- `auth.type` must be `delegated`.
|
|
65
65
|
- The connector is pinned to an adapter version.
|
|
66
66
|
- There is no `metadata.json` entity list to generate from.
|
|
67
67
|
|
|
68
68
|
After adding, exercise the connector with [`connector invoke`](./invoke.md) rather than writing entity code.
|
|
69
69
|
|
|
70
|
+
## Installing the packages
|
|
71
|
+
|
|
72
|
+
`connector add` scaffolds files but installs nothing, and the generated `schema.ts` imports packages a fresh app does not declare.
|
|
73
|
+
On success the command prints the exact, version-pinned install to run:
|
|
74
|
+
|
|
75
|
+
```text
|
|
76
|
+
📦 Install the packages this connector needs:
|
|
77
|
+
npm install @microsoft/rayfin-connector-kusto@1.35.0-alpha
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Run it verbatim.
|
|
81
|
+
**Do not drop the version.** Connector packages ship in lockstep with the CLI, but their npm `latest` and `preview` tags lag the published release, so an unversioned install resolves to an older connector that hard-pins its own `@microsoft/rayfin-data` — leaving two Rayfin version lines in one app.
|
|
82
|
+
|
|
83
|
+
With `--json`, the same information is available under `install`:
|
|
84
|
+
|
|
85
|
+
```json
|
|
86
|
+
{
|
|
87
|
+
"install": {
|
|
88
|
+
"packages": ["@microsoft/rayfin-connector-kusto@1.35.0-alpha"],
|
|
89
|
+
"command": "npm install @microsoft/rayfin-connector-kusto@1.35.0-alpha"
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
70
94
|
## Related commands
|
|
71
95
|
|
|
72
96
|
```bash
|
|
@@ -0,0 +1,675 @@
|
|
|
1
|
+
---
|
|
2
|
+
sidebar_position: 8
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Category A — GraphQL entity connectors
|
|
6
|
+
|
|
7
|
+
Reference for `fabric-sqlanalytics` (Lakehouse SQL endpoint), `fabric-warehouse`, and `fabric-sqldatabase`.
|
|
8
|
+
These types surface Fabric SQL as typed entities with read/create/update/delete operations and `@role` row-level security.
|
|
9
|
+
|
|
10
|
+
For `kusto` and `fabric-semanticmodel`, read [Category B](./category-b-function-bridge.md) instead — none of this page applies to them.
|
|
11
|
+
|
|
12
|
+
This page picks up after the connector has been added and its `operations:` narrowed.
|
|
13
|
+
Add and scope the connector first: `rayfin connector add --type <type> --workspace-id <ws> --item-id <item> [--operations read,update]`.
|
|
14
|
+
|
|
15
|
+
## Generate entity files — you, not the CLI
|
|
16
|
+
|
|
17
|
+
`rayfin connector add` writes `rayfin/connectors/<name>/metadata.json` and a placeholder `schema.ts`, then stops.
|
|
18
|
+
The per-table entity `.ts` files are produced by **you** by reading `metadata.json` and following the [Entity generation contract](#entity-generation-contract).
|
|
19
|
+
The CLI does not emit them.
|
|
20
|
+
|
|
21
|
+
What to do, in order:
|
|
22
|
+
|
|
23
|
+
1. Read `rayfin/connectors/<name>/metadata.json` (shape: `SchemaMetadata` — see the [metadata.json reference](#metadatajson-reference)).
|
|
24
|
+
2. Pick the tables in scope.
|
|
25
|
+
- **Full set** — every table under `schemas[].tables[]`.
|
|
26
|
+
- **Subset** — only the tables the user named. Filter `schemas[].tables[]` by `tableName` before generating.
|
|
27
|
+
3. For each surviving table, write a file at `rayfin/connectors/<name>/<EntityName>.ts` following the [Entity generation contract](#entity-generation-contract) exactly.
|
|
28
|
+
4. Overwrite the placeholder `rayfin/connectors/<name>/schema.ts` with the [aggregate schema](#aggregate-connector-schema).
|
|
29
|
+
5. Surface every warning the contract emits (no-FK-metadata notes, unknown SQL types). Do not drop them silently.
|
|
30
|
+
|
|
31
|
+
When the user asks for a refresh ("regenerate everything"), re-do the same flow against the existing `metadata.json`.
|
|
32
|
+
Only re-run `rayfin connector remove` and `rayfin connector add` when the *source schema* changed and you need fresh metadata.
|
|
33
|
+
|
|
34
|
+
## Scope `@role(...)` on entities
|
|
35
|
+
|
|
36
|
+
The YAML `operations:` controls which actions reach the source.
|
|
37
|
+
The entity-level `@role(...)` decorator controls which roles can perform which actions on that entity.
|
|
38
|
+
|
|
39
|
+
The entity's `actions` list **must be a subset** of the YAML `operations:` for that connector.
|
|
40
|
+
The host's settings validator does not catch the mismatch today — DAB will fail at `rayfin up connector apply` time.
|
|
41
|
+
Always narrow YAML first, then add `@role(...)` to entities.
|
|
42
|
+
|
|
43
|
+
Example: a `fabric-warehouse` connector with `operations: [read, update]` and an `Order` entity locked to read-only:
|
|
44
|
+
|
|
45
|
+
```ts
|
|
46
|
+
import { entity, int, text, decimal, Source } from '@microsoft/rayfin-core/experimental';
|
|
47
|
+
import { role } from '@microsoft/rayfin-core';
|
|
48
|
+
|
|
49
|
+
@role('authenticated', ['read'])
|
|
50
|
+
@entity()
|
|
51
|
+
export class Order extends Source({ schema: 'dbo', table: 'Order', primaryKey: ['orderId'] }) {
|
|
52
|
+
@int({ column: 'OrderID' }) orderId!: number;
|
|
53
|
+
@text() customerEmail!: string;
|
|
54
|
+
@decimal({ precision: 18, scale: 2 }) total!: number;
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Legal `actions`: `'read' | 'create' | 'update' | 'delete' | '*'`.
|
|
59
|
+
Stack multiple `@role(...)` decorators to give different roles different actions on the same entity.
|
|
60
|
+
|
|
61
|
+
## Row-level policies
|
|
62
|
+
|
|
63
|
+
Use a `policy` callback on `@role(...)` for row-level security.
|
|
64
|
+
Policies use the typed `claims` / `item` DSL — never raw SQL strings.
|
|
65
|
+
|
|
66
|
+
```ts
|
|
67
|
+
@role('authenticated', ['read', 'update'], {
|
|
68
|
+
policy: (claims, item) => claims.sub.eq(item.owner_id),
|
|
69
|
+
})
|
|
70
|
+
@entity()
|
|
71
|
+
export class Todo extends Source({ schema: 'dbo', table: 'Todo', primaryKey: ['id'] }) {
|
|
72
|
+
@uuid() id!: string;
|
|
73
|
+
@uuid() owner_id!: string;
|
|
74
|
+
@text() body?: string;
|
|
75
|
+
}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
DSL surface: `claims.sub | email | role`, `item.<columnName>`, `.eq(...)`, `.and(...)`, `.or(...)`.
|
|
79
|
+
Use any claim the same way — for example `claims.email.eq(item.user_email)`.
|
|
80
|
+
`RoleDeclarationOptions` also accepts `include` and `exclude` arrays for field-level allow/block lists.
|
|
81
|
+
|
|
82
|
+
When to prompt the user: inspect `metadata.json` for columns named `owner_id`, `user_id`, `tenant_id`, `created_by`.
|
|
83
|
+
If you see one, ask whether that column should restrict rows so each authenticated user only sees their own.
|
|
84
|
+
|
|
85
|
+
## Aggregate connector schema
|
|
86
|
+
|
|
87
|
+
Install the connector packages first.
|
|
88
|
+
`rayfin connector add` scaffolds files but installs nothing, and the aggregate `schema.ts` imports two packages a fresh app does not yet declare.
|
|
89
|
+
|
|
90
|
+
`connector add` prints the exact command to run — copy it from that output, or rebuild it from the `packages` array in `rayfin connector types --json`, which carries both the package names and the version:
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
# Shape only. Use the version connector add printed, not this one.
|
|
94
|
+
npm install @microsoft/rayfin-connector-fabric-graphql@1.35.0-alpha @microsoft/rayfin-connectors@1.35.0-alpha
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
**Always pin the version.** Connector packages ship in lockstep with the CLI, but their npm `latest` and `preview` tags lag behind.
|
|
98
|
+
An unversioned `npm install` resolves to an older release that hard-pins its own `@microsoft/rayfin-data`, leaving two Rayfin version lines in one app.
|
|
99
|
+
|
|
100
|
+
- `@microsoft/rayfin-connector-fabric-graphql` — provides `GraphQLBackedConnector`. Not a dependency of `@microsoft/rayfin-client`, so it is always missing on a fresh app.
|
|
101
|
+
- `@microsoft/rayfin-connectors` — provides `ConnectorConfig`. It ships transitively under `@microsoft/rayfin-client`, but declare it directly so strict resolvers (pnpm) do not treat it as a phantom dependency.
|
|
102
|
+
|
|
103
|
+
Even though both imports are `import type`, TypeScript still needs the packages present at compile time.
|
|
104
|
+
|
|
105
|
+
The aggregate `rayfin/connectors/<name>/schema.ts` is what the app imports from.
|
|
106
|
+
The CLI leaves a placeholder there; overwrite it so it exports **three** things:
|
|
107
|
+
|
|
108
|
+
1. **Entity re-exports** — re-export every entity class you generated for this connector, **as types** (`export type { ... }`).
|
|
109
|
+
2. **`<Name>Schema` (type)** — a `GraphQLBackedConnector<TSchema, typeof connectorConfig>` marker.
|
|
110
|
+
`TSchema` maps each in-scope entity name to its class via `typeof`; the second argument is `typeof connectorConfig`, from which the marker derives both the CRUD operation union and the dialect.
|
|
111
|
+
This is what makes `client.connectors.<name>.<Entity>.select(...)` strongly typed and gates CRUD methods to the declared operations at compile time.
|
|
112
|
+
3. **`connectorConfig` (value)** — a `ConnectorConfig` carrying the runtime `connector` type, the same `operations` allow-list, and an **`entities` map** keyed exactly as in `TSchema` and covering every entity there.
|
|
113
|
+
The runtime routes on `connector`, gates CRUD verbs by `operations`, and uses `entities` as the default selection so a read or write without an explicit `select` returns the full row.
|
|
114
|
+
Give the entries as **string arrays** of entity property names, not entity classes — see below.
|
|
115
|
+
Only the class form carries relationship cardinality, so a relationship `select` against the array form throws `ENTITIES_REQUIRED_FOR_RELATIONSHIP_SELECT`.
|
|
116
|
+
|
|
117
|
+
`GraphQLBackedConnector<TSchema, typeof connectorConfig>` **is** the published typed marker for Category A connectors.
|
|
118
|
+
Never invent per-type names like `FabricWarehouse` or `FabricSqlAnalytics` — they do not exist.
|
|
119
|
+
|
|
120
|
+
### Import entities as types, never as values
|
|
121
|
+
|
|
122
|
+
This file is imported by browser code, because the app client reads `connectorConfig` from it.
|
|
123
|
+
A value import pulls the decorated entity classes into the browser bundle with it.
|
|
124
|
+
|
|
125
|
+
That breaks the app. `vite build` still exits 0, type-check passes, lint passes, and deploy succeeds, but the bundler lowers the decorators into an invalid class expression (`var Order = @Source({...})`), the emitted bundle fails to parse, and the page renders blank with `Uncaught SyntaxError: Invalid or unexpected token`.
|
|
126
|
+
Entity classes are build-time declarations consumed by DAB config generation. They have no business in a browser bundle.
|
|
127
|
+
|
|
128
|
+
Every entity reference in this file is a `typeof` lookup, which is pure type information, so `import type` and `export type` are sufficient:
|
|
129
|
+
|
|
130
|
+
```ts
|
|
131
|
+
import type { Order } from './Order.js'; // correct
|
|
132
|
+
import { Order } from './Order.js'; // wrong — ships the decorated class
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
The same applies to the re-exports. Use `export type { Order }`, not `export { Order }`.
|
|
136
|
+
|
|
137
|
+
This is also why `entities` takes column names rather than classes.
|
|
138
|
+
|
|
139
|
+
### Single source of truth for operations
|
|
140
|
+
|
|
141
|
+
For one connector, the `connectorConfig.operations` array and the YAML `operations:` must describe the same verb set — that pair is the connector-wide ceiling.
|
|
142
|
+
Each entity's `@role(...)` actions stay a **subset** of it, so a read-only entity keeps `['read']` on a read/update connector.
|
|
143
|
+
Narrow YAML first, mirror it into `connectorConfig.operations`, then grant each entity only the verbs it actually needs.
|
|
144
|
+
Never widen a decorator to match the connector's full verb set.
|
|
145
|
+
|
|
146
|
+
### Subset rule
|
|
147
|
+
|
|
148
|
+
In subset mode, include in `TSchema` only the entities you actually generated.
|
|
149
|
+
Never reference an entity class you did not generate — the `typeof` lookup and its import would dangle.
|
|
150
|
+
|
|
151
|
+
```ts
|
|
152
|
+
// rayfin/connectors/inventory/schema.ts
|
|
153
|
+
import type { GraphQLBackedConnector } from '@microsoft/rayfin-connector-fabric-graphql';
|
|
154
|
+
import type { ConnectorConfig } from '@microsoft/rayfin-connectors';
|
|
155
|
+
|
|
156
|
+
import type { Order } from './Order.js';
|
|
157
|
+
import type { Customer } from './Customer.js';
|
|
158
|
+
|
|
159
|
+
export type { Order } from './Order.js';
|
|
160
|
+
export type { Customer } from './Customer.js';
|
|
161
|
+
|
|
162
|
+
// Use `as const satisfies` (not a `: ConnectorConfig` annotation) so the
|
|
163
|
+
// `connector` and `operations` literals survive — the marker reads them to
|
|
164
|
+
// derive the permitted operations and the dialect.
|
|
165
|
+
export const connectorConfig = {
|
|
166
|
+
connector: 'fabric-warehouse',
|
|
167
|
+
operations: ['read', 'update'],
|
|
168
|
+
entities: {
|
|
169
|
+
Order: ['orderId', 'customerId', 'total', 'placedUtc'],
|
|
170
|
+
Customer: ['customerId', 'email'],
|
|
171
|
+
},
|
|
172
|
+
} as const satisfies ConnectorConfig;
|
|
173
|
+
|
|
174
|
+
export type InventorySchema = GraphQLBackedConnector<
|
|
175
|
+
{ Order: typeof Order; Customer: typeof Customer },
|
|
176
|
+
typeof connectorConfig
|
|
177
|
+
>;
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
The pattern scales to any number of entities under any names — add one `import type` line, one `export type` line, and one `TSchema` key per entity.
|
|
181
|
+
|
|
182
|
+
The `<Name>Schema` name is the PascalCase connector name plus `Schema` (connector `inventory` → `InventorySchema`).
|
|
183
|
+
For a read-only Lakehouse (`fabric-sqlanalytics`), `operations` is `['read']`.
|
|
184
|
+
|
|
185
|
+
### The `entities` map
|
|
186
|
+
|
|
187
|
+
`connectorConfig.entities` gives the runtime each entity's field names, and it is what makes a call without an explicit `select` work:
|
|
188
|
+
|
|
189
|
+
- Reads — `findMany(filter?)`, `findFirst(filter?)` and `findByKey` use it as the default selection to return the full row.
|
|
190
|
+
- Writes on `fabric-sqldatabase` — `create` / `update` / `delete` use it to read the row back, including server-generated columns the caller never sent.
|
|
191
|
+
|
|
192
|
+
**Omitting it is not optional.** Without it, every no-selection read throws `SELECTION_REQUIRED`, on every dialect.
|
|
193
|
+
|
|
194
|
+
Give the names as **string arrays**, and list every entity in `TSchema`.
|
|
195
|
+
The map also accepts entity classes, but do not use that form here: it puts the decorated classes back in the browser bundle and reintroduces the blank page above.
|
|
196
|
+
|
|
197
|
+
**Use the entity's property names, not the database column names.**
|
|
198
|
+
These are the names on the generated class — the same keys that appear in the row type — not the `columnName` values in `metadata.json`.
|
|
199
|
+
For `@uuid({ column: 'ProductID' }) productId!: string`, the entry is `productId`.
|
|
200
|
+
The one exception is a field that declares `graphqlName`, in which case use that name.
|
|
201
|
+
Include every scalar field and leave relationship fields out, since those need nested sub-selections.
|
|
202
|
+
|
|
203
|
+
**Relationship selects need the class form.**
|
|
204
|
+
Cardinality (`@one` versus `@many`) lives in the decorator metadata, which a name array cannot carry, so selecting a relationship path against the array form throws `ENTITIES_REQUIRED_FOR_RELATIONSHIP_SELECT`.
|
|
205
|
+
Scalar reads and writes are unaffected.
|
|
206
|
+
If an app needs relationship selects in browser code, select the foreign key scalar and fetch the related entity separately, or move that query server-side where the class can be imported.
|
|
207
|
+
|
|
208
|
+
## Wire connectors into the app client
|
|
209
|
+
|
|
210
|
+
Declaring and deploying a connector does not make it callable from app code.
|
|
211
|
+
Expose it through a `ConnectorsRayfinClient` so it shows up as `client.connectors.<name>`.
|
|
212
|
+
|
|
213
|
+
`ConnectorsRayfinClient` is experimental — import it only from the `@microsoft/rayfin-client/experimental` subpath, never the stable `@microsoft/rayfin-client` entry.
|
|
214
|
+
|
|
215
|
+
The connector key must be identical in three places: the `name` in `rayfin.yml`, the property in `AppConnectorsSchema`, and the property in the `connectors` option.
|
|
216
|
+
|
|
217
|
+
```ts
|
|
218
|
+
import { ConnectorsRayfinClient } from '@microsoft/rayfin-client/experimental';
|
|
219
|
+
import { ProductDbSchema, connectorConfig as productDbConfig } from '../../rayfin/connectors/productDb/schema';
|
|
220
|
+
import { InventorySchema, connectorConfig as inventoryConfig } from '../../rayfin/connectors/inventory/schema';
|
|
221
|
+
|
|
222
|
+
type AppConnectorsSchema = {
|
|
223
|
+
productDb: ProductDbSchema;
|
|
224
|
+
inventory: InventorySchema;
|
|
225
|
+
};
|
|
226
|
+
|
|
227
|
+
export function getRayfinClient() {
|
|
228
|
+
// Type params are <DataSchema, FunctionsSchema, ConnectorsSchema> — set the
|
|
229
|
+
// unused slots to Record<string, never>, or pass real schema types if the app
|
|
230
|
+
// also has a data model / functions.
|
|
231
|
+
return new ConnectorsRayfinClient<Record<string, never>, Record<string, never>, AppConnectorsSchema>({
|
|
232
|
+
baseUrl: import.meta.env.VITE_RAYFIN_API_URL,
|
|
233
|
+
publishableKey: import.meta.env.VITE_RAYFIN_PUBLISHABLE_KEY,
|
|
234
|
+
authStorage: true,
|
|
235
|
+
connectors: {
|
|
236
|
+
productDb: productDbConfig,
|
|
237
|
+
inventory: inventoryConfig,
|
|
238
|
+
},
|
|
239
|
+
});
|
|
240
|
+
}
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
Verify by type-checking the project (`tsc --noEmit` or the app build).
|
|
244
|
+
A mismatch between a connector key in `AppConnectorsSchema` and the `connectors` option surfaces there.
|
|
245
|
+
|
|
246
|
+
## Reading and writing connector entities
|
|
247
|
+
|
|
248
|
+
Each entity is reached through `client.connectors.<name>.<Entity>`.
|
|
249
|
+
What the surface exposes depends on the connector's `operations:` (which gate the verbs) and the connector type/dialect (which shapes what writes return).
|
|
250
|
+
|
|
251
|
+
### Reads
|
|
252
|
+
|
|
253
|
+
Reads work identically on Lakehouse, Warehouse, and SQL Database — only writes differ.
|
|
254
|
+
|
|
255
|
+
```ts
|
|
256
|
+
// Query chain: select -> where -> orderBy -> execute
|
|
257
|
+
const orders = await client.connectors.inventory.Order
|
|
258
|
+
.select(['orderId', 'customerEmail', 'total'])
|
|
259
|
+
.where({ total: { gt: 100 } })
|
|
260
|
+
.orderBy({ total: 'desc' })
|
|
261
|
+
.execute();
|
|
262
|
+
|
|
263
|
+
// By-key read. Pass a key OBJECT (all parts of the composite PK) AND a
|
|
264
|
+
// required scalar-only `select`. The result is Pick<Row, selected> | null.
|
|
265
|
+
const order = await client.connectors.inventory.Order.findByKey(
|
|
266
|
+
{ orderId: 'o-1' },
|
|
267
|
+
['orderId', 'customerEmail', 'total'],
|
|
268
|
+
);
|
|
269
|
+
|
|
270
|
+
const lineItem = await client.connectors.sales.OrderItem.findByKey(
|
|
271
|
+
{ orderId: 'o-1', productId: 'p-9' }, // both parts of the composite PK required
|
|
272
|
+
['orderId', 'productId', 'quantity'],
|
|
273
|
+
);
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
`findByKey` takes the key object and a **required** `select` of scalar columns, returning `Pick<Row, selected> | null`.
|
|
277
|
+
Relationships cannot be selected via `findByKey` — read them through the query chain.
|
|
278
|
+
A composite key requires all its parts in the key object; omitting one is a compile error.
|
|
279
|
+
|
|
280
|
+
### Reading related columns
|
|
281
|
+
|
|
282
|
+
`select` can pull columns from a related entity — a `@one` (forward FK) or `@many` (reverse FK) navigation field — by naming them as a **dotted path**.
|
|
283
|
+
The builder expands each path into the nested selection the server expects and unwraps the response so the related rows sit inline on the result.
|
|
284
|
+
|
|
285
|
+
```ts
|
|
286
|
+
const products = await client.connectors.inventory.Product
|
|
287
|
+
.select(['name', 'category.name', 'orderItems.quantity'])
|
|
288
|
+
.where({ stock: { gt: 0 } })
|
|
289
|
+
.execute();
|
|
290
|
+
|
|
291
|
+
products[0].category.name; // @one — related row inline
|
|
292
|
+
products[0].orderItems[0].quantity; // @many — related rows inline (unwrapped)
|
|
293
|
+
|
|
294
|
+
// Paths nest to arbitrary depth, hopping across entities.
|
|
295
|
+
await client.connectors.inventory.Product
|
|
296
|
+
.select(['name', 'orderItems.order.customerEmail'])
|
|
297
|
+
.execute();
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
- Name related columns as a dotted path (`category.name`), never the bare relationship (`category`) — a navigation field is not a selectable leaf and is a compile error.
|
|
301
|
+
- Each path is validated segment-by-segment against the schema, so a wrong hop (`category.nope`) fails to compile; depth is unbounded.
|
|
302
|
+
- `findByKey` cannot select relationships — it is scalar-only.
|
|
303
|
+
|
|
304
|
+
Self-referencing foreign keys generate an entity and a DAB relationship, but the client cannot query across one.
|
|
305
|
+
Do not select a dotted path over a self-relationship.
|
|
306
|
+
|
|
307
|
+
### Writes
|
|
308
|
+
|
|
309
|
+
`create` / `update` / `delete` exist only when the connector's `operations:` and the entity `@role(...)` allow them, and what they return is fixed by the connector's dialect:
|
|
310
|
+
|
|
311
|
+
| Connector type | Dialect | `create`/`update`/`delete` return | Notes |
|
|
312
|
+
| --- | --- | --- | --- |
|
|
313
|
+
| `fabric-sqlanalytics` (Lakehouse) | — | **not available** | Read-only at the host; only `select`/`findByKey`/query chain exist. Writes are a compile error. |
|
|
314
|
+
| `fabric-sqldatabase` (SQL Database) | row-returning | the **full entity row** | Reads the whole row back after the write — every column, including server-generated ones the caller did not send. Needs the [`entities` map](#the-entities-map). |
|
|
315
|
+
| `fabric-warehouse` (Warehouse) | no `OUTPUT` clause | `DbOperationResult { result: string }` | DWSQL cannot read the row back, so mutations resolve to a status object. On success `result` is `"success"`; a failed write throws a GraphQL error. |
|
|
316
|
+
|
|
317
|
+
```ts
|
|
318
|
+
// SQL Database — mutation returns the full row, with every server-generated
|
|
319
|
+
// column filled in by the database.
|
|
320
|
+
const created = await client.connectors.orders.Order.create({
|
|
321
|
+
quantity: 3, // supply only the columns you own…
|
|
322
|
+
unitPrice: 19.99,
|
|
323
|
+
});
|
|
324
|
+
created.id; // server-assigned identity — returned, though never sent
|
|
325
|
+
created.createdUtc; // server default (SYSUTCDATETIME()) — returned
|
|
326
|
+
created.lineTotal; // computed (quantity * unitPrice) — returned
|
|
327
|
+
|
|
328
|
+
// Warehouse — mutation returns DbOperationResult, NOT the row
|
|
329
|
+
const result = await client.connectors.inventory.Order.update(
|
|
330
|
+
{ orderId: 'o-1' }, // key object (all composite parts)
|
|
331
|
+
{ total: 250 }, // partial update
|
|
332
|
+
);
|
|
333
|
+
result.result; // "success" — status string, not the row
|
|
334
|
+
|
|
335
|
+
// Delete by key (both dialects) — full composite key required
|
|
336
|
+
await client.connectors.sales.OrderItem.delete({ orderId: 'o-1', productId: 'p-9' });
|
|
337
|
+
|
|
338
|
+
// Lakehouse — writes do not exist
|
|
339
|
+
client.connectors.analytics.Sales.create({ /* ... */ }); // compile error: read-only
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
**Server-generated columns.**
|
|
343
|
+
These are the columns marked `AutoGenerated<T>` on the entity: `IDENTITY`, any `DEFAULT`, and computed (`AS (...)`) columns.
|
|
344
|
+
They are optional on `create` / `update` and come back populated in the returned row.
|
|
345
|
+
Most cannot be written: passing an `IDENTITY` or computed value is rejected by the database, and rowversion / temporal columns are server-maintained.
|
|
346
|
+
The one exception is a plain `DEFAULT` column — omit it to get the default, or pass a value to override it.
|
|
347
|
+
|
|
348
|
+
By-key `update`/`delete` take the same full key object as `findByKey`.
|
|
349
|
+
A keyless entity (`primaryKey` omitted or `[]`) exposes no `findByKey` / `update` / `delete` at all — it is read-only regardless of the connector type.
|
|
350
|
+
|
|
351
|
+
### Quick decision guide
|
|
352
|
+
|
|
353
|
+
- **Lakehouse (`fabric-sqlanalytics`)** — reads only. Model entities, `select`, `findByKey`, query chain. No `@role` write actions.
|
|
354
|
+
- **SQL Database (`fabric-sqldatabase`)** — full CRUD; mutations hand back the full row, so you can render the created/updated record directly.
|
|
355
|
+
- **Warehouse (`fabric-warehouse`)** — full CRUD; mutations hand back a `DbOperationResult`, so re-query with `findByKey`/`select` if you need the persisted values.
|
|
356
|
+
|
|
357
|
+
For aggregation, `groupBy`, or year/quarter/month time-bucketing, search the docs (`rayfin docs search`).
|
|
358
|
+
|
|
359
|
+
## Worked example — a use case on specific entities
|
|
360
|
+
|
|
361
|
+
The user already ran `rayfin connector add` for a `fabric-warehouse` connector named `sales` and says *"I just need users to read and update orders and their line items."*
|
|
362
|
+
|
|
363
|
+
- **Scope and narrow.** Operations are `read`, `update`; set `operations:` on the `sales` entry to just those.
|
|
364
|
+
- **Generate the subset.** Filter `metadata.json` to `Order` and `OrderItem`, write both per the contract. Drop any `@one(() => Customer, ...)` to an out-of-scope table (no dangling import).
|
|
365
|
+
- **Roles and aggregate.** `@role('authenticated', ['read', 'update'])` on both; aggregate `SalesSchema = GraphQLBackedConnector<{ Order: typeof Order; OrderItem: typeof OrderItem }, typeof connectorConfig>`, with both entities imported and re-exported as **types**, and `connectorConfig` declared `as const satisfies ConnectorConfig` with `operations: ['read', 'update']`.
|
|
366
|
+
- **Wire and verify.** Add `sales` to `AppConnectorsSchema` and the `connectors` option. Type-check: `client.connectors.sales.Order.create(...)` is now a compile error — the proof the scope took effect. Deploy with `rayfin up`.
|
|
367
|
+
|
|
368
|
+
## Entity generation contract
|
|
369
|
+
|
|
370
|
+
When you generate `rayfin/connectors/<name>/<EntityName>.ts` from `metadata.json`, follow these rules end-to-end.
|
|
371
|
+
`metadata.json` is the only source of truth for physical keys and relationships — never infer either from column names, values, naming conventions, or table shape.
|
|
372
|
+
The CLI itself does not emit entity files.
|
|
373
|
+
|
|
374
|
+
### 1. File name and class name
|
|
375
|
+
|
|
376
|
+
- `className = pascalCase(table.tableName)` (for example `product_category` → `ProductCategory`).
|
|
377
|
+
- **Pluralization is allowed, but must be idempotent — never double-pluralize.**
|
|
378
|
+
Pluralizing a singular table name is fine (`Order` → `Orders`, `Category` → `Categories`).
|
|
379
|
+
But first check whether the name is already plural: if the source table is already plural (`Orders`, `Categories`, `sales_line_items`), keep it exactly as-is — do not add another plural suffix.
|
|
380
|
+
Whatever name you settle on, the `@entity` name and the `client.connectors.<name>.<Entity>` access path must stay consistent with it.
|
|
381
|
+
- File name is `<className>.ts`. One file per table; no nesting.
|
|
382
|
+
- **Cross-connector uniqueness — disambiguate only on collision.**
|
|
383
|
+
GraphQL type names are global across every connector in the app: two connectors that each produce an entity with the same name collide at `rayfin up connector apply` time even when they point at different physical tables.
|
|
384
|
+
Before finalizing a name, scan every other `rayfin/connectors/*/` directory and the entities you have already generated for the current connector for a matching class / `@entity` name.
|
|
385
|
+
If — and only if — the base name is already taken, prefix it with the connector's source database name (PascalCased, from `metadata.json` `source`): `Product` → `SalesDbProduct`.
|
|
386
|
+
Disambiguation is required even when the colliding entities refer to the same physical source table.
|
|
387
|
+
If that database name is itself shared across the colliding connectors, fall back to the PascalCased connector name (the `rayfin.yml` `name`, which the host guarantees unique).
|
|
388
|
+
Never rename a name that does not collide, and never blanket-prefix every entity.
|
|
389
|
+
When you rename an entity, use the disambiguated name everywhere: the class name, the file name, the `@entity` name, the `TSchema` key and re-export, and the `client.connectors.<name>.<Entity>` access path.
|
|
390
|
+
Surface a one-line note to the user for each rename.
|
|
391
|
+
|
|
392
|
+
### 2. Primary keys from metadata
|
|
393
|
+
|
|
394
|
+
When `table.primaryKeyColumns` is present and non-empty, declare exactly those columns on `Source(...)`, using each column's TypeScript property name and preserving `table.primaryKeyColumns` (`ORDINAL_POSITION`) order.
|
|
395
|
+
This rule applies to single-column and composite keys.
|
|
396
|
+
Keep one property per key column and preserve each SQL column name with `column:` when it differs.
|
|
397
|
+
|
|
398
|
+
```ts
|
|
399
|
+
export class OrderItem extends Source({
|
|
400
|
+
schema: 'dbo',
|
|
401
|
+
table: 'OrderItem',
|
|
402
|
+
primaryKey: ['orderId', 'productId'],
|
|
403
|
+
}) {
|
|
404
|
+
@uuid({ column: 'OrderID' }) orderId!: string;
|
|
405
|
+
@uuid({ column: 'ProductID' }) productId!: string;
|
|
406
|
+
@int() quantity!: number;
|
|
407
|
+
}
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
Every key part is then required in the by-key methods (`findByKey`/`update`/`delete`).
|
|
411
|
+
|
|
412
|
+
When `primaryKeyColumns` is absent or empty, emit `primaryKey: []`.
|
|
413
|
+
The entity is keyless and exposes no `findByKey`/`update`/`delete` methods.
|
|
414
|
+
Do not choose a column as a synthetic key, including columns named `id`, columns that appear unique in sampled data, or the first non-nullable column.
|
|
415
|
+
|
|
416
|
+
Lakehouse SQL endpoints commonly omit PK metadata.
|
|
417
|
+
For `fabric-sqlanalytics`, this means generating a read-only, keyless entity unless `primaryKeyColumns` is actually present in `metadata.json`.
|
|
418
|
+
Never add a logical PK on the agent's own initiative; a Builder may deliberately declare one later when they know the source's logical uniqueness contract.
|
|
419
|
+
|
|
420
|
+
### 3. Key-field validation
|
|
421
|
+
|
|
422
|
+
- Resolve every `primaryKeyColumns` entry against `table.columns` by exact SQL column name before converting it to a TypeScript property name.
|
|
423
|
+
- Preserve the metadata order for composite keys.
|
|
424
|
+
- A key column must be non-nullable. If metadata names a nullable or missing column, do not guess a replacement; stop generation for that table and report the inconsistency.
|
|
425
|
+
- An empty table or a table without PK metadata is keyless.
|
|
426
|
+
|
|
427
|
+
The key's property name and datatype come from the matching column metadata; the PK is not renamed to `id`.
|
|
428
|
+
Preserve the on-disk SQL column name via the `column:` option.
|
|
429
|
+
|
|
430
|
+
### 4. SQL type to decorator mapping
|
|
431
|
+
|
|
432
|
+
Look up `column.dataType` (case-insensitive) in this table:
|
|
433
|
+
|
|
434
|
+
| SQL type family | Decorator | TS type |
|
|
435
|
+
| --- | --- | --- |
|
|
436
|
+
| `int`, `bigint`, `smallint`, `tinyint` | `@int()` | `number` |
|
|
437
|
+
| `decimal`, `numeric`, `money`, `smallmoney`, `float`, `real` | `@decimal({ precision, scale })` | `number` |
|
|
438
|
+
| `bit` | `@boolean()` | `boolean` |
|
|
439
|
+
| `date`, `datetime`, `datetime2`, `smalldatetime`, `datetimeoffset`, `time` | `@date()` | `Date` |
|
|
440
|
+
| `uniqueidentifier` | `@uuid()` | `string` |
|
|
441
|
+
| `varchar`, `nvarchar`, `char`, `nchar`, `text`, `ntext` | `@text()` | `string` |
|
|
442
|
+
| Anything else (`geography`, `hierarchyid`, `xml`, vector) | `@text()` + warning | `string` |
|
|
443
|
+
|
|
444
|
+
For the fallback case, emit a warning: `Unknown SQL type <dataType> for <table>.<column>; falling back to @text().`
|
|
445
|
+
|
|
446
|
+
### 4a. Server-generated columns
|
|
447
|
+
|
|
448
|
+
`metadata.json` flags the columns the database fills in.
|
|
449
|
+
On each column, check for these server-generation markers:
|
|
450
|
+
|
|
451
|
+
- `identity` — an `IDENTITY(seed, increment)` key.
|
|
452
|
+
- `default` — a column `DEFAULT` (for example `newid()`, `sysutcdatetime()`, `NEXT VALUE FOR <seq>`).
|
|
453
|
+
- `computed` — a computed column (`AS (<expr>)`).
|
|
454
|
+
- `serverManaged` — a column the server maintains with no user-facing expression: `'rowversion'`, `'temporalRowStart'`, or `'temporalRowEnd'`.
|
|
455
|
+
|
|
456
|
+
If any of these is present, the column is server-generated: wrap its TS type from section 4 in `AutoGenerated<…>`.
|
|
457
|
+
The decorator and its `column:` option are unchanged; do not emit a `default:` option — the marker is type-only and the connector never writes these columns.
|
|
458
|
+
|
|
459
|
+
`AutoGenerated<T>` makes the column optional on create/update input and read back as plain `T` in the returned row.
|
|
460
|
+
Nullability is unchanged: a non-null server-generated column still uses `!:`; a nullable one uses `?:`.
|
|
461
|
+
|
|
462
|
+
Optional on input does not mean "accepts a value".
|
|
463
|
+
For most server-generated columns, passing a value is rejected by the database:
|
|
464
|
+
|
|
465
|
+
- `identity` — inserting an explicit value fails. Never send it.
|
|
466
|
+
- `computed` — a computed column cannot be written; setting it is a server error.
|
|
467
|
+
- `serverManaged` — server-maintained; writing is rejected.
|
|
468
|
+
- plain `default` — the one exception: omit it to get the default, or pass a value to override it.
|
|
469
|
+
|
|
470
|
+
```ts
|
|
471
|
+
@int({ column: 'Id' })
|
|
472
|
+
id!: AutoGenerated<number>; // IDENTITY — omit on write, server assigns
|
|
473
|
+
|
|
474
|
+
@uuid({ column: 'PublicId' })
|
|
475
|
+
publicId!: AutoGenerated<string>; // DEFAULT newid()
|
|
476
|
+
|
|
477
|
+
@decimal({ optional: true, column: 'LineTotal', precision: 28, scale: 2 })
|
|
478
|
+
lineTotal?: AutoGenerated<number>; // computed: AS ([Quantity] * [UnitPrice])
|
|
479
|
+
|
|
480
|
+
@text({ column: 'RowVer' })
|
|
481
|
+
rowVer!: AutoGenerated<string>; // rowversion (serverManaged) — read-back-only
|
|
482
|
+
```
|
|
483
|
+
|
|
484
|
+
### 5. Field-level decorator options
|
|
485
|
+
|
|
486
|
+
For every column, the decorator option object is built in this order (omit keys you do not need):
|
|
487
|
+
|
|
488
|
+
1. `optional: true` — if `column.isNullable` is true.
|
|
489
|
+
2. `column: '<columnName>'` — when the SQL column name differs from the TS property name. Single-quote the value; escape embedded `'`.
|
|
490
|
+
3. **Text only:** `max: <maxLength>` — when the decorator is `text` and `column.maxLength > 0`.
|
|
491
|
+
4. **Decimal only:** `precision: <p>, scale: <s>` — when the decorator is `decimal` and both are present in metadata.
|
|
492
|
+
5. **Integer:** do not emit `min`/`max` based on SQL precision; those are value bounds, not storage capacity. Plain `@int()` is correct.
|
|
493
|
+
|
|
494
|
+
If the option object is empty, emit `@text()` rather than `@text({})`.
|
|
495
|
+
|
|
496
|
+
Property name is always `camelCase(column.columnName)` — the PK is not special-cased or renamed to `id`.
|
|
497
|
+
Nullable columns use `?:`; non-nullable use `!:`.
|
|
498
|
+
|
|
499
|
+
### 5a. Server-generated columns have no decorator option
|
|
500
|
+
|
|
501
|
+
`metadata.json` carries `column.identity`, `column.default`, and `column.computed`, but no field decorator has an option to mark a column as server-generated.
|
|
502
|
+
Do not invent one.
|
|
503
|
+
|
|
504
|
+
Declare the column normally — those markers change nothing about the emitted decorator, beyond the `AutoGenerated<T>` type wrapper in section 4a.
|
|
505
|
+
What they do change is what you tell the Builder: surface a one-line note for each such column (for example, `Column '<table>.<column>' is server-generated (identity); the source will populate it, so omit it on create()`).
|
|
506
|
+
|
|
507
|
+
An identity or computed column that is also a declared key follows the normal [primary key rules](#2-primary-keys-from-metadata) unchanged — being server-generated never exempts it from key-field validation.
|
|
508
|
+
|
|
509
|
+
### 6. Global-type shadow rule
|
|
510
|
+
|
|
511
|
+
If `pascalCase(table.tableName)` equals a column's TS type (for example a table named `Date` with a `datetime2` column), the unqualified `Date` in the annotation resolves to the entity class, not the global.
|
|
512
|
+
Render the type as `globalThis.Date` in that one field annotation only.
|
|
513
|
+
|
|
514
|
+
### 7. Relationships
|
|
515
|
+
|
|
516
|
+
Forward (`@one`) and reverse (`@many`) relationships are emitted from foreign-key metadata **only**.
|
|
517
|
+
Do not infer Lakehouse relationships from matching column names, star-schema naming, or sampled values.
|
|
518
|
+
|
|
519
|
+
Schema discovery records an FK on the referencing table only, so `table.foreignKeys` governs **forward** relationships alone.
|
|
520
|
+
When it is absent or empty, emit no `@one` — but still emit every `@many` the reverse-FK index below produces, because a parent table legitimately has no FKs of its own while being the target of another table's.
|
|
521
|
+
A table with neither its own FKs nor any incoming constraint gets no relationship decorators at all.
|
|
522
|
+
|
|
523
|
+
Before generating relationships, group `table.foreignKeys` entries by `constraintName`.
|
|
524
|
+
One group is one FK relationship; rows in the same group are the ordered column pairs of a composite FK.
|
|
525
|
+
Preserve their metadata order.
|
|
526
|
+
|
|
527
|
+
**`@one` — one per FK constraint on this table.** For each FK constraint group:
|
|
528
|
+
|
|
529
|
+
- All rows must reference the same schema and table. If they do not, report inconsistent metadata and skip the constraint.
|
|
530
|
+
- `fieldName = camelCase(singularize(referencedTableName))`. If more than one constraint on the table would produce the same field name, derive a stable disambiguated name from `constraintName` rather than overwriting one.
|
|
531
|
+
- `sourceFields = group.map(fk => camelCase(fk.columnName))`.
|
|
532
|
+
- `targetFields = group.map(fk => camelCase(fk.referencedColumnName))`.
|
|
533
|
+
- The relationship is optional when any source column in the group is nullable. Emit `{ optional: true }` and `?:` in that case.
|
|
534
|
+
|
|
535
|
+
```ts
|
|
536
|
+
@one(() => <ReferencedClass>, { sourceFields: ['<src1>', '<src2>'], targetFields: ['<tgt1>', '<tgt2>'] })
|
|
537
|
+
<fieldName>!: <ReferencedClass>;
|
|
538
|
+
|
|
539
|
+
// Optional relationship
|
|
540
|
+
@one(() => <ReferencedClass>, { optional: true, sourceFields: ['<src>'], targetFields: ['<tgt>'] })
|
|
541
|
+
<fieldName>?: <ReferencedClass>;
|
|
542
|
+
```
|
|
543
|
+
|
|
544
|
+
Self-referencing FK constraints follow the same rules and must be optional.
|
|
545
|
+
Use the current class directly in the resolver and add no sibling import.
|
|
546
|
+
The entity and DAB relationship generate fine, but the client does not currently support selecting a dotted path across a self-relationship.
|
|
547
|
+
|
|
548
|
+
If the referenced table is not in the table index, skip the relationship and warn: `Foreign key <table>.<column> references unknown table <refSchema>.<refTable>; relationship skipped.`
|
|
549
|
+
|
|
550
|
+
In subset mode, apply the same skip when the referenced table exists in `metadata.json` but is not in the set you are generating: omit the `@one` and its sibling import, and warn that the relationship was dropped because the target is out of scope.
|
|
551
|
+
Never import a `./<Class>.js` file you did not write.
|
|
552
|
+
|
|
553
|
+
**`@many` — reverse-direction relationships pointing at this table.**
|
|
554
|
+
Build a reverse-FK index across all tables: every grouped FK constraint from another table to this table becomes one `@many` on this table.
|
|
555
|
+
|
|
556
|
+
- `fieldName = camelCase(pluralize(otherTable.tableName))` — `pluralize` is idempotent, so an already-plural table (`Orders`) stays `orders`, never `orderses`.
|
|
557
|
+
- `sourceFields = group.map(fk => camelCase(fk.referencedColumnName))` — this table's referenced properties.
|
|
558
|
+
- `targetFields = group.map(fk => camelCase(fk.columnName))` — the other table's FK properties.
|
|
559
|
+
|
|
560
|
+
```ts
|
|
561
|
+
@many(() => <OtherClass>, { sourceFields: ['<src>'], targetFields: ['<tgt>'] })
|
|
562
|
+
<fieldName>!: <OtherClass>[];
|
|
563
|
+
```
|
|
564
|
+
|
|
565
|
+
In subset mode, only emit a `@many` when the other table is also being generated.
|
|
566
|
+
|
|
567
|
+
`singularize` / `pluralize` are the same simplified rules the generator uses:
|
|
568
|
+
|
|
569
|
+
- `singularize`: `ies → y` (length > 3); `xes|ses|ches|shes → drop -es`; `<non-s>s → drop trailing -s`; else unchanged.
|
|
570
|
+
- `pluralize` (idempotent): if the name is already plural (ends in `s`, `es`, or `ies`), return it unchanged; otherwise `<non-vowel>y → -ies`; `x|z|ch|sh → +es`; else `+s`.
|
|
571
|
+
|
|
572
|
+
### 8. Missing key and relationship metadata
|
|
573
|
+
|
|
574
|
+
When `primaryKeyColumns` is absent or empty, emit one warning to the user: `No PK metadata available for <tableName>; generated as a keyless entity.`
|
|
575
|
+
|
|
576
|
+
When `foreignKeys` is absent or empty and no reverse FK constraint points at the table, emit one warning: `No FK metadata available for <tableName>; relationships omitted.`
|
|
577
|
+
|
|
578
|
+
For Lakehouse, these warnings describe a known metadata limitation, not a request to infer schema.
|
|
579
|
+
The entity file still generates with `primaryKey: []` and without `@one`/`@many`.
|
|
580
|
+
If the Builder knows logical keys or relationships the endpoint does not expose, explain that they may add those declarations manually; never add them without that explicit input.
|
|
581
|
+
|
|
582
|
+
### 9. Imports
|
|
583
|
+
|
|
584
|
+
Build the import line deterministically:
|
|
585
|
+
|
|
586
|
+
- Always include `entity` and `Source`.
|
|
587
|
+
- Then append, in this exact order, any decorator names actually used in the file: `boolean`, `date`, `decimal`, `int`, `text`, `uuid`, `one`, `many`. Skip any not used.
|
|
588
|
+
- All from `@microsoft/rayfin-core/experimental`.
|
|
589
|
+
- If any column is server-generated, also add `import type { AutoGenerated } from '@microsoft/rayfin-core/experimental';`.
|
|
590
|
+
- For every relationship target that is a different class, add a sibling import: `import { <SiblingClass> } from './<SiblingClass>.js';`. Sibling imports are alphabetised; self-references get no sibling import.
|
|
591
|
+
- In subset mode, only relationships that survived the subset skip contribute sibling imports.
|
|
592
|
+
|
|
593
|
+
### 10. Canonical example
|
|
594
|
+
|
|
595
|
+
```ts
|
|
596
|
+
// @generated — do not edit.
|
|
597
|
+
|
|
598
|
+
import { entity, uuid, text, int, date, one, many, Source } from '@microsoft/rayfin-core/experimental';
|
|
599
|
+
import type { AutoGenerated } from '@microsoft/rayfin-core/experimental';
|
|
600
|
+
import { Category } from './Category.js';
|
|
601
|
+
import { OrderItem } from './OrderItem.js';
|
|
602
|
+
|
|
603
|
+
@entity()
|
|
604
|
+
export class Product extends Source({ schema: 'dbo', table: 'Product', primaryKey: ['productId'] }) {
|
|
605
|
+
@uuid({ column: 'ProductID' })
|
|
606
|
+
productId!: string;
|
|
607
|
+
|
|
608
|
+
@text()
|
|
609
|
+
name!: string;
|
|
610
|
+
|
|
611
|
+
@int()
|
|
612
|
+
stock!: number;
|
|
613
|
+
|
|
614
|
+
// DEFAULT sysutcdatetime() — server-generated, so AutoGenerated: optional on
|
|
615
|
+
// write, read back in the returned row.
|
|
616
|
+
@date({ column: 'CreatedUtc' })
|
|
617
|
+
createdUtc!: AutoGenerated<Date>;
|
|
618
|
+
|
|
619
|
+
@one(() => Category, { sourceFields: ['categoryId'], targetFields: ['categoryId'] })
|
|
620
|
+
category!: Category;
|
|
621
|
+
|
|
622
|
+
@many(() => OrderItem, { sourceFields: ['productId'], targetFields: ['productId'] })
|
|
623
|
+
orderItems!: OrderItem[];
|
|
624
|
+
}
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
## metadata.json reference
|
|
628
|
+
|
|
629
|
+
Path: `rayfin/connectors/<name>/metadata.json`.
|
|
630
|
+
Written by `connector add`; never edit by hand.
|
|
631
|
+
It is regenerated only by `rayfin connector remove <name>` followed by `rayfin connector add ...`.
|
|
632
|
+
|
|
633
|
+
Top-level: `SchemaMetadata { source, connector, connectionString, discoveredAt, schemas[] }`.
|
|
634
|
+
Each schema entry: `{ schemaName, tables[] }`.
|
|
635
|
+
Each table: `{ tableName, columns[], foreignKeys?, primaryKeyColumns? }`.
|
|
636
|
+
|
|
637
|
+
Columns carry `columnName`, `dataType`, `isNullable`, and optional `maxLength` / `precision` / `scale`.
|
|
638
|
+
Server-generated columns additionally carry `identity` (`{ seed, increment }`), `default` (the SQL default expression), `computed` (the `AS (...)` expression), `serverManaged`, and `datePrecision`.
|
|
639
|
+
Foreign keys carry `constraintName`, `columnName`, and the `referencedTableSchema` / `referencedTableName` / `referencedColumnName` triple.
|
|
640
|
+
|
|
641
|
+
`primaryKeyColumns`, when available, contains single-column or composite PKs in `ORDINAL_POSITION` order.
|
|
642
|
+
Multiple `foreignKeys` entries with the same `constraintName` form one composite FK and must be generated as one relationship.
|
|
643
|
+
|
|
644
|
+
Fabric SQL Database generally exposes PK/FK and server-generation metadata.
|
|
645
|
+
Warehouse and Lakehouse may omit some catalog metadata; Lakehouse commonly omits PK/FK constraints entirely.
|
|
646
|
+
Absence means "unknown / not exposed", not permission to synthesize keys or relationships.
|
|
647
|
+
|
|
648
|
+
## Category A anti-patterns
|
|
649
|
+
|
|
650
|
+
- Never leave `schema.ts` as bare re-exports — the client import of `<Name>Schema` and `connectorConfig` would fail.
|
|
651
|
+
- Never import or re-export entity classes as **values** in `schema.ts`. Use `import type` / `export type`; a value import ships decorated classes to the browser and the page renders blank.
|
|
652
|
+
- Populate `connectorConfig.entities` with **column-name arrays**, never entity classes. Omitting it makes every no-selection read throw `SELECTION_REQUIRED`; using classes puts them back in the bundle.
|
|
653
|
+
- In subset mode, list in `TSchema` only entities you actually generated.
|
|
654
|
+
- Keep `connectorConfig.operations` identical to the YAML `operations:`, and keep each entity `@role(...)` action a subset of that pair. Narrow YAML first; never widen above the type's catalog allowlist, and never widen a decorator just to match the connector.
|
|
655
|
+
- Policies use the typed `claims` / `item` DSL — never raw SQL or DAB-policy strings like `"@claims.sub eq @item.owner_id"`.
|
|
656
|
+
- Never double-pluralize an entity/class name.
|
|
657
|
+
- GraphQL type names are global across connectors — disambiguate duplicates, never blanket-prefix.
|
|
658
|
+
- `metadata.json` is the only source of truth for keys and relationships. Never synthesize a primary key or infer a relationship from column names, sampled values, or naming conventions — absent metadata means keyless and relationship-free.
|
|
659
|
+
- When the user asks for one entity, read `metadata.json` and filter — do not regenerate every table.
|
|
660
|
+
- Never edit `metadata.json` or `dab-config.json` by hand — both are regenerated.
|
|
661
|
+
|
|
662
|
+
## Troubleshooting
|
|
663
|
+
|
|
664
|
+
| Symptom | Likely cause | Fix |
|
|
665
|
+
| --- | --- | --- |
|
|
666
|
+
| `rayfin up connector apply` fails on a role action | Entity `@role(...)` includes an action not in the YAML `operations:` | Narrow the decorator to match YAML. |
|
|
667
|
+
| `rayfin up connector apply` fails with a duplicate / redefined GraphQL type | Two connectors generated an entity with the same name — type names are global | Disambiguate the colliding entity by prefixing its source database name; update the class, file name, `@entity` name, `TSchema` key, and access path together, then re-apply. |
|
|
668
|
+
| `rayfin connector add` writes the YAML entry but no entity files | Expected — the CLI never emits entity `.ts` files | Generate them from `metadata.json` per the contract. If `metadata.json` is also missing, schema discovery failed. |
|
|
669
|
+
| `Property '<name>' does not exist on connectors` | Connector key in `AppConnectorsSchema` does not match the key in the `connectors` option | Use the `rayfin.yml` `name` in all three places. |
|
|
670
|
+
| A CRUD method is missing from autocomplete | Expected: `<Name>Schema` narrows methods to the connector's `operations` | Widen the allow-list via `rayfin connector add --operations`, then re-import. |
|
|
671
|
+
| Import of `ConnectorsRayfinClient` fails to resolve | Imported from the stable `@microsoft/rayfin-client` entry | Import from `@microsoft/rayfin-client/experimental`. |
|
|
672
|
+
| `Cannot find module '@microsoft/rayfin-connector-fabric-graphql'` | The connector packages were never installed — `connector add` does not add them | Run the pinned `npm install` command `connector add` printed, or rebuild it from `rayfin connector types --json`. |
|
|
673
|
+
| Deployed page is blank with `Uncaught SyntaxError: Invalid or unexpected token`, though build, tests and deploy all passed | `schema.ts` imports or re-exports entity classes as values, so the decorated classes were bundled into the browser build | Switch every entity import and re-export in `schema.ts` to `import type` / `export type`, and give `connectorConfig.entities` column-name arrays instead of classes. |
|
|
674
|
+
| A read throws `SELECTION_REQUIRED` | `connectorConfig.entities` is missing, so there is no default column list | Add the entity's property names, or pass an explicit `select([...])`. |
|
|
675
|
+
| A read returns null or errors on a field that exists in the database | `entities` lists database column names instead of the entity's property names | Use the names declared on the generated class (or `graphqlName` where set), for example `productId`, not `ProductID`. |
|
|
@@ -0,0 +1,269 @@
|
|
|
1
|
+
---
|
|
2
|
+
sidebar_position: 7
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# Category B — function-bridge connectors
|
|
6
|
+
|
|
7
|
+
Reference for `kusto` (Fabric KQL Database) and `fabric-semanticmodel` (Power BI semantic model).
|
|
8
|
+
A Category B connector is a named-operation surface backed by a small, platform-owned function (UDF).
|
|
9
|
+
The Builder never writes or sees the function code.
|
|
10
|
+
|
|
11
|
+
The Builder declares the connector and calls a typed method; the Fabric app backend injects connector configuration and delegated authentication before forwarding to the function.
|
|
12
|
+
There are no GraphQL entities, so do **not** generate entity files, `@role` policies, or `metadata.json` entities for these connector types.
|
|
13
|
+
|
|
14
|
+
For the entity-generating types (`fabric-sqlanalytics`, `fabric-warehouse`, `fabric-sqldatabase`), read [Category A](./category-a-entities.md) instead.
|
|
15
|
+
|
|
16
|
+
## What each type exposes
|
|
17
|
+
|
|
18
|
+
| Type | Operations | Query language | Auth |
|
|
19
|
+
| --- | --- | --- | --- |
|
|
20
|
+
| `kusto` | `executeQuery`, `executeCommand` | KQL | `delegated` only |
|
|
21
|
+
| `fabric-semanticmodel` | `executeQuery` | DAX | `delegated` only |
|
|
22
|
+
|
|
23
|
+
Both are pinned to an adapter version (`version: '1'` today).
|
|
24
|
+
Delegated authentication runs every call as the signed-in user through the on-behalf-of flow.
|
|
25
|
+
|
|
26
|
+
## Add the connector
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
rayfin connector add \
|
|
30
|
+
--type kusto \
|
|
31
|
+
--workspace-id <ws-id> \
|
|
32
|
+
--item-id <kql-database-item-id> \
|
|
33
|
+
--name telemetry
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`--item-id` identifies the Fabric item to query: a KQL Database for `kusto`, a semantic model for `fabric-semanticmodel`.
|
|
37
|
+
Omit `--name` to derive the connector name from the Fabric item's display name.
|
|
38
|
+
|
|
39
|
+
The CLI verifies the item, writes the `rayfin.yml` entry, and scaffolds `rayfin/connectors/<name>/schema.ts`.
|
|
40
|
+
|
|
41
|
+
## Resulting rayfin.yml
|
|
42
|
+
|
|
43
|
+
```yaml
|
|
44
|
+
connectors:
|
|
45
|
+
- name: telemetry
|
|
46
|
+
type: kusto
|
|
47
|
+
version: '1'
|
|
48
|
+
config:
|
|
49
|
+
workspaceId: <ws-id>
|
|
50
|
+
itemId: <kql-database-item-id>
|
|
51
|
+
auth:
|
|
52
|
+
type: delegated
|
|
53
|
+
operations:
|
|
54
|
+
- name: executeQuery
|
|
55
|
+
- name: executeCommand
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
The `config` block is the single source of truth for what to query and is not sent on the client wire.
|
|
59
|
+
`kusto` allows `executeQuery` and `executeCommand`; `fabric-semanticmodel` allows `executeQuery`.
|
|
60
|
+
In every case `auth.type` must be `delegated`.
|
|
61
|
+
|
|
62
|
+
## The generated schema.ts
|
|
63
|
+
|
|
64
|
+
`connector add` writes `rayfin/connectors/<name>/schema.ts` with a `// @generated — do not edit.` banner.
|
|
65
|
+
Do not hand-edit it.
|
|
66
|
+
Regenerate it by removing and re-adding the connector (`rayfin connector remove <name>`, then `rayfin connector add ...`).
|
|
67
|
+
|
|
68
|
+
For `fabric-semanticmodel` the file exports the typed marker plus a generic runtime config:
|
|
69
|
+
|
|
70
|
+
```ts
|
|
71
|
+
import type { ConnectorConfig } from '@microsoft/rayfin-connectors';
|
|
72
|
+
import type { FabricSemanticModel } from '@microsoft/rayfin-connector-fabric-semanticmodel';
|
|
73
|
+
|
|
74
|
+
export type SalesModelSchema = FabricSemanticModel<'executeQuery'>;
|
|
75
|
+
|
|
76
|
+
export const connectorConfig = {
|
|
77
|
+
connector: 'fabric-semanticmodel',
|
|
78
|
+
} as const satisfies ConnectorConfig;
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Kusto bakes cluster routing into the generated file
|
|
82
|
+
|
|
83
|
+
`connector add --type kusto` resolves the KQL Database's cluster query endpoint and database name from `(workspaceId, itemId)` at add time and writes both into the generated `connectorConfig`:
|
|
84
|
+
|
|
85
|
+
```ts
|
|
86
|
+
import type { Kusto, KustoConnectorConfig } from '@microsoft/rayfin-connector-kusto';
|
|
87
|
+
|
|
88
|
+
export type TelemetrySchema = Kusto<'executeQuery' | 'executeCommand'>;
|
|
89
|
+
|
|
90
|
+
export const connectorConfig = {
|
|
91
|
+
connector: 'kusto',
|
|
92
|
+
queryServiceUri: 'https://<cluster>.kusto.fabric.microsoft.com',
|
|
93
|
+
databaseName: '<database>',
|
|
94
|
+
} as const satisfies KustoConnectorConfig;
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Three names appear here and nowhere else:
|
|
98
|
+
|
|
99
|
+
- `queryServiceUri` — the resolved Kusto cluster query endpoint.
|
|
100
|
+
- `databaseName` — the resolved KQL database name.
|
|
101
|
+
- `KustoConnectorConfig` — the Kusto-specific config type these two keys satisfy, exported from `@microsoft/rayfin-connector-kusto` rather than `@microsoft/rayfin-connectors`.
|
|
102
|
+
|
|
103
|
+
These keys live only in the file the Kusto scaffold writes.
|
|
104
|
+
They are **not** part of the shared `rayfin.yml` schema — never write a cluster URI or database name into `rayfin.yml`, and never send either value from app code.
|
|
105
|
+
If the resolved values look wrong, re-add the connector rather than editing the generated file; the values come from Fabric, not from anything you can fix by hand.
|
|
106
|
+
|
|
107
|
+
The Kusto scaffold imports both its marker and `KustoConnectorConfig` from `@microsoft/rayfin-connector-kusto`, so it does not import `@microsoft/rayfin-connectors` at all.
|
|
108
|
+
|
|
109
|
+
## Install the packages the generated file imports
|
|
110
|
+
|
|
111
|
+
`connector add` scaffolds files but installs nothing. It prints the exact pinned command — copy it from that output, or rebuild it from the `packages` array in `rayfin connector types --json`, which carries both the package names and the version:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
# Shape only. Use the version connector add printed, not this one.
|
|
115
|
+
|
|
116
|
+
# kusto — marker and config type both come from this one package
|
|
117
|
+
npm install @microsoft/rayfin-connector-kusto@1.35.0-alpha
|
|
118
|
+
|
|
119
|
+
# fabric-semanticmodel
|
|
120
|
+
npm install @microsoft/rayfin-connector-fabric-semanticmodel@1.35.0-alpha @microsoft/rayfin-connectors@1.35.0-alpha
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
**Always pin the version.** Connector packages ship in lockstep with the CLI, but their npm `latest` and `preview` tags lag behind, so an unversioned install pulls an older release and its own mismatched `@microsoft/rayfin-data`.
|
|
124
|
+
|
|
125
|
+
## Wire into the app client
|
|
126
|
+
|
|
127
|
+
Use the same `ConnectorsRayfinClient` wiring as Category A, but with no entity re-exports and no `GraphQLBackedConnector`.
|
|
128
|
+
Import the generated `<Name>Schema` type and `connectorConfig` value, key both maps by the exact `rayfin.yml` connector name, and pass the config through the client's `connectors` option.
|
|
129
|
+
|
|
130
|
+
Category B additionally needs a **connector runtime map** as the client's second constructor argument.
|
|
131
|
+
This is not optional: the runtime is what injects the generated routing and decodes the response.
|
|
132
|
+
|
|
133
|
+
- `kusto()` merges the generated `queryServiceUri` and `databaseName` into the outbound payload. Without it, `executeQuery` and `executeCommand` cannot route to the cluster.
|
|
134
|
+
- `fabricSemanticModel()` decodes the Arrow response. Without it, `executeQuery` results cannot be read.
|
|
135
|
+
|
|
136
|
+
Key the runtime map by the same connector name, and call the factory once per connector:
|
|
137
|
+
|
|
138
|
+
```ts
|
|
139
|
+
import { ConnectorsRayfinClient } from '@microsoft/rayfin-client/experimental';
|
|
140
|
+
import { kusto } from '@microsoft/rayfin-connector-kusto';
|
|
141
|
+
import { fabricSemanticModel } from '@microsoft/rayfin-connector-fabric-semanticmodel';
|
|
142
|
+
import {
|
|
143
|
+
type TelemetrySchema,
|
|
144
|
+
connectorConfig as telemetryConfig,
|
|
145
|
+
} from '../../rayfin/connectors/telemetry/schema';
|
|
146
|
+
import {
|
|
147
|
+
type SalesModelSchema,
|
|
148
|
+
connectorConfig as salesModelConfig,
|
|
149
|
+
} from '../../rayfin/connectors/salesModel/schema';
|
|
150
|
+
|
|
151
|
+
type AppConnectorsSchema = {
|
|
152
|
+
telemetry: TelemetrySchema;
|
|
153
|
+
salesModel: SalesModelSchema;
|
|
154
|
+
};
|
|
155
|
+
|
|
156
|
+
const client = new ConnectorsRayfinClient<
|
|
157
|
+
Record<string, never>,
|
|
158
|
+
Record<string, never>,
|
|
159
|
+
AppConnectorsSchema
|
|
160
|
+
>(
|
|
161
|
+
{
|
|
162
|
+
baseUrl: import.meta.env.VITE_RAYFIN_API_URL,
|
|
163
|
+
publishableKey: import.meta.env.VITE_RAYFIN_PUBLISHABLE_KEY,
|
|
164
|
+
connectors: {
|
|
165
|
+
telemetry: telemetryConfig,
|
|
166
|
+
salesModel: salesModelConfig,
|
|
167
|
+
},
|
|
168
|
+
},
|
|
169
|
+
// Second argument: per-connector runtime hooks, keyed by connector name.
|
|
170
|
+
{
|
|
171
|
+
telemetry: kusto(),
|
|
172
|
+
salesModel: fabricSemanticModel(),
|
|
173
|
+
}
|
|
174
|
+
);
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
`ConnectorsRayfinClient` is experimental — import it only from the `@microsoft/rayfin-client/experimental` subpath, never the stable `@microsoft/rayfin-client` entry.
|
|
178
|
+
|
|
179
|
+
The connector key must be identical in four places: the `name` in `rayfin.yml`, the property in `AppConnectorsSchema`, the property in the `connectors` option, and the property in the runtime map.
|
|
180
|
+
|
|
181
|
+
## Calling a Kusto connector
|
|
182
|
+
|
|
183
|
+
Correlation ids are not part of the response body — the connector function relays the Kusto bytes untouched — so generate the `clientRequestId` yourself and pass the same value to both `executeQuery` and `toQueryResult`:
|
|
184
|
+
|
|
185
|
+
```ts
|
|
186
|
+
const clientRequestId = `KPC.rayfin_kusto_v1;${crypto.randomUUID()}`;
|
|
187
|
+
|
|
188
|
+
const response = await client.connectors.telemetry.executeQuery({
|
|
189
|
+
query: 'StormEvents | summarize count() by State | top 10 by count_',
|
|
190
|
+
clientRequestId,
|
|
191
|
+
});
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
Optionally normalize the raw Kusto v1 `{ Tables }` document into a discriminated result that preserves every returned Kusto table:
|
|
195
|
+
|
|
196
|
+
```ts
|
|
197
|
+
import { toQueryResult } from '@microsoft/rayfin-connector-kusto';
|
|
198
|
+
|
|
199
|
+
const result = toQueryResult(response, { clientRequestId });
|
|
200
|
+
if (result.status === 'success') {
|
|
201
|
+
renderTables(result.tables);
|
|
202
|
+
} else {
|
|
203
|
+
showError(result.error.code, result.error.message);
|
|
204
|
+
}
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
Successful results contain `tables` plus the `clientRequestId` you passed in (empty when you pass none) and an optional `activityId`; correlation is never on the wire.
|
|
208
|
+
Each table contains named typed `columns` and row-major `rows`.
|
|
209
|
+
Error results contain `error.message` and an optional `error.code`.
|
|
210
|
+
|
|
211
|
+
### Management commands
|
|
212
|
+
|
|
213
|
+
`executeCommand` runs a Kusto management (control) command — the command text starts with a leading dot.
|
|
214
|
+
It returns the same native Kusto v1 `{ Tables }` document as `executeQuery`, so normalize it with `toQueryResult` the same way, and pass a matching `clientRequestId` to correlate end to end:
|
|
215
|
+
|
|
216
|
+
```ts
|
|
217
|
+
const clientRequestId = `KPC.rayfin_kusto_v1;${crypto.randomUUID()}`;
|
|
218
|
+
|
|
219
|
+
const databases = await client.connectors.telemetry.executeCommand({
|
|
220
|
+
command: '.show databases',
|
|
221
|
+
clientRequestId,
|
|
222
|
+
});
|
|
223
|
+
|
|
224
|
+
const result = toQueryResult(databases, { clientRequestId });
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
## Calling a semantic model connector
|
|
228
|
+
|
|
229
|
+
`executeQuery` on `fabric-semanticmodel` runs DAX and accepts an optional `resultSetRowCountLimit`.
|
|
230
|
+
There is no default.
|
|
231
|
+
Omit it and every row comes back, so ask the user for a bound rather than inventing one.
|
|
232
|
+
|
|
233
|
+
Prefer it over wrapping the DAX in `TOPN` when the user wants a guard rather than a deliberately ranked subset.
|
|
234
|
+
Exceeding it fails the query with an `'overflow'` error, so a truncated result announces itself, where a `TOPN` returns a complete-looking partial answer.
|
|
235
|
+
|
|
236
|
+
Run `rayfin docs search "resultSetRowCountLimit"` for version-locked details, since this behavior is owned by the connector package rather than the CLI.
|
|
237
|
+
|
|
238
|
+
## Exercising a Category B connector from the CLI
|
|
239
|
+
|
|
240
|
+
`rayfin connector invoke <name> <operation>` is the loop for Category B.
|
|
241
|
+
See [`connector invoke`](./invoke.md) for payload input, transports, token handling, and the output contract.
|
|
242
|
+
|
|
243
|
+
- `connector inspect` supports `fabric-semanticmodel` but **not** `kusto` — a `kusto` connector errors with `Unsupported connector type: kusto`.
|
|
244
|
+
There is no ad-hoc query path for Kusto connectors today.
|
|
245
|
+
- `connector invoke` on `fabric-semanticmodel` calls Fabric/Power BI directly under the developer's identity, so it works with or without `rayfin up`.
|
|
246
|
+
Every other type, `kusto` included, POSTs to the deployed item and requires a prior `rayfin up`.
|
|
247
|
+
- `connector invoke` on `fabric-semanticmodel` returns an already-normalized result, because that connector normalizes inside its `invoke` middleware.
|
|
248
|
+
Do not apply `toQueryResult` to it again.
|
|
249
|
+
- A resolved `connector invoke` call is not automatically a success.
|
|
250
|
+
A connector that normalizes reports failure as `status: 'error'`; one that returns the raw service envelope reports it as `status: 'Failed'`.
|
|
251
|
+
The CLI converts either into a non-zero exit.
|
|
252
|
+
|
|
253
|
+
## Verify
|
|
254
|
+
|
|
255
|
+
`rayfin dev` parses the `connectors:` block but does not wire Category B calls.
|
|
256
|
+
A real `executeQuery` requires `rayfin up`.
|
|
257
|
+
|
|
258
|
+
## Troubleshooting
|
|
259
|
+
|
|
260
|
+
| Symptom | Likely cause | Fix |
|
|
261
|
+
| --- | --- | --- |
|
|
262
|
+
| `connector inspect` errors with `Unsupported connector type: kusto` | `connector inspect` has no Kusto path | Use `rayfin connector invoke <name> executeQuery` instead. |
|
|
263
|
+
| `Cannot find module '@microsoft/rayfin-connector-kusto'` | `connector add` scaffolds but does not install | Run the pinned `npm install` command `connector add` printed; never install unversioned. |
|
|
264
|
+
| The generated `schema.ts` has `queryServiceUri` / `databaseName` you did not expect | Expected — Kusto cluster routing is resolved at add time and baked in | Do not edit the file. Re-add the connector to re-resolve. |
|
|
265
|
+
| `rayfin up` rejects `auth.type: application` | Category B connectors are delegated-only | Set `auth.type: delegated`. |
|
|
266
|
+
| A Kusto query fails to reach the cluster, or the request carries no `queryServiceUri` | The runtime map was omitted, so nothing injected the generated routing | Pass `{ <name>: kusto() }` as the client's second constructor argument. |
|
|
267
|
+
| A semantic-model `executeQuery` result cannot be read or decoded | `fabricSemanticModel()` was not registered, so the Arrow response is never decoded | Pass `{ <name>: fabricSemanticModel() }` as the client's second constructor argument. |
|
|
268
|
+
| `client.connectors.<name>` is not typed | Connector key differs between `rayfin.yml`, `AppConnectorsSchema`, and the `connectors` option | Use the `rayfin.yml` `name` in all three. |
|
|
269
|
+
| A Category B call works locally under `rayfin dev` | It does not — `rayfin dev` only parses the block | Deploy with `rayfin up` and retest. |
|
|
@@ -6,14 +6,26 @@ sidebar_position: 6
|
|
|
6
6
|
|
|
7
7
|
Connectors let a Rayfin app read from — and, for some types, write to — Microsoft Fabric data sources: warehouses, SQL databases, Lakehouse SQL analytics endpoints, semantic models, and KQL databases.
|
|
8
8
|
|
|
9
|
-
## Prerequisite — the `connector` command group
|
|
9
|
+
## Prerequisite — enabling the `connector` command group
|
|
10
10
|
|
|
11
|
-
`rayfin connector ...` is only registered when the
|
|
11
|
+
Connectors are behind a feature flag, so `rayfin connector ...` is only registered when the project opts in. Prefer the declarative setting in `rayfin/rayfin.yml`:
|
|
12
|
+
|
|
13
|
+
```yaml
|
|
14
|
+
services:
|
|
15
|
+
connectors:
|
|
16
|
+
enabled: true
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
With that setting present the command group is available from the very first `connector add`, with nothing to configure in the shell — the path generated apps and agents should use.
|
|
20
|
+
|
|
21
|
+
The flag also activates automatically once `rayfin.yml` has a `connectors:` block (written by `connector add`), and it can still be turned on for a single command with the environment variable:
|
|
12
22
|
|
|
13
23
|
```bash
|
|
14
24
|
RAYFIN_FEATURE_FLAGS=connectors npx rayfin connector search --help
|
|
15
25
|
```
|
|
16
26
|
|
|
27
|
+
Without any of the three the commands do not exist and the CLI reports an unknown command.
|
|
28
|
+
|
|
17
29
|
Each command has its own reference page:
|
|
18
30
|
|
|
19
31
|
- [`connector search`](./search.md) — discover Fabric sources the signed-in identity can add.
|
|
@@ -21,6 +33,11 @@ Each command has its own reference page:
|
|
|
21
33
|
- [`connector inspect`](./inspect.md) — run a single read-only sample query against a source.
|
|
22
34
|
- [`connector invoke`](./invoke.md) — run one named operation against a configured connector.
|
|
23
35
|
|
|
36
|
+
Each category has its own contract page — read the one that matches your connector type, not both:
|
|
37
|
+
|
|
38
|
+
- [Category A — GraphQL entity connectors](./category-a-entities.md) — entity generation, `@role` policies, the aggregate schema, and the per-dialect read/write matrix.
|
|
39
|
+
- [Category B — function-bridge connectors](./category-b-function-bridge.md) — the `kusto` and `fabric-semanticmodel` contract, including the Kusto cluster routing baked into the generated `schema.ts`.
|
|
40
|
+
|
|
24
41
|
A typical loop is search → add → inspect (Category A) or search → add → invoke (Category B).
|
|
25
42
|
|
|
26
43
|
## Two categories of connector
|
|
@@ -55,7 +55,7 @@ If you omit it, the server generates a UUID automatically.
|
|
|
55
55
|
|
|
56
56
|
**Scope:** the rules above apply to entities you own under `rayfin/data/`.
|
|
57
57
|
Connector entities under `rayfin/connectors/` model an existing, external Fabric SQL source and follow different rules — the primary key is whatever the source actually declares, of any name and any datatype, including composite keys or no key at all.
|
|
58
|
-
See
|
|
58
|
+
See [Entity generation contract](../cli/connectors/category-a-entities.md#entity-generation-contract) for the connector-path key rules.
|
|
59
59
|
|
|
60
60
|
```typescript
|
|
61
61
|
@entity()
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@microsoft/rayfin-guide",
|
|
3
|
-
"version": "1.35.0-alpha.
|
|
3
|
+
"version": "1.35.0-alpha.1541",
|
|
4
4
|
"description": "Cross-cutting Builder guides for the Rayfin platform — discovered by `@microsoft/rayfin-docs` via the `rayfinDocs` package.json field convention.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"files": [
|