@microsoft/rayfin-guide 1.35.0-alpha.1374 → 1.35.0-alpha.1541

package/assets/docs/cli/connectors/category-b-function-bridge.md

@@ -0,0 +1,269 @@
+---
+sidebar_position: 7
+---
+
+# Category B — function-bridge connectors
+
+Reference for `kusto` (Fabric KQL Database) and `fabric-semanticmodel` (Power BI semantic model).
+A Category B connector is a named-operation surface backed by a small, platform-owned function (UDF).
+The Builder never writes or sees the function code.
+
+The Builder declares the connector and calls a typed method; the Fabric app backend injects connector configuration and delegated authentication before forwarding to the function.
+There are no GraphQL entities, so do **not** generate entity files, `@role` policies, or `metadata.json` entities for these connector types.
+
+For the entity-generating types (`fabric-sqlanalytics`, `fabric-warehouse`, `fabric-sqldatabase`), read [Category A](./category-a-entities.md) instead.
+
+## What each type exposes
+
+| Type | Operations | Query language | Auth |
+| --- | --- | --- | --- |
+| `kusto` | `executeQuery`, `executeCommand` | KQL | `delegated` only |
+| `fabric-semanticmodel` | `executeQuery` | DAX | `delegated` only |
+
+Both are pinned to an adapter version (`version: '1'` today).
+Delegated authentication runs every call as the signed-in user through the on-behalf-of flow.
+
+## Add the connector
+
+```bash
+rayfin connector add \
+  --type kusto \
+  --workspace-id <ws-id> \
+  --item-id <kql-database-item-id> \
+  --name telemetry
+```
+
+`--item-id` identifies the Fabric item to query: a KQL Database for `kusto`, a semantic model for `fabric-semanticmodel`.
+Omit `--name` to derive the connector name from the Fabric item's display name.
+
+The CLI verifies the item, writes the `rayfin.yml` entry, and scaffolds `rayfin/connectors/<name>/schema.ts`.
+
+## Resulting rayfin.yml
+
+```yaml
+connectors:
+  - name: telemetry
+    type: kusto
+    version: '1'
+    config:
+      workspaceId: <ws-id>
+      itemId: <kql-database-item-id>
+    auth:
+      type: delegated
+    operations:
+      - name: executeQuery
+      - name: executeCommand
+```
+
+The `config` block is the single source of truth for what to query and is not sent on the client wire.
+`kusto` allows `executeQuery` and `executeCommand`; `fabric-semanticmodel` allows `executeQuery`.
+In every case `auth.type` must be `delegated`.
+
+## The generated schema.ts
+
+`connector add` writes `rayfin/connectors/<name>/schema.ts` with a `// @generated — do not edit.` banner.
+Do not hand-edit it.
+Regenerate it by removing and re-adding the connector (`rayfin connector remove <name>`, then `rayfin connector add ...`).
+
+For `fabric-semanticmodel` the file exports the typed marker plus a generic runtime config:
+
+```ts
+import type { ConnectorConfig } from '@microsoft/rayfin-connectors';
+import type { FabricSemanticModel } from '@microsoft/rayfin-connector-fabric-semanticmodel';
+
+export type SalesModelSchema = FabricSemanticModel<'executeQuery'>;
+
+export const connectorConfig = {
+  connector: 'fabric-semanticmodel',
+} as const satisfies ConnectorConfig;
+```
+
+### Kusto bakes cluster routing into the generated file
+
+`connector add --type kusto` resolves the KQL Database's cluster query endpoint and database name from `(workspaceId, itemId)` at add time and writes both into the generated `connectorConfig`:
+
+```ts
+import type { Kusto, KustoConnectorConfig } from '@microsoft/rayfin-connector-kusto';
+
+export type TelemetrySchema = Kusto<'executeQuery' | 'executeCommand'>;
+
+export const connectorConfig = {
+  connector: 'kusto',
+  queryServiceUri: 'https://<cluster>.kusto.fabric.microsoft.com',
+  databaseName: '<database>',
+} as const satisfies KustoConnectorConfig;
+```
+
+Three names appear here and nowhere else:
+
+- `queryServiceUri` — the resolved Kusto cluster query endpoint.
+- `databaseName` — the resolved KQL database name.
+- `KustoConnectorConfig` — the Kusto-specific config type these two keys satisfy, exported from `@microsoft/rayfin-connector-kusto` rather than `@microsoft/rayfin-connectors`.
+
+These keys live only in the file the Kusto scaffold writes.
+They are **not** part of the shared `rayfin.yml` schema — never write a cluster URI or database name into `rayfin.yml`, and never send either value from app code.
+If the resolved values look wrong, re-add the connector rather than editing the generated file; the values come from Fabric, not from anything you can fix by hand.
+
+The Kusto scaffold imports both its marker and `KustoConnectorConfig` from `@microsoft/rayfin-connector-kusto`, so it does not import `@microsoft/rayfin-connectors` at all.
+
+## Install the packages the generated file imports
+
+`connector add` scaffolds files but installs nothing. It prints the exact pinned command — copy it from that output, or rebuild it from the `packages` array in `rayfin connector types --json`, which carries both the package names and the version:
+
+```bash
+# Shape only. Use the version connector add printed, not this one.
+
+# kusto — marker and config type both come from this one package
+npm install @microsoft/rayfin-connector-kusto@1.35.0-alpha
+
+# fabric-semanticmodel
+npm install @microsoft/rayfin-connector-fabric-semanticmodel@1.35.0-alpha @microsoft/rayfin-connectors@1.35.0-alpha
+```
+
+**Always pin the version.** Connector packages ship in lockstep with the CLI, but their npm `latest` and `preview` tags lag behind, so an unversioned install pulls an older release and its own mismatched `@microsoft/rayfin-data`.
+
+## Wire into the app client
+
+Use the same `ConnectorsRayfinClient` wiring as Category A, but with no entity re-exports and no `GraphQLBackedConnector`.
+Import the generated `<Name>Schema` type and `connectorConfig` value, key both maps by the exact `rayfin.yml` connector name, and pass the config through the client's `connectors` option.
+
+Category B additionally needs a **connector runtime map** as the client's second constructor argument.
+This is not optional: the runtime is what injects the generated routing and decodes the response.
+
+- `kusto()` merges the generated `queryServiceUri` and `databaseName` into the outbound payload. Without it, `executeQuery` and `executeCommand` cannot route to the cluster.
+- `fabricSemanticModel()` decodes the Arrow response. Without it, `executeQuery` results cannot be read.
+
+Key the runtime map by the same connector name, and call the factory once per connector:
+
+```ts
+import { ConnectorsRayfinClient } from '@microsoft/rayfin-client/experimental';
+import { kusto } from '@microsoft/rayfin-connector-kusto';
+import { fabricSemanticModel } from '@microsoft/rayfin-connector-fabric-semanticmodel';
+import {
+  type TelemetrySchema,
+  connectorConfig as telemetryConfig,
+} from '../../rayfin/connectors/telemetry/schema';
+import {
+  type SalesModelSchema,
+  connectorConfig as salesModelConfig,
+} from '../../rayfin/connectors/salesModel/schema';
+
+type AppConnectorsSchema = {
+  telemetry: TelemetrySchema;
+  salesModel: SalesModelSchema;
+};
+
+const client = new ConnectorsRayfinClient<
+  Record<string, never>,
+  Record<string, never>,
+  AppConnectorsSchema
+>(
+  {
+    baseUrl: import.meta.env.VITE_RAYFIN_API_URL,
+    publishableKey: import.meta.env.VITE_RAYFIN_PUBLISHABLE_KEY,
+    connectors: {
+      telemetry: telemetryConfig,
+      salesModel: salesModelConfig,
+    },
+  },
+  // Second argument: per-connector runtime hooks, keyed by connector name.
+  {
+    telemetry: kusto(),
+    salesModel: fabricSemanticModel(),
+  }
+);
+```
+
+`ConnectorsRayfinClient` is experimental — import it only from the `@microsoft/rayfin-client/experimental` subpath, never the stable `@microsoft/rayfin-client` entry.
+
+The connector key must be identical in four places: the `name` in `rayfin.yml`, the property in `AppConnectorsSchema`, the property in the `connectors` option, and the property in the runtime map.
+
+## Calling a Kusto connector
+
+Correlation ids are not part of the response body — the connector function relays the Kusto bytes untouched — so generate the `clientRequestId` yourself and pass the same value to both `executeQuery` and `toQueryResult`:
+
+```ts
+const clientRequestId = `KPC.rayfin_kusto_v1;${crypto.randomUUID()}`;
+
+const response = await client.connectors.telemetry.executeQuery({
+  query: 'StormEvents | summarize count() by State | top 10 by count_',
+  clientRequestId,
+});
+```
+
+Optionally normalize the raw Kusto v1 `{ Tables }` document into a discriminated result that preserves every returned Kusto table:
+
+```ts
+import { toQueryResult } from '@microsoft/rayfin-connector-kusto';
+
+const result = toQueryResult(response, { clientRequestId });
+if (result.status === 'success') {
+  renderTables(result.tables);
+} else {
+  showError(result.error.code, result.error.message);
+}
+```
+
+Successful results contain `tables` plus the `clientRequestId` you passed in (empty when you pass none) and an optional `activityId`; correlation is never on the wire.
+Each table contains named typed `columns` and row-major `rows`.
+Error results contain `error.message` and an optional `error.code`.
+
+### Management commands
+
+`executeCommand` runs a Kusto management (control) command — the command text starts with a leading dot.
+It returns the same native Kusto v1 `{ Tables }` document as `executeQuery`, so normalize it with `toQueryResult` the same way, and pass a matching `clientRequestId` to correlate end to end:
+
+```ts
+const clientRequestId = `KPC.rayfin_kusto_v1;${crypto.randomUUID()}`;
+
+const databases = await client.connectors.telemetry.executeCommand({
+  command: '.show databases',
+  clientRequestId,
+});
+
+const result = toQueryResult(databases, { clientRequestId });
+```
+
+## Calling a semantic model connector
+
+`executeQuery` on `fabric-semanticmodel` runs DAX and accepts an optional `resultSetRowCountLimit`.
+There is no default.
+Omit it and every row comes back, so ask the user for a bound rather than inventing one.
+
+Prefer it over wrapping the DAX in `TOPN` when the user wants a guard rather than a deliberately ranked subset.
+Exceeding it fails the query with an `'overflow'` error, so a truncated result announces itself, where a `TOPN` returns a complete-looking partial answer.
+
+Run `rayfin docs search "resultSetRowCountLimit"` for version-locked details, since this behavior is owned by the connector package rather than the CLI.
+
+## Exercising a Category B connector from the CLI
+
+`rayfin connector invoke <name> <operation>` is the loop for Category B.
+See [`connector invoke`](./invoke.md) for payload input, transports, token handling, and the output contract.
+
+- `connector inspect` supports `fabric-semanticmodel` but **not** `kusto` — a `kusto` connector errors with `Unsupported connector type: kusto`.
+  There is no ad-hoc query path for Kusto connectors today.
+- `connector invoke` on `fabric-semanticmodel` calls Fabric/Power BI directly under the developer's identity, so it works with or without `rayfin up`.
+  Every other type, `kusto` included, POSTs to the deployed item and requires a prior `rayfin up`.
+- `connector invoke` on `fabric-semanticmodel` returns an already-normalized result, because that connector normalizes inside its `invoke` middleware.
+  Do not apply `toQueryResult` to it again.
+- A resolved `connector invoke` call is not automatically a success.
+  A connector that normalizes reports failure as `status: 'error'`; one that returns the raw service envelope reports it as `status: 'Failed'`.
+  The CLI converts either into a non-zero exit.
+
+## Verify
+
+`rayfin dev` parses the `connectors:` block but does not wire Category B calls.
+A real `executeQuery` requires `rayfin up`.
+
+## Troubleshooting
+
+| Symptom | Likely cause | Fix |
+| --- | --- | --- |
+| `connector inspect` errors with `Unsupported connector type: kusto` | `connector inspect` has no Kusto path | Use `rayfin connector invoke <name> executeQuery` instead. |
+| `Cannot find module '@microsoft/rayfin-connector-kusto'` | `connector add` scaffolds but does not install | Run the pinned `npm install` command `connector add` printed; never install unversioned. |
+| The generated `schema.ts` has `queryServiceUri` / `databaseName` you did not expect | Expected — Kusto cluster routing is resolved at add time and baked in | Do not edit the file. Re-add the connector to re-resolve. |
+| `rayfin up` rejects `auth.type: application` | Category B connectors are delegated-only | Set `auth.type: delegated`. |
+| A Kusto query fails to reach the cluster, or the request carries no `queryServiceUri` | The runtime map was omitted, so nothing injected the generated routing | Pass `{ <name>: kusto() }` as the client's second constructor argument. |
+| A semantic-model `executeQuery` result cannot be read or decoded | `fabricSemanticModel()` was not registered, so the Arrow response is never decoded | Pass `{ <name>: fabricSemanticModel() }` as the client's second constructor argument. |
+| `client.connectors.<name>` is not typed | Connector key differs between `rayfin.yml`, `AppConnectorsSchema`, and the `connectors` option | Use the `rayfin.yml` `name` in all three. |
+| A Category B call works locally under `rayfin dev` | It does not — `rayfin dev` only parses the block | Deploy with `rayfin up` and retest. |

package/assets/docs/cli/connectors/index.md

@@ -6,14 +6,26 @@ sidebar_position: 6
 
 Connectors let a Rayfin app read from — and, for some types, write to — Microsoft Fabric data sources: warehouses, SQL databases, Lakehouse SQL analytics endpoints, semantic models, and KQL databases.
 
-## Prerequisite — the `connector` command group is feature-flagged
+## Prerequisite — enabling the `connector` command group
 
-`rayfin connector ...` is only registered when the `RAYFIN_FEATURE_FLAGS` environment variable contains `connectors`. Without it the commands do not exist and the CLI reports an unknown command.
+Connectors are behind a feature flag, so `rayfin connector ...` is only registered when the project opts in. Prefer the declarative setting in `rayfin/rayfin.yml`:
+
+```yaml
+services:
+  connectors:
+    enabled: true
+```
+
+With that setting present the command group is available from the very first `connector add`, with nothing to configure in the shell — the path generated apps and agents should use.
+
+The flag also activates automatically once `rayfin.yml` has a `connectors:` block (written by `connector add`), and it can still be turned on for a single command with the environment variable:
 
 ```bash
 RAYFIN_FEATURE_FLAGS=connectors npx rayfin connector search --help
 ```
 
+Without any of the three the commands do not exist and the CLI reports an unknown command.
+
 Each command has its own reference page:
 
 - [`connector search`](./search.md) — discover Fabric sources the signed-in identity can add.
@@ -21,6 +33,11 @@ Each command has its own reference page:
 - [`connector inspect`](./inspect.md) — run a single read-only sample query against a source.
 - [`connector invoke`](./invoke.md) — run one named operation against a configured connector.
 
+Each category has its own contract page — read the one that matches your connector type, not both:
+
+- [Category A — GraphQL entity connectors](./category-a-entities.md) — entity generation, `@role` policies, the aggregate schema, and the per-dialect read/write matrix.
+- [Category B — function-bridge connectors](./category-b-function-bridge.md) — the `kusto` and `fabric-semanticmodel` contract, including the Kusto cluster routing baked into the generated `schema.ts`.
+
 A typical loop is search → add → inspect (Category A) or search → add → invoke (Category B).
 
 ## Two categories of connector

package/assets/docs/cli/env-interpolation.md

@@ -145,10 +145,6 @@ services:
     enabled: ${AUTH_ENABLED:-false}
     issuer: ${AUTH_ISSUER}
     audience: ${AUTH_AUDIENCE:-https://api.example.com}
-
-  storage:
-    enabled: ${STORAGE_ENABLED:-true}
-    accountName: ${STORAGE_ACCOUNT:-devstoreaccount1}
 ```
 
 ```bash
@@ -159,7 +155,6 @@ DB_PORT=5432
 DB_NAME=todos_prod
 AUTH_ENABLED=true
 AUTH_ISSUER=https://auth.example.com
-STORAGE_ACCOUNT=prodstorageaccount
 ```
 
 ## Security Best Practices

package/assets/docs/cli/environment-variables.md

@@ -171,7 +171,6 @@ Read by the Rayfin WebService container via the ASP.NET Core configuration syste
 | --- | --- | --- |
 | `Auth__Enabled` | `services.auth.enabled` | `true` / `false` |
 | `Data__Enabled` | `services.data.enabled` | `true` / `false` |
-| `Storage__Enabled` | `services.storage.enabled` | `true` / `false` |
 
 The following signing-key variables are set to dev-mode defaults by `rayfin up` and are not typically edited:
 
@@ -190,7 +189,7 @@ These variables are read from the shell environment and are never written to fil
 | `RAYFIN_WORKSPACE_ID` | Fabric workspace ID for non-interactive setup. Used with `RAYFIN_TOKEN`. |
 | `RAYFIN_TENANT_ID` | Entra ID tenant used by `rayfin up` for portal URLs and the `ctid` query parameter. Equivalent to the `-t, --tenant <id>` flag (precedence: flag > env var > signed-in tenant). |
 | `RAYFIN_ENCRYPTION_FALLBACK_ENABLED` | Set to `true` to allow plaintext token cache on systems without OS credential storage. Development only. |
-| `RAYFIN_FEATURE_FLAGS` | Comma-separated list of experimental feature names to enable (case-insensitive). Recognized values include `docker-local-dev`, `storage`, `functions`, and `postgresql`. |
+| `RAYFIN_FEATURE_FLAGS` | Comma-separated list of experimental feature names to enable (case-insensitive). Recognized values include `docker-local-dev`, `functions`, and `postgresql`. |
 | `RAYFIN_WEBSERVICE_IMAGE_NAME` | **Experimental.** Override the webservice container image used by `rayfin dev --provider docker` and Docker Compose. Defaults to `ghcr.io/microsoft/project-rayfin/webservice:cli-<version>`. |
 | `RAYFIN_APPINSIGHTS_CONNECTION_STRING` | Override the telemetry endpoint for the CLI and VS Code extension. |
 
@@ -199,7 +198,6 @@ These variables are read from the shell environment and are never written to fil
 | Flag | Effect |
 | --- | --- |
 | `docker-local-dev` | Allows `rayfin dev --provider docker` and the Docker maintenance commands. Bare `rayfin dev` remains available without this flag and defaults to Fabric. |
-| `storage` | Exposes storage commands (`rayfin dev storage *`) and storage prompts during `rayfin init`. |
 | `functions` | Exposes Functions service prompts during `rayfin init`. |
 | `postgresql` | Adds PostgreSQL as a selectable dialect during `rayfin init` and `rayfin init` with bundled templates. |
 

package/assets/docs/data/overview.md

@@ -53,6 +53,10 @@ If you omit it, the server generates a UUID automatically.
 - You may supply your own UUID at creation time if you prefer client-generated identifiers.
 - Composite or non-`id` primary keys are not supported.
 
+**Scope:** the rules above apply to entities you own under `rayfin/data/`.
+Connector entities under `rayfin/connectors/` model an existing, external Fabric SQL source and follow different rules — the primary key is whatever the source actually declares, of any name and any datatype, including composite keys or no key at all.
+See [Entity generation contract](../cli/connectors/category-a-entities.md#entity-generation-contract) for the connector-path key rules.
+
 ```typescript
 @entity()
 export class Todo {

package/assets/docs/data/permissions.md

@@ -145,23 +145,6 @@ export class SecureDocument {
 }
 ```
 
-## Storage permissions
-
-The same `@role` decorator works with storage entities.
-When applied to a `@blob()` class, Rayfin generates a storage policy instead of a database policy:
-
-```typescript
-import { blob, role } from '@microsoft/rayfin-core';
-
-@blob()
-@role('authenticated', '*', {
-  policy: (claims, item) => claims.sub.eq(item.owner_id),
-})
-export class ProfileImage {
-  owner_id!: string;
-}
-```
-
 ## How it works
 
 - The `@role` decorator collects permission metadata at class definition time.

package/assets/docs/getting-started/project-structure.md

@@ -78,8 +78,6 @@ services:
   data:
     enabled: true
     dialect: mssql
-  storage:
-    enabled: false
   staticHosting:
     enabled: true
     root: .
@@ -160,12 +158,6 @@ Configure an email provider for magic links, password resets, and email verifica
 | `useStartTls` | `boolean` | `false` | Use STARTTLS for the SMTP connection. |
 | `webPort` | `number` | `1080` | MailDev web UI port (local development only). |
 
-#### `services.storage`
-
-| Field | Type | Default | Description |
-| --- | --- | --- | --- |
-| `enabled` | `boolean` | `false` | Enable the storage service. |
-
 #### `services.staticHosting`
 
 | Field | Type | Default | Description |

package/assets/docs/hosting/index.md

@@ -133,7 +133,7 @@ The deploy tool updates the configuration and pushes it to the backend during de
 
 - The compressed ZIP archive must not exceed **100 MB**.
 - The CLI uses maximum compression to minimize upload size.
-- If your build output exceeds the limit, consider excluding large assets or using the storage service for binary files.
+- If your build output exceeds the limit, consider excluding large assets from the deployed bundle.
 
 ## Complete example
 
@@ -182,7 +182,6 @@ If the ZIP exceeds 100 MB:
 
 - Review your build output for unnecessary files (source maps, unoptimized images).
 - Configure your bundler to exclude development artifacts from the production build.
-- Move large binary assets to Rayfin storage instead of bundling them as static content.
 
 ### No remote endpoint configured
 

package/assets/docs/preview/local-dev-docker.md

@@ -24,9 +24,9 @@ This command:
 - Validates that Docker and Docker Compose are available.
 - Generates `rayfin/.temp/docker-compose.yml` from your project configuration.
 - Allocates ports for each service.
-- Starts containers for enabled services (WebService, database, and optional storage).
+- Starts containers for enabled services (WebService and database).
 - Runs health checks and waits for all services to be healthy.
-- Applies the project's declared data and storage configuration to the local backend.
+- Applies the project's declared data configuration to the local backend.
 - Starts the frontend with `npm run dev:frontend` when that script exists, falling back to `npm run dev` for existing projects.
 - Resolves that script from `services.staticHosting.path` when the frontend lives in a nested package, otherwise from the project root.
 - Builds and starts the configured local Functions host when `services.functions.enabled` is `true`.
@@ -100,14 +100,6 @@ npx rayfin dev db apply --force
 Run this after making changes to entities in `rayfin/data/`.
 Use `--force` to regenerate configuration even if no changes are detected.
 
-### `rayfin dev storage apply`
-
-Generate and apply storage configuration to the local development server.
-
-```bash
-npx rayfin dev storage apply
-```
-
 ### `rayfin dev status`
 
 Display the status of the local development environment.
@@ -120,7 +112,7 @@ Shows container health, port assignments, and service readiness.
 
 ### `rayfin dev watch`
 
-Watch `./rayfin/data` or `./rayfin/storage` and auto-apply configuration changes.
+Watch `./rayfin/data` and auto-apply configuration changes.
 
 ```bash
 npx rayfin dev watch
@@ -153,11 +145,5 @@ Set the feature flag in your environment:
 export RAYFIN_FEATURE_FLAGS=docker-local-dev
 ```
 
-Or combine with other flags:
-
-```bash
-export RAYFIN_FEATURE_FLAGS=docker-local-dev,storage
-```
-
 This feature flag gates only Docker provider selection and Docker maintenance commands.
 Bare `rayfin dev` is available without preview flags and uses Fabric.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@microsoft/rayfin-guide",
-  "version": "1.35.0-alpha.1374",
+  "version": "1.35.0-alpha.1541",
   "description": "Cross-cutting Builder guides for the Rayfin platform — discovered by `@microsoft/rayfin-docs` via the `rayfinDocs` package.json field convention.",
   "type": "module",
   "files": [