@tailor-platform/sdk 1.44.1 → 1.45.0

package/docs/services/tailordb-migration.md

@@ -0,0 +1,418 @@
+# TailorDB Migrations
+
+> **Beta:** The `tailordb migration` command and the migration runtime are beta features. They may introduce breaking changes in future releases. The CLI emits a beta warning on every invocation.
+
+The migration system tracks changes to your TailorDB type definitions over time and applies them to deployed workspaces with optional data transformation scripts.
+
+For the CLI command reference, see [`tailordb migration`](../cli/tailordb.md#tailordb-migration). This document covers concepts, workflows, and operational guidance.
+
+## Overview
+
+**Key Properties**
+
+- **Local snapshot–based diff detection** — each migration is generated by diffing your current type definitions against the previous snapshot stored in `migrations/<NNNN>/`.
+- **Transaction-wrapped data migrations** — each `migrate.ts` script runs inside a database transaction on the platform; if the script throws, all changes in that migration roll back.
+- **Automatic execution during `apply`** — `tailor-sdk apply` detects pending migrations, runs the two-stage type update (pre-migration → script → post-migration), and updates the migration checkpoint label.
+- **Type-safe scripts** — the generated `db.ts` provides Kysely types that reflect the schema state **before** the migration runs, so transformations are written against the actual data shape.
+
+**Files in `migrations/`**
+
+```
+migrations/
+├── 0000/                    # Initial schema snapshot
+│   └── schema.json
+├── 0001/                    # First change
+│   ├── diff.json            # Field-level diff from 0000
+│   ├── migrate.ts           # Data migration script (only if breaking)
+│   └── db.ts                # Kysely types for the script (pre-migration shape)
+├── 0002/
+│   └── diff.json            # No script — non-breaking changes only
+└── ...
+```
+
+`0000` always contains a full snapshot. `0001` and onward contain a diff plus, for breaking changes, a script and its types. **Commit the entire `migrations/` directory to version control.**
+
+## Initial Setup
+
+### New project
+
+When you start with no `migrations/` directory:
+
+1. Add the `migration` block to `tailor.config.ts` (see [Configuration](#configuration)).
+2. Define your initial types in `tailordb/`.
+3. Generate the initial migration:
+   ```bash
+   tailor-sdk tailordb migration generate
+   ```
+   This creates `migrations/0000/schema.json` from your current types.
+4. Run `tailor-sdk apply`. The migration label is set to `0000` on the deployed namespace.
+
+### Adding migrations to an existing project
+
+If you already have a deployed workspace whose schema matches your local type definitions:
+
+1. Add the `migration` block to `tailor.config.ts`.
+2. Run `tailor-sdk tailordb migration generate` to create `0000/schema.json` from current local types.
+3. Run `tailor-sdk apply`. Because remote schema already matches, no script runs; only the migration label is set.
+
+If your local types and remote schema have **diverged**, reconcile them before introducing migrations — either update local types to match remote, or accept that the first non-`0000` migration will reflect that gap.
+
+### Resetting
+
+`tailor-sdk tailordb migration generate --init` deletes the existing `migrations/` directory and starts over from `0000`. Use this only on projects that are not yet deployed, or when you have decided to re-baseline (the next `apply` will see all migrations as new and require coordination — see [Resetting a deployed project](#resetting-a-deployed-project)).
+
+## Migration Workflow
+
+A typical change cycle:
+
+1. **Modify a type definition.**
+
+   ```typescript
+   // tailordb/user.ts
+   export const user = db.type("User", {
+     name: db.string(),
+     email: db.string(), // ← new required field
+     ...db.fields.timestamps(),
+   });
+   ```
+
+2. **Generate the migration.**
+
+   ```bash
+   tailor-sdk tailordb migration generate --name "add email to user"
+   ```
+
+   Output:
+
+   ```
+   Generated migration 0001
+     Diff file: ./migrations/0001/diff.json
+     Migration script: ./migrations/0001/migrate.ts
+     DB types: ./migrations/0001/db.ts
+   ```
+
+   If `EDITOR` or `VISUAL` is set, `migrate.ts` opens automatically.
+
+3. **Edit `migrate.ts`** to populate data for the new required field:
+
+   ```typescript
+   import type { Transaction } from "./db";
+
+   export async function main(trx: Transaction): Promise<void> {
+     await trx
+       .updateTable("User")
+       .set({ email: "default@example.com" })
+       .where("email", "is", null)
+       .execute();
+   }
+   ```
+
+4. **Apply.**
+   ```bash
+   tailor-sdk apply
+   ```
+   The pre-migration phase relaxes the new field to optional, the script runs and populates values, then the post-migration phase enforces `required: true`.
+
+## Configuration
+
+```typescript
+// tailor.config.ts
+export default defineConfig({
+  name: "my-app",
+  db: {
+    tailordb: {
+      files: ["./tailordb/*.ts"],
+      migration: {
+        directory: "./migrations",
+        // Optional. Defaults to the first machine user in auth.machineUsers.
+        machineUser: "admin-machine-user",
+      },
+    },
+  },
+});
+```
+
+| Option                  | Type   | Description                                                                                               |
+| ----------------------- | ------ | --------------------------------------------------------------------------------------------------------- |
+| `migration.directory`   | string | Directory path for migration files. Required when migrations are enabled for the namespace.               |
+| `migration.machineUser` | string | Machine user used to run migration scripts. Optional; defaults to the first entry in `auth.machineUsers`. |
+
+## Generated Files
+
+| File               | When generated                   | Description                                                                                               |
+| ------------------ | -------------------------------- | --------------------------------------------------------------------------------------------------------- |
+| `0000/schema.json` | First `migration generate`       | Full snapshot of all types in the namespace.                                                              |
+| `XXXX/diff.json`   | Every subsequent migration       | Field-level diff against the previous snapshot.                                                           |
+| `XXXX/migrate.ts`  | Only for breaking changes        | Data transformation script. The `main` export receives a Kysely `Transaction`.                            |
+| `XXXX/db.ts`       | Generated alongside `migrate.ts` | Kysely types reflecting the schema **before** this migration. Re-generated on every `migration generate`. |
+
+`db.ts` reflects the pre-migration schema because the script runs after the pre-migration phase has temporarily relaxed breaking constraints (e.g., a new `required` field is added as `optional` first), so the data being read still matches the previous shape.
+
+## Migration Script Anatomy
+
+```typescript
+import type { Transaction } from "./db";
+
+export async function main(trx: Transaction): Promise<void> {
+  // SELECT is supported.
+  const users = await trx.selectFrom("User").select(["id", "name"]).execute();
+
+  // Loop and transform.
+  for (const u of users) {
+    await trx
+      .updateTable("User")
+      .set({ displayName: u.name.toUpperCase() })
+      .where("id", "=", u.id)
+      .execute();
+  }
+}
+```
+
+**Rules**
+
+- Always use the `trx` argument for database access. Anything that bypasses `trx` is not part of the transaction and will not roll back on failure.
+- Do not import resolvers, executors, or other SDK runtime services. The migration script runs as a standalone bundle on the platform with only Kysely access; SDK service helpers are not available.
+- Standard Node-compatible packages are bundled. Keep dependencies minimal — every import is shipped to the platform.
+- `console.log` / `console.error` output is captured and surfaced under `Logs:` in the apply output. Use it sparingly for progress markers on long-running migrations.
+- The script is idempotency-friendly by default because it runs inside a transaction, but **plan for re-execution**: if a later migration in the same `apply` fails, the platform may retry the apply, and you may want your script to tolerate already-migrated rows (e.g., `where("email", "is", null)` instead of unconditional updates).
+
+## Supported Schema Changes
+
+| Change Type                    | Breaking? | Migration Script? | Notes                                                                                 |
+| ------------------------------ | --------- | ----------------- | ------------------------------------------------------------------------------------- |
+| Add optional field             | No        | No                | Schema change only                                                                    |
+| Add required field             | Yes       | Yes               | Script populates default values                                                       |
+| Remove field                   | No        | No                | Schema change only — data is preserved server-side                                    |
+| Change optional → required     | Yes       | Yes               | Script sets defaults for null values                                                  |
+| Change required → optional     | No        | No                | Schema change only                                                                    |
+| Add index                      | No        | No                | Schema change only                                                                    |
+| Remove index                   | No        | No                | Schema change only                                                                    |
+| Add unique constraint          | Yes       | Yes               | Script must resolve duplicate values                                                  |
+| Remove unique constraint       | No        | No                | Schema change only                                                                    |
+| Add enum value                 | No        | No                | Schema change only                                                                    |
+| Remove enum value              | Yes       | Yes               | Script migrates records with removed values                                           |
+| Add type                       | No        | No                | Schema change only                                                                    |
+| Remove type                    | No        | No                | Schema change only — data is preserved server-side                                    |
+| Change foreign key target type | Yes       | Yes               | Script updates references to the new target                                           |
+| Change field type              | -         | -                 | **Not supported** — see [3-step migration](#3-step-migration-for-unsupported-changes) |
+| Change array → single value    | -         | -                 | **Not supported** — see [3-step migration](#3-step-migration-for-unsupported-changes) |
+| Change single value → array    | -         | -                 | **Not supported** — see [3-step migration](#3-step-migration-for-unsupported-changes) |
+
+### 3-step migration for unsupported changes
+
+Field type changes (e.g., `string` → `integer`) and array-cardinality changes are not detected by the diff engine. Use a 3-step strategy:
+
+1. **Migration N**: Add a new field with the desired type (e.g., `fieldName_new`). Write a script that populates it from the old field.
+2. **Migration N+1**: Remove the old field.
+3. **Migration N+2**: Add the field back with the original name and the new type. Script copies from the temporary field, then remove the temporary field in migration N+3 (or in the same step if you can express it).
+
+The same pattern works for switching between scalar and array.
+
+## Automatic Migration Execution
+
+When you run `tailor-sdk apply`, the SDK detects pending migrations (anything past the current `sdk-migration` label on the deployed namespace) and runs them in order before continuing with the rest of the apply.
+
+### Per-migration phases
+
+For each pending migration:
+
+1. **Pre-migration**: Type changes that would be breaking are applied in a relaxed form first. Newly-required fields are added as optional; fields whose `optional → required` transition is breaking are temporarily kept optional. Non-breaking changes that are part of the same migration are also applied here.
+2. **Script execution**: If `diff.requiresMigrationScript` is true, `migrate.ts` is bundled and sent to the platform via the script execution API. It runs as the configured machine user inside a transaction.
+3. **Post-migration**: Required constraints are enforced; field/type deletions are applied; the `sdk-migration` label is bumped to this migration's number.
+
+This split is what allows existing rows to be backfilled before the database starts rejecting nulls.
+
+### Schema verification
+
+Before running migrations, `apply` performs two checks:
+
+1. **Local schema check** — your current type definitions must match the latest snapshot in `migrations/`. If they don't, you forgot to run `migration generate`.
+2. **Remote schema check** — the deployed schema is reconstructed from migration history; the actual remote schema must match. Drift here means someone applied a different set of migrations or edited the schema out-of-band.
+
+On drift you'll see something like:
+
+```
+✖ Remote schema drift detected:
+Namespace: tailordb
+  Remote migration: 0007
+  Differences:
+  Type 'User':
+    - Field 'email': required: remote=false, expected=true
+```
+
+To bypass both checks (not recommended outside of recovery scenarios):
+
+```bash
+tailor-sdk apply --no-schema-check
+```
+
+### Example output
+
+```
+ℹ Found 2 pending migration(s) to execute.
+ℹ Executing 2 pending migration(s)...
+ℹ Using machine user: admin-machine-user for namespace 'tailordb'
+
+✔ Migration tailordb/0002 completed successfully
+✔ Migration tailordb/0003 completed successfully
+
+✔ All migrations completed successfully.
+✔ Successfully applied changes.
+```
+
+## `migration set` Semantics
+
+`tailor-sdk tailordb migration set <N>` updates the `sdk-migration` label on the deployed namespace's metadata. **It does not modify any data or schema.** It only changes which migrations the next `apply` will consider pending.
+
+| Movement                         | Effect on next `apply`                                                    | Effect on data                                                |
+| -------------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------- |
+| Forward (e.g., `0001` → `0003`)  | Migrations `0002` and `0003` are skipped — they will not run.             | None.                                                         |
+| Backward (e.g., `0003` → `0001`) | Migrations `0002` and `0003` become pending and will re-execute on apply. | None directly — but the re-executed scripts may rewrite data. |
+
+Use cases:
+
+- **Recovery from drift** — you investigated, manually fixed the remote, and want the SDK's bookkeeping to reflect reality.
+- **Re-running a faulty migration** in a development workspace — set backward, fix `migrate.ts`, apply.
+- **Skipping a migration** that you know was already applied out-of-band.
+
+`migration set` does not perform a true rollback. To undo a schema/data change in production, write a new forward migration that reverses it (see [Rollback Strategy](#rollback-strategy)).
+
+## Team Workflow and CI/CD
+
+### Branch coordination
+
+Migration numbers are assigned sequentially, so two developers branching off the same point and each generating `0005` will collide. Conventions that work:
+
+- **Don't generate migrations on long-lived feature branches.** Generate them just before merge, after rebasing onto main.
+- **Resolve collisions by re-generating.** If your branch has `0005` but main now has `0005` from another PR, delete your `0005/` directory, rebase, and run `migration generate` again. Re-edit the resulting `migrate.ts`.
+- **Treat migration files as merge-conflict-prone.** They are committed JSON and TypeScript, so review them in PRs. The `diff.json` is the source of truth — if review focuses there, regenerating after rebase is straightforward.
+
+### CI / CD
+
+- For non-interactive environments, pass `--yes` to `migration generate` and `--yes` to `apply`. `apply` runs migrations automatically when the `migrations/` directory is configured.
+- Run `tailor-sdk tailordb migration status` in CI to detect "developer forgot to commit a migration" situations early. The exit code is non-zero only on errors, so check the output.
+- Avoid running migrations in parallel against the same workspace — there is no locking. Serialize deploys per environment.
+
+### Resetting a deployed project
+
+`migration generate --init` is destructive locally but does not touch the deployed workspace. Re-baselining a deployed project requires:
+
+1. Run `migration generate --init` to start over from `0000`.
+2. Run `tailor-sdk tailordb migration set 0` against the deployed namespace.
+3. Run `tailor-sdk apply` — the new `0000` becomes the baseline.
+
+Coordinate this with your team because everyone else's local migrations will be invalidated.
+
+## Failure Recovery
+
+If a `migrate.ts` throws:
+
+- **The transaction rolls back** for that migration's script. Database changes the script made are undone.
+- **The pre-migration phase already ran** before the script. Type-level relaxations (e.g., a field changed to optional) **are not undone**. The post-migration phase, including the label bump, does not run.
+- The whole `apply` aborts. Subsequent migrations in the same run do not execute.
+
+After a failure:
+
+1. Read the `Logs:` block in the apply output to find the cause.
+2. Fix `migrate.ts` (or the data it depends on).
+3. Re-run `tailor-sdk apply`. The same migration runs again because its label was never bumped.
+4. If the pre-migration relaxation is causing problems for application code in the meantime, accept the temporary optionality or roll forward with a fix; do not try to manually re-tighten the schema, or you'll create remote drift.
+
+If a migration **succeeds in script** but the post-migration phase fails (rare; usually due to constraint violation that the script should have prevented), the situation is the same as above plus the data changes from the script are persisted. Investigate, fix, and re-run.
+
+## Rollback Strategy
+
+There is no automatic down-migration. To roll back a schema/data change in production, write a new forward migration that reverses the previous one. For example, to undo a `0005` that added a required `email` field:
+
+1. Edit your type definitions to remove the field.
+2. `migration generate --name "rollback 0005 email"` produces `0006` with a removal diff.
+3. Apply.
+
+In **development workspaces**, a quicker option is to fix `0005/migrate.ts` in place, run `migration set <previous>` to re-mark it pending, and apply. Do not do this on production — it confuses migration history across environments.
+
+## Machine User and Permissions
+
+Migration scripts execute server-side under a machine user identity. The CLI selects the user in this priority order:
+
+1. `db.<namespace>.migration.machineUser` if set in `tailor.config.ts`.
+2. The first entry in `auth.machineUsers` otherwise.
+
+The CLI logs the selected user before running scripts (`Using machine user: ...`).
+
+**Permissions required**
+
+The machine user needs read/write access to every type the migration script touches. If your migrations alter data across multiple types, the simplest path is to give the migration user broad access (e.g., an `ADMIN` role) and restrict day-to-day machine users separately. If the user lacks permission, the script fails with a permission error in `Logs:`.
+
+If you see `No machine user available for migration execution`, either:
+
+- Add `machineUsers: { ... }` to your auth config and `tailor-sdk apply` it, or
+- Set `migration.machineUser` to an existing machine user name in the db config.
+
+## Multi-Namespace Coordination
+
+If your project defines multiple TailorDB namespaces (`db: { ns1: { ... }, ns2: { ... } }`), each has its own `migrations/` directory and its own migration label. During `apply`:
+
+- Migrations are grouped by namespace and executed namespace by namespace.
+- Within a namespace, migrations run sequentially in number order.
+- There is no cross-namespace ordering guarantee. Do not write a migration in `ns1` that depends on data produced by a migration in `ns2` running first.
+- Each namespace can specify its own `migration.machineUser`.
+
+## Performance and Large Tables
+
+The migration script runs in a single transaction. For tables with many rows:
+
+- Prefer set-based SQL (`updateTable(...).set(...).where(...)`) over per-row loops.
+- If a per-row loop is unavoidable, batch by primary key range. Avoid `OFFSET`-based pagination — it scans previously-seen rows on every page.
+- Long-running transactions can hit platform timeouts and hold locks. For very large backfills, consider splitting the work across multiple migrations, each operating on a subset.
+- Add `LIMIT` and resumability (idempotent `where` clauses) so a re-run after a transient failure converges.
+
+## Testing Migrations Locally
+
+The platform-side execution path is hard to fully replicate locally, but you can sanity-check `migrate.ts` logic:
+
+- The bundle that ships to the platform is plain JavaScript exporting `main(trx)`. You can import it in a Vitest test, pass a Kysely-compatible mock or a local SQLite database with the same schema as `db.ts`, and assert the resulting state.
+- Run `migration generate` on a clean working copy first, review `diff.json`, then run again after editing types to ensure the diff matches what you intended.
+- For non-trivial migrations, apply against a scratch workspace before promoting to staging or production.
+
+## Environment-Specific Strategies
+
+A migration script is a function — branching on environment (e.g., to skip a backfill in dev) is just normal TypeScript. The migration runtime injects nothing environment-specific into the script itself; if you need environment awareness, query a sentinel record or rely on data shape. Avoid env-vars inside `migrate.ts` because the script is bundled and shipped to the platform; the platform-side environment is what counts.
+
+For genuinely different schemas across environments, prefer separate workspaces with the same migration history rather than divergent `migrations/` directories.
+
+## Troubleshooting
+
+### Remote schema drift detected
+
+**Cause:** Remote schema doesn't match what the migration history says it should be.
+
+**Resolution:**
+
+1. `tailor-sdk tailordb migration status` to see local vs remote.
+2. Compare with teammates — has someone applied different migrations?
+3. If remote was changed manually, decide whether to update local migrations to match or to use `migration set <N>` to align bookkeeping.
+4. As a last resort in non-production environments, `--no-schema-check` skips both checks. Do not use this as a routine workaround.
+
+### "No machine user available for migration execution"
+
+**Cause:** Neither `migration.machineUser` is set nor are there any machine users in `auth.machineUsers`.
+
+**Resolution:** Add a machine user to auth, apply auth changes, then re-run.
+
+### "Machine user not found"
+
+**Cause:** `migration.machineUser` references a name that doesn't exist in the deployed auth config.
+
+**Resolution:** Either add the machine user to `auth.machineUsers` and apply, or change `migration.machineUser` to a valid name.
+
+### Migration script execution fails
+
+**Cause:** Runtime error in your `migrate.ts`, a permission error from the machine user, or a constraint violation when post-migration tightens types.
+
+**Resolution:** Read the `Logs:` block. Fix the script or the data assumption it relies on, and re-run `tailor-sdk apply`. The label is not bumped on failure, so the same migration retries.
+
+### `migrate.ts` not found for a migration that needs one
+
+**Cause:** `diff.requiresMigrationScript` is true but `migrate.ts` is missing from the migration directory.
+
+**Resolution:** Either re-run `migration generate` (it skips already-generated diffs but will fill in a missing script), or restore the file from version control.

package/docs/services/tailordb.md

@@ -586,3 +586,9 @@ db.type("User", {
 ```
 
 **Warning**: Do not use `unsafeAllowAllTypePermission` or `unsafeAllowAllGqlPermission` in production environments as they effectively disable authorization checks.
+
+## Migrations
+
+When you change a TailorDB type definition, the SDK can generate a migration that captures the diff and, for breaking changes, runs a data transformation script during `tailor-sdk apply`. See the [TailorDB Migrations guide](./tailordb-migration.md) for the full workflow, configuration, supported change types, team coordination, and troubleshooting.
+
+For the CLI command reference, see [`tailordb migration`](../cli/tailordb.md#tailordb-migration).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tailor-platform/sdk",
-  "version": "1.44.1",
+  "version": "1.45.0",
   "description": "Tailor Platform SDK - The SDK to work with Tailor Platform",
   "license": "MIT",
   "repository": {
@@ -112,7 +112,7 @@
     "mime-types": "3.0.2",
     "multiline-ts": "4.0.1",
     "open": "11.0.0",
-    "ora": "9.3.0",
+    "ora": "9.4.0",
     "oxc-parser": "0.127.0",
     "p-limit": "7.3.0",
     "pathe": "2.0.3",
@@ -138,7 +138,7 @@
     "@types/mime-types": "3.0.1",
     "@types/node": "24.12.2",
     "@types/semver": "7.7.1",
-    "@typescript/native-preview": "7.0.0-dev.20260424.1",
+    "@typescript/native-preview": "7.0.0-dev.20260424.2",
     "@vitest/coverage-v8": "4.1.5",
     "eslint": "10.2.1",
     "eslint-plugin-jsdoc": "62.9.0",

package/dist/application-Cot5-3g3.mjs

@@ -1,4 +0,0 @@
-
-import { n as generatePluginFilesIfNeeded, r as loadApplication, t as defineApplication } from "./application-vJxYH5LE.mjs";
-
-export { defineApplication };