@tailor-platform/sdk 2.0.1 → 2.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +117 -0
- package/{dist/cli/index.mjs → bin/tailor.mjs} +1 -1
- package/dist/{application-CM6hKnSK.mjs → application-BpLeawBg.mjs} +126 -28
- package/dist/application-BpLeawBg.mjs.map +1 -0
- package/dist/application-BxLLiLsr.mjs +3 -0
- package/dist/cli/commands/deploy/tailordb/index.d.mts +2 -1
- package/dist/cli/commands/deploy/types.d.mts +2 -1
- package/dist/cli/commands/tailordb/migrate/diff-calculator.d.mts +11 -4
- package/dist/cli/commands/tailordb/migrate/file-state.d.mts +1 -0
- package/dist/cli/commands/tailordb/migrate/schema-checks.d.mts +2 -2
- package/dist/cli/commands/tailordb/migrate/snapshot-types.d.mts +8 -1
- package/dist/cli/commands/tailordb/migrate/snapshot.d.mts +5 -3
- package/dist/cli/lib.d.mts +4 -4
- package/dist/cli/lib.mjs +3 -482
- package/dist/cli/lib.mjs.map +1 -1
- package/dist/cli/main.mjs +997 -202
- package/dist/cli/main.mjs.map +1 -1
- package/dist/completion/zsh-worker.zsh +145 -2
- package/dist/configure/config/types.d.mts +12 -1
- package/dist/configure/index.d.mts +3 -3
- package/dist/configure/services/index.d.mts +3 -2
- package/dist/configure/services/resolver/index.d.mts +1 -1
- package/dist/configure/services/resolver/resolver.d.mts +1 -1
- package/dist/configure/services/tailordb/index.d.mts +2 -1
- package/dist/configure/services/tailordb/schema.d.mts +46 -46
- package/dist/configure/services/tailordb/types.d.mts +30 -4
- package/dist/{crashreport-B5LqCTfF.mjs → crashreport-CIIfLnoO.mjs} +2 -2
- package/dist/{crashreport-B5LqCTfF.mjs.map → crashreport-CIIfLnoO.mjs.map} +1 -1
- package/dist/{crashreport-eI7ty7ok.mjs → crashreport-CINZ5dVN.mjs} +1 -1
- package/dist/errors-CWj21238.mjs +75 -0
- package/dist/errors-CWj21238.mjs.map +1 -0
- package/dist/{globals-TfAVItuK.mjs → globals-B2nlxBrz.mjs} +2 -10
- package/dist/globals-B2nlxBrz.mjs.map +1 -0
- package/dist/kysely/index.d.mts +52 -5
- package/dist/kysely/index.mjs +7 -2
- package/dist/kysely/index.mjs.map +1 -1
- package/dist/{logger-CXQq9YIp.mjs → logger-DSdExfor.mjs} +83 -42
- package/dist/logger-DSdExfor.mjs.map +1 -0
- package/dist/plugin/builtin/seed/index.mjs +1 -1
- package/dist/{register-ts-hook-LYV7zH-e.mjs → register-ts-hook-CTth1eqj.mjs} +1587 -272
- package/dist/register-ts-hook-CTth1eqj.mjs.map +1 -0
- package/dist/runtime/globals.d.mts +0 -7
- package/dist/runtime/workflow.d.mts +2 -7
- package/dist/schema--xYWRGfe.mjs.map +1 -1
- package/dist/seed/index.d.mts +52 -3
- package/dist/seed/index.mjs +200 -44
- package/dist/seed/index.mjs.map +1 -1
- package/dist/{seed-B-yckgQY.mjs → seed-xqM9WIHP.mjs} +8 -8
- package/dist/seed-xqM9WIHP.mjs.map +1 -0
- package/dist/{service-B5WynBHQ.mjs → service-CCwl3Avt.mjs} +3 -3
- package/dist/{service-B5WynBHQ.mjs.map → service-CCwl3Avt.mjs.map} +1 -1
- package/dist/service-_XmjSEGr.mjs +3 -0
- package/dist/{service-k1xy_A5G.mjs → service-goqlJJgA.mjs} +2 -2
- package/dist/{service-k1xy_A5G.mjs.map → service-goqlJJgA.mjs.map} +1 -1
- package/dist/{service_pb-D5--My_M.mjs → service_pb-BEWj1pOV.mjs} +2 -2
- package/dist/{service_pb-D5--My_M.mjs.map → service_pb-BEWj1pOV.mjs.map} +1 -1
- package/dist/{service_pb-i8CmnQP4.mjs → service_pb-BjlhvK9E.mjs} +1 -1
- package/dist/utils/test/index.d.mts +9 -3
- package/dist/utils/test/index.mjs +49 -25
- package/dist/utils/test/index.mjs.map +1 -1
- package/dist/vitest/environment.mjs +1 -1
- package/dist/vitest/index.d.mts +2 -1
- package/dist/vitest/index.mjs +84 -2
- package/dist/vitest/index.mjs.map +1 -1
- package/dist/vitest/mocks/file.d.mts +1 -1
- package/dist/vitest/pglite-kysely.d.mts +43 -0
- package/dist/vitest/setup.mjs +1 -1
- package/dist/workflow-Bamae_Yc.mjs.map +1 -1
- package/dist/{workspace_resource_pb-DXnO2AT-.mjs → workspace_resource_pb-HNIL_Vfj.mjs} +10 -2
- package/dist/{workspace_resource_pb-DXnO2AT-.mjs.map → workspace_resource_pb-HNIL_Vfj.mjs.map} +1 -1
- package/docs/cli/tailordb.md +87 -22
- package/docs/cli-reference.md +13 -11
- package/docs/configuration.md +4 -1
- package/docs/migration/v2.md +1 -1
- package/docs/plugin/custom.md +48 -48
- package/docs/plugin/index.md +24 -24
- package/docs/services/auth.md +10 -10
- package/docs/services/executor.md +1 -1
- package/docs/services/resolver.md +34 -2
- package/docs/services/tailordb-migration.md +375 -76
- package/docs/services/tailordb.md +28 -28
- package/docs/testing.md +3 -1
- package/package.json +13 -8
- package/dist/application-CM6hKnSK.mjs.map +0 -1
- package/dist/application-CXNaUNhv.mjs +0 -3
- package/dist/errors-D7583Zz7.mjs +0 -58
- package/dist/errors-D7583Zz7.mjs.map +0 -1
- package/dist/globals-TfAVItuK.mjs.map +0 -1
- package/dist/logger-CXQq9YIp.mjs.map +0 -1
- package/dist/register-ts-hook-LYV7zH-e.mjs.map +0 -1
- package/dist/seed-B-yckgQY.mjs.map +0 -1
- package/dist/service-M7OYziQP.mjs +0 -3
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
> **Beta:** The `tailordb migration` command and the migration runtime are beta features. They may introduce breaking changes in future releases. The CLI emits a beta warning on every invocation.
|
|
4
4
|
|
|
5
|
-
The migration system tracks changes to your TailorDB
|
|
5
|
+
The migration system tracks changes to your TailorDB table definitions over time and applies them to deployed workspaces with optional data transformation scripts.
|
|
6
6
|
|
|
7
7
|
For the CLI command reference, see [`tailordb migration`](../cli/tailordb.md#tailordb-migration). This document covers concepts, workflows, and operational guidance.
|
|
8
8
|
|
|
@@ -10,9 +10,9 @@ For the CLI command reference, see [`tailordb migration`](../cli/tailordb.md#tai
|
|
|
10
10
|
|
|
11
11
|
**Key Properties**
|
|
12
12
|
|
|
13
|
-
- **Local snapshot–based diff detection** — each migration is generated by diffing your current
|
|
13
|
+
- **Local snapshot–based diff detection** — each migration is generated by diffing your current table definitions against the previous snapshot stored in `migrations/<NNNN>/`.
|
|
14
14
|
- **Transaction-wrapped data migrations** — each `migrate.ts` script runs inside a database transaction on the platform; if the script throws, all changes in that migration roll back.
|
|
15
|
-
- **Automatic execution during `apply`** — `tailor deploy` detects pending migrations, runs the two-stage
|
|
15
|
+
- **Automatic execution during `apply`** — `tailor deploy` detects pending migrations, runs the two-stage table update (pre-migration → script → post-migration), and updates the migration checkpoint label.
|
|
16
16
|
- **Type-safe scripts** — the generated `db.ts` provides Kysely types that reflect the schema state **before** the migration runs, so transformations are written against the actual data shape.
|
|
17
17
|
|
|
18
18
|
**Files in `migrations/`**
|
|
@@ -39,33 +39,33 @@ migrations/
|
|
|
39
39
|
When you start with no `migrations/` directory:
|
|
40
40
|
|
|
41
41
|
1. Add the `migration` block to `tailor.config.ts` (see [Configuration](#configuration)).
|
|
42
|
-
2. Define your initial
|
|
42
|
+
2. Define your initial tables in `tailordb/`.
|
|
43
43
|
3. Generate the initial migration:
|
|
44
44
|
```bash
|
|
45
45
|
tailor tailordb migration generate
|
|
46
46
|
```
|
|
47
|
-
This creates `migrations/0000/schema.json` from your current
|
|
47
|
+
This creates `migrations/0000/schema.json` from your current tables.
|
|
48
48
|
4. Run `tailor deploy`. The migration label is set to `0000` on the deployed namespace.
|
|
49
49
|
|
|
50
50
|
### Adding migrations to an existing project
|
|
51
51
|
|
|
52
|
-
If you already have a deployed workspace whose schema matches your local
|
|
52
|
+
If you already have a deployed workspace whose schema matches your local table definitions:
|
|
53
53
|
|
|
54
54
|
1. Add the `migration` block to `tailor.config.ts`.
|
|
55
|
-
2. Run `tailor tailordb migration generate` to create `0000/schema.json` from current local
|
|
55
|
+
2. Run `tailor tailordb migration generate` to create `0000/schema.json` from current local tables.
|
|
56
56
|
3. Run `tailor deploy`. Because remote schema already matches, no script runs; only the migration label is set.
|
|
57
57
|
|
|
58
|
-
If your local
|
|
58
|
+
If your local tables and remote schema have **diverged**, reconcile them before introducing migrations — either update local tables to match remote, or accept that the first non-`0000` migration will reflect that gap.
|
|
59
59
|
|
|
60
60
|
### Resetting
|
|
61
61
|
|
|
62
|
-
`tailor tailordb migration generate --init` deletes the existing `migrations/` directory and
|
|
62
|
+
`tailor tailordb migration generate --init` deletes the existing `migrations/` directory and creates `0000` from the current local types. Use it only before the project is deployed. For a deployed migration history, use [`migration rebaseline`](#re-baselining-a-deployed-migration-history), which verifies the history and connected workspace before replacing any files.
|
|
63
63
|
|
|
64
64
|
## Migration Workflow
|
|
65
65
|
|
|
66
66
|
A typical change cycle:
|
|
67
67
|
|
|
68
|
-
1. **Modify a
|
|
68
|
+
1. **Modify a table definition.**
|
|
69
69
|
|
|
70
70
|
```typescript
|
|
71
71
|
// tailordb/user.ts
|
|
@@ -115,31 +115,39 @@ A typical change cycle:
|
|
|
115
115
|
|
|
116
116
|
### Warnings and optional migration scripts
|
|
117
117
|
|
|
118
|
-
Some non-breaking changes can still cause data loss — most notably removing a field (`field_removed`) or removing a
|
|
118
|
+
Some non-breaking changes can still cause data loss — most notably removing a field (`field_removed`) or removing a table (`type_removed`). `migration generate` reports these as **warnings**:
|
|
119
119
|
|
|
120
120
|
```
|
|
121
121
|
Warning: data loss possible:
|
|
122
122
|
|
|
123
|
-
- User.legacyParentId: Field removed (existing data will be
|
|
123
|
+
- User.legacyParentId: Field removed (existing data will no longer be accessible through the schema after the post-migration phase)
|
|
124
124
|
```
|
|
125
125
|
|
|
126
|
-
No `migrate.ts` is generated automatically because the schema change itself is non-breaking, but the existing data is
|
|
126
|
+
No `migrate.ts` is generated automatically because the schema change itself is non-breaking, but the existing data is no longer accessible through the active schema after the post-migration phase. The platform may retain a removed field's underlying stored value, so do not rely on removal to clear data before reusing the same field name. If you need to preserve, transform, or clear that data first, add a script with:
|
|
127
127
|
|
|
128
128
|
```bash
|
|
129
129
|
tailor tailordb migration script 0002
|
|
130
130
|
```
|
|
131
131
|
|
|
132
|
-
This writes `migrations/0002/migrate.ts` and `migrations/0002/db.ts` next to the existing `diff.json
|
|
132
|
+
This writes `migrations/0002/migrate.ts` and `migrations/0002/db.ts` next to the existing `diff.json` (add `--with-test` to also scaffold a `migrate.test.ts` — see [Testing Migrations Locally](#testing-migrations-locally)). The removed field stays readable inside `migrate.ts` because the pre-migration phase keeps it on the table until the script finishes (see [Per-migration phases](#per-migration-phases)). The next `tailor deploy` runs the script automatically — `migrate.ts` is executed whenever the file exists on disk, regardless of whether the diff itself required it.
|
|
133
|
+
|
|
134
|
+
If the data loss is intentional and no script is needed, record that decision the same way as for breaking changes (see [Breaking changes without a script](#breaking-changes-without-a-script)):
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
tailor tailordb migration script 0002 --no-script --reason "column no longer needed, data can be dropped"
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
In an interactive session, `migration generate` offers to record the reason on the spot when it detects warnings. The acknowledgment is stored in `diff.json`, so it is reviewable in the PR, and it satisfies `migration validate --strict` — useful for enforcing in CI that destructive changes are explicitly acknowledged before merge (see [Schema verification](#schema-verification)).
|
|
133
141
|
|
|
134
142
|
### Breaking changes without a script
|
|
135
143
|
|
|
136
|
-
Breaking changes require `migrate.ts`. If it is missing at deploy time (for example, the generated script was deleted), `tailor deploy` fails before applying the migration or anything after it. When there is genuinely nothing to migrate — say, the affected
|
|
144
|
+
Breaking changes require `migrate.ts`. If it is missing at deploy time (for example, the generated script was deleted), `tailor deploy` fails before applying the migration or anything after it. When there is genuinely nothing to migrate — say, the affected table holds no data yet — record an explicit acknowledgment instead of keeping an empty script:
|
|
137
145
|
|
|
138
146
|
```bash
|
|
139
147
|
tailor tailordb migration script 0002 --no-script --reason "no data yet, safe to skip"
|
|
140
148
|
```
|
|
141
149
|
|
|
142
|
-
This stores the reason in `migrations/0002/diff.json` (commit the change). The next `tailor deploy` applies the schema change as usual, skips only the script step, and logs the recorded reason. The command refuses to record a skip while `migrate.ts` exists — delete the script first. If `migrate.ts` is added back later, the script takes precedence over the acknowledgment and runs.
|
|
150
|
+
This stores the reason in `migrations/0002/diff.json` (commit the change). The next `tailor deploy` applies the schema change as usual, skips only the script step, and logs the recorded reason. The command refuses to record a skip while `migrate.ts` exists — delete the script first. If `migrate.ts` is added back later, the script takes precedence over the acknowledgment and runs; run `tailor tailordb migration script 0002` again to clear the now-stale acknowledgment from `diff.json`.
|
|
143
151
|
|
|
144
152
|
## Configuration
|
|
145
153
|
|
|
@@ -167,15 +175,24 @@ export default defineConfig({
|
|
|
167
175
|
|
|
168
176
|
## Generated Files
|
|
169
177
|
|
|
170
|
-
| File
|
|
171
|
-
|
|
|
172
|
-
| `0000/schema.json`
|
|
173
|
-
| `XXXX/diff.json`
|
|
174
|
-
| `XXXX/migrate.ts`
|
|
175
|
-
| `XXXX/db.ts`
|
|
178
|
+
| File | When generated | Description |
|
|
179
|
+
| ---------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
|
|
180
|
+
| `0000/schema.json` | First `migration generate` | Full snapshot of all tables in the namespace. |
|
|
181
|
+
| `XXXX/diff.json` | Every subsequent migration | Field-level diff against the previous snapshot. |
|
|
182
|
+
| `XXXX/migrate.ts` | Auto-generated for breaking changes; added manually via `tailordb migration script` for warning-tier changes | Data transformation script. The `main` export receives a Kysely `Transaction`. |
|
|
183
|
+
| `XXXX/db.ts` | Generated once when `migrate.ts` is created | Kysely types reflecting the schema **before** this migration. Exports `Database`, `Transaction`, and `MigrationContext`. |
|
|
184
|
+
| `XXXX/migrate.test.ts` | Added via `tailordb migration script --with-test` | Unit-test scaffold for `migrate.ts` (see [Testing Migrations Locally](#testing-migrations-locally)). Never deployed. |
|
|
176
185
|
|
|
177
186
|
`db.ts` reflects the pre-migration schema because the script runs after the pre-migration phase has temporarily relaxed breaking constraints (e.g., a new `required` field is added as `optional` first), so the data being read still matches the previous shape.
|
|
178
187
|
|
|
188
|
+
### Migration file format compatibility
|
|
189
|
+
|
|
190
|
+
Migration files are versioned independently of the SDK package. This SDK writes format version `2` and reads versions `1` through `2`. It normalizes supported older formats in memory; it never rewrites applied migration files on disk.
|
|
191
|
+
|
|
192
|
+
If a future SDK can no longer replay an old migration format, re-baseline while using an SDK version that still supports the complete history, commit the new baseline, deploy it to every environment, and then upgrade the SDK. A file from a newer unsupported format instead requires upgrading the SDK first. The CLI rejects both cases with guidance rather than attempting a best-effort replay.
|
|
193
|
+
|
|
194
|
+
There is no migration-file conversion command. Keeping applied files unchanged preserves the record of what ran, while `migration rebaseline` provides the escape hatch when the supported replay window changes.
|
|
195
|
+
|
|
179
196
|
## Migration Script Anatomy
|
|
180
197
|
|
|
181
198
|
```typescript
|
|
@@ -196,6 +213,34 @@ export async function main(trx: Transaction): Promise<void> {
|
|
|
196
213
|
}
|
|
197
214
|
```
|
|
198
215
|
|
|
216
|
+
**Worked example: backfilling a new required enum field**
|
|
217
|
+
|
|
218
|
+
Adding a required field is a breaking change, so `migration generate` scaffolds `migrate.ts`. Given this table change:
|
|
219
|
+
|
|
220
|
+
```typescript
|
|
221
|
+
// tailordb/user.ts
|
|
222
|
+
export const user = db.table("User", {
|
|
223
|
+
name: db.string(),
|
|
224
|
+
email: db.string(),
|
|
225
|
+
role: db.enum(["MANAGER", "STAFF"]), // ← new required field
|
|
226
|
+
...db.fields.timestamps(),
|
|
227
|
+
});
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
existing `User` rows have no `role` value yet, so the script assigns one before the post-migration phase enforces the constraint:
|
|
231
|
+
|
|
232
|
+
```typescript
|
|
233
|
+
import type { Transaction } from "./db";
|
|
234
|
+
|
|
235
|
+
export async function main(trx: Transaction): Promise<void> {
|
|
236
|
+
await trx.updateTable("User").set({ role: "MANAGER" }).where("role", "is", null).execute();
|
|
237
|
+
}
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
The `where("role", "is", null)` guard keeps the script idempotent — rows that already have a value are untouched if the script re-runs.
|
|
241
|
+
|
|
242
|
+
Reference scripts for other breaking-change patterns live in the repository's [migration fixture templates](https://github.com/tailor-platform/sdk/tree/main/example/tests/migration-fixtures/templates): backfilling fields that become required and migrating rows off a removed enum value ([0005](https://github.com/tailor-platform/sdk/blob/main/example/tests/migration-fixtures/templates/0005/migrate.ts)), and de-duplicating values before a unique constraint is added ([0006](https://github.com/tailor-platform/sdk/blob/main/example/tests/migration-fixtures/templates/0006/migrate.ts)). They show the shape of each migration, not drop-in logic — adapt them to your data (the suffix strategy in `0006`, for example, assumes the suffixed names are not already taken).
|
|
243
|
+
|
|
199
244
|
**Accessing environment variables**
|
|
200
245
|
|
|
201
246
|
The migration `main` receives an optional second argument exposing the variables defined in `defineConfig({ env })` — the same values available via `context.env` in resolvers. The `MigrationContext` type is exported from the generated `./db`:
|
|
@@ -223,39 +268,158 @@ The `env` values are injected at bundle time (the same mechanism as resolvers/ex
|
|
|
223
268
|
|
|
224
269
|
## Supported Schema Changes
|
|
225
270
|
|
|
226
|
-
| Change Type
|
|
227
|
-
|
|
|
228
|
-
| Add optional field
|
|
229
|
-
| Add required field
|
|
230
|
-
| Remove field
|
|
231
|
-
| Change optional → required
|
|
232
|
-
| Change required → optional
|
|
233
|
-
| Add index (non-unique)
|
|
234
|
-
| Add unique index
|
|
235
|
-
| Change unique index fields
|
|
236
|
-
| Remove index
|
|
237
|
-
| Add unique constraint
|
|
238
|
-
| Remove unique constraint
|
|
239
|
-
| Change decimal scale
|
|
240
|
-
| Add enum value
|
|
241
|
-
| Remove enum value
|
|
242
|
-
| Add
|
|
243
|
-
| Remove
|
|
244
|
-
| Change foreign key target
|
|
245
|
-
| Change field type
|
|
246
|
-
| Change
|
|
247
|
-
| Change single value
|
|
271
|
+
| Change Type | Breaking? | Migration Script? | Notes |
|
|
272
|
+
| --------------------------------- | --------- | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
273
|
+
| Add optional field | No | No | Schema change only |
|
|
274
|
+
| Add required field | Yes | Yes | Script populates default values |
|
|
275
|
+
| Remove field | No | Optional | Warning tier — no script is auto-generated, but you can add one with `tailordb migration script` to preserve or clear data before the field leaves the active schema. The field stays readable from `migrate.ts` during Pre-migration. |
|
|
276
|
+
| Change optional → required | Yes | Yes | Script sets defaults for null values |
|
|
277
|
+
| Change required → optional | No | No | Schema change only |
|
|
278
|
+
| Add index (non-unique) | No | No | Schema change only |
|
|
279
|
+
| Add unique index | Yes | Yes | Script must resolve duplicate value combinations across the index fields |
|
|
280
|
+
| Change unique index fields | Yes | Yes | Treated like adding a new unique constraint over the new field set |
|
|
281
|
+
| Remove index | No | No | Schema change only (removing the unique constraint from an index is also non-breaking) |
|
|
282
|
+
| Add unique constraint | Yes | Yes | Script must resolve duplicate values |
|
|
283
|
+
| Remove unique constraint | No | No | Schema change only |
|
|
284
|
+
| Change decimal scale | Yes | Yes | Auto-generated script re-saves existing rows under the new scale. Decreasing scale rounds values half-up and can lose precision. If the same change adds a unique constraint, duplicate handling runs after re-saving. |
|
|
285
|
+
| Add enum value | No | No | Schema change only |
|
|
286
|
+
| Remove enum value | Yes | Yes | Script migrates records with removed values |
|
|
287
|
+
| Add table | No | No | Schema change only |
|
|
288
|
+
| Remove table | No | Optional | Warning tier — no script is auto-generated, but you can add one with `tailordb migration script` to preserve data before the table leaves the active schema. The table stays readable from `migrate.ts` during Pre-migration. |
|
|
289
|
+
| Change foreign key target table | Yes | Yes | Script updates references to the new target |
|
|
290
|
+
| Change field type (verified pair) | Yes | Yes | In-place for `uuid` → `string`, `enum` → `string`, `decimal` → `string`, and `integer` → `float`; review the generated normalization scaffold and customize it only when existing values need transformation |
|
|
291
|
+
| Change field type (other pair) | - | - | **Not supported** — see [3-step migration](#3-step-migration-for-unsupported-changes) |
|
|
292
|
+
| Change array → single value | - | - | **Not supported** — see [3-step migration](#3-step-migration-for-unsupported-changes) |
|
|
293
|
+
| Change single value → array | - | - | **Not supported** — see [3-step migration](#3-step-migration-for-unsupported-changes) |
|
|
294
|
+
|
|
295
|
+
### Generated normalization script for field type changes
|
|
296
|
+
|
|
297
|
+
For example, changing `User.age` from `integer` to `float` generates a `migrate.ts` that scans non-null values in batches of 100:
|
|
298
|
+
|
|
299
|
+
```typescript
|
|
300
|
+
import type { Transaction } from "./db";
|
|
301
|
+
|
|
302
|
+
export async function main(trx: Transaction): Promise<void> {
|
|
303
|
+
// Normalize User.age from integer to float while the previous type is still active
|
|
304
|
+
{
|
|
305
|
+
let lastId: string | undefined;
|
|
306
|
+
while (true) {
|
|
307
|
+
let query = trx
|
|
308
|
+
.selectFrom("User")
|
|
309
|
+
.select(["id", "age"])
|
|
310
|
+
.where("age", "is not", null)
|
|
311
|
+
.orderBy("id", "asc")
|
|
312
|
+
.limit(100);
|
|
313
|
+
if (lastId) {
|
|
314
|
+
query = query.where("id", ">", lastId);
|
|
315
|
+
}
|
|
316
|
+
const rows = await query.execute();
|
|
317
|
+
if (rows.length === 0) break;
|
|
318
|
+
|
|
319
|
+
for (const row of rows) {
|
|
320
|
+
// TODO(tailor-migration-review): Remove this marker and the `never` annotation after reviewing the normalization.
|
|
321
|
+
// Keep the value accepted by the active integer type and castable to float.
|
|
322
|
+
const sourceValue = row.age;
|
|
323
|
+
if (sourceValue === null) continue;
|
|
324
|
+
const normalizedValue: never = sourceValue;
|
|
325
|
+
if (Object.is(normalizedValue, sourceValue)) continue;
|
|
326
|
+
await trx
|
|
327
|
+
.updateTable("User")
|
|
328
|
+
.set({ ["age"]: normalizedValue })
|
|
329
|
+
.where("id", "=", row.id)
|
|
330
|
+
.execute();
|
|
331
|
+
}
|
|
332
|
+
lastId = rows[rows.length - 1]!.id;
|
|
333
|
+
}
|
|
334
|
+
}
|
|
335
|
+
}
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
The generated `never` annotation intentionally causes a TypeScript error until you review the normalization. If the existing values are already suitable for the target type, remove the annotation and review marker to accept the identity transformation; it does not write any rows. If values need application-specific normalization, replace the expression and remove the annotation and marker while keeping the result valid for both the active source type and the target type. The source field contract remains active until the script finishes; for example, an `integer` → `float` script cannot write fractional values during this phase.
|
|
248
339
|
|
|
249
340
|
### 3-step migration for unsupported changes
|
|
250
341
|
|
|
251
|
-
Field type changes (
|
|
342
|
+
Field type changes outside the verified in-place pairs (for example, `string` → `integer`) and array-cardinality changes are detected but rejected by the diff engine. Use an expand-contract strategy:
|
|
252
343
|
|
|
253
|
-
1. **Migration N**: Add
|
|
344
|
+
1. **Migration N**: Add an optional field with the desired type (e.g., `fieldName_new`). If the old field is required, make it optional in the same migration. Write a script that copies and converts every non-null old value into the temporary field, then sets the old field to `null` in the same row update.
|
|
254
345
|
2. **Migration N+1**: Remove the old field.
|
|
255
346
|
3. **Migration N+2**: Add the field back with the original name and the new type. Script copies from the temporary field, then remove the temporary field in migration N+3 (or in the same step if you can express it).
|
|
256
347
|
|
|
257
348
|
The same pattern works for switching between scalar and array.
|
|
258
349
|
|
|
350
|
+
> **Do not skip clearing the old field.** Removing a field from the schema does not necessarily remove its stored JSON value. Re-adding the same name with an incompatible type can deploy successfully while leaving stale values that make subsequent reads fail. Verify that every old value is `null` before removing and re-adding the field name.
|
|
351
|
+
|
|
352
|
+
## Testing Pending Migrations
|
|
353
|
+
|
|
354
|
+
`tailor tailordb migration test` runs every migration pending in the source workspace against an isolated workspace. The source workspace is selected by `--workspace-id` or the active profile and is never modified.
|
|
355
|
+
|
|
356
|
+
The command performs the following sequence:
|
|
357
|
+
|
|
358
|
+
1. Reads each migration-enabled namespace's `sdk-migration` checkpoint from the source workspace and reconstructs that exact snapshot from local migration history.
|
|
359
|
+
2. Creates a temporary workspace in the same region, organization, and folder as the source, unless `--target-workspace-id` names an existing throwaway workspace.
|
|
360
|
+
3. Deploys the checkpoint snapshots and writes their checkpoint labels.
|
|
361
|
+
4. Loads fixture data or clones source records.
|
|
362
|
+
5. Runs the normal deployment pipeline, including every pending pre-migration, `migrate.ts`, and post-migration phase.
|
|
363
|
+
6. Optionally runs an assertion script against the migrated data.
|
|
364
|
+
7. Deletes an automatically-created workspace after success or failure.
|
|
365
|
+
|
|
366
|
+
Both the pre-migration and final TailorDB schemas come from committed migration snapshots. Ungenerated changes in the current type source are not included in the rehearsal.
|
|
367
|
+
|
|
368
|
+
Executors are omitted from the baseline deployment so loading fixture or cloned records cannot trigger current event handlers against the older schema. Auth user profiles are also deferred until the pending migrations finish, while configured machine users remain available to run seed and migration scripts. The final deployment restores the configured executors and user profiles. Static websites are deployed so configuration references to their URLs resolve, but their workspace-bound custom domains are omitted from migration-test deployments.
|
|
369
|
+
|
|
370
|
+
### Seed mode
|
|
371
|
+
|
|
372
|
+
Seed mode is the default and uses the JSONL files produced by the configured `seedPlugin`. Run `tailor generate` after adding the plugin or changing seed types, then populate its `data/*.jsonl` files:
|
|
373
|
+
|
|
374
|
+
```bash
|
|
375
|
+
tailor tailordb migration test --data seed
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
Rows are loaded only for types present in the deployed pre-migration snapshots (and current schemas without migrations), in foreign-key dependency order. Fields introduced by pending migrations, including timestamp and nested fields, are removed before insertion so current fixtures can be loaded into the baseline schema. Missing type files are treated as empty. IdP `_User` fixtures are not loaded by this command.
|
|
379
|
+
|
|
380
|
+
Use `--machine-user` to override the seed plugin's `machineUserName`, the namespace migration setting, and the first configured Auth machine user for seed and assertion execution.
|
|
381
|
+
|
|
382
|
+
### Clone mode
|
|
383
|
+
|
|
384
|
+
Clone mode copies TailorDB records from the source workspace after the identical application, namespace names, and pre-migration schemas exist in the target. For namespaces without migration history, the command reproduces the deployed source schema rather than uncommitted local type changes:
|
|
385
|
+
|
|
386
|
+
```bash
|
|
387
|
+
tailor tailordb migration test --data clone
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
The platform clone API is feature-gated and requires editor access to both same-region workspaces. It copies TailorDB records only: IdP users, file blobs, and metadata labels are not copied. File fields therefore retain references whose blobs are absent. The command polls the asynchronous operation and reports platform failures; if clone is unavailable, use seed mode.
|
|
391
|
+
|
|
392
|
+
The source schema is re-verified immediately before cloning; if the source workspace was deployed or otherwise changed after the test started, the command aborts instead of cloning data that no longer matches the deployed baseline.
|
|
393
|
+
|
|
394
|
+
### Assertions and retained targets
|
|
395
|
+
|
|
396
|
+
Pass a TypeScript file with `--assert`. Its exported `main` function uses the same Kysely transaction signature as `migrate.ts`, runs after all pending migrations, and must throw when an invariant fails:
|
|
397
|
+
|
|
398
|
+
```bash
|
|
399
|
+
tailor tailordb migration test \
|
|
400
|
+
--data seed \
|
|
401
|
+
--assert ./tests/assert-customer-email.ts \
|
|
402
|
+
--assert-namespace tailordb
|
|
403
|
+
```
|
|
404
|
+
|
|
405
|
+
`--assert-namespace` is inferred when only one namespace has pending migrations and is required otherwise.
|
|
406
|
+
|
|
407
|
+
To inspect the result after a run, pass `--keep` so the automatically created workspace survives instead of being deleted, on success and on failure:
|
|
408
|
+
|
|
409
|
+
```bash
|
|
410
|
+
tailor tailordb migration test --keep
|
|
411
|
+
```
|
|
412
|
+
|
|
413
|
+
Alternatively, provide an empty designated throwaway workspace. This mode never deletes the target and requires explicit acknowledgment:
|
|
414
|
+
|
|
415
|
+
```bash
|
|
416
|
+
tailor tailordb migration test \
|
|
417
|
+
--target-workspace-id 00000000-0000-4000-8000-000000000000 \
|
|
418
|
+
--yes
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
Do not target a shared development or production workspace: baseline deployment reconciles its managed resources and schemas before the migration test runs.
|
|
422
|
+
|
|
259
423
|
## Automatic Migration Execution
|
|
260
424
|
|
|
261
425
|
When you run `tailor deploy`, the SDK detects pending migrations (anything past the current `sdk-migration` label on the deployed namespace) and runs them in order before continuing with the rest of the apply.
|
|
@@ -264,9 +428,10 @@ When you run `tailor deploy`, the SDK detects pending migrations (anything past
|
|
|
264
428
|
|
|
265
429
|
For each pending migration:
|
|
266
430
|
|
|
267
|
-
1. **Pre-migration**:
|
|
431
|
+
1. **Pre-migration**: Schema changes that would be breaking are applied in a relaxed form first. A verified in-place field type change keeps its complete previous field contract until Post-migration, including field and type-level hooks or validators changed by the same migration. Newly-required fields are added as optional; fields whose `optional → required` transition is breaking are temporarily kept optional. Fields that are being removed in this migration are temporarily kept on the table so that `migrate.ts` can still read them (for example, to `innerJoin` through a foreign key that is about to be dropped). Breaking type-level index changes are relaxed the same way: a newly-added unique index is withheld, and an index gaining a unique constraint (or a unique index changing its field set) keeps its previous definition, so `migrate.ts` can resolve duplicates first. Non-breaking changes that are part of the same migration are also applied here.
|
|
268
432
|
2. **Script execution**: If `migrate.ts` exists on disk for this migration, it is bundled and sent to the platform via the script execution API and runs as the configured machine user inside a transaction. The script is hard-required for breaking changes (`diff.requiresMigrationScript`) — deploy fails if the file is missing, unless a `--no-script` acknowledgment was recorded (see [Breaking changes without a script](#breaking-changes-without-a-script)). It is also executed when present for warning-tier diffs — see [Warnings and optional migration scripts](#warnings-and-optional-migration-scripts).
|
|
269
|
-
3. **Post-migration**: Required constraints
|
|
433
|
+
3. **Post-migration schema**: Required constraints and the target field definitions are applied. Do not assume that removing a field clears its underlying stored JSON value.
|
|
434
|
+
4. **Checkpoint and cleanup**: The `sdk-migration` label is bumped to this migration's number, then removed GQL permissions and tables are deleted. Advancing the checkpoint first prevents a failed checkpoint write from requiring the SDK to recreate irreversibly deleted records.
|
|
270
435
|
|
|
271
436
|
This split is what allows existing rows to be backfilled before the database starts rejecting nulls, and what lets `migrate.ts` traverse foreign-key fields that the same migration removes.
|
|
272
437
|
|
|
@@ -274,7 +439,7 @@ This split is what allows existing rows to be backfilled before the database sta
|
|
|
274
439
|
|
|
275
440
|
Before running migrations, `apply` performs two checks:
|
|
276
441
|
|
|
277
|
-
1. **Local schema check** — your current
|
|
442
|
+
1. **Local schema check** — your current table definitions must match the latest snapshot in `migrations/`. If they don't, you forgot to run `migration generate`.
|
|
278
443
|
2. **Remote schema check** — the deployed schema is reconstructed from migration history; the actual remote schema must match. Drift here means someone applied a different set of migrations or edited the schema out-of-band.
|
|
279
444
|
|
|
280
445
|
On drift you'll see something like:
|
|
@@ -290,7 +455,7 @@ Namespace: tailordb
|
|
|
290
455
|
|
|
291
456
|
The error also points you at `migration status`, `migration generate`, `migration sync`, and `migration set` — see [Remote schema drift detected](#remote-schema-drift-detected) for which one applies.
|
|
292
457
|
|
|
293
|
-
To run the same checks without deploying — plus migration file integrity (numbering, parseable contents,
|
|
458
|
+
To run the same checks without deploying — plus migration file integrity (numbering, parseable contents, a `migrate.ts` or a recorded `--no-script` acknowledgment for every migration that requires a script, and no unresolved generated normalization review markers):
|
|
294
459
|
|
|
295
460
|
```bash
|
|
296
461
|
tailor tailordb migration validate
|
|
@@ -298,6 +463,8 @@ tailor tailordb migration validate
|
|
|
298
463
|
|
|
299
464
|
It reports issues per namespace, exits with a non-zero code when any check fails, and supports `--json` for machine-readable output.
|
|
300
465
|
|
|
466
|
+
With `--strict`, validation additionally fails when a migration not yet applied to the remote has data-loss warnings (see [Warnings and optional migration scripts](#warnings-and-optional-migration-scripts)) but neither a `migrate.ts` nor a recorded `--no-script` acknowledgment. The failure names the affected type and field and prints the exact command to record the acknowledgment.
|
|
467
|
+
|
|
301
468
|
To bypass both checks during deploy (not recommended outside of recovery scenarios):
|
|
302
469
|
|
|
303
470
|
```bash
|
|
@@ -318,9 +485,31 @@ tailor deploy --no-schema-check
|
|
|
318
485
|
✔ Successfully applied changes.
|
|
319
486
|
```
|
|
320
487
|
|
|
488
|
+
## Re-baselining a deployed migration history
|
|
489
|
+
|
|
490
|
+
`tailor tailordb migration rebaseline` collapses the complete history into a new `0000/schema.json` in the current migration format. It does not modify the deployed schema or data.
|
|
491
|
+
|
|
492
|
+
Before running it:
|
|
493
|
+
|
|
494
|
+
1. Apply the latest migration to every environment. The CLI verifies the connected workspace, but it cannot inspect other workspaces.
|
|
495
|
+
2. Commit or otherwise preserve the existing migration history. Files after `0000`, including `migrate.ts` and `db.ts`, disappear from the working tree; Git history retains committed files.
|
|
496
|
+
3. Make sure local type changes have been captured with `tailor tailordb migration generate`.
|
|
497
|
+
|
|
498
|
+
Then re-baseline one namespace:
|
|
499
|
+
|
|
500
|
+
```bash
|
|
501
|
+
tailor tailordb migration rebaseline --namespace tailordb
|
|
502
|
+
```
|
|
503
|
+
|
|
504
|
+
The command validates the migration files, verifies that replaying the latest migration exactly reproduces the local types, and checks that the connected workspace is at that latest migration with no schema drift. After confirmation, it replaces the local history with the reconstructed baseline, records a new migration history ID in both `0000/schema.json` and remote metadata, and resets the connected workspace's `sdk-migration` label to `0000`. Use `--yes` only after arranging the same operational preconditions in non-interactive automation.
|
|
505
|
+
|
|
506
|
+
Commit the resulting `migrations/` change before generating any new migrations. For another environment still carrying the exact checkpoint and history ID that the new baseline replaced, the next `tailor deploy` checks whether its remote schema exactly matches the new `0000`. If it does, deploy offers to reset the checkpoint to `0000` and move the environment to the new history ID before applying any later local migrations. A markerless history is eligible only for the first rebaseline, at the exact migration recorded as replaced. Any other checkpoint or history ID is rejected without changing remote metadata, even if its schema happens to match the baseline.
|
|
507
|
+
|
|
508
|
+
Partial squashing is not supported: re-baselining always replaces the full history for one namespace.
|
|
509
|
+
|
|
321
510
|
## `migration set` Semantics
|
|
322
511
|
|
|
323
|
-
`tailor tailordb migration set <N>` updates the `sdk-migration` label on the deployed namespace's metadata. **It does not modify any data or schema.** It only changes which migrations the next `apply` will consider pending.
|
|
512
|
+
`tailor tailordb migration set <N>` updates the `sdk-migration` label on the deployed namespace's metadata. **It does not modify any data or schema.** It only changes which migrations the next `apply` will consider pending. The command also aligns the remote migration history ID with the local baseline, removing a stale ID when the local history predates re-baselining.
|
|
324
513
|
|
|
325
514
|
The migration number is validated before anything is sent to the remote: it must be a 4-digit value (e.g. `0001`) or a bare integer (e.g. `1`) within 0–9999, and must exist in the working tree's migration history, which is itself validated (a gapped history is rejected). `0` is always accepted as the baseline (even when no migrations directory exists yet), provided the history passes validation.
|
|
326
515
|
|
|
@@ -339,14 +528,14 @@ Use cases:
|
|
|
339
528
|
|
|
340
529
|
## `migration sync` Semantics
|
|
341
530
|
|
|
342
|
-
`tailor tailordb migration sync <N>` reconstructs the schema snapshot at migration `N` from the working tree's migration history and **overwrites the remote schema to match it**, then sets the `sdk-migration` label to `N
|
|
531
|
+
`tailor tailordb migration sync <N>` reconstructs the schema snapshot at migration `N` from the working tree's migration history and **overwrites the remote schema to match it**, then sets the `sdk-migration` label to `N` and aligns the remote migration history ID with the local baseline. Unlike `migration set`, it changes the remote schema as well as the bookkeeping. Like `set`, it never runs `migrate.ts` scripts itself — it only changes what the next `apply` considers pending:
|
|
343
532
|
|
|
344
|
-
| Movement | Effect on next `apply` | Effect on data
|
|
345
|
-
| -------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
346
|
-
| Backward (e.g., `0003` → `0001`) | Migrations `0002` and `0003` become pending and re-execute, including their `migrate.ts`. |
|
|
347
|
-
| Forward (e.g., `0001` → `0003`) | Migrations `0002` and `0003` are skipped — their `migrate.ts` scripts will not run. | Data the skipped scripts would have migrated stays as-is.
|
|
533
|
+
| Movement | Effect on next `apply` | Effect on data |
|
|
534
|
+
| -------------------------------- | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
|
535
|
+
| Backward (e.g., `0003` → `0001`) | Migrations `0002` and `0003` become pending and re-execute, including their `migrate.ts`. | Tables absent from snapshot `0001` are deleted along with their data; re-executed scripts may rewrite data. |
|
|
536
|
+
| Forward (e.g., `0001` → `0003`) | Migrations `0002` and `0003` are skipped — their `migrate.ts` scripts will not run. | Data the skipped scripts would have migrated stays as-is. |
|
|
348
537
|
|
|
349
|
-
Before anything is sent to the remote, `sync` verifies that replaying the full migration history reproduces the current local
|
|
538
|
+
Before anything is sent to the remote, `sync` verifies that replaying the full migration history reproduces the current local table definitions. If it does not — because migration files were edited and no longer match, or because a schema change has not been recorded with `migration generate` yet — the command fails without touching the remote. This means a rewritten migration history is validated before it can overwrite the deployed schema.
|
|
350
539
|
|
|
351
540
|
Because syncing backward causes already-applied scripts to re-execute on the next deploy, **write `migrate.ts` scripts to be idempotent** (see [Performance and Large Tables](#performance-and-large-tables) for resumable `where` clauses).
|
|
352
541
|
|
|
@@ -359,35 +548,40 @@ The main use case is recovering from drift after a `deploy --no-schema-check` fr
|
|
|
359
548
|
Migration numbers are assigned sequentially, so two developers branching off the same point and each generating `0005` will collide. Conventions that work:
|
|
360
549
|
|
|
361
550
|
- **Don't generate migrations on long-lived feature branches.** Generate them just before merge, after rebasing onto main.
|
|
362
|
-
- **Resolve collisions by re-generating.** If your branch has `0005` but main now has `0005` from another PR,
|
|
551
|
+
- **Resolve collisions by re-generating.** If your branch has `0005` but main now has `0005` from another PR, regenerate yours as `0006` — see [Resolving a migration number conflict](#resolving-a-migration-number-conflict).
|
|
363
552
|
- **Treat migration files as merge-conflict-prone.** They are committed JSON and TypeScript, so review them in PRs. The `diff.json` is the source of truth — if review focuses there, regenerating after rebase is straightforward.
|
|
364
553
|
|
|
554
|
+
### Resolving a migration number conflict
|
|
555
|
+
|
|
556
|
+
When your branch and main each generated the same number, merging or rebasing stops with an add/add conflict on `migrations/0005/diff.json`. Resolve it by re-generating your migration on top of main's:
|
|
557
|
+
|
|
558
|
+
1. **Save your script edits aside.** If you customized `0005/migrate.ts`, keep a copy before touching the directory — during a rebase, `git show ORIG_HEAD:migrations/0005/migrate.ts` prints the version from your pre-rebase branch tip.
|
|
559
|
+
2. **Take main's `0005/` directory in full.** Accept main's version of every conflicting file. Then check for files only your side added: if your migration has a `migrate.ts` and main's does not, that file never conflicts — it silently stays next to main's `diff.json`. Delete such leftovers explicitly.
|
|
560
|
+
3. **Finish the rebase or merge, then re-run `migration generate`.** With main's migration now part of local history, the diff is computed against the correct base — including main's changes — and your migration lands as the next number (`0006`).
|
|
561
|
+
4. **Port your script.** Copy the logic saved in step 1 into the newly scaffolded `0006/migrate.ts`. For a warning-tier change, `migration generate` does not scaffold a script — recreate it first with `tailor tailordb migration script 0006`.
|
|
562
|
+
|
|
563
|
+
**When a plain rename is enough.** If the two migrations touch disjoint tables and fields, renaming your directory to the next free number (keeping main's `0005/`) can be acceptable. Run `tailor tailordb migration validate` after the rename: if it reports a mismatch, the migrations were not disjoint — discard the rename and re-generate as above. A passing check covers only the schema history, not your script: `migrate.ts` now runs after main's migration, so confirm it does not read or write tables that migration touches — when in doubt, re-generate.
|
|
564
|
+
|
|
365
565
|
### CI / CD
|
|
366
566
|
|
|
367
567
|
- For non-interactive environments, pass `--yes` to `migration generate` and `--yes` to `apply`. `apply` runs migrations automatically when the `migrations/` directory is configured.
|
|
368
|
-
- Run `tailor tailordb migration validate` in CI to catch uncommitted migrations, broken migration files, and remote schema drift before deploying. It exits with a non-zero code when validation fails and supports `--json`.
|
|
369
|
-
- `tailor tailordb migration status` shows applied and pending migrations for a human-readable comparison. Its exit code is non-zero
|
|
568
|
+
- Run `tailor tailordb migration validate` in CI to catch uncommitted migrations, broken migration files, unreviewed generated normalization logic, and remote schema drift before deploying. It exits with a non-zero code when validation fails and supports `--json`. Add `--strict` to also require an explicit acknowledgment (a `migrate.ts` or a recorded `--no-script` reason) for every pending migration that can drop data, so destructive changes cannot merge unnoticed.
|
|
569
|
+
- `tailor tailordb migration status` validates file-format compatibility across the full local history, compares its history ID with the deployed namespace, and shows applied and pending migrations for a human-readable comparison. Its exit code is non-zero on incompatible files, migration history mismatches, and remote read errors, so check the output.
|
|
370
570
|
- Avoid running migrations in parallel against the same workspace — there is no locking. Serialize deploys per environment.
|
|
371
571
|
|
|
372
572
|
### Resetting a deployed project
|
|
373
573
|
|
|
374
|
-
`migration generate --init`
|
|
375
|
-
|
|
376
|
-
1. Run `migration generate --init` to start over from `0000`.
|
|
377
|
-
2. Run `tailor tailordb migration set 0` against the deployed namespace.
|
|
378
|
-
3. Run `tailor deploy` — the new `0000` becomes the baseline.
|
|
379
|
-
|
|
380
|
-
Coordinate this with your team because everyone else's local migrations will be invalidated.
|
|
574
|
+
Use `tailor tailordb migration rebaseline` rather than combining `migration generate --init` with a manual checkpoint change. See [Re-baselining a deployed migration history](#re-baselining-a-deployed-migration-history) for the required cross-environment coordination and verification.
|
|
381
575
|
|
|
382
576
|
## Failure Recovery
|
|
383
577
|
|
|
384
578
|
If the pre-migration phase or `migrate.ts` fails:
|
|
385
579
|
|
|
386
580
|
- **The transaction rolls back** for that migration's script. Database changes the script made are undone.
|
|
387
|
-
- **The pre-migration schema changes are rolled back** to the prior checkpoint:
|
|
581
|
+
- **The pre-migration schema changes are rolled back** to the prior checkpoint: tables that already existed are restored to their previous shape, and tables the migration newly introduced are dropped. The workspace is left at its prior checkpoint and prior schema — not half-applied.
|
|
388
582
|
- The whole `apply` aborts and the checkpoint label is not bumped. Subsequent migrations in the same run do not execute.
|
|
389
583
|
|
|
390
|
-
The rollback is best-effort per
|
|
584
|
+
The rollback is best-effort per table; if reverting a table fails, a warning is logged and the original migration error is still reported.
|
|
391
585
|
|
|
392
586
|
After a failure:
|
|
393
587
|
|
|
@@ -395,13 +589,17 @@ After a failure:
|
|
|
395
589
|
2. Fix `migrate.ts` (or the data it depends on).
|
|
396
590
|
3. Re-run `tailor deploy`. The same migration runs again because its label was never bumped, and the prior-checkpoint schema is a clean baseline to retry against.
|
|
397
591
|
|
|
398
|
-
If a migration **succeeds in script** but
|
|
592
|
+
If a migration **succeeds in script** but its reversible **post-migration schema update** fails (rare; usually a constraint violation the script should have prevented), the SDK makes the same best-effort restoration to the prior-checkpoint schema. The script's committed data changes remain, so write migration scripts to tolerate re-execution.
|
|
593
|
+
|
|
594
|
+
The checkpoint is advanced only after the reversible post-migration schema updates succeed. If the checkpoint write reports an error, the SDK reads it back: a matching value is treated as committed. Any other observed value leaves the post-migration schema unchanged rather than risk rolling back a concurrent deployment; a value beyond the current migration confirms a concurrent deploy, while an older or missing value means the checkpoint must be repaired before retrying. If read-back also fails, the SDK likewise leaves the post-migration schema unchanged; verify the remote checkpoint before retrying.
|
|
595
|
+
|
|
596
|
+
Removed tables are deleted only after the checkpoint is committed. If that cleanup fails, the checkpoint remains at the new migration and the SDK fails closed: the leftover table is reported as remote schema drift on the next deploy. Remove the leftover GQL permission and table manually, verify the remote schema, and then retry. The SDK does not automatically ignore or delete a same-named remote table because it cannot distinguish failed cleanup from a table recreated after cleanup completed.
|
|
399
597
|
|
|
400
598
|
## Rollback Strategy
|
|
401
599
|
|
|
402
600
|
There is no automatic down-migration. To roll back a schema/data change in production, write a new forward migration that reverses the previous one. For example, to undo a `0005` that added a required `email` field:
|
|
403
601
|
|
|
404
|
-
1. Edit your
|
|
602
|
+
1. Edit your table definitions to remove the field.
|
|
405
603
|
2. `migration generate --name "rollback 0005 email"` produces `0006` with a removal diff.
|
|
406
604
|
3. Apply.
|
|
407
605
|
|
|
@@ -418,7 +616,7 @@ The CLI logs the selected user before running scripts (`Using machine user: ...`
|
|
|
418
616
|
|
|
419
617
|
**Permissions required**
|
|
420
618
|
|
|
421
|
-
The machine user needs read/write access to every
|
|
619
|
+
The machine user needs read/write access to every table the migration script touches. If your migrations alter data across multiple tables, the simplest path is to give the migration user broad access (e.g., an `ADMIN` role) and restrict day-to-day machine users separately. If the user lacks permission, the script fails with a permission error in `Logs:`.
|
|
422
620
|
|
|
423
621
|
If you see `No machine user available for migration execution`, either:
|
|
424
622
|
|
|
@@ -445,10 +643,105 @@ The migration script runs in a single transaction. For tables with many rows:
|
|
|
445
643
|
|
|
446
644
|
## Testing Migrations Locally
|
|
447
645
|
|
|
448
|
-
|
|
646
|
+
### Unit-testing migrate.ts
|
|
449
647
|
|
|
450
|
-
-
|
|
451
|
-
|
|
648
|
+
`main` is a plain function, so you can unit-test it with Vitest before the first deploy ever runs it. `createKyselyMock` from `@tailor-platform/sdk/vitest` compiles queries to the same SQL as the deployed migration, so a test verifies the exact statements the script issues — SQL, parameters, and order. Type the mock with the `Database` interface exported from the generated `db.ts`.
|
|
649
|
+
|
|
650
|
+
Scaffold a ready-to-fill test next to the script with:
|
|
651
|
+
|
|
652
|
+
```bash
|
|
653
|
+
tailor tailordb migration script 0005 --with-test
|
|
654
|
+
```
|
|
655
|
+
|
|
656
|
+
When `migrate.ts` already exists (the usual case for breaking changes, where `migration generate` creates it), the command adds only `migrate.test.ts`. Or write the test by hand:
|
|
657
|
+
|
|
658
|
+
```typescript
|
|
659
|
+
// migrations/0005/migrate.test.ts
|
|
660
|
+
import { createKyselyMock } from "@tailor-platform/sdk/vitest";
|
|
661
|
+
import { describe, expect, test } from "vitest";
|
|
662
|
+
import type { Database } from "./db";
|
|
663
|
+
import { main } from "./migrate";
|
|
664
|
+
|
|
665
|
+
describe("0005 add required email", () => {
|
|
666
|
+
test("backfills null emails", async () => {
|
|
667
|
+
const mock = createKyselyMock<Database>();
|
|
668
|
+
|
|
669
|
+
await mock.withTx((trx) => main(trx));
|
|
670
|
+
|
|
671
|
+
expect(mock.updates).toHaveLength(1);
|
|
672
|
+
expect(mock.updates[0]?.updateValues()).toEqual({ email: "unknown@example.com" });
|
|
673
|
+
expect(mock.updates[0]?.sql).toContain('where "email" is null');
|
|
674
|
+
});
|
|
675
|
+
});
|
|
676
|
+
```
|
|
677
|
+
|
|
678
|
+
Stage the rows each query returns with `mock.enqueueResult(...)` or `mock.setQueryResolver(...)` when the script reads before writing; call `main(trx, { env: { ... } })` when the script takes a `MigrationContext`. See [Kysely-layer mock](../testing.md#kysely-layer-mock-createkyselymock) for the full mock API.
|
|
679
|
+
|
|
680
|
+
These tests need no platform connection and no `tailor-runtime` environment — they run in a plain Vitest setup. Vitest's default `include` pattern already picks up `migrations/**/migrate.test.ts`; if your config narrows `include`, add the migrations directory. The test file is ignored by `tailor deploy` and never ships to the platform.
|
|
681
|
+
|
|
682
|
+
### Executing migrate.ts against a local Postgres (PGlite)
|
|
683
|
+
|
|
684
|
+
A statement-level test verifies what the script issues, not what it does to data (e.g., whether a `where` clause matches the rows you intended). To run `main` against real rows locally, back Kysely with [`@electric-sql/pglite`](https://pglite.dev/) — an in-memory PostgreSQL — via `createKyselyPGlite` from `@tailor-platform/sdk/vitest`:
|
|
685
|
+
|
|
686
|
+
```bash
|
|
687
|
+
npm install -D @electric-sql/pglite
|
|
688
|
+
```
|
|
689
|
+
|
|
690
|
+
Create the tables the script touches (matching the shape in the generated `db.ts`), stage rows, then run the script in a transaction:
|
|
691
|
+
|
|
692
|
+
```typescript
|
|
693
|
+
// migrations/0005/migrate.pglite.test.ts
|
|
694
|
+
import { PGlite } from "@electric-sql/pglite";
|
|
695
|
+
import { sql } from "@tailor-platform/sdk/kysely";
|
|
696
|
+
import { createKyselyPGlite } from "@tailor-platform/sdk/vitest";
|
|
697
|
+
import { afterAll, beforeAll, describe, expect, test } from "vitest";
|
|
698
|
+
import type { Database } from "./db";
|
|
699
|
+
import { main } from "./migrate";
|
|
700
|
+
|
|
701
|
+
const db = createKyselyPGlite<Database>(new PGlite());
|
|
702
|
+
|
|
703
|
+
beforeAll(async () => {
|
|
704
|
+
await sql`
|
|
705
|
+
CREATE TABLE "User" (
|
|
706
|
+
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
707
|
+
"name" text NOT NULL,
|
|
708
|
+
"email" text
|
|
709
|
+
)
|
|
710
|
+
`.execute(db);
|
|
711
|
+
});
|
|
712
|
+
|
|
713
|
+
afterAll(async () => {
|
|
714
|
+
await db.destroy();
|
|
715
|
+
});
|
|
716
|
+
|
|
717
|
+
describe("0005 add required email", () => {
|
|
718
|
+
test("backfills null emails and keeps existing ones", async () => {
|
|
719
|
+
await sql`
|
|
720
|
+
INSERT INTO "User" ("name", "email")
|
|
721
|
+
VALUES ('a', NULL), ('b', 'b@example.com')
|
|
722
|
+
`.execute(db);
|
|
723
|
+
|
|
724
|
+
await db.transaction().execute((trx) => main(trx));
|
|
725
|
+
|
|
726
|
+
const rows = await db.selectFrom("User").select(["name", "email"]).orderBy("name").execute();
|
|
727
|
+
expect(rows).toEqual([
|
|
728
|
+
{ name: "a", email: "unknown@example.com" },
|
|
729
|
+
{ name: "b", email: "b@example.com" },
|
|
730
|
+
]);
|
|
731
|
+
});
|
|
732
|
+
});
|
|
733
|
+
```
|
|
734
|
+
|
|
735
|
+
Two caveats keep this from replacing a scratch workspace:
|
|
736
|
+
|
|
737
|
+
- PGlite runs full PostgreSQL, while TailorDB supports [a subset of it](https://docs.tailor.tech/guides/function/accessing-tailordb#supported-sql-queries) — a statement that passes here can still be rejected on deploy.
|
|
738
|
+
- The `CREATE TABLE` statements are yours, so they can drift from the schema the platform actually has.
|
|
739
|
+
|
|
740
|
+
### Beyond unit tests
|
|
741
|
+
|
|
742
|
+
A unit test verifies which statements the script issues; a PGlite test verifies what they do to the rows you staged. Neither runs against your actual data. To cover that:
|
|
743
|
+
|
|
744
|
+
- Run `migration generate` on a clean working copy first, review `diff.json`, then run again after editing tables to ensure the diff matches what you intended.
|
|
452
745
|
- For non-trivial migrations, apply against a scratch workspace before promoting to staging or production.
|
|
453
746
|
|
|
454
747
|
## Environment-Specific Strategies
|
|
@@ -481,6 +774,12 @@ For genuinely different schemas across environments, prefer separate workspaces
|
|
|
481
774
|
2. Restore the file from version control (`git checkout -- <path>`), or regenerate migration files with `migration generate` / `migration script`.
|
|
482
775
|
3. Do not hand-edit `schema.json` or `diff.json`; they are managed by the CLI.
|
|
483
776
|
|
|
777
|
+
### "Unsupported migration file format version" error
|
|
778
|
+
|
|
779
|
+
**Cause:** A `schema.json` or `diff.json` file is older or newer than the format versions supported by the installed SDK.
|
|
780
|
+
|
|
781
|
+
**Resolution:** Follow the ordering in the error message. For an older history, restore an SDK version that can read every file, run `migration rebaseline`, commit and deploy the new baseline everywhere, and then upgrade. For a file produced by a newer SDK, upgrade the SDK that is reading it. Do not hand-edit the version field.
|
|
782
|
+
|
|
484
783
|
### "No machine user available for migration execution"
|
|
485
784
|
|
|
486
785
|
**Cause:** Neither `migration.machineUser` is set nor are there any machine users in `auth.machineUsers`.
|
|
@@ -495,7 +794,7 @@ For genuinely different schemas across environments, prefer separate workspaces
|
|
|
495
794
|
|
|
496
795
|
### Migration script execution fails
|
|
497
796
|
|
|
498
|
-
**Cause:** Runtime error in your `migrate.ts`, a permission error from the machine user, or a constraint violation when post-migration tightens
|
|
797
|
+
**Cause:** Runtime error in your `migrate.ts`, a permission error from the machine user, or a constraint violation when post-migration tightens tables.
|
|
499
798
|
|
|
500
799
|
**Resolution:** Read the `Logs:` block. Fix the script or the data assumption it relies on, and re-run `tailor deploy`. The label is not bumped on failure, so the same migration retries.
|
|
501
800
|
|