qubu 0.4.2 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/{canonical-BXUguqfo.mjs → canonical-DMvR9yBe.mjs} +4 -4
- package/dist/codegen.d.mts +1 -1
- package/dist/codegen.mjs +7 -4
- package/dist/{column-hqKr7-1I.mjs → column-DmazTL67.mjs} +15 -2
- package/dist/{complete-WYyVozgK.mjs → complete-DP7pliuY.mjs} +6 -6
- package/dist/{complete-types-IjEn5VPN.d.mts → complete-types-CY0KbzNw.d.mts} +1 -1
- package/dist/core.d.mts +1 -1
- package/dist/core.mjs +3 -3
- package/dist/diff.d.mts +201 -1
- package/dist/diff.mjs +8 -8
- package/dist/{index-1DpA3mUh.d.mts → index-C-480HmV.d.mts} +12 -12
- package/dist/index.d.mts +2 -2
- package/dist/index.mjs +226 -44
- package/dist/introspection.d.mts +2 -2
- package/dist/introspection.mjs +24 -13
- package/dist/mysql.d.mts +2 -2
- package/dist/{on-conflict-hfPW0KmQ.mjs → on-conflict-jPsl9l0K.mjs} +6 -3
- package/dist/postgres.d.mts +2 -2
- package/dist/postgres.mjs +3 -3
- package/dist/{registry-BRMLYwDp.mjs → registry-BGqa05et.mjs} +2 -2
- package/dist/{relational-BZ3WDPzC.mjs → relational-x3BDVX9e.mjs} +2 -2
- package/dist/schema.d.mts +2 -2
- package/dist/schema.mjs +5 -5
- package/dist/{snapshot-C-W65HEd.mjs → snapshot-BSraiLtH.mjs} +2 -2
- package/dist/snapshot.d.mts +4 -4
- package/dist/snapshot.mjs +585 -5
- package/dist/{source-BcS2AsIg.mjs → source-C4Vmu5bb.mjs} +1 -1
- package/dist/{sqlite-Cg0nwYEH.mjs → sqlite-CsIUtZ2Q.mjs} +6 -6
- package/dist/sqlite.d.mts +30 -3
- package/dist/sqlite.mjs +28 -2
- package/dist/{table-Bp5irMSj.mjs → table-DLQ7YWth.mjs} +2 -2
- package/dist/{types-BK1COGZe.d.mts → types-C0VkiwpR.d.mts} +146 -41
- package/dist/{types-JSZHpUEj.d.mts → types-CTCqtFlS.d.mts} +1 -1
- package/dist/{types-JM3FcAnX.mjs → types-CTENnDh9.mjs} +2 -2
- package/dist/{value-Bi71Agyf.mjs → value-BEEj_Ayd.mjs} +1 -1
- package/dist/vite/ambient.d.ts +6 -0
- package/docs/config.json +1 -0
- package/docs/dialects-and-execution.md +47 -0
- package/docs/guides/drizzle.md +2 -1
- package/docs/index.md +6 -2
- package/docs/migrations/adapters.md +70 -0
- package/docs/migrations/artifacts-and-policy.md +123 -0
- package/docs/migrations/index.md +46 -0
- package/docs/migrations/lotta-adoption.md +50 -0
- package/docs/migrations/operations.md +133 -0
- package/docs/migrations/recovery.md +121 -0
- package/docs/reference/introspection-support.md +1 -1
- package/docs/reference/supported-surface.md +59 -44
- package/docs/schema/catalog-model.md +1 -1
- package/docs/schema/columns-and-writes.md +14 -0
- package/docs/schema/ddl-emission.md +15 -10
- package/docs/schema/introspection.md +4 -3
- package/docs/schema/migration-plans.md +6 -6
- package/docs/schema/snapshots.md +2 -2
- package/package.json +5 -14
- package/dist/ddl.d.mts +0 -118
- package/dist/ddl.mjs +0 -1119
- package/dist/index-CPvfEheG.d.mts +0 -202
- package/dist/migration.d.mts +0 -214
- package/dist/migration.mjs +0 -1160
- package/dist/mysql-B_cYzzX2.mjs +0 -585
package/docs/guides/drizzle.md
CHANGED
|
@@ -75,7 +75,8 @@ Unix timestamps and Drizzle must continue reading and writing `Date` values:
|
|
|
75
75
|
|
|
76
76
|
```ts
|
|
77
77
|
import { schema, table } from "qubu"
|
|
78
|
-
import {
|
|
78
|
+
import { toSqliteDrizzleSchema } from "@qubu/drizzle/sqlite"
|
|
79
|
+
import { sqliteTimestamp } from "qubu/sqlite"
|
|
79
80
|
|
|
80
81
|
const events = table("events", {
|
|
81
82
|
createdAt: sqliteTimestamp({
|
package/docs/index.md
CHANGED
|
@@ -46,6 +46,8 @@ define a table, build a `SELECT`, and inspect its SQL and parameters.
|
|
|
46
46
|
data before DDL emission.
|
|
47
47
|
- [Emit DDL](schema/ddl-emission.md) from an approved migration plan without
|
|
48
48
|
handing Qubu a database connection.
|
|
49
|
+
- [Operate migrations](migrations/index.md) with versioned artifacts, verified
|
|
50
|
+
adapter profiles, baselines, a portable executor, and explicit recovery.
|
|
49
51
|
|
|
50
52
|
## The query pipeline
|
|
51
53
|
|
|
@@ -76,6 +78,7 @@ needs to preserve a fact across composition:
|
|
|
76
78
|
| ------------------------ | --------------------------------------------------- | ----------------------------------------------------------------------------------- |
|
|
77
79
|
| Query model | [Source scope](query-model/source-scope.md) | Source identity, result shapes, fragments, metadata, and query composition |
|
|
78
80
|
| Schema model | [Tables and names](schema/tables-and-names.md) | Tables, snapshots, diffs, migration plans, and DDL emission |
|
|
81
|
+
| Migration operations | [Migration operations](migrations/index.md) | Artifacts, adapters, CLI policy, baselines, execution, and recovery |
|
|
79
82
|
| Database introspection | [Database introspection](schema/introspection.md) | Catalog readers, Snapshot v1 mapping, identities, diagnostics, and support limits |
|
|
80
83
|
| Schema source generation | [Generate a schema](schema/code-generation.md) | Machine-owned TypeScript, identity handoff, controlled mappings, and v1 exclusions |
|
|
81
84
|
| Rendering and execution | [Dialects and execution](dialects-and-execution.md) | Placeholder and identifier policies, capabilities, adapters, and raw-SQL boundaries |
|
|
@@ -104,6 +107,7 @@ The inferred row is `{ id: number; name: string }`. The value `7` stays out
|
|
|
104
107
|
of the SQL text and appears in the `parameters` array in placeholder order.
|
|
105
108
|
|
|
106
109
|
The [supported features](reference/supported-surface.md) page is the canonical
|
|
107
|
-
package-entrypoint and ownership map.
|
|
108
|
-
migration
|
|
110
|
+
package-entrypoint and ownership map. Applications may use Qubu's portable
|
|
111
|
+
migration executor while retaining credentials, approval policy, custom SQL,
|
|
112
|
+
and deployment lifecycle ownership. [Troubleshooting](troubleshooting.md)
|
|
109
113
|
starts from common errors and points to the concept page behind each one.
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# Adapter capability profiles
|
|
2
|
+
|
|
3
|
+
> Select a migration adapter from capabilities proven by its driver and environment, not from dialect name alone.
|
|
4
|
+
|
|
5
|
+
Every executable migration adapter opens one pinned migration session and
|
|
6
|
+
advertises the exact behavior the executor may use:
|
|
7
|
+
|
|
8
|
+
| Field | Contract |
|
|
9
|
+
| -------------------------------------- | ----------------------------------------------------------------------------------- |
|
|
10
|
+
| `dialect`, `serverVersion` | Physical target and optional version used for compatibility checks |
|
|
11
|
+
| `session` | Must be `pinned` until `close()` resolves |
|
|
12
|
+
| `transactionalDdl` | Whether DDL effects can roll back |
|
|
13
|
+
| `optionalTransactions`, `transactions` | Whether optional phases join a transaction and which requirements are proven |
|
|
14
|
+
| `lease`, `leaseKind` | Database-backed exclusion of another migration runner |
|
|
15
|
+
| `locks` | Independently supported program DDL lock requirements |
|
|
16
|
+
| `journal` | Database storage, head compare-and-swap, and atomic applied-record/head advancement |
|
|
17
|
+
| `parameters` | Supported tagged parameter kinds |
|
|
18
|
+
| `commitAmbiguity` | Ambiguous commit becomes `recovery-required` |
|
|
19
|
+
| `forbiddenPhases` | Checkpointed support or explicit rejection |
|
|
20
|
+
| `features` | Named constraints an artifact may require |
|
|
21
|
+
|
|
22
|
+
The migrator lease and a program's DDL lock are different controls. The lease
|
|
23
|
+
excludes another Qubu runner; a DDL lock protects the database operation. The
|
|
24
|
+
executor never treats one as proof of the other.
|
|
25
|
+
|
|
26
|
+
## Current profiles
|
|
27
|
+
|
|
28
|
+
The following stable profiles have live conformance coverage in this checkout:
|
|
29
|
+
|
|
30
|
+
| Migration entrypoint | Dialect | Transactions | Locks | Forbidden phases | Notes |
|
|
31
|
+
| ------------------------------------- | ---------- | ----------------------------- | --------------- | ---------------- | --------------------------------------------------- |
|
|
32
|
+
| `@qubu/adapter-libsql/migration` | SQLite | required, optional | none, exclusive | unsupported | Pinned application-owned client |
|
|
33
|
+
| `@qubu/adapter-node-sqlite/migration` | SQLite | required, optional | none, exclusive | unsupported | Pinned application-owned `DatabaseSync` |
|
|
34
|
+
| `@qubu/adapter-pg/migration` | PostgreSQL | required, optional, forbidden | none, exclusive | checkpointed | Caller supplies an already-pinned client |
|
|
35
|
+
| `@qubu/adapter-postgresjs/migration` | PostgreSQL | required, optional, forbidden | none, exclusive | checkpointed | Reserves and releases one connection |
|
|
36
|
+
| `@qubu/adapter-pglite/migration` | PostgreSQL | required, optional, forbidden | none, exclusive | checkpointed | Uses the database query queue as the pinned session |
|
|
37
|
+
|
|
38
|
+
All five support every current tagged parameter kind (`null`, `boolean`,
|
|
39
|
+
`string`, `number`, `bigint`, `bytes`, and `json`), a database journal and
|
|
40
|
+
lease, atomic applied-record/head advancement, and recovery-required commit
|
|
41
|
+
ambiguity classification. Support still depends on the artifact's server,
|
|
42
|
+
feature, transaction, and lock constraints.
|
|
43
|
+
|
|
44
|
+
These exported profiles are unavailable and must not be passed to the
|
|
45
|
+
executor:
|
|
46
|
+
|
|
47
|
+
| Export | Status | Reason |
|
|
48
|
+
| --------------------------- | ----------------- | ----------------------------------------------------------------------------------- |
|
|
49
|
+
| `d1MigrationProfile` | `incompatible` | D1 exposes no pinned interactive transaction/session contract |
|
|
50
|
+
| `mysql2MigrationProfile` | `not-yet-written` | MySQL implicit-commit lease, checkpoint, and recovery semantics are not live-proven |
|
|
51
|
+
| `bunSqliteMigrationProfile` | `not-yet-written` | A Bun-native pinned-session and journal conformance run is missing |
|
|
52
|
+
|
|
53
|
+
Unavailable profiles expose `reason` and `missingCapabilities`; they do not
|
|
54
|
+
fall back to a generic executor.
|
|
55
|
+
|
|
56
|
+
For libSQL, let the migration entrypoint exclude all reserved journal objects
|
|
57
|
+
during strict inspection:
|
|
58
|
+
|
|
59
|
+
```ts
|
|
60
|
+
import { createClient } from "@libsql/client"
|
|
61
|
+
import { libsqlMigrationAdapter, readLibsqlMigrationSnapshot } from "@qubu/adapter-libsql/migration"
|
|
62
|
+
|
|
63
|
+
const client = createClient({ url: process.env.DATABASE_URL! })
|
|
64
|
+
const adapter = libsqlMigrationAdapter(client, {
|
|
65
|
+
readSnapshot: readLibsqlMigrationSnapshot,
|
|
66
|
+
})
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`DATABASE_URL` remains application configuration; neither the adapter nor CLI
|
|
70
|
+
assigns deployment-provider meaning to it.
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# Artifacts and approval policy
|
|
2
|
+
|
|
3
|
+
> Review exactly what is authenticated and executable before an artifact enters a repository.
|
|
4
|
+
|
|
5
|
+
Qubu has two strict artifact kinds. An executable migration contains a reviewed
|
|
6
|
+
plan and authoritative program. A verified baseline records an observed schema
|
|
7
|
+
without pretending that historical SQL ran.
|
|
8
|
+
|
|
9
|
+
## Published formats
|
|
10
|
+
|
|
11
|
+
These format versions are independent of npm package semver:
|
|
12
|
+
|
|
13
|
+
| Value | Current version | Meaning |
|
|
14
|
+
| --------------------------- | --------------: | ---------------------------------------- |
|
|
15
|
+
| `qubu-executable-migration` | 1 | Executable artifact envelope |
|
|
16
|
+
| `qubu-verified-baseline` | 1 | Non-executable verified starting point |
|
|
17
|
+
| `qubu-migration-program` | 1 | Ordered executable phases and statements |
|
|
18
|
+
| `qubu-migration-plan` | 2 | Dialect-neutral reviewed plan |
|
|
19
|
+
| `qubu-canonical-json` | 1 | Canonical JSON byte encoding |
|
|
20
|
+
| `sha-256` | 1 | Operational digest algorithm contract |
|
|
21
|
+
| `qubu-migration-journal` | 1 | Logical journal format |
|
|
22
|
+
|
|
23
|
+
Strict decoders reject unknown keys, malformed values, non-canonical encoded
|
|
24
|
+
text, unsupported versions, and digest mismatches. There is no compatibility
|
|
25
|
+
decoder for provisional application formats.
|
|
26
|
+
|
|
27
|
+
### Executable artifact schema
|
|
28
|
+
|
|
29
|
+
An executable artifact records:
|
|
30
|
+
|
|
31
|
+
- `id`, zero-based `sequence`, and `parentArtifactDigest` lineage;
|
|
32
|
+
- canonicalization and digest descriptors, dialect, and optional minimum server
|
|
33
|
+
version or required capability constraints;
|
|
34
|
+
- the plan plus `planDigest`;
|
|
35
|
+
- renderer identity, the program, and `programDigest`;
|
|
36
|
+
- before/after snapshot descriptors, each with a strong digest and either an
|
|
37
|
+
embedded snapshot or a reference;
|
|
38
|
+
- operation-scoped approvals and custom-program provenance;
|
|
39
|
+
- artifact provenance and `artifactDigest`.
|
|
40
|
+
|
|
41
|
+
The program—not `emitMigrationPlan(...).sql` and not joined statement text—is
|
|
42
|
+
the execution authority. Each phase declares its position, dependencies,
|
|
43
|
+
transaction and lock requirements, preconditions, postconditions, and ordered
|
|
44
|
+
statements. Each statement declares its operation ID, dependencies, SQL, and
|
|
45
|
+
tagged parameters.
|
|
46
|
+
|
|
47
|
+
### Baseline artifact schema
|
|
48
|
+
|
|
49
|
+
A baseline records `id`, sequence and parent lineage, encoding descriptors,
|
|
50
|
+
dialect and optional constraints, one verified snapshot descriptor,
|
|
51
|
+
`verifiedAt`, provenance, optional operator metadata, and `artifactDigest`. It
|
|
52
|
+
has no migration plan, program, or SQL digest.
|
|
53
|
+
|
|
54
|
+
Artifact IDs are stable identities, not repository order. Sequence and parent
|
|
55
|
+
digest establish the linear chain. Renumbering therefore changes lineage and
|
|
56
|
+
the artifact digest.
|
|
57
|
+
|
|
58
|
+
## Canonical bytes and digest domains
|
|
59
|
+
|
|
60
|
+
`encodeCanonical()` sorts object keys by Unicode code-point order, preserves
|
|
61
|
+
array order, emits compact JSON as UTF-8, normalizes `-0` to `0`, rejects
|
|
62
|
+
non-finite numbers, and adds one LF at EOF. `digestCanonical()` prefixes those
|
|
63
|
+
bytes with the UTF-8 bytes for:
|
|
64
|
+
|
|
65
|
+
```text
|
|
66
|
+
qubu:migrate:v1:<domain>\0
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The five domains are `artifact`, `baseline`, `migration-plan`,
|
|
70
|
+
`migration-program`, and `schema-snapshot`. Domain separation prevents equal
|
|
71
|
+
JSON values used for different purposes from sharing an integrity identity.
|
|
72
|
+
Operational digests have the form `sha256:` plus 64 lowercase hexadecimal
|
|
73
|
+
digits and are recomputed while sealing or decoding.
|
|
74
|
+
|
|
75
|
+
Snapshot and plan `fingerprint` APIs are deterministic FNV-1a64 change
|
|
76
|
+
detectors. They remain useful for caches and fixture assertions, but they are
|
|
77
|
+
not cryptographic integrity evidence and are never valid journal heads,
|
|
78
|
+
artifact parents, or substitutes for a SHA-256 field.
|
|
79
|
+
|
|
80
|
+
## Exact approval policy
|
|
81
|
+
|
|
82
|
+
Preview planning may retain unresolved safety findings. Sealing is stricter:
|
|
83
|
+
|
|
84
|
+
| Operation state | Sealing rule |
|
|
85
|
+
| -------------------------------------- | --------------------------------------------------------------------------------- |
|
|
86
|
+
| Safe and supported | No approval required |
|
|
87
|
+
| Review-required, destructive, or lossy | Exact `operationId` approval with a non-empty reason |
|
|
88
|
+
| Unknown or unsupported | Exact custom-program substitution and `custom-program` approval |
|
|
89
|
+
| Explicit custom SQL | Always requires exact review and a reason |
|
|
90
|
+
| Skipped operation | Recompute the target snapshot before sealing; never preserve a false after-digest |
|
|
91
|
+
|
|
92
|
+
An approval also records the operation's safety and the exact sorted finding
|
|
93
|
+
codes. It may record `approvedBy` and `approvedAt`. A mismatched operation ID,
|
|
94
|
+
safety classification, finding set, or decision fails compilation. A broad
|
|
95
|
+
"allow unsafe" option on the preview DDL emitter is not an artifact-sealing
|
|
96
|
+
approval.
|
|
97
|
+
|
|
98
|
+
Custom programs replace one exact operation. They must declare transaction and
|
|
99
|
+
lock requirements, statements, tagged parameters, and any pre/postconditions,
|
|
100
|
+
plus source and reason provenance. Qubu does not split arbitrary SQL, infer its
|
|
101
|
+
effects, or approve it automatically.
|
|
102
|
+
|
|
103
|
+
## Version and golden-fixture maintenance
|
|
104
|
+
|
|
105
|
+
Treat canonical bytes and the meaning of every published field as release
|
|
106
|
+
contracts. When changing an encoder, tag, ordering rule, parameter encoding,
|
|
107
|
+
renderer meaning, or schema:
|
|
108
|
+
|
|
109
|
+
1. Keep golden values for every already-published artifact, baseline, program,
|
|
110
|
+
plan, snapshot, canonicalization, and digest version.
|
|
111
|
+
2. Verify the same canonical bytes and SHA-256 results in Node.js, Bun, and a
|
|
112
|
+
worker-compatible Web Crypto runtime.
|
|
113
|
+
3. If encoded bytes or semantics change, introduce a new explicit format,
|
|
114
|
+
canonicalization, digest, plan, program, renderer, or artifact version as
|
|
115
|
+
appropriate. Do not silently update a version 1 fixture.
|
|
116
|
+
4. Keep old fixtures as decode/verification evidence for supported published
|
|
117
|
+
versions; add a new fixture for the new version.
|
|
118
|
+
5. Run artifact tamper/non-canonical tests and packed-package checks before
|
|
119
|
+
release.
|
|
120
|
+
|
|
121
|
+
The current canonical SHA-256 vectors live with
|
|
122
|
+
`packages/migrate/test/artifact.test.ts`; changes to them require the same
|
|
123
|
+
intentional version decision as file-based golden fixtures.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Migration operations
|
|
2
|
+
|
|
3
|
+
> Choose the package entrypoint that owns each migration concern without pulling database or Node.js behavior into pure schema code.
|
|
4
|
+
|
|
5
|
+
Qubu migrations are split across explicit ownership boundaries:
|
|
6
|
+
|
|
7
|
+
| Owner | Imports | Responsibility |
|
|
8
|
+
| --------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
|
|
9
|
+
| `qubu` | `qubu/snapshot`, `qubu/diff`, `qubu/introspection` | Pure schema snapshots, comparison, and catalog mapping |
|
|
10
|
+
| `@qubu/migrate` | Focused subpaths listed below | Pure planning and compilation plus portable artifacts, journals, execution, status, baselines, and bootstrap |
|
|
11
|
+
| `@qubu/cli` | `@qubu/cli/config`, `@qubu/cli/repository` | Node.js configuration loading, artifact files, commands, output, and process exit behavior |
|
|
12
|
+
| Adapter package | `@qubu/adapter-*/migration` | Pinned driver sessions, parameter binding, transactions, leases, locks, database journal storage, and failure classification |
|
|
13
|
+
| Application | Its own configuration and deployment code | Credentials, environment selection, approval policy, custom SQL, rollout timing, and legacy cutover decisions |
|
|
14
|
+
|
|
15
|
+
The pre-alpha `qubu/migration` and `qubu/ddl` entrypoints no longer exist. Use
|
|
16
|
+
the extracted compiler entrypoints:
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
import { createMigrationPlan } from "@qubu/migrate/plan"
|
|
20
|
+
import { emitMigrationPlan } from "@qubu/migrate/ddl"
|
|
21
|
+
import { compileMigrationProgram, sealExecutableArtifact } from "@qubu/migrate/artifact"
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
The `@qubu/migrate` root intentionally exports only format/version constants
|
|
25
|
+
and the central plan and artifact types. Import behavior from its focused
|
|
26
|
+
entrypoint:
|
|
27
|
+
|
|
28
|
+
| Entrypoint | Use it for |
|
|
29
|
+
| -------------------------- | -------------------------------------------------------------------------- |
|
|
30
|
+
| `@qubu/migrate/plan` | Create, encode, decode, fingerprint, and validate migration plans |
|
|
31
|
+
| `@qubu/migrate/ddl` | Preview deterministic dialect SQL without opening a database |
|
|
32
|
+
| `@qubu/migrate/artifact` | Compile programs; canonicalize, digest, seal, encode, and decode artifacts |
|
|
33
|
+
| `@qubu/migrate/repository` | Verify a complete artifact chain and its journal prefix |
|
|
34
|
+
| `@qubu/migrate/journal` | Implement or inspect the storage-neutral journal contract |
|
|
35
|
+
| `@qubu/migrate/executor` | Apply artifacts and reconcile uncertain attempts |
|
|
36
|
+
| `@qubu/migrate/baseline` | Verify and record the initial non-executable baseline |
|
|
37
|
+
| `@qubu/migrate/status` | Inspect pending work, drift, requirements, and interrupted attempts |
|
|
38
|
+
| `@qubu/migrate/bootstrap` | Plan a fresh SQLite database through the normal compiler |
|
|
39
|
+
| `@qubu/migrate/testing` | Test adapter capabilities and deterministic failure boundaries |
|
|
40
|
+
|
|
41
|
+
Start with [Artifacts and approval policy](artifacts-and-policy.md) when
|
|
42
|
+
reviewing a migration format. Check [Adapter capability
|
|
43
|
+
profiles](adapters.md), use [Command line operations](operations.md) to
|
|
44
|
+
configure an application, then keep [Recovery and reconciliation](recovery.md)
|
|
45
|
+
with the deployment runbook. [Lotta Games adoption](lotta-adoption.md) records
|
|
46
|
+
the downstream cutover boundary and current combo-matrix blocker.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Lotta Games adoption
|
|
2
|
+
|
|
3
|
+
> Replace Lotta's provisional runner with Qubu while preserving product-owned deployment policy and historical truth.
|
|
4
|
+
|
|
5
|
+
Adopt the released `@qubu/migrate`, `@qubu/cli`, and libSQL migration entrypoint
|
|
6
|
+
as a hard cutover. Do not add an upstream decoder for Lotta's provisional JSON,
|
|
7
|
+
FNV artifact digests, journal, or broad unsafe flags.
|
|
8
|
+
|
|
9
|
+
Before changing downstream state, inspect every environment for a provisional
|
|
10
|
+
journal or baseline row. Regenerate unreleased migrations in the Qubu artifact
|
|
11
|
+
format. For an existing database, create one standard baseline only after strict
|
|
12
|
+
live introspection matches the intended Qubu snapshot. That baseline records a
|
|
13
|
+
verified starting state; it does not claim that old migrations ran through
|
|
14
|
+
Qubu.
|
|
15
|
+
|
|
16
|
+
Keep these concerns in Lotta:
|
|
17
|
+
|
|
18
|
+
- Turso credentials and environment selection;
|
|
19
|
+
- Cloudflare deployment waiting and migrate-first/migrate-last prompts;
|
|
20
|
+
- the destructive-deny approval policy and product-specific custom programs;
|
|
21
|
+
- rollout timing and the one-time legacy journal cutover decision.
|
|
22
|
+
|
|
23
|
+
Replace the private migration and runner modules with thin configuration and
|
|
24
|
+
CLI/library invocation. Make the release script apply the complete pending
|
|
25
|
+
repository chain and preserve non-zero failures. Generate fresh local test
|
|
26
|
+
databases with `schema bootstrap`; keep connection PRAGMAs in the test harness.
|
|
27
|
+
|
|
28
|
+
> [!WARNING]
|
|
29
|
+
> Keep Drizzle migration SQL and snapshots read-only. Do not import or replay
|
|
30
|
+
> Drizzle history. A verified baseline is the handoff from historical state to
|
|
31
|
+
> Qubu lineage.
|
|
32
|
+
|
|
33
|
+
The downstream verification set should cover a fresh bootstrap, an already
|
|
34
|
+
baselined database, a no-op deploy, pending migrations in both deployment
|
|
35
|
+
timing modes, drift refusal, concurrent invocation, rollback, and explicit
|
|
36
|
+
recovery.
|
|
37
|
+
|
|
38
|
+
## Combo-matrix release blocker
|
|
39
|
+
|
|
40
|
+
The main repository currently pins the `combos` submodule gitlink to
|
|
41
|
+
`2e2856e5692ef5cbef03a055fa71e5baab8ec10a`. Commit `d9d2e02` added and verified
|
|
42
|
+
migration profiles in the main repository without advancing that gitlink, so
|
|
43
|
+
the orphan-branch adapter × environment matrix is not yet synchronized with the
|
|
44
|
+
new profile claims. This is an unresolved pinned-SHA issue, not evidence that
|
|
45
|
+
the matrix has verified the new migration entrypoints.
|
|
46
|
+
|
|
47
|
+
Do not edit the gitlink or matrix as part of Lotta adoption. Before release,
|
|
48
|
+
update the `combos` branch in its own review, run the declared environments and
|
|
49
|
+
real database round trips, classify every candidate pair, then advance the main
|
|
50
|
+
repository gitlink to that reviewed commit in a separate change.
|
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
# Command line operations
|
|
2
|
+
|
|
3
|
+
> Configure, inspect, baseline, and apply a complete migration chain with stable non-interactive behavior.
|
|
4
|
+
|
|
5
|
+
Install the CLI, migration library, and one verified migration adapter. For a
|
|
6
|
+
libSQL application:
|
|
7
|
+
|
|
8
|
+
```bash
|
|
9
|
+
pnpm add @qubu/cli @qubu/migrate @qubu/adapter-libsql @libsql/client
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
The `qubu` binary is implemented with `@alloc/cmd-ts`. It loads
|
|
13
|
+
`qubu.config.js` by default; `--config <path>` selects another application-owned
|
|
14
|
+
module. Every command accepts `--format human|json` (default `human`) and
|
|
15
|
+
`--non-interactive`. Commands do not prompt today; `--non-interactive` records
|
|
16
|
+
the deployment contract and missing explicit input still fails.
|
|
17
|
+
|
|
18
|
+
## Configuration
|
|
19
|
+
|
|
20
|
+
Export a typed config and keep credentials inside the adapter factory:
|
|
21
|
+
|
|
22
|
+
```ts
|
|
23
|
+
import { createClient } from "@libsql/client"
|
|
24
|
+
import { libsqlMigrationAdapter, readLibsqlMigrationSnapshot } from "@qubu/adapter-libsql/migration"
|
|
25
|
+
import { defineConfig } from "@qubu/cli/config"
|
|
26
|
+
import snapshot from "./schema.snapshot.js"
|
|
27
|
+
|
|
28
|
+
const url = process.env.DATABASE_URL
|
|
29
|
+
if (!url) throw new Error("DATABASE_URL is required")
|
|
30
|
+
|
|
31
|
+
export default defineConfig({
|
|
32
|
+
artifacts: "./migrations",
|
|
33
|
+
snapshot,
|
|
34
|
+
environment: "production",
|
|
35
|
+
adapter: () =>
|
|
36
|
+
libsqlMigrationAdapter(createClient({ url }), {
|
|
37
|
+
readSnapshot: readLibsqlMigrationSnapshot,
|
|
38
|
+
}),
|
|
39
|
+
provenance: { source: "my-service" },
|
|
40
|
+
})
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
`snapshot` may be a value or async factory. Alternatively provide both
|
|
44
|
+
`schema` and `snapshotFromSchema`. Optional configuration owns operation
|
|
45
|
+
approvals, custom programs, renderer/server constraints, baseline operator
|
|
46
|
+
metadata, and reconciliation proof. `artifacts` is resolved from the CLI
|
|
47
|
+
working directory.
|
|
48
|
+
|
|
49
|
+
| Field | Required | Meaning |
|
|
50
|
+
| ------------------------------------------ | ------------------- | -------------------------------------------------------------------------------------------- |
|
|
51
|
+
| `artifacts` | yes | Artifact directory, relative to the command working directory unless absolute |
|
|
52
|
+
| `snapshot` | one snapshot source | Snapshot value or sync/async factory |
|
|
53
|
+
| `schema` + `snapshotFromSchema` | one snapshot source | Application-owned conversion when the source is a Qubu `Schema` |
|
|
54
|
+
| `adapter` | database commands | Sync/async factory returning a migration adapter |
|
|
55
|
+
| `approvals` | no | Sync/async operation policy; receives the operation, finding codes, and requested CLI reason |
|
|
56
|
+
| `customPrograms` | no | Exact operation substitutions with execution requirements and provenance |
|
|
57
|
+
| `renderer`, `serverVersion`, `constraints` | no | Renderer identity and target compatibility constraints |
|
|
58
|
+
| `provenance` | no | Artifact source/revision/actor/metadata; defaults to `{ source: "@qubu/cli" }` |
|
|
59
|
+
| `environment` | no | `development`, `test`, `staging`, or `production`; context only |
|
|
60
|
+
| `baselineOperator` | no | JSON-safe operator metadata stored in a baseline |
|
|
61
|
+
| `verifyReconciliation` | reconcile only | Application-owned proof of the selected live outcome |
|
|
62
|
+
|
|
63
|
+
## Commands
|
|
64
|
+
|
|
65
|
+
| Syntax | Reads or writes | Important failure behavior |
|
|
66
|
+
| --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
|
|
67
|
+
| `qubu migrate create <id> [--approve <operation-id=reason>...] [--approved-by <actor>] [--dry-run]` | Verifies the full repository, plans from its embedded final snapshot, seals, then writes one canonical artifact unless dry-run | Unknown operation IDs or missing exact approvals fail policy |
|
|
68
|
+
| `qubu migrate verify` | Strictly decodes and verifies every artifact and the complete chain | Any malformed, tampered, forked, gapped, or mismatched artifact fails validation |
|
|
69
|
+
| `qubu migrate status` | Opens a session and lease; reports managed drift, unmanaged objects, pending artifacts, interrupted attempts, and incompatible requirements | Recovery, validation, drift, and capability policy are distinct failures |
|
|
70
|
+
| `qubu migrate apply [--dry-run]` | Applies the complete verified pending chain; dry-run performs status/preflight only | It never limits discovery to Git-added or branch-diff files |
|
|
71
|
+
| `qubu migrate baseline <id> --confirm <fact>... [--dry-run]` | Without dry-run, strictly compares the live managed schema, initializes an empty journal, records baseline, then writes the artifact | Requires an empty artifact repository and all seven exact confirmations; dry-run does not inspect the database |
|
|
72
|
+
| `qubu migrate reconcile <attempt-id> --outcome applied\|rolled_back --reason <text>` | Runs application-owned verification, then records the explicit outcome | Requires `verifyReconciliation` in config; no automatic inference |
|
|
73
|
+
| `qubu schema bootstrap [--approve <operation-id=reason>...] [--dry-run]` | Plans an empty SQLite snapshot through diff/plan/program; executes through the normal executor unless dry-run | Currently rejects non-SQLite targets |
|
|
74
|
+
|
|
75
|
+
JSON output is stable, newline-terminated, recursively key-sorted, and redacts
|
|
76
|
+
credential-like keys and credentials or secrets embedded in URLs. Human output
|
|
77
|
+
is deliberately terse. Signals propagate through adapters; an abort exits 130.
|
|
78
|
+
|
|
79
|
+
| Exit | Meaning |
|
|
80
|
+
| ---: | ----------------------------------------------------------------------------- |
|
|
81
|
+
| 0 | Success |
|
|
82
|
+
| 2 | CLI usage or argument error |
|
|
83
|
+
| 3 | Artifact, repository, journal, or other validation failure |
|
|
84
|
+
| 4 | Policy or adapter-capability refusal |
|
|
85
|
+
| 5 | Managed schema drift |
|
|
86
|
+
| 6 | Recovery or reconciliation required |
|
|
87
|
+
| 7 | Adapter, concurrency, rollback, uncertain-outcome, or other execution failure |
|
|
88
|
+
| 130 | Aborted |
|
|
89
|
+
|
|
90
|
+
## Status, drift, and bootstrap
|
|
91
|
+
|
|
92
|
+
Status compares managed physical schema facts against the embedded expected
|
|
93
|
+
snapshot. Logical IDs help reporting but do not prove equality. Objects not
|
|
94
|
+
owned by the managed snapshot are returned separately as `unmanagedObjects`;
|
|
95
|
+
Qubu journal objects are excluded by migration snapshot readers.
|
|
96
|
+
|
|
97
|
+
`schema bootstrap` is for a fresh SQLite database. It produces the same
|
|
98
|
+
versioned program and validation path as a migration. SQLite inline constraints
|
|
99
|
+
are compiled into table creation, while table rebuilds are explicit phases with
|
|
100
|
+
copy/postcondition checks. Session settings such as SQLite PRAGMAs remain in
|
|
101
|
+
the application or adapter setup.
|
|
102
|
+
|
|
103
|
+
## Baseline and cutover checklist
|
|
104
|
+
|
|
105
|
+
A baseline is a statement about the live database now, not a replay of its
|
|
106
|
+
history. Before supplying all seven confirmations, the operator must verify:
|
|
107
|
+
|
|
108
|
+
- `database-target`: the connection names the intended environment;
|
|
109
|
+
- `snapshot-source`: the reviewed snapshot is the intended source of truth;
|
|
110
|
+
- `zero-managed-drift`: strict inspection reports no managed mismatch;
|
|
111
|
+
- `backup-restore-ready`: backup and restore procedures are ready;
|
|
112
|
+
- `other-migrators-stopped`: no other migration runner can race the cutover;
|
|
113
|
+
- `application-compatible`: deployed code is compatible with the live schema;
|
|
114
|
+
- `legacy-history-cutover`: the team accepts the new baseline as the lineage start.
|
|
115
|
+
|
|
116
|
+
For example, repeat `--confirm` once per exact value. The CLI rejects missing
|
|
117
|
+
or unknown confirmation names even outside production; `environment` is
|
|
118
|
+
reported as context rather than used to weaken the policy.
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
qubu migrate baseline lotta-cutover \
|
|
122
|
+
--confirm database-target \
|
|
123
|
+
--confirm snapshot-source \
|
|
124
|
+
--confirm zero-managed-drift \
|
|
125
|
+
--confirm backup-restore-ready \
|
|
126
|
+
--confirm other-migrators-stopped \
|
|
127
|
+
--confirm application-compatible \
|
|
128
|
+
--confirm legacy-history-cutover \
|
|
129
|
+
--format json --non-interactive
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
After success, preserve the written baseline artifact with the repository. Its
|
|
133
|
+
sequence is zero, its parent is null, and later migrations extend its digest.
|
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
# Recovery and reconciliation
|
|
2
|
+
|
|
3
|
+
> Stop after an ambiguous attempt, prove the live outcome, and repair journal lineage without replaying SQL.
|
|
4
|
+
|
|
5
|
+
The journal has one versioned metadata row with an atomic head, immutable
|
|
6
|
+
applied artifact records, mutable attempts, phase/statement checkpoints, and
|
|
7
|
+
append-only reconciliation records. Adapter implementations store it in the
|
|
8
|
+
same database and reserve `__qubu_migration_`-prefixed objects from managed
|
|
9
|
+
schema inspection.
|
|
10
|
+
|
|
11
|
+
| Record | Fields and invariant |
|
|
12
|
+
| ---------------- | -------------------------------------------------------------------------------------------------------------- |
|
|
13
|
+
| Metadata | `format`, `version`, and nullable SHA-256 `head`; the head equals the final applied digest |
|
|
14
|
+
| Applied artifact | `artifactId`, `sequence`, artifact/parent digests, `kind`, `attemptId`, and `appliedAt`; records are immutable |
|
|
15
|
+
| Attempt | ID, artifact ID/digest, expected head, state, timestamps, and optional redacted failure |
|
|
16
|
+
| Checkpoint | Attempt/phase IDs, optional statement ID, `started` or `completed`, and timestamp |
|
|
17
|
+
| Reconciliation | Attempt ID, proven `applied` or `rolled_back` outcome, non-empty reason, and timestamp |
|
|
18
|
+
|
|
19
|
+
## Attempt state machine
|
|
20
|
+
|
|
21
|
+
The diagram shows every legal state transition. `started`, `running`, and
|
|
22
|
+
`recovery_required` all block later migrations until the attempt reaches a
|
|
23
|
+
terminal state.
|
|
24
|
+
|
|
25
|
+
```mermaid
|
|
26
|
+
stateDiagram-v2
|
|
27
|
+
[*] --> started
|
|
28
|
+
started --> running
|
|
29
|
+
started --> rolled_back: definite failure
|
|
30
|
+
started --> recovery_required: interrupted or uncertain
|
|
31
|
+
running --> applied: history + head recorded
|
|
32
|
+
running --> rolled_back: definite rollback
|
|
33
|
+
running --> recovery_required: effect or outcome uncertain
|
|
34
|
+
recovery_required --> applied: verified reconciliation
|
|
35
|
+
recovery_required --> rolled_back: verified reconciliation
|
|
36
|
+
applied --> [*]
|
|
37
|
+
rolled_back --> [*]
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
An `applied` attempt must have a matching immutable applied record. Applied
|
|
41
|
+
records must form a zero-based linear sequence whose final digest equals the
|
|
42
|
+
metadata head. The journal must be an exact prefix of the artifact repository.
|
|
43
|
+
Duplicate IDs or digests, forks, gaps, parent mismatches, tampering, a stale
|
|
44
|
+
head, or a non-prefix repository fail before any statement executes.
|
|
45
|
+
|
|
46
|
+
## Execution and concurrency guarantees
|
|
47
|
+
|
|
48
|
+
For each invocation, the executor verifies the entire repository, opens one
|
|
49
|
+
pinned session, checks capabilities, acquires the migrator lease, validates the
|
|
50
|
+
journal and repository prefix, checks the live before-snapshot digest, then
|
|
51
|
+
applies each pending artifact. Within an artifact it creates an attempt,
|
|
52
|
+
executes ordered phases with preconditions and postconditions, writes durable
|
|
53
|
+
checkpoints, appends immutable history, and compare-and-swaps the head.
|
|
54
|
+
Resources are released in reverse order: DDL lock, migrator lease, then session.
|
|
55
|
+
|
|
56
|
+
A second runner cannot rely on the lease alone. Atomic applied-record/head
|
|
57
|
+
advancement uses the expected parent as a compare-and-swap guard. A runner that
|
|
58
|
+
observes the already-matching head exits idempotently; a conflicting head is a
|
|
59
|
+
structured `concurrency` error.
|
|
60
|
+
|
|
61
|
+
| Phase requirement | Executor behavior |
|
|
62
|
+
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
63
|
+
| `required` | Refuses an adapter without proven transactional DDL; runs that phase transactionally, with the applied/head update in the final phase's transaction |
|
|
64
|
+
| `optional` | Uses a transaction when `optionalTransactions` is true; otherwise relies on checkpoints |
|
|
65
|
+
| `forbidden` | Runs outside a transaction only on a profile advertising checkpointed forbidden phases |
|
|
66
|
+
| Mixed program | Reports phase-level behavior and returns `mixed` atomicity rather than claiming whole-migration atomicity |
|
|
67
|
+
|
|
68
|
+
Program compilation has no `unknown` transaction state: unresolved requirements
|
|
69
|
+
must be resolved by the renderer or explicit custom program before sealing.
|
|
70
|
+
Transactions are phase-scoped. Do not infer that an earlier committed phase
|
|
71
|
+
will roll back because a later phase fails.
|
|
72
|
+
|
|
73
|
+
Errors use stable codes: `validation`, `policy`, `drift`, `concurrency`,
|
|
74
|
+
`capability`, `definite-rollback`, `uncertain-outcome`, `recovery-required`,
|
|
75
|
+
`aborted`, and `adapter`. Context may include artifact, attempt, phase, and
|
|
76
|
+
statement identifiers. Persisted failures omit SQL parameters and credentials.
|
|
77
|
+
|
|
78
|
+
Do not automatically retry after any statement may have taken effect. Only an
|
|
79
|
+
error explicitly marked `retry: "safe"`—normally validation or a failure proven
|
|
80
|
+
before execution—is retryable. A definite transaction rollback ends as
|
|
81
|
+
`rolled_back`; an interrupted non-transactional phase, ambiguous commit or
|
|
82
|
+
rollback, or unproven effect ends as `recovery_required`.
|
|
83
|
+
|
|
84
|
+
## Reconciliation runbook
|
|
85
|
+
|
|
86
|
+
1. Stop deploys and every migration runner for the target database. Preserve
|
|
87
|
+
logs, the artifact repository, the database, and journal rows.
|
|
88
|
+
2. Run `qubu migrate status --format json --non-interactive`. Record the
|
|
89
|
+
interrupted attempt ID, artifact digest, checkpoints, journal head, pending
|
|
90
|
+
chain, managed drift, and incompatible requirements.
|
|
91
|
+
3. Find the exact artifact by digest. Verify the repository again; do not edit,
|
|
92
|
+
renumber, or reseal it to make the chain pass.
|
|
93
|
+
4. Inspect the live database through adapter/application-owned checks. Use
|
|
94
|
+
completed checkpoints only as evidence of where to inspect, not as proof
|
|
95
|
+
that a statement committed. Verify the artifact's preconditions,
|
|
96
|
+
postconditions, and expected snapshot.
|
|
97
|
+
5. Decide `applied` only if the application can prove the artifact's complete
|
|
98
|
+
post-state. Decide `rolled_back` only if it can prove the complete pre-state
|
|
99
|
+
and absence of all intended effects. If neither is provable, restore or
|
|
100
|
+
repair under an application-specific incident plan; do not guess.
|
|
101
|
+
6. Configure `verifyReconciliation` to repeat that proof, then run:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
qubu migrate reconcile <attempt-id> \
|
|
105
|
+
--outcome applied \
|
|
106
|
+
--reason "Verified every postcondition against incident INC-123" \
|
|
107
|
+
--format json --non-interactive
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Use `--outcome rolled_back` only for a proven pre-state. The reconciler
|
|
111
|
+
requires a non-empty reason and the application verifier to return true.
|
|
112
|
+
|
|
113
|
+
7. Run `migrate status` again. Confirm no interrupted attempt remains, the head
|
|
114
|
+
and applied history form the repository prefix, managed drift is zero, and
|
|
115
|
+
only the intended pending artifacts remain.
|
|
116
|
+
8. Resume with `migrate apply`. Never replay individual SQL statements from the
|
|
117
|
+
uncertain artifact.
|
|
118
|
+
|
|
119
|
+
When reconciliation records `applied`, Qubu appends the missing immutable
|
|
120
|
+
history/head for the exact artifact if needed. When it records `rolled_back`,
|
|
121
|
+
the head stays at the previous artifact. Both outcomes append an audit record.
|
|
@@ -151,7 +151,7 @@ The mapper can emit these facts in canonical Snapshot v1:
|
|
|
151
151
|
- stable logical IDs with physical names preserved separately.
|
|
152
152
|
|
|
153
153
|
Strict mode returns no snapshot when a supported table fact cannot be mapped
|
|
154
|
-
soundly. Lossy mode is explicit and marks warnings in the result. A
|
|
154
|
+
soundly. Lossy mode is explicit and marks warnings in the result. A fingerprint is
|
|
155
155
|
canonical content, not an identity or rename marker.
|
|
156
156
|
|
|
157
157
|
## Deferred and limited features
|