@zukhruf/sqlite-drift 0.0.0-stage → 0.3.30

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ezzabuzaid
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,108 @@
-# Temporary Holding Version
+# @zukhruf/sqlite-drift
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+A package keeps tables, indexes, views and triggers in a SQLite database. A later version of the package can declare them otherwise, for example with one more column. The database still stores the objects as the earlier version made them, and the new code fails on them, for example with `no such column`. This package compares the stored objects with their declaration, and makes the database store them as declared. No person keeps a version number. This is also known as declarative schema management, and the difference that it finds is known as schema drift.
+
+The words in these documents have one meaning each. See the glossary in [CONTEXT.md](./CONTEXT.md).
+
+## Use it
+
+This project is an experiment. It uses only `node:sqlite`.
+
+```sh
+npm install @zukhruf/sqlite-drift
+```
+
+```ts
+import { mkdtempDisposable } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { DatabaseSync } from 'node:sqlite';
+
+import { Declaration } from '@zukhruf/sqlite-drift';
+
+await using scratch = await mkdtempDisposable(join(tmpdir(), 'drift-'));
+using database = new DatabaseSync(join(scratch.path, 'app.sqlite'));
+// An earlier version made the table without failed_at.
+database.exec(`CREATE TABLE failures (directory TEXT PRIMARY KEY, error TEXT NOT NULL);
+  INSERT INTO failures VALUES ('/data/notes', 'denied')`);
+
+const failures = new Declaration([
+  'CREATE TABLE failures (directory TEXT PRIMARY KEY, error TEXT NOT NULL, failed_at TEXT NOT NULL)',
+]);
+console.log(failures.drift(database));
+failures.apply(database);
+console.log(failures.drift(database));
+console.log(database.prepare('SELECT count(*) AS rows FROM failures').get());
+```
+
+Output:
+
+```
+stale
+fits
+[Object: null prototype] { rows: 0 }
+```
+
+## A declaration compares the text that SQLite stores
+
+- `new Declaration(statements)` makes the objects once in an empty database in memory, and keeps the text that SQLite stores for each one. [ADR 0001](./docs/adr/0001-compare-the-text-that-sqlite-stores.md) tells why.
+- `drift(database)` gives `missing` when the database stores none of the objects, `fits` when it stores each one as declared, and `stale` for all other cases. It only reads, so it also works on a database opened read-only.
+- Each change to a CREATE statement is drift: a column, a type, a CHECK, STRICT, the query of a view.
+- A change of meaning with no change of text is not drift. When the meaning of a column changes, change its name too.
+- SQLite takes names that are different only in ASCII case as one name.
+
+## Apply makes the database store the declaration
+
+- `apply(database)` does nothing when the drift is `fits`.
+- When no stored object is stale, `apply` makes each missing object, and the other objects keep their rows.
+- When one stored object is stale, `apply` replaces every object of the declaration, and their rows are gone. [ADR 0002](./docs/adr/0002-a-stale-object-replaces-the-whole-declaration.md) tells why.
+- `apply` does not drop an object that the declaration does not name.
+- In a transaction of the caller, `apply` works in a savepoint, and a rollback of the caller removes its changes. With no transaction, `apply` writes in its own `BEGIN IMMEDIATE` transaction.
+- A caller that must keep the rows of a stale table asks `drift`, and changes the table itself.
+
+## Write a declaration
+
+- List a table before the tables that refer to it.
+- A declaration must stand alone in an empty database. Do not declare a trigger or an index on a table that the declaration does not name, and do not use a function that only your connection registers.
+- Give each table its own declaration when it must keep its rows while another table changes.
+
+## Errors
+
+| Error                    | When                                                                                  |
+| ------------------------ | ------------------------------------------------------------------------------------- |
+| The error of SQLite      | `new Declaration` fails when a statement does not run in an empty database.           |
+| `database is locked`     | `apply` with no transaction waits for the busy timeout of the connection, then fails. |
+| The error of a statement | `apply` fails, and the database stays as it was.                                      |
+
+## The same concepts in other systems
+
+| System                               | What it does                                                                                                                              |
+| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
+| Röthlisberger's SQLite migrator      | It makes the declaration in memory and compares `sqlite_schema`. It changes a stale table with the 12 steps of SQLite and keeps the rows. |
+| Airbyte destinations                 | Each sync compares the final table with the stream (`existingSchemaMatchesStreamConfig`), and a soft reset makes a stale table again.     |
+| Drizzle Kit `push`, Prisma `db push` | They compare a declaration in TypeScript or in the Prisma language with the database, and change the database.                            |
+| Android Room                         | It keeps a version number for migrations, and a hash of the declaration that finds a number that a person forgot to raise.                |
+| Kubernetes controllers               | Each loop compares the desired state with the state that exists, and changes what is different.                                           |
+
+## Documentation
+
+- [0001 Compare the text that SQLite stores](./docs/adr/0001-compare-the-text-that-sqlite-stores.md)
+- [0002 A stale object replaces the whole declaration](./docs/adr/0002-a-stale-object-replaces-the-whole-declaration.md)
+
+## Development
+
+This package is part of the [zukhruf](../../../README.md) workspace. Run the commands from the workspace root.
+
+```sh
+npm install
+npx nx run sqlite-drift:test        # builds, then runs the tests in src/
+npx nx run sqlite-drift:typecheck   # formats, lints, then type checks
+npx nx run sqlite-drift:build       # compiles src/ to dist/
+```
+
+```
+src/
+  index.ts               the exports
+  declaration.ts         Declaration and Drift
+  declaration.test.ts    the tests, on real SQLite files
+```

package/dist/declaration.d.ts

@@ -0,0 +1,9 @@
+import { DatabaseSync } from 'node:sqlite';
+export type Drift = 'missing' | 'stale' | 'fits';
+export declare class Declaration {
+    #private;
+    constructor(statements: readonly string[]);
+    drift(database: DatabaseSync): Drift;
+    apply(database: DatabaseSync): void;
+}
+//# sourceMappingURL=declaration.d.ts.map

package/dist/declaration.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"declaration.d.ts","sourceRoot":"","sources":["../src/declaration.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAI3C,MAAM,MAAM,KAAK,GAAG,SAAS,GAAG,OAAO,GAAG,MAAM,CAAC;AAajD,qBAAa,WAAW;;gBAGV,UAAU,EAAE,SAAS,MAAM,EAAE;IAezC,KAAK,CAAC,QAAQ,EAAE,YAAY,GAAG,KAAK;IAgBpC,KAAK,CAAC,QAAQ,EAAE,YAAY,GAAG,IAAI;CAiCpC"}

package/dist/declaration.js

@@ -0,0 +1,83 @@
+import { DatabaseSync } from 'node:sqlite';
+// The tables, indexes, views and triggers a caller owns in a SQLite database,
+// by the CREATE statements that make them. SQLite keeps the text of each
+// CREATE in sqlite_schema. A declaration makes its objects once in an empty
+// database in memory, so the declared text and the stored text both come from
+// SQLite, and they compare as they are.
+export class Declaration {
+    #objects;
+    constructor(statements) {
+        using blank = new DatabaseSync(':memory:');
+        for (const statement of statements)
+            blank.exec(statement);
+        this.#objects = blank
+            .prepare('SELECT "type", "name", "sql" FROM sqlite_schema WHERE "sql" IS NOT NULL ORDER BY rowid')
+            .all()
+            .map(({ type, name, sql }) => ({
+            type: String(type),
+            name: String(name),
+            sql: String(sql),
+        }));
+    }
+    drift(database) {
+        const pairs = this.#objects.map((object) => ({
+            object,
+            found: stored(database, object.name),
+        }));
+        if (pairs.every(({ found }) => found === undefined))
+            return 'missing';
+        return pairs.every(({ object, found }) => found?.sql === object.sql)
+            ? 'fits'
+            : 'stale';
+    }
+    // Creates the objects the database does not store. When any stored object
+    // differs from its declaration, replaces every object of the declaration,
+    // with its rows: a table can refer to another, and SQLite drops a table only
+    // after nothing refers to its rows. Works inside the caller's transaction,
+    // or in a write of its own. Objects the declaration does not name stay.
+    apply(database) {
+        if (this.drift(database) === 'fits')
+            return;
+        const ownsTransaction = !database.isTransaction;
+        database.exec(ownsTransaction ? 'BEGIN IMMEDIATE' : 'SAVEPOINT sqlite_drift');
+        try {
+            const stale = this.#objects.some((object) => {
+                const found = stored(database, object.name);
+                return found !== undefined && found.sql !== object.sql;
+            });
+            if (stale)
+                for (const object of this.#objects.toReversed()) {
+                    const found = stored(database, object.name);
+                    if (found !== undefined)
+                        database.exec(`DROP ${found.type.toUpperCase()} ${quote(found.name)}`);
+                }
+            for (const object of this.#objects)
+                if (stored(database, object.name) === undefined)
+                    database.exec(object.sql);
+            database.exec(ownsTransaction ? 'COMMIT' : 'RELEASE sqlite_drift');
+        }
+        catch (error) {
+            if (!ownsTransaction) {
+                database.exec('ROLLBACK TO sqlite_drift');
+                database.exec('RELEASE sqlite_drift');
+            }
+            else if (database.isTransaction) {
+                database.exec('ROLLBACK');
+            }
+            throw error;
+        }
+    }
+}
+// SQLite takes names that differ only in ASCII case for one object.
+function stored(database, name) {
+    const row = database
+        .prepare('SELECT "type", "name", "sql" FROM sqlite_schema WHERE lower("name") = lower(?)')
+        .get(name);
+    return row === undefined
+        ? undefined
+        : { type: String(row.type), name: String(row.name), sql: String(row.sql) };
+}
+function quote(name) {
+    return `"${name.replaceAll('"', '""')}"`;
+}
+//# sourceMappingURL=declaration.js.map

package/dist/declaration.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"declaration.js","sourceRoot":"","sources":["../src/declaration.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAY3C,8EAA8E;AAC9E,yEAAyE;AACzE,4EAA4E;AAC5E,8EAA8E;AAC9E,wCAAwC;AACxC,MAAM,OAAO,WAAW;IACb,QAAQ,CAA0B;IAE3C,YAAY,UAA6B;QACvC,MAAM,KAAK,GAAG,IAAI,YAAY,CAAC,UAAU,CAAC,CAAC;QAC3C,KAAK,MAAM,SAAS,IAAI,UAAU;YAAE,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QAC1D,IAAI,CAAC,QAAQ,GAAG,KAAK;aAClB,OAAO,CACN,wFAAwF,CACzF;aACA,GAAG,EAAE;aACL,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,IAAI,EAAE,GAAG,EAAE,EAAE,EAAE,CAAC,CAAC;YAC7B,IAAI,EAAE,MAAM,CAAC,IAAI,CAAC;YAClB,IAAI,EAAE,MAAM,CAAC,IAAI,CAAC;YAClB,GAAG,EAAE,MAAM,CAAC,GAAG,CAAC;SACjB,CAAC,CAAC,CAAC;IACR,CAAC;IAED,KAAK,CAAC,QAAsB;QAC1B,MAAM,KAAK,GAAG,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;YAC3C,MAAM;YACN,KAAK,EAAE,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC;SACrC,CAAC,CAAC,CAAC;QACJ,IAAI,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,CAAC,KAAK,KAAK,SAAS,CAAC;YAAE,OAAO,SAAS,CAAC;QACtE,OAAO,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,MAAM,EAAE,KAAK,EAAE,EAAE,EAAE,CAAC,KAAK,EAAE,GAAG,KAAK,MAAM,CAAC,GAAG,CAAC;YAClE,CAAC,CAAC,MAAM;YACR,CAAC,CAAC,OAAO,CAAC;IACd,CAAC;IAED,0EAA0E;IAC1E,0EAA0E;IAC1E,6EAA6E;IAC7E,2EAA2E;IAC3E,wEAAwE;IACxE,KAAK,CAAC,QAAsB;QAC1B,IAAI,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,KAAK,MAAM;YAAE,OAAO;QAC5C,MAAM,eAAe,GAAG,CAAC,QAAQ,CAAC,aAAa,CAAC;QAChD,QAAQ,CAAC,IAAI,CACX,eAAe,CAAC,CAAC,CAAC,iBAAiB,CAAC,CAAC,CAAC,wBAAwB,CAC/D,CAAC;QACF,IAAI,CAAC;YACH,MAAM,KAAK,GAAG,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,EAAE;gBAC1C,MAAM,KAAK,GAAG,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC;gBAC5C,OAAO,KAAK,KAAK,SAAS,IAAI,KAAK,CAAC,GAAG,KAAK,MAAM,CAAC,GAAG,CAAC;YACzD,CAAC,CAAC,CAAC;YACH,IAAI,KAAK;gBACP,KAAK,MAAM,MAAM,IAAI,IAAI,CAAC,QAAQ,CAAC,UAAU,EAAE,EAAE,CAAC;oBAChD,MAAM,KAAK,GAAG,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC;oBAC5C,IAAI,KAAK,KAAK,SAAS;wBACrB,QAAQ,CAAC,IAAI,CACX,QAAQ,KAAK,CAAC,IAAI,CAAC,WAAW,EAAE,IAAI,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CACxD,CAAC;gBACN,CAAC;YACH,KAAK,MAAM,MAAM,IAAI,IAAI,CAAC,QAAQ;gBAChC,IAAI,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,SAAS;oBAC7C,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;YAC9B,QAAQ,CAAC,IAAI,CAAC,eAAe,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,sBAAsB,CAAC,CAAC;QACrE,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,IAAI,CAAC,eAAe,EAAE,CAAC;gBACrB,QAAQ,CAAC,IAAI,CAAC,0BAA0B,CAAC,CAAC;gBAC1C,QAAQ,CAAC,IAAI,CAAC,sBAAsB,CAAC,CAAC;YACxC,CAAC;iBAAM,IAAI,QAAQ,CAAC,aAAa,EAAE,CAAC;gBAClC,QAAQ,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;YAC5B,CAAC;YACD,MAAM,KAAK,CAAC;QACd,CAAC;IACH,CAAC;CACF;AAED,oEAAoE;AACpE,SAAS,MAAM,CACb,QAAsB,EACtB,IAAY;IAEZ,MAAM,GAAG,GAAG,QAAQ;SACjB,OAAO,CACN,gFAAgF,CACjF;SACA,GAAG,CAAC,IAAI,CAAC,CAAC;IACb,OAAO,GAAG,KAAK,SAAS;QACtB,CAAC,CAAC,SAAS;QACX,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAC;AAC/E,CAAC;AAED,SAAS,KAAK,CAAC,IAAY;IACzB,OAAO,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC;AAC3C,CAAC"}

package/dist/index.d.ts

@@ -0,0 +1,2 @@
+export { Declaration, type Drift } from './declaration.ts';
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,KAAK,KAAK,EAAE,MAAM,kBAAkB,CAAC"}

package/dist/index.js

@@ -0,0 +1,2 @@
+export { Declaration } from "./declaration.js";
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAc,MAAM,kBAAkB,CAAC"}

package/package.json

@@ -1,6 +1,44 @@
 {
   "name": "@zukhruf/sqlite-drift",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.3.30",
+  "description": "Declarative schema management for SQLite. A declaration lists the CREATE statements of the tables, indexes, views and triggers that one caller owns. The package compares them with the text that SQLite stores, gives the drift (missing, fits or stale), and makes the database store each object as declared. No person keeps a version number.",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/JanuaryLabs/zukhruf",
+    "directory": "packages/sqlite/drift"
+  },
+  "bugs": {
+    "url": "https://github.com/JanuaryLabs/zukhruf/issues"
+  },
+  "homepage": "https://github.com/JanuaryLabs/zukhruf/tree/main/packages/sqlite/drift#readme",
+  "keywords": [
+    "sqlite",
+    "node-sqlite",
+    "schema",
+    "schema-drift",
+    "declarative-schema",
+    "migration",
+    "sqlite-schema",
+    "ddl"
+  ],
+  "type": "module",
+  "exports": {
+    "./package.json": "./package.json",
+    ".": {
+      "@zukhruf/source": "./src/index.ts",
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js",
+      "default": "./dist/index.js"
+    }
+  },
+  "files": [
+    "dist",
+    "!dist/spec/**",
+    "!**/*.tsbuildinfo",
+    "!**/*.test.*"
+  ],
+  "publishConfig": {
+    "access": "public"
+  }
+}