@monlite/core 1.0.0 → 1.2.0

package/README.md

@@ -389,8 +389,21 @@ definition supports `index`, `unique`, `notNull`, `default`, and `references`.
 
 **Migrations are automatic for additive changes.** Re-opening a collection with a
 new declared column adds it (`ALTER TABLE ADD COLUMN`) on declaration — give
-`NOT NULL` columns a `default` so existing rows can be backfilled. Destructive
-changes (rename/drop/type-change) still need a manual migration.
+`NOT NULL` columns a `default` so existing rows can be backfilled.
+
+For **destructive changes** — dropping, renaming, or changing a column's
+type/constraints — call `$migrate()`. It safely rebuilds the table (in a
+transaction, preserving data and indexes) to match the new declared schema:
+
+```ts
+// v2 schema: `fullname` → `name`, `age` is now INTEGER, `legacy` removed.
+const users = db.collection("users", { schema: { name: "TEXT", age: "INTEGER" } });
+await users.$migrate({ rename: { fullname: "name" }, drop: ["legacy"] });
+```
+
+A column that exists on disk but isn't in the new schema (and isn't listed in
+`drop`) throws — so you never lose data by accident. Run migrations at startup,
+before using the collection.
 
 ### Do I have to care: JSON vs native columns?
 
@@ -551,6 +564,40 @@ Write your own against the `MonlitePlugin` interface (`init` / `afterWrite` /
 
 ---
 
+## The local backend for AI agents
+
+monlite aims to be your **entire local data layer** — one embedded `.db`, one
+install — collapsing the services you'd otherwise run (Mongo, Qdrant, Redis) for
+a local/edge/desktop agent. Documents and vectors are core + a plugin; the
+Redis-style primitives are small companion packages:
+
+| Package | Replaces (locally) | Provides |
+|---|---|---|
+| [`@monlite/kv`](https://www.npmjs.com/package/@monlite/kv) | Redis cache | Synchronous `get/set/incr` KV with TTLs |
+| [`@monlite/queue`](https://www.npmjs.com/package/@monlite/queue) | Redis / BullMQ | Durable job queue — retries, backoff, delays, priorities, concurrency |
+| [`@monlite/cron`](https://www.npmjs.com/package/@monlite/cron) | cron / scheduler | Persisted cron schedules; composes with the queue |
+
+```ts
+import { kv } from "@monlite/kv";
+import { createQueue } from "@monlite/queue";
+import { createCron } from "@monlite/cron";
+
+const cache = kv(db);
+cache.set("session:42", { user: "ali" }, { ttl: 60_000 });
+
+const queue = createQueue(db, { maxAttempts: 3 });
+queue.process("email", async (job) => send(job.payload), { concurrency: 5 });
+queue.add("email", { to: "ali@example.com" });
+
+createCron(db).schedule("nightly", "0 0 * * *", () => queue.add("report", {}));
+```
+
+These target **local / edge / desktop** runtimes — not a distributed cloud-scale
+Redis/Mongo/Qdrant replacement. For scale, keep the real services and
+[`@monlite/sync`](https://www.npmjs.com/package/@monlite/sync) to them.
+
+---
+
 ## Drivers & zero dependencies
 
 monlite talks to SQLite through a tiny driver adapter, so it runs on two
@@ -578,6 +625,30 @@ based on your Node version and whether you want the extra dependency.
 
 ---
 
+## Encryption at rest
+
+Encrypt the whole database file with a key. Install the drop-in cipher driver
+and pass an `encryption` option:
+
+```bash
+npm install better-sqlite3-multiple-ciphers
+```
+
+```ts
+const db = createDb("./secure.db", { encryption: { key: process.env.DB_KEY } });
+// ...use db exactly as normal — everything on disk is encrypted.
+
+db.rekey(newKey); // rotate the key
+```
+
+- A **wrong or missing key throws `MonliteEncryptionError`** when opening.
+- Optional `cipher` selects the scheme (`"sqlcipher"`, `"chacha20"`,
+  `"aes256cbc"`, …); the default is ChaCha20-Poly1305.
+- Encryption requires `better-sqlite3-multiple-ciphers` (a drop-in for
+  `better-sqlite3`) and is **not** available on the `node:sqlite` backend.
+
+---
+
 ## How it works
 
 Every collection is a single SQLite table:
@@ -610,6 +681,22 @@ future-proofing.
 
 ---
 
+## Examples
+
+Runnable demos live in [`examples/`](examples/): a notes app (CRUD + full-text
+search + live queries), AI-agent memory (vector + hybrid search), and local-first
+sync. `cd examples && npm install && node notes.mjs`.
+
+## Benchmarks
+
+[`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) compares monlite to the raw SQLite
+driver, NeDB, and lowdb (`pnpm bench` to reproduce). In short: ~150k–250k
+ops/sec, roughly 2× the raw-driver overhead for the full document API, and it
+**stays flat on indexed reads where JSON-file stores degrade** (lowdb point reads
+are ~15× slower at 10k docs).
+
+---
+
 ## License
 
 MIT 🌙

package/dist/index.cjs

@@ -119,6 +119,12 @@ var MonliteQueryError = class extends MonliteError {
     this.name = "MonliteQueryError";
   }
 };
+var MonliteEncryptionError = class extends MonliteError {
+  constructor(message, options) {
+    super(message, options);
+    this.name = "MonliteEncryptionError";
+  }
+};
 var MonliteConstraintError = class extends MonliteError {
   collection;
   constructor(message, options) {
@@ -787,6 +793,124 @@ var Collection = class {
       }
     }
   }
+  foreignKeysOn() {
+    try {
+      const row = this.db.prepare(`PRAGMA foreign_keys`).get();
+      return !!row?.foreign_keys;
+    } catch {
+      return true;
+    }
+  }
+  declaredIndexDdl() {
+    const out = [];
+    for (const field of this.columnOrder) {
+      if (this.columnDefs[field].index) {
+        out.push(
+          `CREATE INDEX IF NOT EXISTS "idx_${this.name}_${field}" ON "${this.name}"("${field}")`
+        );
+      }
+    }
+    return out;
+  }
+  /**
+   * Reconcile the physical table to the declared schema, performing the changes
+   * the auto-additive path can't: **dropping** columns, **renaming** them, and
+   * **changing a column's type/constraints** — via a safe, transactional table
+   * rebuild that preserves data. Structured collections only.
+   *
+   * Pass `rename` to map an existing physical column to a new declared name, and
+   * `drop` to acknowledge columns that the new schema removes (an unacknowledged
+   * column drop throws, so data is never lost by accident).
+   *
+   * ```ts
+   * const users = db.collection("users", { schema: { name: "TEXT", age: "INTEGER" } });
+   * await users.$migrate({ rename: { fullname: "name" }, drop: ["legacy"] });
+   * ```
+   */
+  async $migrate(options = {}) {
+    if (this.mode !== "structured") {
+      throw new MonliteError(
+        `$migrate() is only available on structured collections (declare a schema).`
+      );
+    }
+    this.ensureTable();
+    const rename = options.rename ?? {};
+    const drop = new Set(options.drop ?? []);
+    const SYSTEM = /* @__PURE__ */ new Set(["_id", "created_at", "updated_at", "data"]);
+    const physical = this.db.prepare(`PRAGMA table_info("${this.name}")`).all().map((r) => r.name).filter((n) => !SYSTEM.has(n));
+    const physicalSet = new Set(physical);
+    const targetSet = new Set(this.columnOrder);
+    const renamedFrom = {};
+    for (const [from, to] of Object.entries(rename)) {
+      if (!physicalSet.has(from)) {
+        throw new MonliteError(
+          `Cannot rename "${from}": no such column in "${this.name}".`
+        );
+      }
+      if (!targetSet.has(to)) {
+        throw new MonliteError(
+          `Cannot rename "${from}" to "${to}": "${to}" is not in the schema.`
+        );
+      }
+      renamedFrom[to] = from;
+    }
+    const renameSources = new Set(Object.keys(rename));
+    for (const col of physical) {
+      if (targetSet.has(col) || renameSources.has(col) || drop.has(col))
+        continue;
+      throw new MonliteError(
+        `Column "${col}" exists in "${this.name}" but isn't in the schema. Add it to the schema, or list it in \`drop\` to remove it.`
+      );
+    }
+    const tmp = `__mon_migrate_${this.name}`;
+    const newCols = [
+      `_id        TEXT    PRIMARY KEY`,
+      `created_at INTEGER NOT NULL`,
+      `updated_at INTEGER NOT NULL`,
+      `data       TEXT    NOT NULL DEFAULT '{}'`,
+      ...this.columnOrder.map((f) => this.columnDdl(f, false))
+    ];
+    const destCols = [
+      "_id",
+      "created_at",
+      "updated_at",
+      "data",
+      ...this.columnOrder.map((f) => `"${f}"`)
+    ].join(", ");
+    const srcCols = [
+      "_id",
+      "created_at",
+      "updated_at",
+      "data",
+      ...this.columnOrder.map((t) => {
+        const src = renamedFrom[t] ?? (physicalSet.has(t) ? t : null);
+        return src ? `"${src}"` : "NULL";
+      })
+    ].join(", ");
+    const fkOn = this.foreignKeysOn();
+    if (fkOn) this.db.exec(`PRAGMA foreign_keys = OFF`);
+    try {
+      this.guard(
+        () => this.db.transaction(() => {
+          this.db.exec(`DROP TABLE IF EXISTS "${tmp}"`);
+          this.db.exec(
+            `CREATE TABLE "${tmp}" (
+  ${newCols.join(",\n  ")}
+)`
+          );
+          this.db.exec(
+            `INSERT INTO "${tmp}" (${destCols}) SELECT ${srcCols} FROM "${this.name}"`
+          );
+          this.db.exec(`DROP TABLE "${this.name}"`);
+          this.db.exec(`ALTER TABLE "${tmp}" RENAME TO "${this.name}"`);
+          for (const ddl of this.declaredIndexDdl()) this.db.exec(ddl);
+        })
+      );
+    } finally {
+      if (fkOn) this.db.exec(`PRAGMA foreign_keys = ON`);
+    }
+    this.insertSqlCache = void 0;
+  }
   /* --------------------------- row <-> doc -------------------------- */
   rowToDoc(row) {
     const doc = this.mode === "document" ? JSON.parse(row.data) : JSON.parse(row.data ?? "{}");
@@ -1317,6 +1441,7 @@ var AutoIndexer = class {
 
 // src/driver/better-sqlite3.ts
 var STMT_CACHE_MAX = 256;
+var quote = (s) => s.replace(/'/g, "''");
 var BetterSqlite3Driver = class {
   name = "better-sqlite3";
   raw;
@@ -1327,6 +1452,9 @@ var BetterSqlite3Driver = class {
     this.raw = new BetterSqlite3(filename, {
       readonly: options.readonly ?? false
     });
+    if (options.encryption) {
+      this.applyKey(options.encryption.key, options.encryption.cipher);
+    }
     this.raw.pragma("foreign_keys = ON");
     this.raw.pragma(`busy_timeout = ${options.busyTimeout ?? 5e3}`);
     if (!options.readonly && (options.wal ?? true)) {
@@ -1355,6 +1483,28 @@ var BetterSqlite3Driver = class {
   transaction(fn) {
     return this.raw.transaction(fn)();
   }
+  /** Apply the encryption key and verify it by reading the schema. */
+  applyKey(key, cipher) {
+    if (cipher) this.raw.pragma(`cipher='${quote(cipher)}'`);
+    this.raw.pragma(`key='${quote(key)}'`);
+    try {
+      this.raw.exec("SELECT count(*) FROM sqlite_master");
+    } catch (err) {
+      this.raw.close();
+      throw new MonliteEncryptionError(
+        "Failed to open the encrypted database: the key is incorrect, or the file is not encrypted.",
+        { cause: err }
+      );
+    }
+  }
+  rekey(key, cipher) {
+    if (cipher) this.raw.pragma(`cipher='${quote(cipher)}'`);
+    const mode = String(this.raw.pragma("journal_mode", { simple: true }));
+    const wasWal = mode.toLowerCase() === "wal";
+    if (wasWal) this.raw.pragma("journal_mode = DELETE");
+    this.raw.pragma(`rekey='${quote(key)}'`);
+    if (wasWal) this.raw.pragma("journal_mode = WAL");
+  }
   close() {
     this.cache.clear();
     this.raw.close();
@@ -1370,6 +1520,11 @@ var NodeSqliteDriver = class {
   cache = /* @__PURE__ */ new Map();
   depth = 0;
   constructor(nodeSqlite, filename, options) {
+    if (options.encryption) {
+      throw new MonliteError(
+        "Encryption is not supported on the node:sqlite backend. Use better-sqlite3 with the better-sqlite3-multiple-ciphers package."
+      );
+    }
     this.verbose = options.verbose;
     const { DatabaseSync } = nodeSqlite;
     this.raw = new DatabaseSync(filename, {
@@ -1445,6 +1600,14 @@ function loadBetterSqlite3() {
     return null;
   }
 }
+function loadCipherSqlite3() {
+  try {
+    const mod = req("better-sqlite3-multiple-ciphers");
+    return mod?.default ?? mod;
+  } catch {
+    return null;
+  }
+}
 function loadNodeSqlite() {
   try {
     return req("node:sqlite");
@@ -1454,6 +1617,20 @@ function loadNodeSqlite() {
 }
 function createDriver(filename, options = {}) {
   const choice = options.driver ?? "auto";
+  if (options.encryption) {
+    if (choice === "node:sqlite") {
+      throw new MonliteError(
+        `Encryption is not supported on the node:sqlite backend. Use better-sqlite3 with the better-sqlite3-multiple-ciphers package.`
+      );
+    }
+    const cipher = loadCipherSqlite3();
+    if (!cipher) {
+      throw new MonliteError(
+        `Encryption requires the "better-sqlite3-multiple-ciphers" package (a drop-in for better-sqlite3). Run \`npm install better-sqlite3-multiple-ciphers\`.`
+      );
+    }
+    return new BetterSqlite3Driver(cipher, filename, options);
+  }
   if (choice === "better-sqlite3") {
     const mod = loadBetterSqlite3();
     if (!mod) {
@@ -1893,6 +2070,7 @@ var Monlite = class {
   $sync;
   collections = /* @__PURE__ */ new Map();
   plugins;
+  encrypted;
   closed = false;
   constructor(filename, options = {}) {
     this.driver = createDriver(filename, {
@@ -1901,8 +2079,10 @@ var Monlite = class {
       wal: options.wal,
       busyTimeout: options.busyTimeout,
       allowExtensions: options.allowExtensions,
+      encryption: options.encryption,
       verbose: options.verbose
     });
+    this.encrypted = options.encryption !== void 0;
     this.autoIndexer = new AutoIndexer(
       this.driver,
       options.autoIndex ?? true,
@@ -2050,6 +2230,19 @@ var Monlite = class {
     this.driver.exec(`VACUUM INTO '${path.replace(/'/g, "''")}'`);
     return Promise.resolve();
   }
+  /**
+   * Rotate the encryption key. Only valid for a database opened with the
+   * `encryption` option; throws otherwise. Pass `cipher` to also change scheme.
+   */
+  rekey(key, cipher) {
+    this.assertOpen();
+    if (!this.encrypted || !this.driver.rekey) {
+      throw new MonliteError(
+        "rekey() requires a database opened with the `encryption` option."
+      );
+    }
+    this.driver.rekey(key, cipher);
+  }
   /** Close the underlying SQLite connection. */
   $disconnect() {
     if (!this.closed) {
@@ -2069,6 +2262,7 @@ function createDb(filename, options) {
 exports.Collection = Collection;
 exports.Monlite = Monlite;
 exports.MonliteConstraintError = MonliteConstraintError;
+exports.MonliteEncryptionError = MonliteEncryptionError;
 exports.MonliteError = MonliteError;
 exports.MonliteForeignKeyError = MonliteForeignKeyError;
 exports.MonliteNotNullError = MonliteNotNullError;