picovolt 1.6.0 → 1.7.1

package/README.md

@@ -1,15 +1,15 @@
 # PicoVolt (PVDB)
 
 [![CI](https://github.com/MiniJe/picovolt/actions/workflows/ci.yml/badge.svg)](https://github.com/MiniJe/picovolt/actions/workflows/ci.yml)
-[![Version](https://img.shields.io/badge/version-1.6.0-blue.svg)](CHANGELOG.md)
+[![crates.io](https://img.shields.io/crates/v/picovolt.svg)](https://crates.io/crates/picovolt)
 [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
-![Status: 1.0 stable](https://img.shields.io/badge/status-1.0%20stable-brightgreen.svg)
+![Status: stable 1.x](https://img.shields.io/badge/status-stable%201.x-brightgreen.svg)
 [![GitHub stars](https://img.shields.io/github/stars/MiniJe/picovolt?style=social)](https://github.com/MiniJe/picovolt)
 
-PicoVolt is an embedded database engine written from scratch in Rust. As of 1.0
-its public API and on-disk format are stable under Semantic Versioning. It is
-young software and has not had an external security audit, so review it and keep
-backups before trusting it with data you cannot regenerate.
+PicoVolt is an embedded database engine written in Rust. Its 1.x public API and
+on-disk format are stable under Semantic Versioning. It is young software and
+has not had an external security audit, so review it and keep backups before
+trusting it with data you cannot regenerate.
 
 If PicoVolt is useful to you, consider starring the repository on GitHub. It is
 the simplest way to help others discover the project.
@@ -18,26 +18,20 @@ The engine decouples query logic from storage representation through a
 Virtualization Layer Engine (VLE) that shifts between two on-disk shapes:
 
 - **Development mode:** a `.pv/` workspace of mutable, append-only chunk files
-  plus a content-addressed blob store, friendly to git and code review.
+  plus a content-addressed blob store and inspectable manifest.
 - **Production mode:** a single contiguous, memory-mappable `.pvdb` file produced
   by `pv_bake()`.
 
-Pages are chameleon. Hot data lands in a slotted row layout for O(1) appends, and
-idle pages can be transposed into a packed columnar layout for compression and
-cache efficiency.
+New records use a slotted row layout for O(1) appends. Idle pages can be
+transposed into a packed columnar layout for compression and cache efficiency.
 
 ## Status
 
-The engine is built out across four phases, all implemented, with over 180 unit and
-integration tests plus doctests passing and a clean `cargo clippy -D warnings` on
-Linux and Windows. Changes are tracked in [CHANGELOG.md](CHANGELOG.md).
-
-| Phase | Scope | Status |
-|-------|-------|--------|
-| 1 | Core memory layouts and error taxonomy | Done |
-| 2 | Page engine, CAS dedup, compression, VLE router | Done |
-| 3 | MVCC and snapshot isolation, WASM runtime | Done |
-| 4 | Public surface (`pv_open_dev` / `pv_open_prod` / `query` / `pv_bake`) | Done |
+The current stable release is exercised by a 240+ test Rust suite plus doctests
+and maintained-binding integration tests. CI also enforces formatting and
+warning-free Clippy builds on Linux and Windows. Shipped changes are tracked in
+[CHANGELOG.md](CHANGELOG.md), and the remaining work toward 2.0 is tracked in
+[ROADMAP.md](ROADMAP.md).
 
 ### Module map
 
@@ -50,7 +44,7 @@ Linux and Windows. Changes are tracked in [CHANGELOG.md](CHANGELOG.md).
 | [`storage/cache.rs`](src/storage/cache.rs) | bounded LRU buffer pool (enables larger-than-RAM reads) |
 | [`storage/cas.rs`](src/storage/cas.rs) | BLAKE3 content-addressable dedup (memory, dev-files, mmap) |
 | [`storage/compress.rs`](src/storage/compress.rs) | Delta-Z, LEB128 varints, dictionary bit-packing |
-| [`storage/index.rs`](src/storage/index.rs) | in-memory ordered secondary index (value to record addresses; point and range) |
+| [`storage/index.rs`](src/storage/index.rs) | ordered secondary-index query structure and its persisted value/address encoding (point and range) |
 | [`storage/record.rs`](src/storage/record.rs) | row and record-body serialization with CAS interception |
 | [`storage/vle.rs`](src/storage/vle.rs) | dev directory store, owned prod snapshot, streamed reads, `bake` |
 | [`engine/mvcc.rs`](src/engine/mvcc.rs) | transaction clock and snapshot visibility |
@@ -122,33 +116,32 @@ Install the first-class CLI with `cargo install picovolt`, then use `pv query`,
 Rust, Python, Go, Node, and browser projects are in [`starters/`](starters/README.md); supported adapters
 are catalogued in [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md).
 
-SQL supported: `CREATE TABLE [IF NOT EXISTS]` with `PRIMARY KEY`, `UNIQUE`, and `NOT NULL`,
-`CREATE [UNIQUE] INDEX ON t (col)`, single- and multi-row `INSERT`,
-`UPDATE ... SET ... WHERE`, `DELETE ... WHERE`, `DROP TABLE [IF EXISTS]`, and
-`SELECT [DISTINCT] {* | col [AS alias], ... | COUNT/SUM/MIN/MAX/AVG(...) [AS alias]}
-FROM t [WHERE <pred>] [GROUP BY cols] [HAVING <pred>] [BEFORE tx]
-[ORDER BY col [ASC|DESC], ...] [LIMIT n] [OFFSET n]`, where `<pred>` combines
-`col <op> value` (`=`, `!=`, `<`, `<=`, `>`, `>=`, `LIKE`, `NOT LIKE`),
-`col [NOT] IN (...)`, `col [NOT] BETWEEN a AND b`, and `col IS [NOT] NULL` with
-`AND`, `OR`, and parentheses. Integer and decimal values compare by magnitude.
-Two-table equality `INNER`/`LEFT JOIN` queries support qualified references,
-projection, aliases, `DISTINCT`, filters, ordering, and pagination. Rust callers
-can cache `Database::prepare(...)` templates and use explicit transaction
-lifecycle methods or atomic `Database::transaction(...)` closures with both
-filesystem and in-memory databases.
+SQL supports the normal PicoVolt CRUD and schema statements plus projection,
+filters, aggregates, grouping, time travel, ordering, and pagination. The
+current query surface includes `AS`/bare table aliases, N-table equality
+`INNER`/`LEFT` joins, searched `CASE WHEN`, and the focused `LOWER`, `UPPER`,
+`TRIM`, `LENGTH`, `ABS`, `COALESCE`, and `NULLIF` scalar functions. Schema-light
+types, literal defaults, named inserts, and persisted `CHECK` constraints cover
+common adapter DDL. See the precise syntax, examples, type behavior, and
+deliberate limits in
+[`docs/SQL.md`](docs/SQL.md). Rust callers can cache `Database::prepare(...)`
+templates; C, WebAssembly, JavaScript, Python, and Go expose the same reusable
+prepared-statement lifecycle. Callers can use explicit transactions or atomic
+`Database::transaction(...)` closures with both filesystem and in-memory
+databases.
 Durability is selectable via `Database::set_durability` (`Fast` OS-cache default,
 or crash-safe `Sync` with fsync and an atomic manifest).
 
 Measured results and the methodology are in [BENCHMARKS.md](BENCHMARKS.md). In
-short, PicoVolt is a page-backed engine with O(1) durable appends (autocommit
+short, PicoVolt is a page-backed engine with O(1) filesystem appends (autocommit
 around 33k rows/s, linear), larger-than-RAM reads through a bounded buffer pool (a
 667-page dataset serves from a 16-page pool), ordered secondary indexes (point
-lookups roughly 11,000 times faster than a scan, plus range predicates), MVCC
+lookups roughly 6,100 times faster than a scan, plus range predicates), MVCC
 time-travel, opt-in crash-safe durability (`Durability::Sync`), and a fast
 compile-and-publish path (CAS dedup, columnar compression, memory-mappable
 single-file artifacts). Current limits include full-workspace transaction
-backups rather than an incremental WAL, only basic two-table equality joins,
-and no concurrent writers.
+backups rather than an incremental WAL, left-deep equality joins rather than a
+general SQL planner, and no concurrent writers.
 
 ## Install and distribution
 
@@ -157,7 +150,8 @@ and no concurrent writers.
 | **Rust** (crates.io) | `cargo add picovolt` |
 | **JavaScript / npm** (WebAssembly, browser and Node) | `npm install picovolt` |
 | **Python** (native wheels) | `python -m pip install picovolt` |
-| **C / Go** (native, via the C ABI) | `cargo build --release --features capi`, then see [`bindings/`](bindings) |
+| **Go** (`database/sql` and direct API) | `go get github.com/MiniJe/picovolt/bindings/go@latest`, then provide the matching native C ABI library described in [`bindings/go/`](bindings/go) |
+| **C** | Download the matching `picovolt-capi-*` bundle from the [latest release](https://github.com/MiniJe/picovolt/releases/latest), or run `cargo build --release --features capi` |
 | **In-memory** (native, no filesystem) | `Database::open_memory()`, export with `bake_to_bytes()` |
 
 PicoVolt runs in the browser through its in-memory backend plus an OPFS persistence
@@ -174,13 +168,14 @@ binding. The bindings suit embedded use, not a concurrent server's primary store
 
 All bindings accept positional `?` parameters
 (`db.query("... WHERE id = ?", [1])`), bound as safely-escaped SQL literals. For
-a familiar surface, drop-in adapters are provided: a `better-sqlite3`-style
-JavaScript API (`import Database from "picovolt/sqlite"`), a Python DB-API 2.0
-module (`import picovolt.dbapi2 as sqlite`), and the Go `database/sql` driver
-([`bindings/go/pvsql`](bindings/go/pvsql)). Shared limits include positional `?`
-only and the intentionally compact SQL grammar; JavaScript and in-memory Rust
-also expose rollback-capable transaction wrappers. Native bindings expose the
-same transaction lifecycle through the C ABI.
+a familiar surface, PicoVolt provides a `better-sqlite3`-inspired JavaScript API
+(`import Database from "picovolt/sqlite"`), a Python DB-API 2.0 module
+(`import picovolt.dbapi2 as sqlite`), and a Go `database/sql` driver
+([`bindings/go/pvsql`](bindings/go/pvsql)). These are interface adapters, not
+drop-in compatibility layers: shared limits include positional `?` only and the
+intentionally compact SQL grammar. JavaScript and in-memory Rust also expose
+rollback-capable transaction wrappers. Native bindings expose the same
+transaction lifecycle through the C ABI.
 
 ## Server mode
 

package/browser.js

@@ -1,10 +1,40 @@
 // Durable browser helper backed by the Origin Private File System (OPFS).
 import { Db } from "./picovolt.js";
 
+export class PersistentStatement {
+  constructor(database, source) {
+    this.database = database;
+    this.source = source;
+    this._prepared = database.db.prepare(source);
+    this.parameterCount = this._prepared.parameterCount;
+  }
+
+  query(params = []) {
+    if (!this._prepared) throw new Error("PicoVolt prepared statement is closed");
+    this.database._assertOpen();
+    return JSON.parse(this._prepared.execute(this.database.db, params));
+  }
+
+  close() {
+    if (!this._prepared) return false;
+    const prepared = this._prepared;
+    this._prepared = undefined;
+    this.database._statements.delete(this);
+    prepared.free();
+    return true;
+  }
+
+  finalize() {
+    return this.close();
+  }
+}
+
 export class PersistentDb {
   constructor(name, db) {
     this.name = name;
     this.db = db;
+    this._closed = false;
+    this._statements = new Set();
   }
 
   static async open(name = "picovolt.pvdb") {
@@ -20,11 +50,20 @@ export class PersistentDb {
   }
 
   query(sql, params) {
+    this._assertOpen();
     const json = params === undefined ? this.db.query(sql) : this.db.query(sql, params);
     return JSON.parse(json);
   }
 
+  prepare(sql) {
+    this._assertOpen();
+    const statement = new PersistentStatement(this, sql);
+    this._statements.add(statement);
+    return statement;
+  }
+
   async save() {
+    this._assertOpen();
     const root = await navigator.storage.getDirectory();
     const handle = await root.getFileHandle(this.name, { create: true });
     const writable = await handle.createWritable();
@@ -33,7 +72,15 @@ export class PersistentDb {
   }
 
   async close() {
+    if (this._closed) return;
     await this.save();
+    for (const statement of [...this._statements]) statement.close();
+    this.db.free();
+    this._closed = true;
+  }
+
+  _assertOpen() {
+    if (this._closed) throw new Error("PicoVolt database is closed");
   }
 }
 

package/package.json

@@ -1,12 +1,12 @@
 {
   "name": "picovolt",
   "type": "module",
-  "description": "PicoVolt (PVDB): a polymorphic embedded database engine in Rust.",
-  "version": "1.6.0",
+  "description": "Embedded SQL database with MVCC history and single-file deployment",
+  "version": "1.7.1",
   "license": "Apache-2.0",
   "repository": {
     "type": "git",
-    "url": "https://github.com/MiniJe/picovolt"
+    "url": "git+https://github.com/MiniJe/picovolt.git"
   },
   "files": [
     "picovolt_bg.wasm",
@@ -18,6 +18,7 @@
     "worker.js"
   ],
   "main": "picovolt.js",
+  "homepage": "https://github.com/MiniJe/picovolt",
   "types": "picovolt.d.ts",
   "sideEffects": [
     "./picovolt.js",

package/picovolt.d.ts

@@ -44,6 +44,12 @@ export class Db {
      * as queries touch them. `totalSize` is the image's byte length.
      */
     static openRemote(read: Function, total_size: number): Db;
+    /**
+     * Validate and retain a reusable SQL template. Preparation verifies the
+     * syntax and records the exact positional-parameter count without running
+     * the statement.
+     */
+    prepare(sql: string): PreparedStatement;
     /**
      * Run one SQL statement, optionally binding `?` placeholders to `params` (a
      * JS array, e.g. `db.query("SELECT * FROM t WHERE id = ?", [1])`). Returns a
@@ -62,3 +68,21 @@ export class Db {
      */
     tables(): string;
 }
+
+/**
+ * A validated, reusable SQL template for the raw WebAssembly API.
+ */
+export class PreparedStatement {
+    private constructor();
+    free(): void;
+    [Symbol.dispose](): void;
+    /**
+     * Execute this statement against `db`, returning the same JSON string as
+     * [`Db::query`]. The statement can be reused with different parameters.
+     */
+    execute(db: Db, params: any): string;
+    /**
+     * Number of positional `?` values required by this statement.
+     */
+    readonly parameterCount: number;
+}

package/picovolt.js

@@ -5,5 +5,5 @@ import { __wbg_set_wasm } from "./picovolt_bg.js";
 __wbg_set_wasm(wasm);
 wasm.__wbindgen_start();
 export {
-    Db
+    Db, PreparedStatement
 } from "./picovolt_bg.js";

package/picovolt_bg.js

@@ -106,6 +106,22 @@ export class Db {
         }
         return Db.__wrap(ret[0]);
     }
+    /**
+     * Validate and retain a reusable SQL template. Preparation verifies the
+     * syntax and records the exact positional-parameter count without running
+     * the statement.
+     * @param {string} sql
+     * @returns {PreparedStatement}
+     */
+    prepare(sql) {
+        const ptr0 = passStringToWasm0(sql, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
+        const len0 = WASM_VECTOR_LEN;
+        const ret = wasm.db_prepare(this.__wbg_ptr, ptr0, len0);
+        if (ret[2]) {
+            throw takeFromExternrefTable0(ret[1]);
+        }
+        return PreparedStatement.__wrap(ret[0]);
+    }
     /**
      * Run one SQL statement, optionally binding `?` placeholders to `params` (a
      * JS array, e.g. `db.query("SELECT * FROM t WHERE id = ?", [1])`). Returns a
@@ -170,6 +186,63 @@ export class Db {
     }
 }
 if (Symbol.dispose) Db.prototype[Symbol.dispose] = Db.prototype.free;
+
+/**
+ * A validated, reusable SQL template for the raw WebAssembly API.
+ */
+export class PreparedStatement {
+    static __wrap(ptr) {
+        const obj = Object.create(PreparedStatement.prototype);
+        obj.__wbg_ptr = ptr;
+        PreparedStatementFinalization.register(obj, obj.__wbg_ptr, obj);
+        return obj;
+    }
+    __destroy_into_raw() {
+        const ptr = this.__wbg_ptr;
+        this.__wbg_ptr = 0;
+        PreparedStatementFinalization.unregister(this);
+        return ptr;
+    }
+    free() {
+        const ptr = this.__destroy_into_raw();
+        wasm.__wbg_preparedstatement_free(ptr, 0);
+    }
+    /**
+     * Execute this statement against `db`, returning the same JSON string as
+     * [`Db::query`]. The statement can be reused with different parameters.
+     * @param {Db} db
+     * @param {any} params
+     * @returns {string}
+     */
+    execute(db, params) {
+        let deferred2_0;
+        let deferred2_1;
+        try {
+            _assertClass(db, Db);
+            const ret = wasm.preparedstatement_execute(this.__wbg_ptr, db.__wbg_ptr, params);
+            var ptr1 = ret[0];
+            var len1 = ret[1];
+            if (ret[3]) {
+                ptr1 = 0; len1 = 0;
+                throw takeFromExternrefTable0(ret[2]);
+            }
+            deferred2_0 = ptr1;
+            deferred2_1 = len1;
+            return getStringFromWasm0(ptr1, len1);
+        } finally {
+            wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
+        }
+    }
+    /**
+     * Number of positional `?` values required by this statement.
+     * @returns {number}
+     */
+    get parameterCount() {
+        const ret = wasm.preparedstatement_parameterCount(this.__wbg_ptr);
+        return ret >>> 0;
+    }
+}
+if (Symbol.dispose) PreparedStatement.prototype[Symbol.dispose] = PreparedStatement.prototype.free;
 export function __wbg___wbindgen_boolean_get_c9c83ebd41b34df3(arg0) {
     const v = arg0;
     const ret = typeof(v) === 'boolean' ? v : undefined;
@@ -240,6 +313,10 @@ export function __wbg_instanceof_Uint8Array_f935dbb0aa7cdeed(arg0) {
     const ret = result;
     return ret;
 }
+export function __wbg_isArray_6339f732981044bf(arg0) {
+    const ret = Array.isArray(arg0);
+    return ret;
+}
 export function __wbg_length_36bd29c6848c2144(arg0) {
     const ret = arg0.length;
     return ret;
@@ -284,6 +361,9 @@ export function __wbindgen_init_externref_table() {
 const DbFinalization = (typeof FinalizationRegistry === 'undefined')
     ? { register: () => {}, unregister: () => {} }
     : new FinalizationRegistry(ptr => wasm.__wbg_db_free(ptr, 1));
+const PreparedStatementFinalization = (typeof FinalizationRegistry === 'undefined')
+    ? { register: () => {}, unregister: () => {} }
+    : new FinalizationRegistry(ptr => wasm.__wbg_preparedstatement_free(ptr, 1));
 
 function addToExternrefTable0(obj) {
     const idx = wasm.__externref_table_alloc();
@@ -291,6 +371,12 @@ function addToExternrefTable0(obj) {
     return idx;
 }
 
+function _assertClass(instance, klass) {
+    if (!(instance instanceof klass)) {
+        throw new Error(`expected instance of ${klass.name}`);
+    }
+}
+
 function debugString(val) {
     // primitive types
     const type = typeof val;

package/sqlite.js

@@ -1,5 +1,6 @@
-// A better-sqlite3-style synchronous API over PicoVolt's WebAssembly engine, so
-// code written for better-sqlite3 can use PicoVolt with minimal change:
+// A better-sqlite3-inspired synchronous API over PicoVolt's WebAssembly engine.
+// It follows the familiar prepare/run/get/all shape while retaining PicoVolt's
+// focused SQL surface:
 //
 //   import Database from "picovolt/sqlite";
 //   const db = new Database();
@@ -30,13 +31,19 @@ function normalizeParams(args) {
 
 class Statement {
   constructor(db, sql) {
+    db._assertOpen();
     this._db = db;
     this.source = sql;
+    this._prepared = db._db.prepare(sql);
+    this.parameterCount = this._prepared.parameterCount;
+    db._statements.add(this);
   }
 
   _exec(args) {
+    if (!this._prepared) throw new Error("PicoVolt prepared statement is closed");
+    this._db._assertOpen();
     const params = normalizeParams(args);
-    const json = params.length ? this._db._db.query(this.source, params) : this._db._db.query(this.source);
+    const json = this._prepared.execute(this._db._db, params);
     return JSON.parse(json);
   }
 
@@ -60,12 +67,27 @@ class Statement {
   *iterate(...args) {
     yield* this.all(...args);
   }
+
+  close() {
+    if (!this._prepared) return false;
+    const prepared = this._prepared;
+    this._prepared = undefined;
+    this._db._statements.delete(this);
+    prepared.free();
+    return true;
+  }
+
+  finalize() {
+    return this.close();
+  }
 }
 
 class Database {
   constructor() {
     this._db = new Db();
     this._inTransaction = false;
+    this._closed = false;
+    this._statements = new Set();
   }
 
   prepare(sql) {
@@ -74,6 +96,7 @@ class Database {
 
   // Run one or more `;`-separated statements with no bound parameters.
   exec(sql) {
+    this._assertOpen();
     for (const stmt of sql.split(";").map((s) => s.trim()).filter(Boolean)) {
       this._db.query(stmt);
     }
@@ -82,11 +105,13 @@ class Database {
 
   // The most recent committed transaction id (upper bound for `... BEFORE tx`).
   get currentTx() {
+    this._assertOpen();
     return this._db.currentTx();
   }
 
   // Export the database as a `.pvdb` byte image (Uint8Array).
   serialize() {
+    this._assertOpen();
     return this._db.export();
   }
 
@@ -98,8 +123,10 @@ class Database {
   // better-sqlite3's common pattern.
   transaction(fn) {
     if (typeof fn !== "function") throw new TypeError("transaction expects a function");
+    this._assertOpen();
     const db = this;
     function wrapped(...args) {
+      db._assertOpen();
       if (db._inTransaction) return fn(...args);
       db._db.beginTransaction();
       db._inTransaction = true;
@@ -121,7 +148,14 @@ class Database {
   }
 
   close() {
-    /* the WebAssembly instance is reclaimed by the GC */
+    if (this._closed) return;
+    for (const statement of [...this._statements]) statement.close();
+    this._db.free();
+    this._closed = true;
+  }
+
+  _assertOpen() {
+    if (this._closed) throw new Error("PicoVolt database is closed");
   }
 }
 

package/worker.js

@@ -3,6 +3,8 @@
 import { PersistentDb } from "./browser.js";
 
 let database;
+let nextStatementId = 1;
+const statements = new Map();
 
 self.addEventListener("message", async ({ data }) => {
   const { id, method } = data ?? {};
@@ -10,6 +12,9 @@ self.addEventListener("message", async ({ data }) => {
     let result;
     switch (method) {
       case "open":
+        if (database) await database.close();
+        database = undefined;
+        statements.clear();
         database = await PersistentDb.open(data.name);
         result = true;
         break;
@@ -17,6 +22,26 @@ self.addEventListener("message", async ({ data }) => {
         if (!database) throw new Error("open the database first");
         result = database.query(data.sql, data.params);
         break;
+      case "prepare": {
+        if (!database) throw new Error("open the database first");
+        const statement = database.prepare(data.sql);
+        const statementId = nextStatementId++;
+        statements.set(statementId, statement);
+        result = { statementId, parameterCount: statement.parameterCount };
+        break;
+      }
+      case "execute": {
+        const statement = statements.get(data.statementId);
+        if (!statement) throw new Error("unknown PicoVolt prepared statement");
+        result = statement.query(data.params ?? []);
+        break;
+      }
+      case "finalize": {
+        const statement = statements.get(data.statementId);
+        result = statement ? statement.close() : false;
+        statements.delete(data.statementId);
+        break;
+      }
       case "save":
         if (!database) throw new Error("open the database first");
         await database.save();
@@ -25,6 +50,7 @@ self.addEventListener("message", async ({ data }) => {
       case "close":
         if (database) await database.close();
         database = undefined;
+        statements.clear();
         result = true;
         break;
       default: