inibase 2.0.0 → 3.0.0

package/README.md

@@ -1,6 +1,6 @@
 # Inibase :pencil:
 
-> A file-based & memory-efficient, serverless, ACID compliant, relational database management system :fire:
+> A file-based & memory-efficient, serverless relational database with **crash-atomic ACID: single-table DML + multi-table transactions** :fire: — per-table writer locks, write-ahead journal + crash recovery, fsync-backed durability, and a `begin`/`commit`/`rollback` transaction API (exact scope in [Durability & crash safety](#durability--crash-safety)).
 
 [![Inibase banner](./.github/assets/banner.jpg)](https://github.com/inicontent/inibase)
 
@@ -12,7 +12,7 @@
 - **Minimalist** :white_circle: (but powerful)
 - **100% TypeScript** :large_blue_diamond:
 - **Super-Fast** :zap: (built-in caching system)
-- **ATOMIC** :lock: File lock for writing
+- **ATOMIC** :lock: Per-table writer locks, write-ahead journal + crash recovery, fsync-backed durability, and multi-table transactions (`begin`/`commit`/`rollback`) for atomic cascades (exact scope below)
 - **Built-in** form validation (+unique values :new: ) :sunglasses:
 - **Suitable for large data** :page_with_curl: (tested with 4M records)
 - **Support Compression** :eight_spoked_asterisk: (using built-in nodejs zlib)
@@ -75,6 +75,98 @@ const users = await db.get("user", { favoriteFoods: "![]Pizza,Burger" });
 
 This structure ensures efficient storage, retrieval, and updates, making our system scalable and high-performing for diverse datasets and applications.
 
+## Durability & crash safety
+
+> [!IMPORTANT]
+> **DML (post / put / delete / truncate) is crash-atomic per table.** A logical
+> operation spans many column files plus a pagination-metadata file; all of
+> those files are committed as one journaled unit, so a table is never observed
+> half-applied. **Multi-table atomicity** is available explicitly through the
+> transaction API (`begin` / `commit` / `rollback`, see below) — cascade
+> deletes and multi-table writes are atomic *inside* a transaction, and
+> best-effort without one.
+
+### How each property is provided
+
+| Property | Mechanism |
+|---|---|
+| **A**tomicity | Write-ahead journal: one journal per table (`.tmp/journal.jsonl`) for single-table DML, plus a database journal (`<db>/.tmp/journal.jsonl`) that transactions append `op` entries to. Intent is fsynced *before* any live file changes; a `commit` marker is fsynced after publication. On crash: no commit marker → roll back (restore backups), commit present → roll forward (complete swaps, discard backups). Recovery runs automatically under the lock before any mutation/read. |
+| **C**onsistency | Schema validation + uniqueness enforcement happen before anything is written; every operation is atomic, so a table is never observed half-applied. |
+| **I**solation | **One writer lock per table** serializes every mutation (single-host and multi-host / NFS); a transaction holds the writer lock of every table it touches for its whole lifetime, plus a database lock to serialize transactions. Reads are **lock-free** with optimistic version-retry: each read snapshots the identity of every column + pagination file, and is re-run if anything changed mid-scan — a reader never sees torn rows. |
+| **D**urability | Durability knob `INIBASE_DURABILITY=full` (default) fsyncs the temp file, the journal (begin/commit), and the affected directories (incl. shell `sed`/`gzip` temp paths and stream pipelines). DDL (create/update table, compression & prepend toggles) is fsynced too. `INIBASE_DURABILITY=none` skips **every** fsync while keeping the exact journal protocol — the data is process-crash safe (a crash leaves an in-flight journal that recovery rolls back or forward) but **not** power-loss safe, because the OS page cache can be lost. |
+
+**Commit-point ordering:** the pagination metadata rename happens *first* (the atomic publication point — the row count flips in a single rename that lock-free readers observe), then column files are swapped `live → backup` + `tmp → live`.
+
+### Transactions (multi-table atomicity)
+
+```js
+await db.begin(["orders", "invoices"]); // pre-lock tables in sorted order (deadlock-free)
+try {
+  await db.post("orders", order);
+  await db.post("invoices", invoice);
+  await db.commit(); // publish everything crash-atomically
+} catch {
+  await db.rollback(); // discard everything; no live file was touched
+}
+```
+
+- **Semantics:** mutations issued inside a transaction are *staged* — their
+  temps are fsynced and an `op` entry appended to the database journal — but no
+  live file changes until `commit()`. `commit()` publishes every staged
+  mutation (per table: pagination rename first, then column swaps) and ends
+  with a single fsynced `commit` marker, making the whole set atomic with
+  respect to crashes and process kills. `rollback()` removes the temps and the
+  journal without touching any live file.
+- **Cascade:** a `delete` inside a transaction cascades into referencing tables
+  *through the same journal*, so the whole parent + children removal is atomic;
+  a failure during the cascade aborts the transaction.
+- **Rules & limits (v1):**
+  - **No read-your-writes:** reads inside a transaction observe the last
+    committed state, not staged writes.
+  - **One mutation per table per transaction.** A second post/put/delete on a
+    table already staged in the same transaction throws `INVALID_PARAMETERS`
+    (composing same-table writes would require read-your-writes).
+  - **DDL is not allowed inside a transaction** (`createTable` / `updateTable`
+    throw `INVALID_PARAMETERS`).
+  - `begin` without a table list locks tables on first touch, in first-touch
+    order; list the tables up front to get sorted, cycle-free ordering.
+  - `returnPostedData` inside a transaction returns the formatted staged rows
+    (no joins); `returnUpdatedData` inside a transaction throws
+    `INVALID_PARAMETERS`.
+  - Transactions are excluded from structural lock-upgrade deadlocks, but two
+    long transactions that touch overlapping tables can only deadlock in the
+    first-touch-order case (pre-listing avoids it).
+
+### Scope & caveats
+
+- **Best-effort without a transaction.** Cross-table cascade deletes outside a
+  transaction stay best-effort: referencing rows are removed *after* the
+  primary delete commits, and a crash in between can leave orphaned references.
+  Wrap them in `begin`/`commit` for atomicity.
+- **NFS:** lock files use `O_EXCL` creation, which is advisory on some NFS
+  servers — two hosts may briefly both believe they hold a lock. Locks store
+  `{pid, host, startedAt}`: a same-host owner that is **provably dead**
+  (`kill(pid, 0)` fails) is stolen immediately; foreign/unknown owners fall
+  back to the TTL (`INIBASE_LOCK_TTL_MS`, default `60000`).
+- **fsync guarantees:** durability assumes the OS/filesystem honours `fsync`.
+  Some disks and virtualized filesystems silently ignore it; directory `fsync`
+  is unsupported on a few platforms (best-effort there). `none` mode
+  deliberately gives up power-loss durability — that is the whole point of the
+  knob.
+- **Cost:** at `full`, every mutation fsyncs the temp file, journal, and
+  directories — expect noticeably slower hot-path writes than pre-durability
+  builds (the benchmark compares `full` vs `none` side by side). The swap
+  protocol also transiently holds temp + backup + live copies (~2–3× a
+  column's size; worst case is a `put` with no `where` — a full-table rewrite).
+- **Cache (`.cache`):** entries are derived, rebuildable artifacts versioned by
+  the row count; they detect staleness but never participate in the ACID
+  guarantee. Non-fsync'd.
+
+Run `pnpm test:durability` for the journal-recovery, multi-process writer,
+live-reader and stale-lock test suite, `pnpm test:transaction` for the
+transaction/cascade/crash-recovery suite, and `pnpm benchmark:durability` for
+the `full` vs `none` throughput comparison.
+
 ## Inibase CLI
 
 ```shell
@@ -793,6 +885,9 @@ await db.get("user", undefined, { sort: {age: -1, username: "asc"} });
 > Default testing uses a table with username, email, and password fields, ensuring password encryption is included in the process<br>
 > Results are measured on a default table plus dedicated tables with `prepend`, `compression`, and `decodeID` configs enabled<br>
 > To run benchmarks, install _typescript_ & _[tsx](https://github.com/privatenumber/tsx)_ globally and run `benchmark` by default bulk, for single use `benchmark --single|-s`
+>
+> > [!WARNING]
+> > The numbers above were measured **before** always-on fsync + write-ahead journaling landed (they no longer reflect current hot-path write costs). Run `pnpm benchmark:durability` for the crash-atomic numbers.
 
 ## Roadmap
 

package/dist/file.d.ts

@@ -1,7 +1,26 @@
 import { type ComparisonOperator, type Field } from "./index.js";
-export declare const lock: (folderPath: string, prefix?: string) => Promise<void>;
+export declare const DURABLE: boolean;
+export declare const lock: (folderPath: string, prefix?: string, ttl?: number) => Promise<void>;
+/**
+ * Non-blocking lock acquisition (used by the read path and the open-time
+ * recovery sweep). Returns true when the lock was acquired (running crash
+ * recovery for prefix-less locks, exactly like `lock`), false when another
+ * process holds it. A single stale-lock steal (dead same-host owner, or aged
+ * foreign/unknown owner) is attempted so a crashed owner can't wedge readers
+ * behind it forever.
+ */
+export declare const tryLock: (folderPath: string, prefix?: string, ttl?: number) => Promise<boolean>;
 export declare const unlock: (folderPath: string, prefix?: string) => Promise<void>;
 export declare const write: (filePath: string, data: any) => Promise<void>;
+/**
+ * fsync an existing file. Used to flush temp files (written via streams or
+ * shell pipelines) before they are renamed into place.
+ */
+export declare const syncFile: (filePath: string) => Promise<void>;
+/**
+ * fsync a directory so that renames performed inside it are durable.
+ */
+export declare const syncDir: (dirPath: string) => Promise<void>;
 export declare const read: (filePath: string) => Promise<string>;
 export declare function escapeShellPath(filePath: string): string;
 /**
@@ -113,6 +132,7 @@ export declare const search: (filePath: string, operator: ComparisonOperator | C
     databasePath?: string;
 }, limit?: number, offset?: number, readWholeFile?: boolean) => Promise<[Record<number, string | number | boolean | null | undefined | (string | number | boolean | null)[]> | null, number, Set<number> | null]>;
 export declare const sum: (fp: string, ln?: number | number[]) => Promise<number>;
+export declare const avg: (fp: string, ln?: number | number[]) => Promise<number>;
 export declare const min: (fp: string, ln?: number | number[]) => Promise<number>;
 export declare const max: (fp: string, ln?: number | number[]) => Promise<number>;
 export declare const getFileDate: (path: string) => Promise<Date>;

package/dist/file.js

@@ -1,4 +1,5 @@
-import { access, appendFile, copyFile, constants as fsConstants, open, readFile, stat, unlink, writeFile, } from "node:fs/promises";
+import { access, appendFile, copyFile, constants as fsConstants, open, readFile, stat, unlink, } from "node:fs/promises";
+import { hostname } from "node:os";
 import { join, resolve } from "node:path";
 import { createInterface } from "node:readline";
 import { Transform } from "node:stream";
@@ -6,37 +7,231 @@ import { pipeline } from "node:stream/promises";
 import { createGunzip, createGzip } from "node:zlib";
 import Inison from "inison";
 import { globalConfig, } from "./index.js";
+import { recover } from "./journal.js";
 import { detectFieldType, isArrayOfObjects, isNumber, isObject, isStringified, isValidID, } from "./utils.js";
 import { compare, decodeID, encodeID, exec, gunzip, gzip, } from "./utils.server.js";
-// Locks older than this are assumed abandoned by a crashed/killed process, not a slow operation.
-const STALE_LOCK_MS = 30_000;
-export const lock = async (folderPath, prefix) => {
-    let lockFile = null;
-    const lockFilePath = join(folderPath, `${prefix ?? ""}.locked`);
+// Locks older than this are candidates for removal. Same-host locks are only
+// stolen when the recorded owner PID is provably dead; foreign-host locks fall
+// back to this TTL (NFS has no reliable cross-host liveness check).
+const DEFAULT_LOCK_TTL_MS = Number(process.env.INIBASE_LOCK_TTL_MS ?? 60_000);
+// Durability knob: `full` (default) fsyncs every temp file, journal entry and
+// directory touch before acknowledging a mutation. `none` skips all fsync
+// calls but keeps the write-ahead journal protocol unchanged, so a *process*
+// crash (page cache survives) still recovers atomically — only power-loss
+// durability is lost. The ACID claim documented in the README holds at `full`.
+const DURABILITY = process.env.INIBASE_DURABILITY ?? "full";
+export const DURABLE = DURABILITY !== "none";
+/** fsync a handle, or no-op when `INIBASE_DURABILITY=none` is set. */
+const maybeSync = async (handle) => {
+    if (DURABLE)
+        await handle.sync();
+};
+const LOCK_RETRY_MS = 13;
+// Per-process reentrancy: the same process may acquire the same lock file
+// multiple times (e.g. post() -> get() -> sort-cache). Depth 1 is a real
+// filesystem acquisition; deeper acquisitions just bump the counter.
+const lockDepths = new Map();
+const lockFilePathFor = (folderPath, prefix) => join(folderPath, `${prefix ?? ""}.locked`);
+const lockOwnerState = async (lockFilePath) => {
     try {
-        lockFile = await open(lockFilePath, "wx");
+        const metadata = JSON.parse(await readFile(lockFilePath, "utf8"));
+        if (typeof metadata.pid === "number" && metadata.host === hostname()) {
+            try {
+                process.kill(metadata.pid, 0);
+            }
+            catch (error) {
+                return error?.code === "ESRCH" ? "dead" : "unknown";
+            }
+            return "alive";
+        }
+    }
+    catch {
+        // No/invalid metadata (e.g. crash between create and metadata write).
+    }
+    return "unknown"; // foreign host or unparsable metadata
+};
+/**
+ * A lock may be stolen when its owner is provably dead on this host
+ * (immediately — a same-host crash must not wedge writers or block crash
+ * recovery) or, for foreign/unknown owners where no liveness check exists
+ * (e.g. NFS), once the recorded lock is older than the TTL.
+ */
+const stealableLock = async (lockFilePath, mtimeMs, ttl) => {
+    const state = await lockOwnerState(lockFilePath);
+    if (state === "alive")
+        return false;
+    if (state === "dead")
+        return true;
+    return Date.now() - mtimeMs > ttl;
+};
+export const lock = async (folderPath, prefix, ttl = DEFAULT_LOCK_TTL_MS) => {
+    const lockFilePath = lockFilePathFor(folderPath, prefix);
+    const resolvedPath = resolve(lockFilePath);
+    const depth = lockDepths.get(resolvedPath);
+    if (depth) {
+        lockDepths.set(resolvedPath, depth + 1);
         return;
     }
-    catch ({ message }) {
-        if (message.split(":")[0] === "EEXIST") {
+    for (;;) {
+        try {
+            const lockFile = await open(lockFilePath, "wx");
+            try {
+                await lockFile.writeFile(JSON.stringify({
+                    pid: process.pid,
+                    host: hostname(),
+                    startedAt: Date.now(),
+                }));
+                await maybeSync(lockFile);
+            }
+            finally {
+                await lockFile.close();
+            }
+            // The global (prefix-less) lock is the writer lock: run crash
+            // recovery for this table while we exclusively hold it. Locked
+            // calls with a prefix are read-side helpers (e.g. sort cache) and
+            // must never roll back an in-flight transaction.
+            if (!prefix) {
+                try {
+                    await recover(folderPath);
+                }
+                catch {
+                    // Recovery is best-effort at lock time; reads retry on
+                    // torn state, writers re-check before mutating.
+                }
+            }
+            lockDepths.set(resolvedPath, 1);
+            return;
+        }
+        catch (error) {
+            const message = String(error?.message ?? error);
+            if (message.split(":")[0] !== "EEXIST")
+                throw error;
             const lockStat = await stat(lockFilePath).catch(() => null);
-            if (lockStat && Date.now() - lockStat.mtimeMs > STALE_LOCK_MS)
+            // Someone else released the lock between our failed open and the
+            // stat: retry immediately instead of counting down the TTL.
+            if (!lockStat)
+                continue;
+            if (await stealableLock(lockFilePath, lockStat.mtimeMs, ttl)) {
                 await unlink(lockFilePath).catch(() => { });
-            return await new Promise((resolve) => setTimeout(() => resolve(lock(folderPath, prefix)), 13));
+            }
+            await new Promise((resolvePromise) => setTimeout(() => resolvePromise(), LOCK_RETRY_MS));
         }
     }
-    finally {
-        await lockFile?.close();
+};
+/**
+ * Non-blocking lock acquisition (used by the read path and the open-time
+ * recovery sweep). Returns true when the lock was acquired (running crash
+ * recovery for prefix-less locks, exactly like `lock`), false when another
+ * process holds it. A single stale-lock steal (dead same-host owner, or aged
+ * foreign/unknown owner) is attempted so a crashed owner can't wedge readers
+ * behind it forever.
+ */
+export const tryLock = async (folderPath, prefix, ttl = DEFAULT_LOCK_TTL_MS) => {
+    const lockFilePath = lockFilePathFor(folderPath, prefix);
+    const resolvedPath = resolve(lockFilePath);
+    const depth = lockDepths.get(resolvedPath);
+    if (depth) {
+        lockDepths.set(resolvedPath, depth + 1);
+        return true;
+    }
+    for (let attempt = 0; attempt < 2; attempt++) {
+        try {
+            const lockFile = await open(lockFilePath, "wx");
+            try {
+                await lockFile.writeFile(JSON.stringify({
+                    pid: process.pid,
+                    host: hostname(),
+                    startedAt: Date.now(),
+                }));
+                await maybeSync(lockFile);
+            }
+            finally {
+                await lockFile.close();
+            }
+            if (!prefix) {
+                try {
+                    await recover(folderPath);
+                }
+                catch {
+                    // Best-effort at lock time, mirroring `lock`.
+                }
+            }
+            lockDepths.set(resolvedPath, 1);
+            return true;
+        }
+        catch (error) {
+            if (String(error?.message ?? error).split(":")[0] !== "EEXIST")
+                throw error;
+            const lockStat = await stat(lockFilePath).catch(() => null);
+            if (!lockStat)
+                continue; // released between open and stat
+            if (attempt === 0 &&
+                (await stealableLock(lockFilePath, lockStat.mtimeMs, ttl)))
+                await unlink(lockFilePath).catch(() => { });
+            else
+                return false; // genuinely held -> don't wait
+        }
     }
+    return false;
 };
 export const unlock = async (folderPath, prefix) => {
+    const lockFilePath = lockFilePathFor(folderPath, prefix);
+    const resolvedPath = resolve(lockFilePath);
+    const depth = lockDepths.get(resolvedPath);
+    if (depth && depth > 1) {
+        lockDepths.set(resolvedPath, depth - 1);
+        return;
+    }
+    lockDepths.delete(resolvedPath);
     try {
-        await unlink(join(folderPath, `${prefix ?? ""}.locked`));
+        await unlink(lockFilePath);
+    }
+    catch {
+        // Already released (stolen by a foreign-host stealer or cleaned up).
     }
-    catch { }
 };
 export const write = async (filePath, data) => {
-    await writeFile(filePath, filePath.endsWith(".gz") ? await gzip(data) : data);
+    const handle = await open(filePath, "w");
+    try {
+        await handle.writeFile(filePath.endsWith(".gz") ? await gzip(data) : data);
+        await maybeSync(handle);
+    }
+    finally {
+        await handle.close();
+    }
+};
+/**
+ * fsync an existing file. Used to flush temp files (written via streams or
+ * shell pipelines) before they are renamed into place.
+ */
+export const syncFile = async (filePath) => {
+    let handle = null;
+    try {
+        handle = await open(filePath, "r+");
+        await maybeSync(handle);
+    }
+    catch {
+        // Unsupported filesystem/platform: best effort.
+    }
+    finally {
+        await handle?.close();
+    }
+};
+/**
+ * fsync a directory so that renames performed inside it are durable.
+ */
+export const syncDir = async (dirPath) => {
+    let handle = null;
+    try {
+        handle = await open(dirPath, "r");
+        await maybeSync(handle);
+    }
+    catch {
+        // Directory fsync is not supported on every platform/filesystem.
+    }
+    finally {
+        await handle?.close();
+    }
 };
 export const read = async (filePath) => filePath.endsWith(".gz")
     ? (await gunzip(await readFile(filePath, "utf8"))).toString()
@@ -132,7 +327,9 @@ const unSecureString = (input) => {
     // Fast path: the common case has no `\n` escape sequence, so avoid
     // allocating a replacement string (and a fresh RegExp) per cell.
     if (typeof input === "string")
-        return input.includes("\\n") ? input.replaceAll("\\n", "\n") || null : input;
+        return input.includes("\\n")
+            ? input.replaceAll("\\n", "\n") || null
+            : input;
     return null;
 };
 /**
@@ -831,11 +1028,11 @@ export const search = async (filePath, operator, comparedAtValue, logicalOperato
     }
 };
 /**
- * Reads the file once and returns either the sum, min or max of the
+ * Reads the file once and returns either the sum, average, min or max of the
  * (optionally-selected) numeric lines.
  *
  * @param filePath   Absolute path of the column file (may be .gz-compressed).
- * @param wanted     Metric to compute: "sum" (default), "min" or "max".
+ * @param wanted     Metric to compute: "sum" (default), "avg", "min" or "max".
  * @param lineNumbers Specific line-number(s) to restrict the scan to.
  *
  * @returns Promise<number>  The requested metric, or 0 if no numeric value found.
@@ -864,7 +1061,7 @@ async function reduceNumbers(filePath, wanted = "sum", lineNumbers) {
             if (Number.isNaN(num))
                 continue;
             processed++;
-            if (wanted === "sum") {
+            if (wanted === "sum" || wanted === "avg") {
                 sum += num;
             }
             else if (wanted === "min") {
@@ -885,10 +1082,17 @@ async function reduceNumbers(filePath, wanted = "sum", lineNumbers) {
     }
     if (processed === 0)
         return 0; // nothing numeric found
-    return wanted === "sum" ? sum : wanted === "min" ? min : max;
+    return wanted === "sum"
+        ? sum
+        : wanted === "avg"
+            ? sum / processed
+            : wanted === "min"
+                ? min
+                : max;
 }
 /* Optional convenience wrappers (signatures unchanged) */
 export const sum = (fp, ln) => reduceNumbers(fp, "sum", ln);
+export const avg = (fp, ln) => reduceNumbers(fp, "avg", ln);
 export const min = (fp, ln) => reduceNumbers(fp, "min", ln);
 export const max = (fp, ln) => reduceNumbers(fp, "max", ln);
 export const getFileDate = (path) => stat(path)

package/dist/index.d.ts

@@ -1,4 +1,5 @@
 import "dotenv/config";
+import { type JournalFileOp } from "./journal.js";
 export interface Data {
     id?: string | number;
     [key: string]: any;
@@ -33,6 +34,27 @@ export interface TableObject {
     schema?: Schema;
     config: TableConfig;
 }
+/**
+ * Per-table state maintained while a database transaction is open: the writer
+ * lock is held for the whole transaction, pagination state is staged in
+ * memory (the live files only change at commit()), and `staged` collects the
+ * journaled ops that commit() publishes (one per table per transaction).
+ */
+export interface TxnTableEntry {
+    locked: boolean;
+    /** Live pagination path the first staged op builds on. */
+    paginationFrom: string;
+    /** Staged last-id / row count (chained across ops in the txn). */
+    lastId: number;
+    total: number;
+    staged: {
+        ops: JournalFileOp[];
+        pagination: {
+            from: string;
+            to: string;
+        } | null;
+    }[];
+}
 export type ComparisonOperator = "=" | "!=" | ">" | "<" | ">=" | "<=" | "*" | "!*" | "[]" | "![]";
 export type pageInfo = {
     total?: number;
@@ -81,6 +103,13 @@ export default class Inibase {
     private databasePath;
     private uniqueMap;
     private schemaFileExtension;
+    /**
+     * Open database transaction (see begin/commit/rollback). Holds the
+     * database lock (`<db>/.tmp/.locked`) for its whole lifetime and the
+     * per-table writer lock of every table it mutates, so mutations stage into
+     * the database journal and publish only on commit().
+     */
+    private transaction;
     constructor(database: string, mainFolder?: string, language?: ErrorLang);
     createError(name: ErrorCode, variable?: string | number | (string | number)[]): Error;
     private validateName;
@@ -116,6 +145,7 @@ export default class Inibase {
     updateTable(tableName: string, schema?: Schema, config?: TableConfig & {
         name?: string;
     }): Promise<void>;
+    private updateTableLocked;
     /**
      * Get table schema and config
      *
@@ -162,6 +192,89 @@ export default class Inibase {
      * @param {string} tableName
      */
     clearCache(tableName: string): Promise<void>;
+    /**
+     * Commit a multi-file mutation crash-atomically:
+     * 1. fsync every freshly-written temp file;
+     * 2. write the journal `begin` entry and fsync it;
+     * 3. rename the pagination metadata file first (atomic publication point:
+     *    the row count flips in a single rename, which is what lock-free
+     *    readers observe) and then swap each live file aside (backup) and
+     *    rename the temp into place;
+     * 4. write the journal `commit` marker and fsync it;
+     * 5. discard backups/temps + the journal, fsync the directories.
+     *
+     * On any failure before `commit`, the journal is rolled back so the table
+     * is left exactly as it was. `renameList` entries are [tempPath, livePath]
+     * pairs; a null tempPath means a pure removal (live file taken out).
+     */
+    private commitFiles;
+    /**
+     * Runs crash recovery for a table (and any crashed database transaction)
+     * before a read. Mutation paths get the same guarantee implicitly (the
+     * writer lock runs recovery on acquire); reads call this explicitly
+     * because they never take the table lock.
+     */
+    private ensureTableRecovered;
+    /**
+     * Blocking database-journal recovery, used by mutation paths (writers are
+     * serialized with live transactions on the database lock anyway).
+     */
+    private ensureDatabaseRecovered;
+    private ensureDatabaseTmpDir;
+    /** Staged per-table entry of the open transaction, or null when none. */
+    private txnTableEntry;
+    /** Lock a table for the open transaction (idempotent per transaction). */
+    private ensureTxnLock;
+    /**
+     * Resolve the pagination state a DML op should build on. Outside a
+     * transaction this reads the live pagination file (as before); inside a
+     * transaction the first touch reads it once and the entry keeps the staged
+     * id/count so chained ops (guarded to one per table) and commit() stay
+     * consistent without publishing anything early.
+     */
+    private resolvePagination;
+    /**
+     * Stage one table mutation into the open transaction: fsync its temps and
+     * append an `op` entry to the database journal (no live file is touched;
+     * commit() performs the actual renames). One staged mutation per table per
+     * transaction (multi-table atomicity; a second touch of the same table
+     * would need read-your-writes composition).
+     */
+    private stageTxnOp;
+    /**
+     * Begin a database transaction. Mutations issued while the transaction is
+     * open (post/put/delete, including cascade deletes) are staged into the
+     * database journal and published atomically at commit(); rollback()
+     * discards them without touching any live file.
+     *
+     * @param tables Optional table names to pre-lock at begin() in sorted
+     * order (the deadlock-free way to span tables). Tables not listed are
+     * locked on first touch, in first-touch order.
+     */
+    begin(tables?: string[]): Promise<void>;
+    /**
+     * Publish every staged mutation atomically: per table (sorted), the
+     * pagination rename comes first (the atomic publication point readers
+     * observe) and then live->backup + tmp->live swaps, before a single fsynced
+     * `commit` marker makes the whole transaction durable. A crash at any
+     * point is recovered by the journal rule (no marker -> roll back all
+     * tables, marker -> roll forward all tables).
+     */
+    commit(): Promise<void>;
+    /**
+     * Discard the open transaction: temps and the journal are removed and no
+     * live file is touched (nothing is published before commit()).
+     */
+    rollback(): Promise<void>;
+    /**
+     * Snapshot the identity (dev:inode:mtime:size) of every column file and
+     * the pagination file. Reading data and then re-verifying this snapshot
+     * lets lock-free readers detect an in-flight writer commit and retry
+     * instead of returning a torn row set.
+     */
+    private snapshotTableFiles;
+    /** True when every snapshotted file is still present and unchanged. */
+    private verifyTableFiles;
     /**
      * Retrieve item(s) from a table
      *
@@ -177,6 +290,7 @@ export default class Inibase {
     get<TData extends Record<string, any> & Partial<Data>>(tableName: string, where?: string | number | (string | number)[] | Criteria, options?: Options, onlyOne?: boolean, onlyLinesNumbers?: false, _whereIsLinesNumbers?: boolean): Promise<(Data & TData)[] | null>;
     get<_TData extends Record<string, any> & Partial<Data>>(tableName: string, where: string | number | (string | number)[] | Criteria | undefined, options: Options | undefined, onlyOne: false | undefined, onlyLinesNumbers: true, _whereIsLinesNumbers?: boolean): Promise<number[] | null>;
     get<_TData extends Record<string, any> & Partial<Data>>(tableName: string, where: string | number | (string | number)[] | Criteria | undefined, options: Options | undefined, onlyOne: true, onlyLinesNumbers: true, _whereIsLinesNumbers?: boolean): Promise<number | null>;
+    private getOnce;
     /**
      * Create new item(s) in a table
      *
@@ -231,6 +345,16 @@ export default class Inibase {
      */
     sum(tableName: string, columns: string, where?: number | string | (number | string)[] | Criteria): Promise<number>;
     sum(tableName: string, columns: string[], where?: number | string | (number | string)[] | Criteria): Promise<Record<string, number>>;
+    /**
+     * Generate average of column(s) in a table
+     *
+     * @param {string} tableName
+     * @param {string} columns
+     * @param {(number | string | (number | string)[] | Criteria)} [where]
+     * @return {*}  {Promise<number | Record<string, number>>}
+     */
+    avg(tableName: string, columns: string, where?: number | string | (number | string)[] | Criteria): Promise<number>;
+    avg(tableName: string, columns: string[], where?: number | string | (number | string)[] | Criteria): Promise<Record<string, number>>;
     /**
      * Generate max of column(s) in a table
      *