@fgv/ts-agent-memory-sqlite-vec 5.1.0-42

package/README.md

@@ -0,0 +1,109 @@
+<!--
+Copyright (c) 2026 Erik Fortune
+SPDX-License-Identifier: MIT
+-->
+
+# @fgv/ts-agent-memory-sqlite-vec
+
+A **persistent, `sqlite-vec`-backed [`IVectorIndex`](https://github.com/ErikFortune/fgv/tree/main/libraries/ts-agent-memory)** for [`@fgv/ts-agent-memory`](https://github.com/ErikFortune/fgv/tree/main/libraries/ts-agent-memory). It is the durable counterpart to the in-package `InMemoryCosineIndex`: embeddings live in a SQLite file, so they **survive a process restart** — opening an existing vault no longer re-embeds every record.
+
+This is a thin **Result-integration boundary** over [`better-sqlite3`](https://github.com/WiseLibs/better-sqlite3) + [`sqlite-vec`](https://github.com/asg017/sqlite-vec): it converts their throwing surface into `Result<T>` and implements the `IVectorIndex` seam. It adds no opinion beyond that — see [Not in scope](#not-in-scope).
+
+The package ships two indexes with the same durability story: `SqliteVecVectorIndex` (one vector per record — the record-granular `IVectorIndex`) and `SqliteVecFragmentIndex` (many vectors per record, each tagged with an in-record `[start, end)` locator — the fragment-granular `IFragmentVectorIndex`, the durable counterpart to `InMemoryFragmentCosineIndex`). See [Fragment-granular index](#fragment-granular-index).
+
+## Why
+
+`InMemoryCosineIndex` is brute-force and in-memory, rebuilt from the store by re-embedding the whole vault on open. That is fine for small per-agent vaults; at v2 durability/scale, re-embedding on every open stops being cheap. This package closes that gap with **zero core `ts-agent-memory` change**: the store already writes to the vector index incrementally on `put`/`delete` and only re-embeds when the consumer explicitly calls `rebuild(store.asRecordSource())`. Back the store with a persistent index instead, and skip that rebuild — the vectors are already on disk.
+
+## Install
+
+```bash
+rush add -p @fgv/ts-agent-memory-sqlite-vec   # or npm/pnpm add
+```
+
+`better-sqlite3` and `sqlite-vec` are **peer dependencies** — bring your own (matching the `@fgv/ts-extras-ollama` / `-transformers` convention). `@fgv/ts-agent-memory` and `@fgv/ts-utils` are peers too; a consumer of this package already has them.
+
+## Quick start
+
+```ts
+import Database from 'better-sqlite3';
+import { SqliteVecVectorIndex } from '@fgv/ts-agent-memory-sqlite-vec';
+import { FileTreeMemoryStore } from '@fgv/ts-agent-memory';
+
+// You own the connection. A file path persists; ':memory:' is ephemeral.
+const db = new Database('/path/to/vault/vectors.db');
+
+const vectorIndex = (await SqliteVecVectorIndex.create({ database: db })).orThrow();
+
+// Wire it into the store exactly where InMemoryCosineIndex would go. On a reopened
+// vault, do NOT call rebuild() — the vectors are already in vectors.db.
+const store = (
+  await FileTreeMemoryStore.create({ root, registry, vectorIndex, embed })
+).orThrow();
+
+// ...use the store; embeddings are written to vectors.db on every put.
+db.close(); // you own the lifecycle — this index never closes your connection.
+```
+
+`SqliteVecVectorIndex` implements the full `IVectorIndex` contract — `add(target, vector)`, `remove(target)`, `query(vector, topK)` — with the **same semantics as `InMemoryCosineIndex`**:
+
+- Keyed by the canonical `edgeTargetKey` (`(scope, id)`), so records that share a filename stem across scopes never collide.
+- The dimension is established by the first `add` (and recovered from the table schema on reopen); a later `add`/`query` of a different dimension fails loudly.
+- Similarity is **cosine** — the returned `score` is `1 − cosineDistance` (cosine similarity in `[-1, 1]`, higher = more similar), byte-for-byte the same scoring contract as the in-memory index.
+- An empty vector, a dimension mismatch, or a bad table name is a `Result.fail`, never a throw.
+
+## Persistence guarantee
+
+Vectors written in one process are present in the next. `SqliteVecVectorIndex.create` recovers the established dimension from the existing `vec0` table, so a reopened index answers queries immediately with **no re-embedding**. That is the entire point of the package. `SqliteVecFragmentIndex` gives the same guarantee for fragments.
+
+## Fragment-granular index
+
+`SqliteVecFragmentIndex` is the durable `IFragmentVectorIndex` — the "discovery" half of a search-then-read contract, where a hit's `(target, locator)` tells you which record **and** which span matched. It is the persistent counterpart to `InMemoryFragmentCosineIndex` and shares its exact semantics.
+
+```ts
+import Database from 'better-sqlite3';
+import { SqliteVecFragmentIndex } from '@fgv/ts-agent-memory-sqlite-vec';
+import { FileTreeMemoryStore, FragmentSemanticRetriever } from '@fgv/ts-agent-memory';
+
+const db = new Database('/path/to/vault/fragments.db');
+const fragmentIndex = (await SqliteVecFragmentIndex.create({ database: db })).orThrow();
+
+// Wire it into the store alongside (or instead of) the record-level vectorIndex.
+// The store chunks + embeds each record via the consumer-supplied fragmentEmbedder
+// and maintains the fragment index on put/delete/evict.
+const store = (
+  await FileTreeMemoryStore.create({ root, registry, fragmentIndex, fragmentEmbedder })
+).orThrow();
+
+// Query for spans, not records:
+const retriever = FragmentSemanticRetriever.create({
+  backend: { fragmentIndex, embedQuery }
+}).orThrow();
+const hits = (await retriever.retrieve({ semantic: 'refund policy', topK: 5, maxPerRecord: 2 })).orThrow();
+// each hit carries { target, locator: { start, end }, score } — read the record, slice the span.
+```
+
+- Keyed on `target_key` as a `vec0` **`PARTITION KEY`** (many rows share it), with the `[start, end)` offsets in auxiliary columns — so `addFragments` is whole-record-replace, `remove` drops every fragment of a target, and both are per-target-clean.
+- `query(vector, topK, maxPerRecord?)` applies the optional per-record cap **during selection, before the `topK` cut**, so one long document cannot crowd others out.
+- Same dimension-establishment, cosine scoring (`score = 1 − cosineDistance`), reopen-recovery, and loud-failure contract as the record index — byte-identical to `InMemoryFragmentCosineIndex`.
+
+Hold the record and fragment indexes in **distinct tables** (default `memory_vectors` vs `memory_fragments`, or supply `tableName`); they are fully independent and may share one database file.
+
+## Not in scope
+
+Deliberately excluded — reach for the upstream libraries (or a different backend) directly if you need these:
+
+- **ANN / large-N indexing.** Query is a brute-force `vec0` KNN scan — correct and durable for the same "thousands of records" regime the in-memory index targets. An approximate-nearest-neighbor structure for very large N is a different backend behind the same `IVectorIndex` / `IFragmentVectorIndex` seam. (This applies to both indexes, including `SqliteVecFragmentIndex`, whose capped query fetches the full ranked set.)
+- **Connection lifecycle.** You open and close the `better-sqlite3` `Database`; this index never does. Pooling, WAL/pragma tuning, backups, and multi-process coordination are yours.
+- **Embedding.** This is a vector *index*, not an embedder — the store's consumer-wired `MemoryEmbedder` produces the vectors (`@fgv/ts-extras/ai-assist` `callProviderEmbedding`, `@fgv/ts-extras-transformers`, etc.).
+- **A browser sibling.** `better-sqlite3` is Node-only. A WASM-SQLite browser variant, if ever needed, is a separate package.
+- **Schema migration across dimension change.** Re-embedding with a different-dimension model against an existing table fails loudly; drop the table (or use a new `tableName`) to re-index.
+
+## Runtime requirements
+
+- Node.js 20+ (`better-sqlite3` native binding).
+- `better-sqlite3` `^12.0.0`, `sqlite-vec` `^0.1.9` (peer dependencies).
+
+## License
+
+MIT

package/config/api-extractor.json

@@ -0,0 +1,38 @@
+{
+  "$schema": "https://developer.microsoft.com/json-schemas/api-extractor/v7/api-extractor.schema.json",
+  "projectFolder": "..",
+  "mainEntryPointFilePath": "<projectFolder>/lib/index.d.ts",
+  "bundledPackages": [],
+  "compiler": {},
+  "apiReport": {
+    "enabled": true
+  },
+  "docModel": {
+    "enabled": true
+  },
+  "dtsRollup": {
+    "enabled": true
+  },
+  "tsdocMetadata": {},
+  "messages": {
+    "compilerMessageReporting": {
+      "default": {
+        "logLevel": "warning"
+      }
+    },
+    "extractorMessageReporting": {
+      "default": {
+        "logLevel": "warning"
+      },
+      "ae-unresolved-link": {
+        "logLevel": "none",
+        "addToApiReportFile": true
+      }
+    },
+    "tsdocMessageReporting": {
+      "default": {
+        "logLevel": "warning"
+      }
+    }
+  }
+}

package/config/jest.config.json

@@ -0,0 +1,13 @@
+{
+  "extends": "@fgv/heft-dual-rig/profiles/default/config/jest.config.json",
+  "coverageThreshold": {
+    "global": {
+      "branches": 100,
+      "functions": 100,
+      "lines": 100,
+      "statements": 100
+    }
+  },
+  "collectCoverage": true,
+  "coverageReporters": ["text", "lcov", "html"]
+}

package/config/rig.json

@@ -0,0 +1,6 @@
+// The "rig.json" file directs tools to look for their config files in an external package.
+// Documentation for this system: https://www.npmjs.com/package/@rushstack/rig-package
+{
+  "$schema": "https://developer.microsoft.com/json-schemas/rig-package/rig.schema.json",
+  "rigPackageName": "@fgv/heft-dual-rig"
+}

package/dist/index.js

@@ -0,0 +1,6 @@
+/*
+ * Copyright (c) 2026 Erik Fortune
+ * SPDX-License-Identifier: MIT
+ */
+export * from './packlets/sqlite-vec-index';
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,cAAc,6BAA6B,CAAC","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nexport * from './packlets/sqlite-vec-index';\n"]}

package/dist/packlets/sqlite-vec-index/index.js

@@ -0,0 +1,8 @@
+/*
+ * Copyright (c) 2026 Erik Fortune
+ * SPDX-License-Identifier: MIT
+ */
+export * from './model';
+export * from './sqliteVecVectorIndex';
+export * from './sqliteVecFragmentIndex';
+//# sourceMappingURL=index.js.map

package/dist/packlets/sqlite-vec-index/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,cAAc,SAAS,CAAC;AACxB,cAAc,wBAAwB,CAAC;AACvC,cAAc,0BAA0B,CAAC","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nexport * from './model';\nexport * from './sqliteVecVectorIndex';\nexport * from './sqliteVecFragmentIndex';\n"]}

package/dist/packlets/sqlite-vec-index/model.js

@@ -0,0 +1,6 @@
+/*
+ * Copyright (c) 2026 Erik Fortune
+ * SPDX-License-Identifier: MIT
+ */
+export {};
+//# sourceMappingURL=model.js.map

package/dist/packlets/sqlite-vec-index/model.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"model.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/model.ts"],"names":[],"mappings":"AAAA;;;GAGG","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nimport type BetterSqlite3 from 'better-sqlite3';\n\n/**\n * Parameters for {@link SqliteVecVectorIndex.create}.\n * @public\n */\nexport interface ISqliteVecVectorIndexCreateParams {\n  /**\n   * A `better-sqlite3` `Database` the consumer owns (bring-your-own, mirroring\n   * the boundary-package convention). The consumer opens it (`new Database(path)`\n   * for a persistent file, or `new Database(':memory:')` for an ephemeral index)\n   * and owns its lifecycle — this index never closes it. `create` loads the\n   * `sqlite-vec` extension onto the connection and, if the vector table already\n   * exists (a reopened persistent file), recovers its established dimension so no\n   * re-embedding is required on open.\n   */\n  readonly database: BetterSqlite3.Database;\n\n  /**\n   * Name of the `vec0` virtual table that holds the embeddings. Must be a simple\n   * SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to `'memory_vectors'`.\n   * Supply a distinct name to hold more than one independent index in a single\n   * database file.\n   */\n  readonly tableName?: string;\n}\n\n/**\n * Parameters for {@link SqliteVecFragmentIndex.create}.\n * @public\n */\nexport interface ISqliteVecFragmentIndexCreateParams {\n  /**\n   * A `better-sqlite3` `Database` the consumer owns (bring-your-own, mirroring\n   * {@link ISqliteVecVectorIndexCreateParams.database}). The consumer opens it and\n   * owns its lifecycle — this index never closes it. `create` loads the\n   * `sqlite-vec` extension onto the connection and, if the fragment table already\n   * exists (a reopened persistent file), recovers its established dimension so no\n   * re-embedding is required on open.\n   */\n  readonly database: BetterSqlite3.Database;\n\n  /**\n   * Name of the `vec0` virtual table that holds the fragment embeddings. Must be a\n   * simple SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to\n   * `'memory_fragments'`. Supply a distinct name (distinct from any record-level\n   * {@link SqliteVecVectorIndex} table) to hold more than one independent index in\n   * a single database file.\n   */\n  readonly tableName?: string;\n}\n"]}

package/dist/packlets/sqlite-vec-index/sqliteVecFragmentIndex.js

@@ -0,0 +1,277 @@
+/*
+ * Copyright (c) 2026 Erik Fortune
+ * SPDX-License-Identifier: MIT
+ */
+import { load as loadSqliteVec } from 'sqlite-vec';
+import { captureResult, fail, succeed } from '@fgv/ts-utils';
+import { edgeTargetKey } from '@fgv/ts-agent-memory';
+/** Default name for the fragment `vec0` virtual table. */
+const DEFAULT_TABLE_NAME = 'memory_fragments';
+/** A simple SQL identifier — the only shape allowed for the table name (it is interpolated into DDL). */
+const IDENTIFIER_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
+/**
+ * A persistent, `sqlite-vec`-backed `IFragmentVectorIndex` (from
+ * `@fgv/ts-agent-memory`) — the fragment-granular sibling of
+ * {@link SqliteVecVectorIndex}, and the **durable** counterpart to the in-memory
+ * `InMemoryFragmentCosineIndex`.
+ *
+ * @remarks
+ * Where {@link SqliteVecVectorIndex} keys one vector per record on a
+ * `target_key` primary key, this index holds **many** vectors per record — one per
+ * in-record `[start, end)` span — so it keys the `vec0` table on `target_key` as a
+ * **`PARTITION KEY`** (many rows may share it) and stores each fragment's locator
+ * offsets in two auxiliary columns (`+start_off`, `+end_off`) that ride alongside
+ * the vector and are returned on query but never filtered. A query is a brute-force
+ * `vec0` KNN scan across all partitions returning per-fragment hits, each carrying
+ * its record `target` and the matched `locator`.
+ *
+ * Semantics match `InMemoryFragmentCosineIndex` exactly: `addFragments` is
+ * whole-record-replace (a single transaction deletes every prior fragment of the
+ * target, then inserts the new set), `remove` drops every fragment of a target,
+ * and `query` applies the optional `maxPerRecord` cap **during selection, before
+ * the topK cut** — so one long document cannot crowd others out. The dimension is
+ * established by the first `addFragments` (the `vec0` column is fixed-width) and
+ * recovered from the table schema when a persistent file is reopened; similarity is
+ * cosine (`score = 1 - cosineDistance`), byte-identical to the in-memory index.
+ * Large-N ANN indexing is explicitly out of scope, same regime as the record index.
+ *
+ * The `better-sqlite3` `Database` is consumer-owned (bring-your-own): this index
+ * loads the `sqlite-vec` extension onto it and reads/writes the table, but never
+ * opens or closes the connection.
+ * @public
+ */
+export class SqliteVecFragmentIndex {
+    constructor(db, table, dimension) {
+        this._db = db;
+        this._table = table;
+        this._dimension = dimension;
+        this._stmts = dimension === undefined ? undefined : this._prepare();
+    }
+    /** The number of records that currently have at least one stored fragment. Zero before the first add. */
+    get recordCount() {
+        if (this._stmts === undefined) {
+            return 0;
+        }
+        // `Number(...)` narrows the count in case the consumer enabled better-sqlite3
+        // safe-integer mode (which returns `count(*)` as a `bigint`).
+        return Number(this._stmts.recordCount.get().c);
+    }
+    /** The total number of fragments currently held across all records. Zero before the first add. */
+    get fragmentCount() {
+        if (this._stmts === undefined) {
+            return 0;
+        }
+        return Number(this._stmts.fragmentCount.get().c);
+    }
+    /**
+     * Family-convention factory. Loads the `sqlite-vec` extension onto the supplied
+     * `better-sqlite3` connection and, if the fragment table already exists (a
+     * reopened persistent file), recovers its established dimension so no
+     * re-embedding is needed on open.
+     *
+     * @param params - See {@link ISqliteVecFragmentIndexCreateParams}.
+     * @returns `Success` with the index, or `Failure` if the table name is not a
+     * simple identifier or the extension fails to load.
+     */
+    static create(params) {
+        var _a;
+        const table = (_a = params.tableName) !== null && _a !== void 0 ? _a : DEFAULT_TABLE_NAME;
+        if (!IDENTIFIER_RE.test(table)) {
+            return Promise.resolve(fail(`sqlite-vec fragment index: table name '${table}' is not a simple SQL identifier`));
+        }
+        return Promise.resolve(captureResult(() => {
+            loadSqliteVec(params.database);
+            const dimension = SqliteVecFragmentIndex._readExistingDimension(params.database, table);
+            return new SqliteVecFragmentIndex(params.database, table, dimension);
+        }).withErrorFormat((e) => `sqlite-vec fragment index: failed to initialize: ${e}`));
+    }
+    /** {@inheritDoc IFragmentVectorIndex.addFragments} */
+    addFragments(target, fragments) {
+        const key = edgeTargetKey(target);
+        // Validate every fragment before touching the database, so a bad fragment never
+        // leaves the record half-replaced or the dimension half-established (whole-record
+        // replace is all-or-nothing). The effective dimension is the established one, or —
+        // on a still-dimensionless index — the first fragment's length; it is committed
+        // (via table creation) only once the whole batch validates.
+        let dimension = this._dimension;
+        for (const fragment of fragments) {
+            if (fragment.vector.length === 0) {
+                return Promise.resolve(fail(`fragment index: cannot add '${key}': empty fragment vector`));
+            }
+            // Locator offsets are persisted as SQLite integers (bound via BigInt). Reject a
+            // non-safe-integer offset up front with a clear message, rather than letting
+            // `BigInt(nonInteger)` throw cryptically inside the write transaction OR storing
+            // a value the read-side `_toOffset` guard would later reject on every query.
+            if (!Number.isSafeInteger(fragment.locator.start) || !Number.isSafeInteger(fragment.locator.end)) {
+                return Promise.resolve(fail(`fragment index: cannot add '${key}': locator [${fragment.locator.start}, ${fragment.locator.end}) offsets must be safe integers`));
+            }
+            if (dimension === undefined) {
+                dimension = fragment.vector.length;
+            }
+            else if (fragment.vector.length !== dimension) {
+                return Promise.resolve(fail(`fragment index: cannot add '${key}': fragment dimension ${fragment.vector.length} does not match index dimension ${dimension}`));
+            }
+        }
+        return Promise.resolve(captureResult(() => {
+            // A same-target re-author (or an empty batch) still needs the table to exist
+            // to delete prior fragments; create it lazily on the first non-empty add.
+            if (this._stmts === undefined) {
+                if (fragments.length === 0) {
+                    // Nothing stored yet and nothing to store: no table, no work.
+                    return 0;
+                }
+                // `fragments` is non-empty here (the empty case returned above), so the
+                // validation loop proved every fragment shares `fragments[0]`'s length —
+                // which IS the dimension to establish. Read it straight from the first
+                // fragment: no cast, no invariant-dependent narrowing.
+                const established = fragments[0].vector.length;
+                this._createTable(established);
+                this._dimension = established;
+                this._stmts = this._prepare();
+            }
+            this._stmts.replace(key, fragments);
+            return fragments.length;
+        }).withErrorFormat((e) => `fragment index: cannot add '${key}': ${e}`));
+    }
+    /** {@inheritDoc IFragmentVectorIndex.remove} */
+    remove(target) {
+        return Promise.resolve(captureResult(() => {
+            // Idempotent: removing a target with no fragments (or before any add created
+            // the table) still succeeds.
+            if (this._stmts !== undefined) {
+                this._stmts.deleteByTarget.run(edgeTargetKey(target));
+            }
+            return target;
+        }).withErrorFormat((e) => `fragment index: cannot remove '${edgeTargetKey(target)}': ${e}`));
+    }
+    /** {@inheritDoc IFragmentVectorIndex.query} */
+    query(vector, topK, maxPerRecord) {
+        if (topK <= 0 || this._stmts === undefined) {
+            return Promise.resolve(succeed([]));
+        }
+        if (vector.length !== this._dimension) {
+            return Promise.resolve(fail(`fragment index: query dimension ${vector.length} does not match index dimension ${this._dimension}`));
+        }
+        const stmts = this._stmts;
+        return Promise.resolve(captureResult(() => {
+            var _a;
+            // With a per-record cap the topK winners may lie past the first topK rows (a
+            // capped record's later fragments are skipped), so fetch the full ranked set
+            // and apply the cap + topK cut here — exactly as the in-memory index does.
+            // Uncapped, KNN's own `k = topK` is already the answer.
+            const fetchK = maxPerRecord === undefined ? topK : Number(stmts.fragmentCount.get().c);
+            if (fetchK <= 0) {
+                return [];
+            }
+            const rows = stmts.query.all(SqliteVecFragmentIndex._toBlob(vector), fetchK);
+            // sqlite-vec returns rows ascending by distance (nearest first); score is
+            // `1 - cosineDistance`, so this order is already descending score.
+            const hits = [];
+            const perRecord = new Map();
+            for (const row of rows) {
+                if (hits.length >= topK) {
+                    break;
+                }
+                if (maxPerRecord !== undefined) {
+                    const used = (_a = perRecord.get(row.target_key)) !== null && _a !== void 0 ? _a : 0;
+                    if (used >= maxPerRecord) {
+                        continue;
+                    }
+                    perRecord.set(row.target_key, used + 1);
+                }
+                const key = row.target_key;
+                hits.push({
+                    target: SqliteVecFragmentIndex._parseKey(key),
+                    score: 1 - row.distance,
+                    locator: {
+                        start: SqliteVecFragmentIndex._toOffset(row.start_off, key),
+                        end: SqliteVecFragmentIndex._toOffset(row.end_off, key)
+                    }
+                });
+            }
+            return hits;
+        }).withErrorFormat((e) => `fragment index: query failed: ${e}`));
+    }
+    /** Create the fragment `vec0` virtual table with the established dimension. */
+    _createTable(dimension) {
+        this._db.exec(`CREATE VIRTUAL TABLE IF NOT EXISTS "${this._table}" USING vec0(` +
+            `target_key TEXT PARTITION KEY, embedding float[${dimension}] distance_metric=cosine, ` +
+            `+start_off integer, +end_off integer)`);
+    }
+    /** Prepare the statements the index reuses. Requires the table to exist. */
+    _prepare() {
+        const del = this._db.prepare(`DELETE FROM "${this._table}" WHERE target_key = ?`);
+        const ins = this._db.prepare(`INSERT INTO "${this._table}"(target_key, embedding, start_off, end_off) VALUES (?, ?, ?, ?)`);
+        // Whole-record replace: drop every prior fragment of the target, then insert the
+        // new set, atomically. An empty set collapses to a pure delete.
+        const replaceTxn = this._db.transaction((key, fragments) => {
+            del.run(key);
+            for (const fragment of fragments) {
+                ins.run(key, SqliteVecFragmentIndex._toBlob(fragment.vector), 
+                // vec0 typed columns reject a JS float; bind the offsets as integers.
+                BigInt(fragment.locator.start), BigInt(fragment.locator.end));
+            }
+        });
+        return {
+            deleteByTarget: del,
+            replace: (key, fragments) => {
+                replaceTxn(key, fragments);
+            },
+            query: this._db.prepare(`SELECT target_key, start_off, end_off, distance FROM "${this._table}" WHERE embedding MATCH ? AND k = ?`),
+            fragmentCount: this._db.prepare(`SELECT count(*) AS c FROM "${this._table}"`),
+            recordCount: this._db.prepare(`SELECT count(DISTINCT target_key) AS c FROM "${this._table}"`)
+        };
+    }
+    /**
+     * Recover the established dimension of an existing fragment `vec0` table from its
+     * stored `CREATE VIRTUAL TABLE` SQL (`float[<n>]`). Returns `undefined` when the
+     * table does not exist yet (a fresh database — dimension is set by the first add).
+     */
+    static _readExistingDimension(db, table) {
+        const row = db
+            .prepare("SELECT sql FROM sqlite_master WHERE type = 'table' AND name = ?")
+            .get(table);
+        if (row === undefined) {
+            return undefined;
+        }
+        const match = row.sql.match(/float\[(\d+)\]/);
+        return match === null ? undefined : Number(match[1]);
+    }
+    /** Pack a `Float32Array` as the little-endian byte blob `vec0` stores. Copies, so the caller may reuse its buffer. */
+    static _toBlob(vector) {
+        return new Uint8Array(Float32Array.from(vector).buffer);
+    }
+    /**
+     * Reverse `edgeTargetKey` — the canonical key is `scope\0id` with NUL excluded
+     * from both components, so the first NUL splits it unambiguously. A key with no
+     * NUL cannot have been written by `edgeTargetKey`; rather than fabricate a wrong
+     * `(scope, id)` from corrupt / externally-edited table data, throw so the query
+     * surfaces it as a loud `Failure`.
+     */
+    static _parseKey(key) {
+        const nul = key.indexOf('\0');
+        if (nul < 0) {
+            throw new Error(`malformed target key '${key}': missing scope/id separator (corrupt persisted data)`);
+        }
+        return {
+            scope: key.slice(0, nul),
+            id: key.slice(nul + 1)
+        };
+    }
+    /**
+     * Coerce a persisted locator offset to a plain `number`. `better-sqlite3` returns
+     * integer columns as `bigint` under safe-integer mode, so an offset can arrive as
+     * either; both narrow to `number` here. A value outside the safe-integer range
+     * (only reachable via corrupt / externally-edited data — the index only ever
+     * writes in-document offsets) throws rather than silently losing precision, so the
+     * query surfaces it as a loud `Failure`.
+     */
+    static _toOffset(value, key) {
+        const n = Number(value);
+        if (!Number.isSafeInteger(n)) {
+            throw new Error(`fragment '${key}': locator offset ${String(value)} is not a safe integer (corrupt persisted data)`);
+        }
+        return n;
+    }
+}
+//# sourceMappingURL=sqliteVecFragmentIndex.js.map

package/dist/packlets/sqlite-vec-index/sqliteVecFragmentIndex.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"sqliteVecFragmentIndex.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/sqliteVecFragmentIndex.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,IAAI,IAAI,aAAa,EAAE,MAAM,YAAY,CAAC;AACnD,OAAO,EAAU,aAAa,EAAE,IAAI,EAAE,OAAO,EAAE,MAAM,eAAe,CAAC;AACrE,OAAO,EAOL,aAAa,EACd,MAAM,sBAAsB,CAAC;AAG9B,0DAA0D;AAC1D,MAAM,kBAAkB,GAAW,kBAAkB,CAAC;AAEtD,yGAAyG;AACzG,MAAM,aAAa,GAAW,0BAA0B,CAAC;AAgBzD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA8BG;AACH,MAAM,OAAO,sBAAsB;IAQjC,YAAoB,EAA0B,EAAE,KAAa,EAAE,SAA6B;QAC1F,IAAI,CAAC,GAAG,GAAG,EAAE,CAAC;QACd,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC;QAC5B,IAAI,CAAC,MAAM,GAAG,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,EAAE,CAAC;IACtE,CAAC;IAED,yGAAyG;IACzG,IAAW,WAAW;QACpB,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,CAAC,CAAC;QACX,CAAC;QACD,8EAA8E;QAC9E,8DAA8D;QAC9D,OAAO,MAAM,CAAE,IAAI,CAAC,MAAM,CAAC,WAAW,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;IAC7E,CAAC;IAED,kGAAkG;IAClG,IAAW,aAAa;QACtB,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,CAAC,CAAC;QACX,CAAC;QACD,OAAO,MAAM,CAAE,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;IAC/E,CAAC;IAED;;;;;;;;;OASG;IACI,MAAM,CAAC,MAAM,CAAC,MAA2C;;QAC9D,MAAM,KAAK,GAAW,MAAA,MAAM,CAAC,SAAS,mCAAI,kBAAkB,CAAC;QAC7D,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC/B,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CAAC,0CAA0C,KAAK,kCAAkC,CAAC,CACxF,CAAC;QACJ,CAAC;QACD,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,aAAa,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAC/B,MAAM,SAAS,GAAuB,sBAAsB,CAAC,sBAAsB,CACjF,MAAM,CAAC,QAAQ,EACf,KAAK,CACN,CAAC;YACF,OAAO,IAAI,sBAAsB,CAAC,MAAM,CAAC,QAAQ,EAAE,KAAK,EAAE,SAAS,CAAC,CAAC;QACvE,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,oDAAoD,CAAC,EAAE,CAAC,CACnF,CAAC;IACJ,CAAC;IAED,sDAAsD;IAC/C,YAAY,CACjB,MAAmB,EACnB,SAA2C;QAE3C,MAAM,GAAG,GAAW,aAAa,CAAC,MAAM,CAAC,CAAC;QAC1C,gFAAgF;QAChF,kFAAkF;QAClF,mFAAmF;QACnF,gFAAgF;QAChF,4DAA4D;QAC5D,IAAI,SAAS,GAAuB,IAAI,CAAC,UAAU,CAAC;QACpD,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,IAAI,QAAQ,CAAC,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBACjC,OAAO,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,+BAA+B,GAAG,0BAA0B,CAAC,CAAC,CAAC;YAC7F,CAAC;YACD,gFAAgF;YAChF,6EAA6E;YAC7E,iFAAiF;YACjF,6EAA6E;YAC7E,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC;gBACjG,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,eAAe,QAAQ,CAAC,OAAO,CAAC,KAAK,KAAK,QAAQ,CAAC,OAAO,CAAC,GAAG,iCAAiC,CAClI,CACF,CAAC;YACJ,CAAC;YACD,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;gBAC5B,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,MAAM,CAAC;YACrC,CAAC;iBAAM,IAAI,QAAQ,CAAC,MAAM,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAChD,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,yBAAyB,QAAQ,CAAC,MAAM,CAAC,MAAM,mCAAmC,SAAS,EAAE,CAChI,CACF,CAAC;YACJ,CAAC;QACH,CAAC;QACD,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,6EAA6E;YAC7E,0EAA0E;YAC1E,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBAC3B,8DAA8D;oBAC9D,OAAO,CAAC,CAAC;gBACX,CAAC;gBACD,wEAAwE;gBACxE,yEAAyE;gBACzE,uEAAuE;gBACvE,uDAAuD;gBACvD,MAAM,WAAW,GAAW,SAAS,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC;gBACvD,IAAI,CAAC,YAAY,CAAC,WAAW,CAAC,CAAC;gBAC/B,IAAI,CAAC,UAAU,GAAG,WAAW,CAAC;gBAC9B,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;YAChC,CAAC;YACD,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YACpC,OAAO,SAAS,CAAC,MAAM,CAAC;QAC1B,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,+BAA+B,GAAG,MAAM,CAAC,EAAE,CAAC,CACvE,CAAC;IACJ,CAAC;IAED,gDAAgD;IACzC,MAAM,CAAC,MAAmB;QAC/B,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,6EAA6E;YAC7E,6BAA6B;YAC7B,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,IAAI,CAAC,MAAM,CAAC,cAAc,CAAC,GAAG,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC;YACxD,CAAC;YACD,OAAO,MAAM,CAAC;QAChB,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,kCAAkC,aAAa,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,CAC5F,CAAC;IACJ,CAAC;IAED,+CAA+C;IACxC,KAAK,CACV,MAAoB,EACpB,IAAY,EACZ,YAAqB;QAErB,IAAI,IAAI,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC3C,OAAO,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC;QACtC,CAAC;QACD,IAAI,MAAM,CAAC,MAAM,KAAK,IAAI,CAAC,UAAU,EAAE,CAAC;YACtC,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,mCAAmC,MAAM,CAAC,MAAM,mCAAmC,IAAI,CAAC,UAAU,EAAE,CACrG,CACF,CAAC;QACJ,CAAC;QACD,MAAM,KAAK,GAAwB,IAAI,CAAC,MAAM,CAAC;QAC/C,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAiC,GAAG,EAAE;;YACjD,6EAA6E;YAC7E,6EAA6E;YAC7E,2EAA2E;YAC3E,wDAAwD;YACxD,MAAM,MAAM,GACV,YAAY,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAE,KAAK,CAAC,aAAa,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;YACtG,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;gBAChB,OAAO,EAAE,CAAC;YACZ,CAAC;YACD,MAAM,IAAI,GAA2B,KAAK,CAAC,KAAK,CAAC,GAAG,CAClD,sBAAsB,CAAC,OAAO,CAAC,MAAM,CAAC,EACtC,MAAM,CACmB,CAAC;YAC5B,0EAA0E;YAC1E,mEAAmE;YACnE,MAAM,IAAI,GAAsB,EAAE,CAAC;YACnC,MAAM,SAAS,GAAwB,IAAI,GAAG,EAAkB,CAAC;YACjE,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;gBACvB,IAAI,IAAI,CAAC,MAAM,IAAI,IAAI,EAAE,CAAC;oBACxB,MAAM;gBACR,CAAC;gBACD,IAAI,YAAY,KAAK,SAAS,EAAE,CAAC;oBAC/B,MAAM,IAAI,GAAW,MAAA,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,UAAU,CAAC,mCAAI,CAAC,CAAC;oBACxD,IAAI,IAAI,IAAI,YAAY,EAAE,CAAC;wBACzB,SAAS;oBACX,CAAC;oBACD,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,UAAU,EAAE,IAAI,GAAG,CAAC,CAAC,CAAC;gBAC1C,CAAC;gBACD,MAAM,GAAG,GAAW,GAAG,CAAC,UAAU,CAAC;gBACnC,IAAI,CAAC,IAAI,CAAC;oBACR,MAAM,EAAE,sBAAsB,CAAC,SAAS,CAAC,GAAG,CAAC;oBAC7C,KAAK,EAAE,CAAC,GAAG,GAAG,CAAC,QAAQ;oBACvB,OAAO,EAAE;wBACP,KAAK,EAAE,sBAAsB,CAAC,SAAS,CAAC,GAAG,CAAC,SAAS,EAAE,GAAG,CAAC;wBAC3D,GAAG,EAAE,sBAAsB,CAAC,SAAS,CAAC,GAAG,CAAC,OAAO,EAAE,GAAG,CAAC;qBACxD;iBACF,CAAC,CAAC;YACL,CAAC;YACD,OAAO,IAAI,CAAC;QACd,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,iCAAiC,CAAC,EAAE,CAAC,CAChE,CAAC;IACJ,CAAC;IAED,+EAA+E;IACvE,YAAY,CAAC,SAAiB;QACpC,IAAI,CAAC,GAAG,CAAC,IAAI,CACX,uCAAuC,IAAI,CAAC,MAAM,eAAe;YAC/D,kDAAkD,SAAS,4BAA4B;YACvF,uCAAuC,CAC1C,CAAC;IACJ,CAAC;IAED,4EAA4E;IACpE,QAAQ;QACd,MAAM,GAAG,GAA4B,IAAI,CAAC,GAAG,CAAC,OAAO,CACnD,gBAAgB,IAAI,CAAC,MAAM,wBAAwB,CACpD,CAAC;QACF,MAAM,GAAG,GAA4B,IAAI,CAAC,GAAG,CAAC,OAAO,CACnD,gBAAgB,IAAI,CAAC,MAAM,kEAAkE,CAC9F,CAAC;QACF,iFAAiF;QACjF,gEAAgE;QAChE,MAAM,UAAU,GAEZ,IAAI,CAAC,GAAG,CAAC,WAAW,CAAC,CAAC,GAAW,EAAE,SAA2C,EAAE,EAAE;YACpF,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;YACb,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;gBACjC,GAAG,CAAC,GAAG,CACL,GAAG,EACH,sBAAsB,CAAC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC;gBAC/C,sEAAsE;gBACtE,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,EAC9B,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,CAC7B,CAAC;YACJ,CAAC;QACH,CAAC,CAAC,CAAC;QACH,OAAO;YACL,cAAc,EAAE,GAAG;YACnB,OAAO,EAAE,CAAC,GAAW,EAAE,SAA2C,EAAQ,EAAE;gBAC1E,UAAU,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YAC7B,CAAC;YACD,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CACrB,yDAAyD,IAAI,CAAC,MAAM,qCAAqC,CAC1G;YACD,aAAa,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,8BAA8B,IAAI,CAAC,MAAM,GAAG,CAAC;YAC7E,WAAW,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,gDAAgD,IAAI,CAAC,MAAM,GAAG,CAAC;SAC9F,CAAC;IACJ,CAAC;IAED;;;;OAIG;IACK,MAAM,CAAC,sBAAsB,CAAC,EAA0B,EAAE,KAAa;QAC7E,MAAM,GAAG,GAAgC,EAAE;aACxC,OAAO,CAAC,iEAAiE,CAAC;aAC1E,GAAG,CAAC,KAAK,CAAgC,CAAC;QAC7C,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;YACtB,OAAO,SAAS,CAAC;QACnB,CAAC;QACD,MAAM,KAAK,GAA4B,GAAG,CAAC,GAAG,CAAC,KAAK,CAAC,gBAAgB,CAAC,CAAC;QACvE,OAAO,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IACvD,CAAC;IAED,sHAAsH;IAC9G,MAAM,CAAC,OAAO,CAAC,MAAoB;QACzC,OAAO,IAAI,UAAU,CAAC,YAAY,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC;IAC1D,CAAC;IAED;;;;;;OAMG;IACK,MAAM,CAAC,SAAS,CAAC,GAAW;QAClC,MAAM,GAAG,GAAW,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;QACtC,IAAI,GAAG,GAAG,CAAC,EAAE,CAAC;YACZ,MAAM,IAAI,KAAK,CAAC,yBAAyB,GAAG,wDAAwD,CAAC,CAAC;QACxG,CAAC;QACD,OAAO;YACL,KAAK,EAAE,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAA8B;YACrD,EAAE,EAAE,GAAG,CAAC,KAAK,CAAC,GAAG,GAAG,CAAC,CAAwB;SAC9C,CAAC;IACJ,CAAC;IAED;;;;;;;OAOG;IACK,MAAM,CAAC,SAAS,CAAC,KAAsB,EAAE,GAAW;QAC1D,MAAM,CAAC,GAAW,MAAM,CAAC,KAAK,CAAC,CAAC;QAChC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,CAAC;YAC7B,MAAM,IAAI,KAAK,CACb,aAAa,GAAG,qBAAqB,MAAM,CAAC,KAAK,CAAC,iDAAiD,CACpG,CAAC;QACJ,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC;CACF","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nimport type BetterSqlite3 from 'better-sqlite3';\nimport { load as loadSqliteVec } from 'sqlite-vec';\nimport { Result, captureResult, fail, succeed } from '@fgv/ts-utils';\nimport {\n  IEdgeTarget,\n  IEmbeddedFragment,\n  IFragmentVectorIndex,\n  IVectorQueryHit,\n  MemoryId,\n  MemoryScopeKey,\n  edgeTargetKey\n} from '@fgv/ts-agent-memory';\nimport { ISqliteVecFragmentIndexCreateParams } from './model';\n\n/** Default name for the fragment `vec0` virtual table. */\nconst DEFAULT_TABLE_NAME: string = 'memory_fragments';\n\n/** A simple SQL identifier — the only shape allowed for the table name (it is interpolated into DDL). */\nconst IDENTIFIER_RE: RegExp = /^[A-Za-z_][A-Za-z0-9_]*$/;\n\n/**\n * One KNN row as returned by the fragment `vec0` MATCH query. The offset columns are\n * typed `number | bigint` because `better-sqlite3` returns integer columns as\n * `bigint` when a consumer enables its safe-integer mode (`defaultSafeIntegers`);\n * {@link SqliteVecFragmentIndex._toOffset} coerces them to a plain `number` (and\n * fails loudly on an out-of-safe-range value) before they reach the public locator.\n */\ninterface IKnnRow {\n  readonly target_key: string;\n  readonly start_off: number | bigint;\n  readonly end_off: number | bigint;\n  readonly distance: number;\n}\n\n/**\n * A persistent, `sqlite-vec`-backed `IFragmentVectorIndex` (from\n * `@fgv/ts-agent-memory`) — the fragment-granular sibling of\n * {@link SqliteVecVectorIndex}, and the **durable** counterpart to the in-memory\n * `InMemoryFragmentCosineIndex`.\n *\n * @remarks\n * Where {@link SqliteVecVectorIndex} keys one vector per record on a\n * `target_key` primary key, this index holds **many** vectors per record — one per\n * in-record `[start, end)` span — so it keys the `vec0` table on `target_key` as a\n * **`PARTITION KEY`** (many rows may share it) and stores each fragment's locator\n * offsets in two auxiliary columns (`+start_off`, `+end_off`) that ride alongside\n * the vector and are returned on query but never filtered. A query is a brute-force\n * `vec0` KNN scan across all partitions returning per-fragment hits, each carrying\n * its record `target` and the matched `locator`.\n *\n * Semantics match `InMemoryFragmentCosineIndex` exactly: `addFragments` is\n * whole-record-replace (a single transaction deletes every prior fragment of the\n * target, then inserts the new set), `remove` drops every fragment of a target,\n * and `query` applies the optional `maxPerRecord` cap **during selection, before\n * the topK cut** — so one long document cannot crowd others out. The dimension is\n * established by the first `addFragments` (the `vec0` column is fixed-width) and\n * recovered from the table schema when a persistent file is reopened; similarity is\n * cosine (`score = 1 - cosineDistance`), byte-identical to the in-memory index.\n * Large-N ANN indexing is explicitly out of scope, same regime as the record index.\n *\n * The `better-sqlite3` `Database` is consumer-owned (bring-your-own): this index\n * loads the `sqlite-vec` extension onto it and reads/writes the table, but never\n * opens or closes the connection.\n * @public\n */\nexport class SqliteVecFragmentIndex implements IFragmentVectorIndex {\n  private readonly _db: BetterSqlite3.Database;\n  private readonly _table: string;\n  /** The dimension of every stored fragment vector; `undefined` until the table exists. */\n  private _dimension: number | undefined;\n  /** Prepared statements; created once the table exists (established or recovered). */\n  private _stmts: IFragmentStatements | undefined;\n\n  private constructor(db: BetterSqlite3.Database, table: string, dimension: number | undefined) {\n    this._db = db;\n    this._table = table;\n    this._dimension = dimension;\n    this._stmts = dimension === undefined ? undefined : this._prepare();\n  }\n\n  /** The number of records that currently have at least one stored fragment. Zero before the first add. */\n  public get recordCount(): number {\n    if (this._stmts === undefined) {\n      return 0;\n    }\n    // `Number(...)` narrows the count in case the consumer enabled better-sqlite3\n    // safe-integer mode (which returns `count(*)` as a `bigint`).\n    return Number((this._stmts.recordCount.get() as { c: number | bigint }).c);\n  }\n\n  /** The total number of fragments currently held across all records. Zero before the first add. */\n  public get fragmentCount(): number {\n    if (this._stmts === undefined) {\n      return 0;\n    }\n    return Number((this._stmts.fragmentCount.get() as { c: number | bigint }).c);\n  }\n\n  /**\n   * Family-convention factory. Loads the `sqlite-vec` extension onto the supplied\n   * `better-sqlite3` connection and, if the fragment table already exists (a\n   * reopened persistent file), recovers its established dimension so no\n   * re-embedding is needed on open.\n   *\n   * @param params - See {@link ISqliteVecFragmentIndexCreateParams}.\n   * @returns `Success` with the index, or `Failure` if the table name is not a\n   * simple identifier or the extension fails to load.\n   */\n  public static create(params: ISqliteVecFragmentIndexCreateParams): Promise<Result<SqliteVecFragmentIndex>> {\n    const table: string = params.tableName ?? DEFAULT_TABLE_NAME;\n    if (!IDENTIFIER_RE.test(table)) {\n      return Promise.resolve(\n        fail(`sqlite-vec fragment index: table name '${table}' is not a simple SQL identifier`)\n      );\n    }\n    return Promise.resolve(\n      captureResult(() => {\n        loadSqliteVec(params.database);\n        const dimension: number | undefined = SqliteVecFragmentIndex._readExistingDimension(\n          params.database,\n          table\n        );\n        return new SqliteVecFragmentIndex(params.database, table, dimension);\n      }).withErrorFormat((e) => `sqlite-vec fragment index: failed to initialize: ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.addFragments} */\n  public addFragments(\n    target: IEdgeTarget,\n    fragments: ReadonlyArray<IEmbeddedFragment>\n  ): Promise<Result<number>> {\n    const key: string = edgeTargetKey(target);\n    // Validate every fragment before touching the database, so a bad fragment never\n    // leaves the record half-replaced or the dimension half-established (whole-record\n    // replace is all-or-nothing). The effective dimension is the established one, or —\n    // on a still-dimensionless index — the first fragment's length; it is committed\n    // (via table creation) only once the whole batch validates.\n    let dimension: number | undefined = this._dimension;\n    for (const fragment of fragments) {\n      if (fragment.vector.length === 0) {\n        return Promise.resolve(fail(`fragment index: cannot add '${key}': empty fragment vector`));\n      }\n      // Locator offsets are persisted as SQLite integers (bound via BigInt). Reject a\n      // non-safe-integer offset up front with a clear message, rather than letting\n      // `BigInt(nonInteger)` throw cryptically inside the write transaction OR storing\n      // a value the read-side `_toOffset` guard would later reject on every query.\n      if (!Number.isSafeInteger(fragment.locator.start) || !Number.isSafeInteger(fragment.locator.end)) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': locator [${fragment.locator.start}, ${fragment.locator.end}) offsets must be safe integers`\n          )\n        );\n      }\n      if (dimension === undefined) {\n        dimension = fragment.vector.length;\n      } else if (fragment.vector.length !== dimension) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': fragment dimension ${fragment.vector.length} does not match index dimension ${dimension}`\n          )\n        );\n      }\n    }\n    return Promise.resolve(\n      captureResult(() => {\n        // A same-target re-author (or an empty batch) still needs the table to exist\n        // to delete prior fragments; create it lazily on the first non-empty add.\n        if (this._stmts === undefined) {\n          if (fragments.length === 0) {\n            // Nothing stored yet and nothing to store: no table, no work.\n            return 0;\n          }\n          // `fragments` is non-empty here (the empty case returned above), so the\n          // validation loop proved every fragment shares `fragments[0]`'s length —\n          // which IS the dimension to establish. Read it straight from the first\n          // fragment: no cast, no invariant-dependent narrowing.\n          const established: number = fragments[0].vector.length;\n          this._createTable(established);\n          this._dimension = established;\n          this._stmts = this._prepare();\n        }\n        this._stmts.replace(key, fragments);\n        return fragments.length;\n      }).withErrorFormat((e) => `fragment index: cannot add '${key}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.remove} */\n  public remove(target: IEdgeTarget): Promise<Result<IEdgeTarget>> {\n    return Promise.resolve(\n      captureResult(() => {\n        // Idempotent: removing a target with no fragments (or before any add created\n        // the table) still succeeds.\n        if (this._stmts !== undefined) {\n          this._stmts.deleteByTarget.run(edgeTargetKey(target));\n        }\n        return target;\n      }).withErrorFormat((e) => `fragment index: cannot remove '${edgeTargetKey(target)}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.query} */\n  public query(\n    vector: Float32Array,\n    topK: number,\n    maxPerRecord?: number\n  ): Promise<Result<ReadonlyArray<IVectorQueryHit>>> {\n    if (topK <= 0 || this._stmts === undefined) {\n      return Promise.resolve(succeed([]));\n    }\n    if (vector.length !== this._dimension) {\n      return Promise.resolve(\n        fail(\n          `fragment index: query dimension ${vector.length} does not match index dimension ${this._dimension}`\n        )\n      );\n    }\n    const stmts: IFragmentStatements = this._stmts;\n    return Promise.resolve(\n      captureResult<ReadonlyArray<IVectorQueryHit>>(() => {\n        // With a per-record cap the topK winners may lie past the first topK rows (a\n        // capped record's later fragments are skipped), so fetch the full ranked set\n        // and apply the cap + topK cut here — exactly as the in-memory index does.\n        // Uncapped, KNN's own `k = topK` is already the answer.\n        const fetchK: number =\n          maxPerRecord === undefined ? topK : Number((stmts.fragmentCount.get() as { c: number | bigint }).c);\n        if (fetchK <= 0) {\n          return [];\n        }\n        const rows: ReadonlyArray<IKnnRow> = stmts.query.all(\n          SqliteVecFragmentIndex._toBlob(vector),\n          fetchK\n        ) as ReadonlyArray<IKnnRow>;\n        // sqlite-vec returns rows ascending by distance (nearest first); score is\n        // `1 - cosineDistance`, so this order is already descending score.\n        const hits: IVectorQueryHit[] = [];\n        const perRecord: Map<string, number> = new Map<string, number>();\n        for (const row of rows) {\n          if (hits.length >= topK) {\n            break;\n          }\n          if (maxPerRecord !== undefined) {\n            const used: number = perRecord.get(row.target_key) ?? 0;\n            if (used >= maxPerRecord) {\n              continue;\n            }\n            perRecord.set(row.target_key, used + 1);\n          }\n          const key: string = row.target_key;\n          hits.push({\n            target: SqliteVecFragmentIndex._parseKey(key),\n            score: 1 - row.distance,\n            locator: {\n              start: SqliteVecFragmentIndex._toOffset(row.start_off, key),\n              end: SqliteVecFragmentIndex._toOffset(row.end_off, key)\n            }\n          });\n        }\n        return hits;\n      }).withErrorFormat((e) => `fragment index: query failed: ${e}`)\n    );\n  }\n\n  /** Create the fragment `vec0` virtual table with the established dimension. */\n  private _createTable(dimension: number): void {\n    this._db.exec(\n      `CREATE VIRTUAL TABLE IF NOT EXISTS \"${this._table}\" USING vec0(` +\n        `target_key TEXT PARTITION KEY, embedding float[${dimension}] distance_metric=cosine, ` +\n        `+start_off integer, +end_off integer)`\n    );\n  }\n\n  /** Prepare the statements the index reuses. Requires the table to exist. */\n  private _prepare(): IFragmentStatements {\n    const del: BetterSqlite3.Statement = this._db.prepare(\n      `DELETE FROM \"${this._table}\" WHERE target_key = ?`\n    );\n    const ins: BetterSqlite3.Statement = this._db.prepare(\n      `INSERT INTO \"${this._table}\"(target_key, embedding, start_off, end_off) VALUES (?, ?, ?, ?)`\n    );\n    // Whole-record replace: drop every prior fragment of the target, then insert the\n    // new set, atomically. An empty set collapses to a pure delete.\n    const replaceTxn: BetterSqlite3.Transaction<\n      (key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => void\n    > = this._db.transaction((key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => {\n      del.run(key);\n      for (const fragment of fragments) {\n        ins.run(\n          key,\n          SqliteVecFragmentIndex._toBlob(fragment.vector),\n          // vec0 typed columns reject a JS float; bind the offsets as integers.\n          BigInt(fragment.locator.start),\n          BigInt(fragment.locator.end)\n        );\n      }\n    });\n    return {\n      deleteByTarget: del,\n      replace: (key: string, fragments: ReadonlyArray<IEmbeddedFragment>): void => {\n        replaceTxn(key, fragments);\n      },\n      query: this._db.prepare(\n        `SELECT target_key, start_off, end_off, distance FROM \"${this._table}\" WHERE embedding MATCH ? AND k = ?`\n      ),\n      fragmentCount: this._db.prepare(`SELECT count(*) AS c FROM \"${this._table}\"`),\n      recordCount: this._db.prepare(`SELECT count(DISTINCT target_key) AS c FROM \"${this._table}\"`)\n    };\n  }\n\n  /**\n   * Recover the established dimension of an existing fragment `vec0` table from its\n   * stored `CREATE VIRTUAL TABLE` SQL (`float[<n>]`). Returns `undefined` when the\n   * table does not exist yet (a fresh database — dimension is set by the first add).\n   */\n  private static _readExistingDimension(db: BetterSqlite3.Database, table: string): number | undefined {\n    const row: { sql: string } | undefined = db\n      .prepare(\"SELECT sql FROM sqlite_master WHERE type = 'table' AND name = ?\")\n      .get(table) as { sql: string } | undefined;\n    if (row === undefined) {\n      return undefined;\n    }\n    const match: RegExpMatchArray | null = row.sql.match(/float\\[(\\d+)\\]/);\n    return match === null ? undefined : Number(match[1]);\n  }\n\n  /** Pack a `Float32Array` as the little-endian byte blob `vec0` stores. Copies, so the caller may reuse its buffer. */\n  private static _toBlob(vector: Float32Array): Uint8Array {\n    return new Uint8Array(Float32Array.from(vector).buffer);\n  }\n\n  /**\n   * Reverse `edgeTargetKey` — the canonical key is `scope\\0id` with NUL excluded\n   * from both components, so the first NUL splits it unambiguously. A key with no\n   * NUL cannot have been written by `edgeTargetKey`; rather than fabricate a wrong\n   * `(scope, id)` from corrupt / externally-edited table data, throw so the query\n   * surfaces it as a loud `Failure`.\n   */\n  private static _parseKey(key: string): IEdgeTarget {\n    const nul: number = key.indexOf('\\0');\n    if (nul < 0) {\n      throw new Error(`malformed target key '${key}': missing scope/id separator (corrupt persisted data)`);\n    }\n    return {\n      scope: key.slice(0, nul) as unknown as MemoryScopeKey,\n      id: key.slice(nul + 1) as unknown as MemoryId\n    };\n  }\n\n  /**\n   * Coerce a persisted locator offset to a plain `number`. `better-sqlite3` returns\n   * integer columns as `bigint` under safe-integer mode, so an offset can arrive as\n   * either; both narrow to `number` here. A value outside the safe-integer range\n   * (only reachable via corrupt / externally-edited data — the index only ever\n   * writes in-document offsets) throws rather than silently losing precision, so the\n   * query surfaces it as a loud `Failure`.\n   */\n  private static _toOffset(value: number | bigint, key: string): number {\n    const n: number = Number(value);\n    if (!Number.isSafeInteger(n)) {\n      throw new Error(\n        `fragment '${key}': locator offset ${String(value)} is not a safe integer (corrupt persisted data)`\n      );\n    }\n    return n;\n  }\n}\n\n/** The prepared statements / helpers the fragment index reuses once its table exists. */\ninterface IFragmentStatements {\n  readonly deleteByTarget: BetterSqlite3.Statement;\n  readonly replace: (key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => void;\n  readonly query: BetterSqlite3.Statement;\n  readonly fragmentCount: BetterSqlite3.Statement;\n  readonly recordCount: BetterSqlite3.Statement;\n}\n"]}