@fgv/ts-agent-memory-sqlite-vec 5.1.0-50 → 5.1.0-51

package/README.md

@@ -25,6 +25,31 @@ rush add -p @fgv/ts-agent-memory-sqlite-vec   # or npm/pnpm add
 
 ## Quick start
 
+Two factories, differing only in **who owns the connection**. If this index is the only
+thing on the file, `open` is the shorter path and needs no `better-sqlite3` import of
+your own:
+
+```ts
+import { SqliteVecVectorIndex } from '@fgv/ts-agent-memory-sqlite-vec';
+import { FileTreeMemoryStore } from '@fgv/ts-agent-memory';
+
+// We open the file and hand back a disposer for the connection we created.
+const handle = (await SqliteVecVectorIndex.open({ path: '/path/to/vault/vectors.db' })).orThrow();
+
+const store = (
+  await FileTreeMemoryStore.create({ root, registry, vectorIndex: handle.index, embed })
+).orThrow();
+
+// ...use the store; embeddings are written to vectors.db on every put.
+handle.close(); // closes the connection THIS open() created; idempotent.
+```
+
+`SqliteVecFragmentIndex.open({ path })` is the identical shape for the fragment lane.
+
+**Two `open` calls on one path give two independent connections, not a shared one.** To
+back a record index *and* a fragment index with a single connection, own it yourself and
+pass it to both `create` methods:
+
 ```ts
 import Database from 'better-sqlite3';
 import { SqliteVecVectorIndex } from '@fgv/ts-agent-memory-sqlite-vec';
@@ -108,7 +133,7 @@ Known instance: the release that added `IEmbeddedFragment.fragmentId` added a `+
 Deliberately excluded — reach for the upstream libraries (or a different backend) directly if you need these:
 
 - **ANN / large-N indexing.** Query is a brute-force `vec0` KNN scan — correct and durable for the same "thousands of records" regime the in-memory index targets. An approximate-nearest-neighbor structure for very large N is a different backend behind the same `IVectorIndex` / `IFragmentVectorIndex` seam. (This applies to both indexes, including `SqliteVecFragmentIndex`, whose capped query fetches the full ranked set.)
-- **Connection lifecycle.** You open and close the `better-sqlite3` `Database`; this index never does. Pooling, WAL/pragma tuning, backups, and multi-process coordination are yours.
+- **Connection lifecycle beyond plain open/close.** With `create({ database })` you open and close the `better-sqlite3` `Database` and this index never does; with `open({ path })` this package opens the file and the returned handle's `close()` disposes of exactly what it opened. Either way, pooling, WAL/pragma tuning, backups, and multi-process coordination are yours.
 - **Embedding.** This is a vector *index*, not an embedder — the store's consumer-wired `MemoryEmbedder` produces the vectors (`@fgv/ts-extras/ai-assist` `callProviderEmbedding`, `@fgv/ts-extras-transformers`, etc.).
 - **A browser sibling.** `better-sqlite3` is Node-only. A WASM-SQLite browser variant, if ever needed, is a separate package.
 - **Schema migration of any kind.** Re-embedding with a different-dimension model against an existing table fails loudly, and a package release that changes a `vec0` table's columns requires a drop-and-re-index — see [Upgrading](#upgrading-vec0-schema-changes-require-a-drop-and-re-index). Drop the table (or use a new `tableName`) to re-index; `vec0` cannot be altered in place.

package/dist/packlets/sqlite-vec-index/connection.js

@@ -0,0 +1,54 @@
+/*
+ * Copyright (c) 2026 Erik Fortune
+ * SPDX-License-Identifier: MIT
+ */
+import { captureAsyncResult, captureResult } from '@fgv/ts-utils';
+/**
+ * Opens a `better-sqlite3` connection this package owns.
+ *
+ * @remarks
+ * **The value import of `better-sqlite3` lives here, and it is deliberately
+ * lazy.** Every other module in this package imports the driver as `import type`
+ * only, so merely importing `@fgv/ts-agent-memory-sqlite-vec` has never loaded the
+ * native binding — a property the path-based factories must not cost consumers who
+ * only ever call `create()`. A static import would move the native load (and its
+ * failure mode) to package-load time for everyone. The dynamic `import()` keeps it
+ * at the one call that actually needs a connection.
+ *
+ * Taking this import is the entire point of the path-based factories: it is the
+ * only place this wrapper leaked its own dependency into consumer source.
+ *
+ * @param path - Filesystem path to the database file. `':memory:'` is accepted and
+ * yields an owned ephemeral connection.
+ * @param label - Package-facing prefix for the failure message.
+ * @returns `Success` with a connection this package owns, or `Failure` if the
+ * driver could not be loaded or the file could not be opened.
+ * @internal
+ */
+export async function openOwnedConnection(path, label) {
+    return (await captureAsyncResult(async () => (await import('better-sqlite3')).default))
+        .withErrorFormat((m) => `${label}: failed to load the 'better-sqlite3' driver: ${m}`)
+        .onSuccess((driver) => captureResult(() => new driver(path)).withErrorFormat((m) => `${label}: failed to open '${path}': ${m}`));
+}
+/**
+ * Closes a connection this package opened.
+ *
+ * @remarks
+ * Only ever called on a connection produced by {@link openOwnedConnection}. A
+ * connection supplied through a `create()` param is the consumer's and is never
+ * closed here — that asymmetry is the reason `close` travels on the handle
+ * returned by `open` rather than sitting on the index class.
+ *
+ * `better-sqlite3`'s own `close()` is safe to call on an already-closed
+ * connection, so a second `close()` on the same handle succeeds rather than
+ * failing.
+ *
+ * @internal
+ */
+export function closeOwnedConnection(db, label) {
+    return captureResult(() => {
+        db.close();
+        return true;
+    }).withErrorFormat((m) => `${label}: failed to close the connection: ${m}`);
+}
+//# sourceMappingURL=connection.js.map

package/dist/packlets/sqlite-vec-index/connection.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"connection.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/connection.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAU,kBAAkB,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAE1E;;;;;;;;;;;;;;;;;;;;;GAqBG;AACH,MAAM,CAAC,KAAK,UAAU,mBAAmB,CACvC,IAAY,EACZ,KAAa;IAEb,OAAO,CAAC,MAAM,kBAAkB,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC,MAAM,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;SACpF,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,KAAK,iDAAiD,CAAC,EAAE,CAAC;SACpF,SAAS,CAAC,CAAC,MAAM,EAAE,EAAE,CACpB,aAAa,CAAC,GAAG,EAAE,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,KAAK,qBAAqB,IAAI,MAAM,CAAC,EAAE,CAAC,CACzG,CAAC;AACN,CAAC;AAED;;;;;;;;;;;;;;GAcG;AACH,MAAM,UAAU,oBAAoB,CAAC,EAA0B,EAAE,KAAa;IAC5E,OAAO,aAAa,CAAO,GAAG,EAAE;QAC9B,EAAE,CAAC,KAAK,EAAE,CAAC;QACX,OAAO,IAAI,CAAC;IACd,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,KAAK,qCAAqC,CAAC,EAAE,CAAC,CAAC;AAC9E,CAAC","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nimport type BetterSqlite3 from 'better-sqlite3';\nimport { Result, captureAsyncResult, captureResult } from '@fgv/ts-utils';\n\n/**\n * Opens a `better-sqlite3` connection this package owns.\n *\n * @remarks\n * **The value import of `better-sqlite3` lives here, and it is deliberately\n * lazy.** Every other module in this package imports the driver as `import type`\n * only, so merely importing `@fgv/ts-agent-memory-sqlite-vec` has never loaded the\n * native binding — a property the path-based factories must not cost consumers who\n * only ever call `create()`. A static import would move the native load (and its\n * failure mode) to package-load time for everyone. The dynamic `import()` keeps it\n * at the one call that actually needs a connection.\n *\n * Taking this import is the entire point of the path-based factories: it is the\n * only place this wrapper leaked its own dependency into consumer source.\n *\n * @param path - Filesystem path to the database file. `':memory:'` is accepted and\n * yields an owned ephemeral connection.\n * @param label - Package-facing prefix for the failure message.\n * @returns `Success` with a connection this package owns, or `Failure` if the\n * driver could not be loaded or the file could not be opened.\n * @internal\n */\nexport async function openOwnedConnection(\n  path: string,\n  label: string\n): Promise<Result<BetterSqlite3.Database>> {\n  return (await captureAsyncResult(async () => (await import('better-sqlite3')).default))\n    .withErrorFormat((m) => `${label}: failed to load the 'better-sqlite3' driver: ${m}`)\n    .onSuccess((driver) =>\n      captureResult(() => new driver(path)).withErrorFormat((m) => `${label}: failed to open '${path}': ${m}`)\n    );\n}\n\n/**\n * Closes a connection this package opened.\n *\n * @remarks\n * Only ever called on a connection produced by {@link openOwnedConnection}. A\n * connection supplied through a `create()` param is the consumer's and is never\n * closed here — that asymmetry is the reason `close` travels on the handle\n * returned by `open` rather than sitting on the index class.\n *\n * `better-sqlite3`'s own `close()` is safe to call on an already-closed\n * connection, so a second `close()` on the same handle succeeds rather than\n * failing.\n *\n * @internal\n */\nexport function closeOwnedConnection(db: BetterSqlite3.Database, label: string): Result<true> {\n  return captureResult<true>(() => {\n    db.close();\n    return true;\n  }).withErrorFormat((m) => `${label}: failed to close the connection: ${m}`);\n}\n"]}

package/dist/packlets/sqlite-vec-index/model.js.map

@@ -1 +1 @@
-{"version":3,"file":"model.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/model.ts"],"names":[],"mappings":"AAAA;;;GAGG","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nimport type BetterSqlite3 from 'better-sqlite3';\n\n/**\n * Parameters for {@link SqliteVecVectorIndex.create}.\n * @public\n */\nexport interface ISqliteVecVectorIndexCreateParams {\n  /**\n   * A `better-sqlite3` `Database` the consumer owns (bring-your-own, mirroring\n   * the boundary-package convention). The consumer opens it (`new Database(path)`\n   * for a persistent file, or `new Database(':memory:')` for an ephemeral index)\n   * and owns its lifecycle — this index never closes it. `create` loads the\n   * `sqlite-vec` extension onto the connection and, if the vector table already\n   * exists (a reopened persistent file), recovers its established dimension so no\n   * re-embedding is required on open.\n   */\n  readonly database: BetterSqlite3.Database;\n\n  /**\n   * Name of the `vec0` virtual table that holds the embeddings. Must be a simple\n   * SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to `'memory_vectors'`.\n   * Supply a distinct name to hold more than one independent index in a single\n   * database file.\n   */\n  readonly tableName?: string;\n}\n\n/**\n * Parameters for {@link SqliteVecFragmentIndex.create}.\n * @public\n */\nexport interface ISqliteVecFragmentIndexCreateParams {\n  /**\n   * A `better-sqlite3` `Database` the consumer owns (bring-your-own, mirroring\n   * {@link ISqliteVecVectorIndexCreateParams.database}). The consumer opens it and\n   * owns its lifecycle — this index never closes it. `create` loads the\n   * `sqlite-vec` extension onto the connection and, if the fragment table already\n   * exists (a reopened persistent file), recovers its established dimension so no\n   * re-embedding is required on open.\n   */\n  readonly database: BetterSqlite3.Database;\n\n  /**\n   * Name of the `vec0` virtual table that holds the fragment embeddings. Must be a\n   * simple SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to\n   * `'memory_fragments'`. Supply a distinct name (distinct from any record-level\n   * {@link SqliteVecVectorIndex} table) to hold more than one independent index in\n   * a single database file.\n   */\n  readonly tableName?: string;\n}\n"]}
+{"version":3,"file":"model.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/model.ts"],"names":[],"mappings":"AAAA;;;GAGG","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nimport type BetterSqlite3 from 'better-sqlite3';\nimport type { Result } from '@fgv/ts-utils';\nimport type { SqliteVecFragmentIndex } from './sqliteVecFragmentIndex';\nimport type { SqliteVecVectorIndex } from './sqliteVecVectorIndex';\n\n/**\n * Parameters for {@link SqliteVecVectorIndex.create}.\n * @public\n */\nexport interface ISqliteVecVectorIndexCreateParams {\n  /**\n   * A `better-sqlite3` `Database` the consumer owns (bring-your-own, mirroring\n   * the boundary-package convention). The consumer opens it (`new Database(path)`\n   * for a persistent file, or `new Database(':memory:')` for an ephemeral index)\n   * and owns its lifecycle — this index never closes it. `create` loads the\n   * `sqlite-vec` extension onto the connection and, if the vector table already\n   * exists (a reopened persistent file), recovers its established dimension so no\n   * re-embedding is required on open.\n   */\n  readonly database: BetterSqlite3.Database;\n\n  /**\n   * Name of the `vec0` virtual table that holds the embeddings. Must be a simple\n   * SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to `'memory_vectors'`.\n   * Supply a distinct name to hold more than one independent index in a single\n   * database file.\n   */\n  readonly tableName?: string;\n}\n\n/**\n * Parameters for {@link SqliteVecVectorIndex.open}.\n * @public\n */\nexport interface ISqliteVecVectorIndexOpenParams {\n  /**\n   * Filesystem path to the database file, opened by this package rather than by\n   * the consumer. Created if it does not exist, exactly as `better-sqlite3` would.\n   * `':memory:'` is accepted and yields an owned ephemeral connection.\n   *\n   * **Two `open` calls on one path produce two independent connections, not a\n   * shared one.** That is legal in SQLite and has a different locking story than\n   * the single-connection case — writes contend, and a reader can see a\n   * `SQLITE_BUSY`. To put a record index and a fragment index on one connection\n   * (the intended shared-handle case), open the connection yourself and pass it to\n   * both `create` methods.\n   */\n  readonly path: string;\n\n  /**\n   * Name of the `vec0` virtual table that holds the embeddings. Must be a simple\n   * SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to `'memory_vectors'`.\n   * Supply a distinct name to hold more than one independent index in a single\n   * database file.\n   */\n  readonly tableName?: string;\n}\n\n/**\n * An index plus the connection {@link SqliteVecVectorIndex.open} opened for it.\n *\n * @public\n */\nexport interface ISqliteVecVectorIndexHandle {\n  /** The index, ready to use. */\n  readonly index: SqliteVecVectorIndex;\n\n  /**\n   * Closes the connection **this `open` call created**. Idempotent — a second\n   * `close` succeeds rather than failing.\n   *\n   * @remarks\n   * The disposer travels on this handle rather than on the index class because an\n   * index built by `create` holds a connection the **consumer** owns, and must stay\n   * incapable of closing it. A `close()` method meaningful on some instances and\n   * forbidden on others would be a lie in the type; here, only the caller that\n   * caused the connection to exist is handed the means to end it.\n   *\n   * The index is unusable afterwards — every operation on it will fail against a\n   * closed connection.\n   */\n  close(): Result<true>;\n}\n\n/**\n * Parameters for {@link SqliteVecFragmentIndex.open}.\n * @public\n */\nexport interface ISqliteVecFragmentIndexOpenParams {\n  /**\n   * Filesystem path to the database file, opened by this package rather than by\n   * the consumer. Created if it does not exist, exactly as `better-sqlite3` would.\n   * `':memory:'` is accepted and yields an owned ephemeral connection.\n   *\n   * **Two `open` calls on one path produce two independent connections, not a\n   * shared one** — see {@link ISqliteVecVectorIndexOpenParams.path}. To put a\n   * fragment index and a record index on one connection, open it yourself and pass\n   * it to both `create` methods.\n   */\n  readonly path: string;\n\n  /**\n   * Name of the `vec0` virtual table that holds the fragment embeddings. Must be a\n   * simple SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to\n   * `'memory_fragments'`.\n   */\n  readonly tableName?: string;\n}\n\n/**\n * An index plus the connection {@link SqliteVecFragmentIndex.open} opened for it.\n *\n * @public\n */\nexport interface ISqliteVecFragmentIndexHandle {\n  /** The index, ready to use. */\n  readonly index: SqliteVecFragmentIndex;\n\n  /**\n   * Closes the connection **this `open` call created**. Idempotent — a second\n   * `close` succeeds rather than failing. See\n   * {@link ISqliteVecVectorIndexHandle.close} for why the disposer lives here\n   * rather than on the index class.\n   */\n  close(): Result<true>;\n}\n\n/**\n * Parameters for {@link SqliteVecFragmentIndex.create}.\n * @public\n */\nexport interface ISqliteVecFragmentIndexCreateParams {\n  /**\n   * A `better-sqlite3` `Database` the consumer owns (bring-your-own, mirroring\n   * {@link ISqliteVecVectorIndexCreateParams.database}). The consumer opens it and\n   * owns its lifecycle — this index never closes it. `create` loads the\n   * `sqlite-vec` extension onto the connection and, if the fragment table already\n   * exists (a reopened persistent file), recovers its established dimension so no\n   * re-embedding is required on open.\n   */\n  readonly database: BetterSqlite3.Database;\n\n  /**\n   * Name of the `vec0` virtual table that holds the fragment embeddings. Must be a\n   * simple SQL identifier (`[A-Za-z_][A-Za-z0-9_]*`). Defaults to\n   * `'memory_fragments'`. Supply a distinct name (distinct from any record-level\n   * {@link SqliteVecVectorIndex} table) to hold more than one independent index in\n   * a single database file.\n   */\n  readonly tableName?: string;\n}\n"]}

package/dist/packlets/sqlite-vec-index/sqliteVecFragmentIndex.js

@@ -6,8 +6,11 @@ import { load as loadSqliteVec } from 'sqlite-vec';
 import { captureResult, fail, failWithDetail, succeed, succeedWithDetail } from '@fgv/ts-utils';
 import { edgeTargetKey } from '@fgv/ts-agent-memory';
 import { invokeHook, tally, withRollbackNote } from './rebuildHelpers';
+import { closeOwnedConnection, openOwnedConnection } from './connection';
 /** Default name for the fragment `vec0` virtual table. */
 const DEFAULT_TABLE_NAME = 'memory_fragments';
+/** Package-facing prefix for this class's failure messages. */
+const LABEL = 'sqlite-vec fragment index';
 /** A simple SQL identifier — the only shape allowed for the table name (it is interpolated into DDL). */
 const IDENTIFIER_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
 /**
@@ -62,9 +65,13 @@ const AUXILIARY_COLUMN_RE = /\+\s*([A-Za-z_][A-Za-z0-9_]*)/g;
  * cosine (`score = 1 - cosineDistance`), byte-identical to the in-memory index.
  * Large-N ANN indexing is explicitly out of scope, same regime as the record index.
  *
- * The `better-sqlite3` `Database` is consumer-owned (bring-your-own): this index
- * loads the `sqlite-vec` extension onto it and reads/writes the table, but never
- * opens or closes the connection.
+ * **Connection ownership depends on which factory you use.** With
+ * {@link SqliteVecFragmentIndex.create} the `Database` is consumer-owned
+ * (bring-your-own): this index loads the `sqlite-vec` extension onto it and
+ * reads/writes the table, but never opens or closes the connection — and that is
+ * the seam for backing this index and a record index with one connection. With
+ * {@link SqliteVecFragmentIndex.open} this package opens the file itself and hands
+ * back a handle carrying the disposer for the connection it created.
  * @public
  */
 export class SqliteVecFragmentIndex {
@@ -115,6 +122,48 @@ export class SqliteVecFragmentIndex {
             return new SqliteVecFragmentIndex(params.database, table, dimension);
         }).withErrorFormat((e) => `sqlite-vec fragment index: failed to initialize: ${e}`));
     }
+    /**
+     * Path-based factory. Opens the database file itself and returns the index
+     * together with a disposer for the connection it created.
+     *
+     * @remarks
+     * The fragment-granular sibling of {@link SqliteVecVectorIndex.open}, and present
+     * for the same reason: a consumer doing sub-document retrieval only would
+     * otherwise still value-import `better-sqlite3` and hand-roll a `captureResult`
+     * around a constructor that throws.
+     *
+     * **Use `create` instead when one connection must back both a fragment index and
+     * a record index** — the intended shared-handle case. Two `open` calls on one path
+     * give two independent connections, not a shared one.
+     *
+     * If initialization fails after the file is opened, the connection is closed
+     * before returning, so a failed `open` does not leak the descriptor it created.
+     * Should that close *itself* fail — the connection is then genuinely leaked — the
+     * returned message says so rather than hiding it. That includes the
+     * auxiliary-column mismatch failure, which is reported by `create` only after the
+     * file is open.
+     *
+     * @param params - See {@link ISqliteVecFragmentIndexOpenParams}.
+     * @returns `Success` with a {@link ISqliteVecFragmentIndexHandle}, or `Failure` if
+     * the driver could not be loaded, the file could not be opened, the table name is
+     * not a simple identifier, the extension fails to load, or the existing table was
+     * written by a version with a different auxiliary-column set.
+     */
+    static async open(params) {
+        return (await openOwnedConnection(params.path, LABEL)).thenOnSuccess(async (database) => (await SqliteVecFragmentIndex.create({ database, tableName: params.tableName }))
+            .onFailure((message) => 
+        // This call opened the connection, so a failure to initialize on top of it
+        // must not leave the file handle behind. A close that ALSO fails is said out
+        // loud rather than swallowed — the same reasoning, and the same helper, as
+        // `withRollbackNote`: silently discarding it would make the "a failed open
+        // leaks nothing" guarantee untrue exactly when it stopped holding, with no
+        // way for a caller to detect it.
+        fail(withRollbackNote(message, closeOwnedConnection(database, LABEL))))
+            .onSuccess((index) => succeed({
+            index,
+            close: () => closeOwnedConnection(database, LABEL)
+        })));
+    }
     /** {@inheritDoc IFragmentVectorIndex.addFragments} */
     addFragments(target, fragments) {
         const key = edgeTargetKey(target);
@@ -278,7 +327,10 @@ export class SqliteVecFragmentIndex {
         return captureResult(() => this._db.prepare(`DELETE FROM "${this._table}"`).run()).onSuccess(() => succeed(true));
     }
     /** {@inheritDoc IFragmentVectorIndex.query} */
-    query(vector, topK, maxPerRecord) {
+    query(vector, topK, options) {
+        const maxPerRecord = options === null || options === void 0 ? void 0 : options.maxPerRecord;
+        const scope = options === null || options === void 0 ? void 0 : options.scope;
+        const id = options === null || options === void 0 ? void 0 : options.id;
         if (topK <= 0 || this._stmts === undefined) {
             return Promise.resolve(succeed([]));
         }
@@ -292,11 +344,33 @@ export class SqliteVecFragmentIndex {
             // capped record's later fragments are skipped), so fetch the full ranked set
             // and apply the cap + topK cut here — exactly as the in-memory index does.
             // Uncapped, KNN's own `k = topK` is already the answer.
-            const fetchK = maxPerRecord === undefined ? topK : Number(stmts.fragmentCount.get().c);
+            // A scope-only narrowing (a versioned kind's per-entity subtree) spans several
+            // records, and `target_key` equality cannot express a prefix, so it is applied
+            // over the full ranked set below. Correct either way — the caller's `topK` is
+            // applied to the NARROWED set, which is the property that matters — but only
+            // the single-record case gets the partition push-down.
+            const recordKey = scope !== undefined && id !== undefined ? edgeTargetKey({ scope, id }) : undefined;
+            // The cap forces the full ranked set ONLY when other records can fill from
+            // behind a capped one. Under a single-record narrowing every row belongs to
+            // that record, so the result is exactly `min(topK, maxPerRecord, fragments)`
+            // and those are the first rows KNN returns — `k = topK` suffices, and
+            // expanding to the table-wide `fragmentCount` would ask an
+            // already-partition-restricted query for far more rows than it can use.
+            const wholeSet = recordKey === undefined && (maxPerRecord !== undefined || scope !== undefined);
+            const fetchK = wholeSet
+                ? Number(stmts.fragmentCount.get().c)
+                : topK;
             if (fetchK <= 0) {
                 return [];
             }
-            const rows = stmts.query.all(SqliteVecFragmentIndex._toBlob(vector), fetchK);
+            const blob = SqliteVecFragmentIndex._toBlob(vector);
+            const rows = (recordKey !== undefined
+                ? stmts.queryScopedToRecord.all(blob, fetchK, recordKey)
+                : stmts.query.all(blob, fetchK));
+            // The scope prefix every record in `scope` shares. `edgeTargetKey` joins with
+            // a NUL, so this cannot collide with a longer scope that merely starts the
+            // same way.
+            const scopePrefix = scope !== undefined && recordKey === undefined ? `${scope}\0` : undefined;
             // sqlite-vec returns rows ascending by distance (nearest first); score is
             // `1 - cosineDistance`, so this order is already descending score.
             const hits = [];
@@ -305,6 +379,9 @@ export class SqliteVecFragmentIndex {
                 if (hits.length >= topK) {
                     break;
                 }
+                if (scopePrefix !== undefined && !row.target_key.startsWith(scopePrefix)) {
+                    continue;
+                }
                 if (maxPerRecord !== undefined) {
                     const used = (_a = perRecord.get(row.target_key)) !== null && _a !== void 0 ? _a : 0;
                     if (used >= maxPerRecord) {
@@ -355,6 +432,12 @@ export class SqliteVecFragmentIndex {
             },
             query: this._db.prepare(`SELECT target_key, start_off, end_off, fragment_id, distance FROM "${this._table}" ` +
                 `WHERE embedding MATCH ? AND k = ?`),
+            // The single-record narrowing constrains `target_key`, which is the table's
+            // PARTITION KEY — so this is a partition-restricted KNN rather than a scan
+            // plus a filter. That is the performance reason this narrowing belongs in the
+            // library instead of in a bigger over-fetch on the caller's side.
+            queryScopedToRecord: this._db.prepare(`SELECT target_key, start_off, end_off, fragment_id, distance FROM "${this._table}" ` +
+                `WHERE embedding MATCH ? AND k = ? AND target_key = ?`),
             fragmentCount: this._db.prepare(`SELECT count(*) AS c FROM "${this._table}"`),
             recordCount: this._db.prepare(`SELECT count(DISTINCT target_key) AS c FROM "${this._table}"`),
             // `LIMIT 1`: membership needs existence, not cardinality.

package/dist/packlets/sqlite-vec-index/sqliteVecFragmentIndex.js.map

@@ -1 +1 @@
-{"version":3,"file":"sqliteVecFragmentIndex.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/sqliteVecFragmentIndex.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,IAAI,IAAI,aAAa,EAAE,MAAM,YAAY,CAAC;AACnD,OAAO,EAGL,aAAa,EACb,IAAI,EACJ,cAAc,EACd,OAAO,EACP,iBAAiB,EAClB,MAAM,eAAe,CAAC;AACvB,OAAO,EAeL,aAAa,EACd,MAAM,sBAAsB,CAAC;AAC9B,OAAO,EAAE,UAAU,EAAE,KAAK,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAGvE,0DAA0D;AAC1D,MAAM,kBAAkB,GAAW,kBAAkB,CAAC;AAEtD,yGAAyG;AACzG,MAAM,aAAa,GAAW,0BAA0B,CAAC;AAEzD;;;;;GAKG;AACH,MAAM,iBAAiB,GAA0B,CAAC,WAAW,EAAE,SAAS,EAAE,aAAa,CAAC,CAAC;AAEzF;;;;GAIG;AACH,MAAM,mBAAmB,GAAW,gCAAgC,CAAC;AA8BrE;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA2CG;AACH,MAAM,OAAO,sBAAsB;IAQjC,YAAoB,EAA0B,EAAE,KAAa,EAAE,SAA6B;QAC1F,IAAI,CAAC,GAAG,GAAG,EAAE,CAAC;QACd,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC;QAC5B,IAAI,CAAC,MAAM,GAAG,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,EAAE,CAAC;IACtE,CAAC;IAED,yGAAyG;IACzG,IAAW,WAAW;QACpB,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,CAAC,CAAC;QACX,CAAC;QACD,8EAA8E;QAC9E,8DAA8D;QAC9D,OAAO,MAAM,CAAE,IAAI,CAAC,MAAM,CAAC,WAAW,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;IAC7E,CAAC;IAED,kGAAkG;IAClG,IAAW,aAAa;QACtB,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,CAAC,CAAC;QACX,CAAC;QACD,OAAO,MAAM,CAAE,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;IAC/E,CAAC;IAED;;;;;;;;;;;;OAYG;IACI,MAAM,CAAC,MAAM,CAAC,MAA2C;;QAC9D,MAAM,KAAK,GAAW,MAAA,MAAM,CAAC,SAAS,mCAAI,kBAAkB,CAAC;QAC7D,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC/B,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CAAC,0CAA0C,KAAK,kCAAkC,CAAC,CACxF,CAAC;QACJ,CAAC;QACD,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,aAAa,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAC/B,MAAM,SAAS,GAAuB,sBAAsB,CAAC,sBAAsB,CACjF,MAAM,CAAC,QAAQ,EACf,KAAK,CACN,CAAC;YACF,OAAO,IAAI,sBAAsB,CAAC,MAAM,CAAC,QAAQ,EAAE,KAAK,EAAE,SAAS,CAAC,CAAC;QACvE,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,oDAAoD,CAAC,EAAE,CAAC,CACnF,CAAC;IACJ,CAAC;IAED,sDAAsD;IAC/C,YAAY,CACjB,MAAmB,EACnB,SAA2C;QAE3C,MAAM,GAAG,GAAW,aAAa,CAAC,MAAM,CAAC,CAAC;QAC1C,gFAAgF;QAChF,kFAAkF;QAClF,mFAAmF;QACnF,gFAAgF;QAChF,4DAA4D;QAC5D,IAAI,SAAS,GAAuB,IAAI,CAAC,UAAU,CAAC;QACpD,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,IAAI,QAAQ,CAAC,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBACjC,OAAO,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,+BAA+B,GAAG,0BAA0B,CAAC,CAAC,CAAC;YAC7F,CAAC;YACD,gFAAgF;YAChF,4EAA4E;YAC5E,uEAAuE;YACvE,IAAI,QAAQ,CAAC,OAAO,KAAK,SAAS,IAAI,QAAQ,CAAC,UAAU,KAAK,SAAS,EAAE,CAAC;gBACxE,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,gEAAgE,CACnG,CACF,CAAC;YACJ,CAAC;YACD,gFAAgF;YAChF,6EAA6E;YAC7E,iFAAiF;YACjF,6EAA6E;YAC7E,wEAAwE;YACxE,IACE,QAAQ,CAAC,OAAO,KAAK,SAAS;gBAC9B,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,EAC9F,CAAC;gBACD,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,eAAe,QAAQ,CAAC,OAAO,CAAC,KAAK,KAAK,QAAQ,CAAC,OAAO,CAAC,GAAG,iCAAiC,CAClI,CACF,CAAC;YACJ,CAAC;YACD,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;gBAC5B,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,MAAM,CAAC;YACrC,CAAC;iBAAM,IAAI,QAAQ,CAAC,MAAM,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAChD,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,yBAAyB,QAAQ,CAAC,MAAM,CAAC,MAAM,mCAAmC,SAAS,EAAE,CAChI,CACF,CAAC;YACJ,CAAC;QACH,CAAC;QACD,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,6EAA6E;YAC7E,0EAA0E;YAC1E,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBAC3B,8DAA8D;oBAC9D,OAAO,CAAC,CAAC;gBACX,CAAC;gBACD,wEAAwE;gBACxE,yEAAyE;gBACzE,uEAAuE;gBACvE,uDAAuD;gBACvD,MAAM,WAAW,GAAW,SAAS,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC;gBACvD,IAAI,CAAC,YAAY,CAAC,WAAW,CAAC,CAAC;gBAC/B,IAAI,CAAC,UAAU,GAAG,WAAW,CAAC;gBAC9B,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;YAChC,CAAC;YACD,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YACpC,OAAO,SAAS,CAAC,MAAM,CAAC;QAC1B,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,+BAA+B,GAAG,MAAM,CAAC,EAAE,CAAC,CACvE,CAAC;IACJ,CAAC;IAED,gDAAgD;IACzC,MAAM,CAAC,MAAmB;QAC/B,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,6EAA6E;YAC7E,6BAA6B;YAC7B,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,IAAI,CAAC,MAAM,CAAC,cAAc,CAAC,GAAG,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC;YACxD,CAAC;YACD,OAAO,MAAM,CAAC;QAChB,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,kCAAkC,aAAa,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,CAC5F,CAAC;IACJ,CAAC;IAED,6CAA6C;IACtC,GAAG,CAAC,MAAmB;QAC5B,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,0EAA0E;YAC1E,gEAAgE;YAChE,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,OAAO,KAAK,CAAC;YACf,CAAC;YACD,OAAO,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,KAAK,SAAS,CAAC;QAClE,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,iCAAiC,aAAa,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,CAC3F,CAAC;IACJ,CAAC;IAED,iDAAiD;IAC1C,KAAK,CAAC,OAAO,CAClB,MAA2B,EAC3B,KAAuB,EACvB,OAA+B;;QAE/B,MAAM,OAAO,GAAY,CAAC,MAAA,OAAO,aAAP,OAAO,uBAAP,OAAO,CAAE,aAAa,mCAAI,MAAM,CAAC,KAAK,MAAM,CAAC;QACvE,6EAA6E;QAC7E,6CAA6C;QAC7C,MAAM,MAAM,GAAiC,MAAM,UAAU,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,CAAC;QACnF,IAAI,MAAM,CAAC,SAAS,EAAE,EAAE,CAAC;YACvB,6EAA6E;YAC7E,6EAA6E;YAC7E,8EAA8E;YAC9E,2CAA2C;YAC3C,OAAO,cAAc,CAAC,mDAAmD,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC;QAC7F,CAAC;QACD,MAAM,OAAO,GAAiB,IAAI,CAAC,MAAM,EAAE,CAAC;QAC5C,IAAI,OAAO,CAAC,SAAS,EAAE,EAAE,CAAC;YACxB,oEAAoE;YACpE,OAAO,cAAc,CAAC,sDAAsD,OAAO,CAAC,OAAO,EAAE,CAAC,CAAC;QACjG,CAAC;QACD,MAAM,OAAO,GAAsB,IAAI,GAAG,EAAgB,CAAC;QAC3D,MAAM,SAAS,GAAsB,IAAI,GAAG,EAAgB,CAAC;QAC7D,MAAM,QAAQ,GAAsB,IAAI,GAAG,EAAgB,CAAC;QAC5D,MAAM,OAAO,GAA2B,EAAE,CAAC;QAC3C,4EAA4E;QAC5E,MAAM,MAAM,GAAG,GAAiC,EAAE,CAAC,CAAC;YAClD,OAAO;YACP,SAAS;YACT,QAAQ;YACR,QAAQ,EAAE,MAAM,CAAC,KAAK,CAAC,QAAQ;YAC/B,OAAO;SACR,CAAC,CAAC;QACH,KAAK,MAAM,MAAM,IAAI,MAAM,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC;YAC1C,MAAM,IAAI,GAAS,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC;YAC/C,2EAA2E;YAC3E,yEAAyE;YACzE,2CAA2C;YAC3C,MAAM,QAAQ,GAA6C,MAAM,UAAU,CAAC,GAAG,EAAE,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;YACxG,IAAI,QAAQ,CAAC,SAAS,EAAE,EAAE,CAAC;gBACzB,MAAM,KAAK,GAAW,sCAAsC,aAAa,CAAC,MAAM,CAAC,MAAM,CAAC,aACtF,QAAQ,CAAC,OACX,EAAE,CAAC;gBACH,IAAI,CAAC,OAAO,EAAE,CAAC;oBACb,iEAAiE;oBACjE,qEAAqE;oBACrE,wBAAwB;oBACxB,OAAO,cAAc,CAAC,gBAAgB,CAAC,KAAK,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;gBAC1E,CAAC;gBACD,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC;gBAC/C,SAAS;YACX,CAAC;YACD,uEAAuE;YACvE,2DAA2D;YAC3D,MAAM,KAAK,GAAmB,MAAM,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,MAAM,EAAE,QAAQ,CAAC,KAAK,CAAC,CAAC;YACrF,IAAI,KAAK,CAAC,SAAS,EAAE,EAAE,CAAC;gBACtB,MAAM,KAAK,GAAW,2BAA2B,KAAK,CAAC,OAAO,EAAE,CAAC;gBACjE,IAAI,CAAC,OAAO,EAAE,CAAC;oBACb,OAAO,cAAc,CAAC,gBAAgB,CAAC,KAAK,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;gBAC1E,CAAC;gBACD,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC;gBAC/C,SAAS;YACX,CAAC;YACD,IAAI,KAAK,CAAC,KAAK,KAAK,CAAC,EAAE,CAAC;gBACtB,KAAK,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;gBACtB,SAAS;YACX,CAAC;YACD,KAAK,CAAC,OAAO,EAAE,IAAI,CAAC,CAAC;YACrB,KAAK,CAAC,SAAS,EAAE,IAAI,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC;QACtC,CAAC;QACD,OAAO,iBAAiB,CAAC,MAAM,EAAE,CAAC,CAAC;IACrC,CAAC;IAED;;;;;;;OAOG;IACK,MAAM;QACZ,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC;QACvB,CAAC;QACD,6EAA6E;QAC7E,iFAAiF;QACjF,OAAO,aAAa,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,gBAAgB,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,SAAS,CAAC,GAAG,EAAE,CAChG,OAAO,CAAC,IAAI,CAAC,CACd,CAAC;IACJ,CAAC;IAED,+CAA+C;IACxC,KAAK,CACV,MAAoB,EACpB,IAAY,EACZ,YAAqB;QAErB,IAAI,IAAI,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC3C,OAAO,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC;QACtC,CAAC;QACD,IAAI,MAAM,CAAC,MAAM,KAAK,IAAI,CAAC,UAAU,EAAE,CAAC;YACtC,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,mCAAmC,MAAM,CAAC,MAAM,mCAAmC,IAAI,CAAC,UAAU,EAAE,CACrG,CACF,CAAC;QACJ,CAAC;QACD,MAAM,KAAK,GAAwB,IAAI,CAAC,MAAM,CAAC;QAC/C,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAiC,GAAG,EAAE;;YACjD,6EAA6E;YAC7E,6EAA6E;YAC7E,2EAA2E;YAC3E,wDAAwD;YACxD,MAAM,MAAM,GACV,YAAY,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAE,KAAK,CAAC,aAAa,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;YACtG,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;gBAChB,OAAO,EAAE,CAAC;YACZ,CAAC;YACD,MAAM,IAAI,GAA2B,KAAK,CAAC,KAAK,CAAC,GAAG,CAClD,sBAAsB,CAAC,OAAO,CAAC,MAAM,CAAC,EACtC,MAAM,CACmB,CAAC;YAC5B,0EAA0E;YAC1E,mEAAmE;YACnE,MAAM,IAAI,GAAsB,EAAE,CAAC;YACnC,MAAM,SAAS,GAAwB,IAAI,GAAG,EAAkB,CAAC;YACjE,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;gBACvB,IAAI,IAAI,CAAC,MAAM,IAAI,IAAI,EAAE,CAAC;oBACxB,MAAM;gBACR,CAAC;gBACD,IAAI,YAAY,KAAK,SAAS,EAAE,CAAC;oBAC/B,MAAM,IAAI,GAAW,MAAA,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,UAAU,CAAC,mCAAI,CAAC,CAAC;oBACxD,IAAI,IAAI,IAAI,YAAY,EAAE,CAAC;wBACzB,SAAS;oBACX,CAAC;oBACD,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,UAAU,EAAE,IAAI,GAAG,CAAC,CAAC,CAAC;gBAC1C,CAAC;gBACD,MAAM,GAAG,GAAW,GAAG,CAAC,UAAU,CAAC;gBACnC,IAAI,CAAC,IAAI,iBACP,MAAM,EAAE,sBAAsB,CAAC,SAAS,CAAC,GAAG,CAAC,EAC7C,KAAK,EAAE,CAAC,GAAG,GAAG,CAAC,QAAQ,IACpB,sBAAsB,CAAC,WAAW,CAAC,GAAG,EAAE,GAAG,CAAC,EAC/C,CAAC;YACL,CAAC;YACD,OAAO,IAAI,CAAC;QACd,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,iCAAiC,CAAC,EAAE,CAAC,CAChE,CAAC;IACJ,CAAC;IAED;;;;OAIG;IACK,YAAY,CAAC,SAAiB;QACpC,IAAI,CAAC,GAAG,CAAC,IAAI,CACX,uCAAuC,IAAI,CAAC,MAAM,eAAe;YAC/D,kDAAkD,SAAS,4BAA4B;YACvF,0DAA0D,CAC7D,CAAC;IACJ,CAAC;IAED,4EAA4E;IACpE,QAAQ;QACd,MAAM,GAAG,GAA4B,IAAI,CAAC,GAAG,CAAC,OAAO,CACnD,gBAAgB,IAAI,CAAC,MAAM,wBAAwB,CACpD,CAAC;QACF,MAAM,GAAG,GAA4B,IAAI,CAAC,GAAG,CAAC,OAAO,CACnD,gBAAgB,IAAI,CAAC,MAAM,4DAA4D;YACrF,wBAAwB,CAC3B,CAAC;QACF,iFAAiF;QACjF,gEAAgE;QAChE,MAAM,UAAU,GAEZ,IAAI,CAAC,GAAG,CAAC,WAAW,CAAC,CAAC,GAAW,EAAE,SAA2C,EAAE,EAAE;;YACpF,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;YACb,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;gBACjC,GAAG,CAAC,GAAG,CACL,GAAG,EACH,sBAAsB,CAAC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC;gBAC/C,yEAAyE;gBACzE,4EAA4E;gBAC5E,2EAA2E;gBAC3E,QAAQ,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,EACtE,QAAQ,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC;gBACpE,0DAA0D;gBAC1D,MAAA,QAAQ,CAAC,UAAU,mCAAI,IAAI,CAC5B,CAAC;YACJ,CAAC;QACH,CAAC,CAAC,CAAC;QACH,OAAO;YACL,cAAc,EAAE,GAAG;YACnB,OAAO,EAAE,CAAC,GAAW,EAAE,SAA2C,EAAQ,EAAE;gBAC1E,UAAU,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YAC7B,CAAC;YACD,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CACrB,sEAAsE,IAAI,CAAC,MAAM,IAAI;gBACnF,mCAAmC,CACtC;YACD,aAAa,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,8BAA8B,IAAI,CAAC,MAAM,GAAG,CAAC;YAC7E,WAAW,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,gDAAgD,IAAI,CAAC,MAAM,GAAG,CAAC;YAC7F,0DAA0D;YAC1D,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,kBAAkB,IAAI,CAAC,MAAM,gCAAgC,CAAC;SACrF,CAAC;IACJ,CAAC;IAED;;;;;;;;;;;OAWG;IACK,MAAM,CAAC,sBAAsB,CAAC,EAA0B,EAAE,KAAa;QAC7E,MAAM,GAAG,GAAgC,EAAE;aACxC,OAAO,CAAC,iEAAiE,CAAC;aAC1E,GAAG,CAAC,KAAK,CAAgC,CAAC;QAC7C,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;YACtB,OAAO,SAAS,CAAC;QACnB,CAAC;QACD,sBAAsB,CAAC,uBAAuB,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;QAC/D,MAAM,KAAK,GAA4B,GAAG,CAAC,GAAG,CAAC,KAAK,CAAC,gBAAgB,CAAC,CAAC;QACvE,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;YACnB,+EAA+E;YAC/E,wEAAwE;YACxE,6EAA6E;YAC7E,qEAAqE;YACrE,MAAM,IAAI,KAAK,CACb,mBAAmB,KAAK,iEAAiE;gBACvF,6EAA6E,CAChF,CAAC;QACJ,CAAC;QACD,OAAO,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1B,CAAC;IAED;;;;;;;;;OASG;IACK,MAAM,CAAC,uBAAuB,CAAC,GAAW,EAAE,KAAa;QAC/D,MAAM,KAAK,GAAa,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,mBAAmB,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACnF,MAAM,QAAQ,GAA0B,iBAAiB,CAAC;QAC1D,MAAM,OAAO,GACX,KAAK,CAAC,MAAM,KAAK,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC;QACzF,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,MAAM,IAAI,KAAK,CACb,mBAAmB,KAAK,4BAA4B,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,mBAAmB;gBACrF,aAAa,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,+CAA+C;gBAC/E,4FAA4F;gBAC5F,sFAAsF;gBACtF,IAAI,KAAK,gFAAgF;gBACzF,0EAA0E,CAC7E,CAAC;QACJ,CAAC;IACH,CAAC;IAED;;;;;;;;OAQG;IACK,MAAM,CAAC,WAAW,CAAC,GAAY,EAAE,GAAW;QAClD,MAAM,OAAO,GAAiC,sBAAsB,CAAC,UAAU,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC;QAC1F,IAAI,OAAO,KAAK,SAAS,IAAI,GAAG,CAAC,WAAW,KAAK,IAAI,EAAE,CAAC;YACtD,MAAM,IAAI,KAAK,CACb,aAAa,GAAG,6EAA6E,CAC9F,CAAC;QACJ,CAAC;QACD,uCACK,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,GAC1C,CAAC,GAAG,CAAC,WAAW,KAAK,IAAI,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,GAAG,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EACpE;IACJ,CAAC;IAED;;;;;;;OAOG;IACK,MAAM,CAAC,UAAU,CAAC,GAAY,EAAE,GAAW;QACjD,MAAM,KAAK,GAA2B,GAAG,CAAC,SAAS,CAAC;QACpD,MAAM,GAAG,GAA2B,GAAG,CAAC,OAAO,CAAC;QAChD,IAAI,KAAK,KAAK,IAAI,IAAI,GAAG,KAAK,IAAI,EAAE,CAAC;YACnC,OAAO,SAAS,CAAC;QACnB,CAAC;QACD,IAAI,KAAK,KAAK,IAAI,IAAI,GAAG,KAAK,IAAI,EAAE,CAAC;YACnC,MAAM,IAAI,KAAK,CACb,aAAa,GAAG,2EAA2E,CAC5F,CAAC;QACJ,CAAC;QACD,OAAO;YACL,KAAK,EAAE,sBAAsB,CAAC,SAAS,CAAC,KAAK,EAAE,GAAG,CAAC;YACnD,GAAG,EAAE,sBAAsB,CAAC,SAAS,CAAC,GAAG,EAAE,GAAG,CAAC;SAChD,CAAC;IACJ,CAAC;IAED,sHAAsH;IAC9G,MAAM,CAAC,OAAO,CAAC,MAAoB;QACzC,OAAO,IAAI,UAAU,CAAC,YAAY,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC;IAC1D,CAAC;IAED;;;;;;OAMG;IACK,MAAM,CAAC,SAAS,CAAC,GAAW;QAClC,MAAM,GAAG,GAAW,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;QACtC,IAAI,GAAG,GAAG,CAAC,EAAE,CAAC;YACZ,MAAM,IAAI,KAAK,CAAC,yBAAyB,GAAG,wDAAwD,CAAC,CAAC;QACxG,CAAC;QACD,OAAO;YACL,KAAK,EAAE,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAA8B;YACrD,EAAE,EAAE,GAAG,CAAC,KAAK,CAAC,GAAG,GAAG,CAAC,CAAwB;SAC9C,CAAC;IACJ,CAAC;IAED;;;;;;;OAOG;IACK,MAAM,CAAC,SAAS,CAAC,KAAsB,EAAE,GAAW;QAC1D,MAAM,CAAC,GAAW,MAAM,CAAC,KAAK,CAAC,CAAC;QAChC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,CAAC;YAC7B,MAAM,IAAI,KAAK,CACb,aAAa,GAAG,qBAAqB,MAAM,CAAC,KAAK,CAAC,iDAAiD,CACpG,CAAC;QACJ,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC;CACF","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nimport type BetterSqlite3 from 'better-sqlite3';\nimport { load as loadSqliteVec } from 'sqlite-vec';\nimport {\n  DetailedResult,\n  Result,\n  captureResult,\n  fail,\n  failWithDetail,\n  succeed,\n  succeedWithDetail\n} from '@fgv/ts-utils';\nimport {\n  FragmentEmbedder,\n  IEdgeTarget,\n  IEmbeddedFragment,\n  IFragmentLocator,\n  IFragmentVectorIndex,\n  IFragmentVectorRebuildReport,\n  IMemoryRecordListing,\n  IMemoryRecordSource,\n  ISkippedVectorRecord,\n  IVectorQueryHit,\n  IVectorRebuildOptions,\n  Kind,\n  MemoryId,\n  MemoryScopeKey,\n  edgeTargetKey\n} from '@fgv/ts-agent-memory';\nimport { invokeHook, tally, withRollbackNote } from './rebuildHelpers';\nimport { ISqliteVecFragmentIndexCreateParams } from './model';\n\n/** Default name for the fragment `vec0` virtual table. */\nconst DEFAULT_TABLE_NAME: string = 'memory_fragments';\n\n/** A simple SQL identifier — the only shape allowed for the table name (it is interpolated into DDL). */\nconst IDENTIFIER_RE: RegExp = /^[A-Za-z_][A-Za-z0-9_]*$/;\n\n/**\n * The auxiliary (`+`-prefixed) columns this version of the index writes. A table\n * created by an earlier version carries a different set; see\n * {@link SqliteVecFragmentIndex._readExistingDimension} for why that has to be\n * detected explicitly rather than migrated.\n */\nconst AUXILIARY_COLUMNS: ReadonlyArray<string> = ['start_off', 'end_off', 'fragment_id'];\n\n/**\n * Matches one `+name` auxiliary-column declaration in a `vec0` `CREATE VIRTUAL TABLE`\n * statement. Only ever consumed via `String.matchAll`, which iterates a clone rather\n * than advancing this instance's `lastIndex`, so the shared `/g` regex is reusable.\n */\nconst AUXILIARY_COLUMN_RE: RegExp = /\\+\\s*([A-Za-z_][A-Za-z0-9_]*)/g;\n\n/**\n * One KNN row as returned by the fragment `vec0` MATCH query. The offset columns are\n * typed `number | bigint` because `better-sqlite3` returns integer columns as\n * `bigint` when a consumer enables its safe-integer mode (`defaultSafeIntegers`);\n * {@link SqliteVecFragmentIndex._toOffset} coerces them to a plain `number` (and\n * fails loudly on an out-of-safe-range value) before they reach the public locator.\n * All three identity columns are nullable: a fragment stored without a locator has\n * `NULL` offsets, and one stored without a `fragmentId` has a `NULL` `fragment_id`.\n */\ninterface IKnnRow {\n  readonly target_key: string;\n  // eslint-disable-next-line @rushstack/no-new-null -- SQLite returns NULL (not undefined) for an absent locator offset\n  readonly start_off: number | bigint | null;\n  // eslint-disable-next-line @rushstack/no-new-null -- SQLite returns NULL (not undefined) for an absent locator offset\n  readonly end_off: number | bigint | null;\n  // eslint-disable-next-line @rushstack/no-new-null -- SQLite returns NULL (not undefined) for an absent fragment id\n  readonly fragment_id: string | null;\n  readonly distance: number;\n}\n\n/**\n * The identity fields of a fragment hit, in `IVectorQueryHit` shape: a field the\n * stored fragment did not carry is *absent*, never present-but-`undefined`, so a hit\n * for a fragment stored without a `fragmentId` is structurally identical to one this\n * index produced before `fragment_id` existed.\n */\ntype FragmentIdentity = Pick<IVectorQueryHit, 'locator' | 'fragmentId'>;\n\n/**\n * A persistent, `sqlite-vec`-backed `IFragmentVectorIndex` (from\n * `@fgv/ts-agent-memory`) — the fragment-granular sibling of\n * {@link SqliteVecVectorIndex}, and the **durable** counterpart to the in-memory\n * `InMemoryFragmentCosineIndex`.\n *\n * @remarks\n * Where {@link SqliteVecVectorIndex} keys one vector per record on a\n * `target_key` primary key, this index holds **many** vectors per record — one per\n * fragment — so it keys the `vec0` table on `target_key` as a **`PARTITION KEY`**\n * (many rows may share it) and stores each fragment's identity in three auxiliary\n * columns (`+start_off`, `+end_off`, `+fragment_id`) that ride alongside the vector\n * and are returned on query but never filtered — in particular `fragment_id` is\n * stored and returned verbatim, never parsed and never part of the query path. A\n * query is a brute-force `vec0` KNN scan across all partitions returning per-fragment\n * hits, each carrying its record `target` plus whichever identity fields the stored\n * fragment was added with (a fragment must carry at least one).\n *\n * **`vec0` schema changes require a drop-and-re-index.** A\n * `CREATE VIRTUAL TABLE IF NOT EXISTS` is a no-op against an existing table (SQLite\n * does not compare schemas) and `vec0` has no `ALTER TABLE ADD COLUMN`, so a database written by an\n * earlier version of this package keeps its old auxiliary columns. `create` detects\n * that by parsing the stored `CREATE VIRTUAL TABLE` SQL and fails with an actionable\n * message naming the expected and found columns, rather than letting a widened\n * `INSERT` surface an opaque `no such column` at statement-prepare time. There are no\n * in-place migrations: drop the table (or use a fresh `tableName`) and re-index.\n * Fragment vectors are re-derivable from the records, so this costs embedding time,\n * never data.\n *\n * Semantics match `InMemoryFragmentCosineIndex` exactly: `addFragments` is\n * whole-record-replace (a single transaction deletes every prior fragment of the\n * target, then inserts the new set), `remove` drops every fragment of a target,\n * and `query` applies the optional `maxPerRecord` cap **during selection, before\n * the topK cut** — so one long document cannot crowd others out. The dimension is\n * established by the first `addFragments` (the `vec0` column is fixed-width) and\n * recovered from the table schema when a persistent file is reopened; similarity is\n * cosine (`score = 1 - cosineDistance`), byte-identical to the in-memory index.\n * Large-N ANN indexing is explicitly out of scope, same regime as the record index.\n *\n * The `better-sqlite3` `Database` is consumer-owned (bring-your-own): this index\n * loads the `sqlite-vec` extension onto it and reads/writes the table, but never\n * opens or closes the connection.\n * @public\n */\nexport class SqliteVecFragmentIndex implements IFragmentVectorIndex {\n  private readonly _db: BetterSqlite3.Database;\n  private readonly _table: string;\n  /** The dimension of every stored fragment vector; `undefined` until the table exists. */\n  private _dimension: number | undefined;\n  /** Prepared statements; created once the table exists (established or recovered). */\n  private _stmts: IFragmentStatements | undefined;\n\n  private constructor(db: BetterSqlite3.Database, table: string, dimension: number | undefined) {\n    this._db = db;\n    this._table = table;\n    this._dimension = dimension;\n    this._stmts = dimension === undefined ? undefined : this._prepare();\n  }\n\n  /** The number of records that currently have at least one stored fragment. Zero before the first add. */\n  public get recordCount(): number {\n    if (this._stmts === undefined) {\n      return 0;\n    }\n    // `Number(...)` narrows the count in case the consumer enabled better-sqlite3\n    // safe-integer mode (which returns `count(*)` as a `bigint`).\n    return Number((this._stmts.recordCount.get() as { c: number | bigint }).c);\n  }\n\n  /** The total number of fragments currently held across all records. Zero before the first add. */\n  public get fragmentCount(): number {\n    if (this._stmts === undefined) {\n      return 0;\n    }\n    return Number((this._stmts.fragmentCount.get() as { c: number | bigint }).c);\n  }\n\n  /**\n   * Family-convention factory. Loads the `sqlite-vec` extension onto the supplied\n   * `better-sqlite3` connection and, if the fragment table already exists (a\n   * reopened persistent file), verifies its auxiliary-column set matches this\n   * version's and recovers its established dimension so no re-embedding is needed on\n   * open.\n   *\n   * @param params - See {@link ISqliteVecFragmentIndexCreateParams}.\n   * @returns `Success` with the index, or `Failure` if the table name is not a\n   * simple identifier, the extension fails to load, or the existing table was\n   * written by a version with a different auxiliary-column set (which requires a\n   * drop-and-re-index — `vec0` cannot be altered in place).\n   */\n  public static create(params: ISqliteVecFragmentIndexCreateParams): Promise<Result<SqliteVecFragmentIndex>> {\n    const table: string = params.tableName ?? DEFAULT_TABLE_NAME;\n    if (!IDENTIFIER_RE.test(table)) {\n      return Promise.resolve(\n        fail(`sqlite-vec fragment index: table name '${table}' is not a simple SQL identifier`)\n      );\n    }\n    return Promise.resolve(\n      captureResult(() => {\n        loadSqliteVec(params.database);\n        const dimension: number | undefined = SqliteVecFragmentIndex._readExistingDimension(\n          params.database,\n          table\n        );\n        return new SqliteVecFragmentIndex(params.database, table, dimension);\n      }).withErrorFormat((e) => `sqlite-vec fragment index: failed to initialize: ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.addFragments} */\n  public addFragments(\n    target: IEdgeTarget,\n    fragments: ReadonlyArray<IEmbeddedFragment>\n  ): Promise<Result<number>> {\n    const key: string = edgeTargetKey(target);\n    // Validate every fragment before touching the database, so a bad fragment never\n    // leaves the record half-replaced or the dimension half-established (whole-record\n    // replace is all-or-nothing). The effective dimension is the established one, or —\n    // on a still-dimensionless index — the first fragment's length; it is committed\n    // (via table creation) only once the whole batch validates.\n    let dimension: number | undefined = this._dimension;\n    for (const fragment of fragments) {\n      if (fragment.vector.length === 0) {\n        return Promise.resolve(fail(`fragment index: cannot add '${key}': empty fragment vector`));\n      }\n      // A fragment carrying neither identity cannot be resolved back to anything by a\n      // consumer holding the hit — the same invariant `embeddedFragmentConverter`\n      // enforces at the untyped boundary, re-checked here at the index seam.\n      if (fragment.locator === undefined && fragment.fragmentId === undefined) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': fragment requires at least one of 'locator' or 'fragmentId'`\n          )\n        );\n      }\n      // Locator offsets are persisted as SQLite integers (bound via BigInt). Reject a\n      // non-safe-integer offset up front with a clear message, rather than letting\n      // `BigInt(nonInteger)` throw cryptically inside the write transaction OR storing\n      // a value the read-side `_toOffset` guard would later reject on every query.\n      // An absent locator persists as a NULL offset pair and skips the check.\n      if (\n        fragment.locator !== undefined &&\n        (!Number.isSafeInteger(fragment.locator.start) || !Number.isSafeInteger(fragment.locator.end))\n      ) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': locator [${fragment.locator.start}, ${fragment.locator.end}) offsets must be safe integers`\n          )\n        );\n      }\n      if (dimension === undefined) {\n        dimension = fragment.vector.length;\n      } else if (fragment.vector.length !== dimension) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': fragment dimension ${fragment.vector.length} does not match index dimension ${dimension}`\n          )\n        );\n      }\n    }\n    return Promise.resolve(\n      captureResult(() => {\n        // A same-target re-author (or an empty batch) still needs the table to exist\n        // to delete prior fragments; create it lazily on the first non-empty add.\n        if (this._stmts === undefined) {\n          if (fragments.length === 0) {\n            // Nothing stored yet and nothing to store: no table, no work.\n            return 0;\n          }\n          // `fragments` is non-empty here (the empty case returned above), so the\n          // validation loop proved every fragment shares `fragments[0]`'s length —\n          // which IS the dimension to establish. Read it straight from the first\n          // fragment: no cast, no invariant-dependent narrowing.\n          const established: number = fragments[0].vector.length;\n          this._createTable(established);\n          this._dimension = established;\n          this._stmts = this._prepare();\n        }\n        this._stmts.replace(key, fragments);\n        return fragments.length;\n      }).withErrorFormat((e) => `fragment index: cannot add '${key}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.remove} */\n  public remove(target: IEdgeTarget): Promise<Result<IEdgeTarget>> {\n    return Promise.resolve(\n      captureResult(() => {\n        // Idempotent: removing a target with no fragments (or before any add created\n        // the table) still succeeds.\n        if (this._stmts !== undefined) {\n          this._stmts.deleteByTarget.run(edgeTargetKey(target));\n        }\n        return target;\n      }).withErrorFormat((e) => `fragment index: cannot remove '${edgeTargetKey(target)}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.has} */\n  public has(target: IEdgeTarget): Promise<Result<boolean>> {\n    return Promise.resolve(\n      captureResult(() => {\n        // Before any add has created the table there is nothing held — a truthful\n        // `false`, matching `remove`'s idempotence and the zero counts.\n        if (this._stmts === undefined) {\n          return false;\n        }\n        return this._stmts.has.get(edgeTargetKey(target)) !== undefined;\n      }).withErrorFormat((e) => `fragment index: cannot check '${edgeTargetKey(target)}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.rebuild} */\n  public async rebuild(\n    source: IMemoryRecordSource,\n    embed: FragmentEmbedder,\n    options?: IVectorRebuildOptions\n  ): Promise<DetailedResult<IFragmentVectorRebuildReport, IFragmentVectorRebuildReport>> {\n    const lenient: boolean = (options?.onRecordError ?? 'fail') === 'skip';\n    // `source` is consumer-supplied, so a throw or rejection becomes a `Failure`\n    // here rather than escaping as an exception.\n    const listed: Result<IMemoryRecordListing> = await invokeHook(() => source.list());\n    if (listed.isFailure()) {\n      // Deliberately BEFORE any clear, matching both siblings: a failed list is no\n      // evidence about the fragments already held, and clearing here would destroy\n      // a healthy PERSISTED index over a transient read error. No detail — there is\n      // nothing this call disturbed to describe.\n      return failWithDetail(`fragment index rebuild: failed to list records: ${listed.message}`);\n    }\n    const cleared: Result<true> = this._clear();\n    if (cleared.isFailure()) {\n      // Also nothing established: the table still holds whatever it held.\n      return failWithDetail(`fragment index rebuild: failed to clear the index: ${cleared.message}`);\n    }\n    const indexed: Map<Kind, number> = new Map<Kind, number>();\n    const fragments: Map<Kind, number> = new Map<Kind, number>();\n    const declined: Map<Kind, number> = new Map<Kind, number>();\n    const skipped: ISkippedVectorRecord[] = [];\n    // Absent stays absent — only the source knows whether it filtered anything.\n    const report = (): IFragmentVectorRebuildReport => ({\n      indexed,\n      fragments,\n      declined,\n      excluded: listed.value.excluded,\n      skipped\n    });\n    for (const scoped of listed.value.records) {\n      const kind: Kind = scoped.record.envelope.kind;\n      // Capture-wrapped: an embedder that throws mid-loop would otherwise escape\n      // past the `'fail'` rollback below, leaving this DURABLE table holding a\n      // partial index that survives the process.\n      const embedded: Result<ReadonlyArray<IEmbeddedFragment>> = await invokeHook(() => embed(scoped.record));\n      if (embedded.isFailure()) {\n        const error: string = `fragment index rebuild: embedding '${edgeTargetKey(scoped.target)}' failed: ${\n          embedded.message\n        }`;\n        if (!lenient) {\n          // A rollback that also fails is said out loud: the `'fail'` path\n          // promises an empty index, and on a DURABLE table a botched rollback\n          // survives the process.\n          return failWithDetail(withRollbackNote(error, this._clear()), report());\n        }\n        skipped.push({ target: scoped.target, error });\n        continue;\n      }\n      // An empty array is this lane's decline, and it is still WRITTEN — the\n      // whole-record-replace is what clears any stale fragments.\n      const added: Result<number> = await this.addFragments(scoped.target, embedded.value);\n      if (added.isFailure()) {\n        const error: string = `fragment index rebuild: ${added.message}`;\n        if (!lenient) {\n          return failWithDetail(withRollbackNote(error, this._clear()), report());\n        }\n        skipped.push({ target: scoped.target, error });\n        continue;\n      }\n      if (added.value === 0) {\n        tally(declined, kind);\n        continue;\n      }\n      tally(indexed, kind);\n      tally(fragments, kind, added.value);\n    }\n    return succeedWithDetail(report());\n  }\n\n  /**\n   * **Empties the rows; does NOT release the table's declared dimension.** That\n   * is a `vec0` constraint rather than a choice — the dimension is schema, and\n   * there is no `ALTER TABLE` for it — so a rebuild at a new dimension fails\n   * here where it would succeed on the in-memory sibling, which forgets its\n   * dimension on reset. Changing dimension needs a drop-and-re-index; see the\n   * note on `IVectorIndex.rebuild`. Tolerates a table that does not exist yet.\n   */\n  private _clear(): Result<true> {\n    if (this._stmts === undefined) {\n      return succeed(true);\n    }\n    // Capture-wrapped like every other statement path: a closed connection or an\n    // I/O error is a `Failure`, not an exception out of a `Result`-returning method.\n    return captureResult(() => this._db.prepare(`DELETE FROM \"${this._table}\"`).run()).onSuccess(() =>\n      succeed(true)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.query} */\n  public query(\n    vector: Float32Array,\n    topK: number,\n    maxPerRecord?: number\n  ): Promise<Result<ReadonlyArray<IVectorQueryHit>>> {\n    if (topK <= 0 || this._stmts === undefined) {\n      return Promise.resolve(succeed([]));\n    }\n    if (vector.length !== this._dimension) {\n      return Promise.resolve(\n        fail(\n          `fragment index: query dimension ${vector.length} does not match index dimension ${this._dimension}`\n        )\n      );\n    }\n    const stmts: IFragmentStatements = this._stmts;\n    return Promise.resolve(\n      captureResult<ReadonlyArray<IVectorQueryHit>>(() => {\n        // With a per-record cap the topK winners may lie past the first topK rows (a\n        // capped record's later fragments are skipped), so fetch the full ranked set\n        // and apply the cap + topK cut here — exactly as the in-memory index does.\n        // Uncapped, KNN's own `k = topK` is already the answer.\n        const fetchK: number =\n          maxPerRecord === undefined ? topK : Number((stmts.fragmentCount.get() as { c: number | bigint }).c);\n        if (fetchK <= 0) {\n          return [];\n        }\n        const rows: ReadonlyArray<IKnnRow> = stmts.query.all(\n          SqliteVecFragmentIndex._toBlob(vector),\n          fetchK\n        ) as ReadonlyArray<IKnnRow>;\n        // sqlite-vec returns rows ascending by distance (nearest first); score is\n        // `1 - cosineDistance`, so this order is already descending score.\n        const hits: IVectorQueryHit[] = [];\n        const perRecord: Map<string, number> = new Map<string, number>();\n        for (const row of rows) {\n          if (hits.length >= topK) {\n            break;\n          }\n          if (maxPerRecord !== undefined) {\n            const used: number = perRecord.get(row.target_key) ?? 0;\n            if (used >= maxPerRecord) {\n              continue;\n            }\n            perRecord.set(row.target_key, used + 1);\n          }\n          const key: string = row.target_key;\n          hits.push({\n            target: SqliteVecFragmentIndex._parseKey(key),\n            score: 1 - row.distance,\n            ...SqliteVecFragmentIndex._toIdentity(row, key)\n          });\n        }\n        return hits;\n      }).withErrorFormat((e) => `fragment index: query failed: ${e}`)\n    );\n  }\n\n  /**\n   * Create the fragment `vec0` virtual table with the established dimension. The\n   * auxiliary columns must stay in sync with `AUXILIARY_COLUMNS`, which\n   * `create` compares against an existing table's stored DDL.\n   */\n  private _createTable(dimension: number): void {\n    this._db.exec(\n      `CREATE VIRTUAL TABLE IF NOT EXISTS \"${this._table}\" USING vec0(` +\n        `target_key TEXT PARTITION KEY, embedding float[${dimension}] distance_metric=cosine, ` +\n        `+start_off integer, +end_off integer, +fragment_id text)`\n    );\n  }\n\n  /** Prepare the statements the index reuses. Requires the table to exist. */\n  private _prepare(): IFragmentStatements {\n    const del: BetterSqlite3.Statement = this._db.prepare(\n      `DELETE FROM \"${this._table}\" WHERE target_key = ?`\n    );\n    const ins: BetterSqlite3.Statement = this._db.prepare(\n      `INSERT INTO \"${this._table}\"(target_key, embedding, start_off, end_off, fragment_id) ` +\n        `VALUES (?, ?, ?, ?, ?)`\n    );\n    // Whole-record replace: drop every prior fragment of the target, then insert the\n    // new set, atomically. An empty set collapses to a pure delete.\n    const replaceTxn: BetterSqlite3.Transaction<\n      (key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => void\n    > = this._db.transaction((key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => {\n      del.run(key);\n      for (const fragment of fragments) {\n        ins.run(\n          key,\n          SqliteVecFragmentIndex._toBlob(fragment.vector),\n          // vec0 typed columns reject a JS float; bind the offsets as integers. An\n          // absent locator binds the pair as NULL — never a partial pair, so the read\n          // side can treat a half-NULL pair as corruption rather than a legal shape.\n          fragment.locator === undefined ? null : BigInt(fragment.locator.start),\n          fragment.locator === undefined ? null : BigInt(fragment.locator.end),\n          // Stored verbatim and never parsed; absent binds as NULL.\n          fragment.fragmentId ?? null\n        );\n      }\n    });\n    return {\n      deleteByTarget: del,\n      replace: (key: string, fragments: ReadonlyArray<IEmbeddedFragment>): void => {\n        replaceTxn(key, fragments);\n      },\n      query: this._db.prepare(\n        `SELECT target_key, start_off, end_off, fragment_id, distance FROM \"${this._table}\" ` +\n          `WHERE embedding MATCH ? AND k = ?`\n      ),\n      fragmentCount: this._db.prepare(`SELECT count(*) AS c FROM \"${this._table}\"`),\n      recordCount: this._db.prepare(`SELECT count(DISTINCT target_key) AS c FROM \"${this._table}\"`),\n      // `LIMIT 1`: membership needs existence, not cardinality.\n      has: this._db.prepare(`SELECT 1 FROM \"${this._table}\" WHERE target_key = ? LIMIT 1`)\n    };\n  }\n\n  /**\n   * Recover the established dimension of an existing fragment `vec0` table from its\n   * stored `CREATE VIRTUAL TABLE` SQL (`float[<n>]`), after checking that the table's\n   * auxiliary columns match `AUXILIARY_COLUMNS`. Returns `undefined` when the\n   * table does not exist yet (a fresh database — dimension is set by the first add).\n   *\n   * Throws when a table of that name exists but is not a usable fragment index (a\n   * mismatched auxiliary-column set, or no `vec0` embedding column); the caller runs\n   * this inside `captureResult`, so it surfaces as a loud `Failure` from `create`.\n   * The same stored DDL answers every one of those questions, so the checks cost\n   * nothing extra.\n   */\n  private static _readExistingDimension(db: BetterSqlite3.Database, table: string): number | undefined {\n    const row: { sql: string } | undefined = db\n      .prepare(\"SELECT sql FROM sqlite_master WHERE type = 'table' AND name = ?\")\n      .get(table) as { sql: string } | undefined;\n    if (row === undefined) {\n      return undefined;\n    }\n    SqliteVecFragmentIndex._verifyAuxiliaryColumns(row.sql, table);\n    const match: RegExpMatchArray | null = row.sql.match(/float\\[(\\d+)\\]/);\n    if (match === null) {\n      // The auxiliary columns matched but there is no `float[<n>]` embedding column,\n      // so this is not a usable fragment index table. Same remedy as a column\n      // mismatch — and failing here beats handing back a dimensionless index whose\n      // first add would `CREATE VIRTUAL TABLE IF NOT EXISTS` into a no-op.\n      throw new Error(\n        `existing table '${table}' has no vec0 embedding column, so it is not a usable fragment ` +\n          `index table. Drop it (or pass a fresh tableName) and re-add every fragment.`\n      );\n    }\n    return Number(match[1]);\n  }\n\n  /**\n   * Compare an existing table's auxiliary columns against `AUXILIARY_COLUMNS`.\n   *\n   * `CREATE VIRTUAL TABLE IF NOT EXISTS` is a no-op against an existing table (SQLite\n   * never compares schemas) and `vec0` has no `ALTER TABLE ADD COLUMN`, so a table\n   * written by an earlier version of this package silently keeps its old columns and\n   * only fails later — as an opaque `no such column` when the widened `INSERT` is\n   * prepared. Detect it here instead and say what to do about it. Order is not\n   * compared: every statement names its columns explicitly, so only the set matters.\n   */\n  private static _verifyAuxiliaryColumns(sql: string, table: string): void {\n    const found: string[] = Array.from(sql.matchAll(AUXILIARY_COLUMN_RE), (m) => m[1]);\n    const expected: ReadonlyArray<string> = AUXILIARY_COLUMNS;\n    const matches: boolean =\n      found.length === expected.length && expected.every((column) => found.includes(column));\n    if (!matches) {\n      throw new Error(\n        `existing table '${table}' has auxiliary columns [${found.join(', ')}] but this index ` +\n          `requires [${expected.join(', ')}] — it was written by a different version of ` +\n          `@fgv/ts-agent-memory-sqlite-vec, or it is not a fragment index table at all. vec0 virtual ` +\n          `tables cannot be altered in place, so this requires a drop-and-re-index: DROP TABLE ` +\n          `\"${table}\" (or pass a fresh tableName) and re-add every fragment. Fragment vectors are ` +\n          `re-derivable from the records, so this costs embedding time, never data.`\n      );\n    }\n  }\n\n  /**\n   * Rebuild the identity fields of a hit from a persisted row, omitting each field\n   * the stored fragment did not carry (so a hit is structurally identical to one this\n   * index produced before `fragment_id` existed).\n   *\n   * A row carrying neither identity violates the write-side invariant and could not\n   * be resolved by the caller, so it fails loudly instead of yielding an anonymous\n   * hit.\n   */\n  private static _toIdentity(row: IKnnRow, key: string): FragmentIdentity {\n    const locator: IFragmentLocator | undefined = SqliteVecFragmentIndex._toLocator(row, key);\n    if (locator === undefined && row.fragment_id === null) {\n      throw new Error(\n        `fragment '${key}': row carries neither a locator nor a fragment id (corrupt persisted data)`\n      );\n    }\n    return {\n      ...(locator !== undefined ? { locator } : {}),\n      ...(row.fragment_id !== null ? { fragmentId: row.fragment_id } : {})\n    };\n  }\n\n  /**\n   * Rebuild a fragment's locator from its persisted offsets, or `undefined` when the\n   * fragment was stored without one (both offsets `NULL`).\n   *\n   * The pair is written all-or-nothing, so a half-`NULL` pair can only come from\n   * corrupt / externally-edited data. Throw rather than coerce — `Number(null)` is\n   * `0`, which would silently fabricate a span starting at the top of the body.\n   */\n  private static _toLocator(row: IKnnRow, key: string): IFragmentLocator | undefined {\n    const start: number | bigint | null = row.start_off;\n    const end: number | bigint | null = row.end_off;\n    if (start === null && end === null) {\n      return undefined;\n    }\n    if (start === null || end === null) {\n      throw new Error(\n        `fragment '${key}': locator has only one of its start/end offsets (corrupt persisted data)`\n      );\n    }\n    return {\n      start: SqliteVecFragmentIndex._toOffset(start, key),\n      end: SqliteVecFragmentIndex._toOffset(end, key)\n    };\n  }\n\n  /** Pack a `Float32Array` as the little-endian byte blob `vec0` stores. Copies, so the caller may reuse its buffer. */\n  private static _toBlob(vector: Float32Array): Uint8Array {\n    return new Uint8Array(Float32Array.from(vector).buffer);\n  }\n\n  /**\n   * Reverse `edgeTargetKey` — the canonical key is `scope\\0id` with NUL excluded\n   * from both components, so the first NUL splits it unambiguously. A key with no\n   * NUL cannot have been written by `edgeTargetKey`; rather than fabricate a wrong\n   * `(scope, id)` from corrupt / externally-edited table data, throw so the query\n   * surfaces it as a loud `Failure`.\n   */\n  private static _parseKey(key: string): IEdgeTarget {\n    const nul: number = key.indexOf('\\0');\n    if (nul < 0) {\n      throw new Error(`malformed target key '${key}': missing scope/id separator (corrupt persisted data)`);\n    }\n    return {\n      scope: key.slice(0, nul) as unknown as MemoryScopeKey,\n      id: key.slice(nul + 1) as unknown as MemoryId\n    };\n  }\n\n  /**\n   * Coerce a persisted locator offset to a plain `number`. `better-sqlite3` returns\n   * integer columns as `bigint` under safe-integer mode, so an offset can arrive as\n   * either; both narrow to `number` here. A value outside the safe-integer range\n   * (only reachable via corrupt / externally-edited data — the index only ever\n   * writes in-document offsets) throws rather than silently losing precision, so the\n   * query surfaces it as a loud `Failure`.\n   */\n  private static _toOffset(value: number | bigint, key: string): number {\n    const n: number = Number(value);\n    if (!Number.isSafeInteger(n)) {\n      throw new Error(\n        `fragment '${key}': locator offset ${String(value)} is not a safe integer (corrupt persisted data)`\n      );\n    }\n    return n;\n  }\n}\n\n/** The prepared statements / helpers the fragment index reuses once its table exists. */\ninterface IFragmentStatements {\n  readonly deleteByTarget: BetterSqlite3.Statement;\n  readonly replace: (key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => void;\n  readonly query: BetterSqlite3.Statement;\n  readonly fragmentCount: BetterSqlite3.Statement;\n  readonly recordCount: BetterSqlite3.Statement;\n  readonly has: BetterSqlite3.Statement;\n}\n"]}
+{"version":3,"file":"sqliteVecFragmentIndex.js","sourceRoot":"","sources":["../../../src/packlets/sqlite-vec-index/sqliteVecFragmentIndex.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAGH,OAAO,EAAE,IAAI,IAAI,aAAa,EAAE,MAAM,YAAY,CAAC;AACnD,OAAO,EAGL,aAAa,EACb,IAAI,EACJ,cAAc,EACd,OAAO,EACP,iBAAiB,EAClB,MAAM,eAAe,CAAC;AACvB,OAAO,EAgBL,aAAa,EACd,MAAM,sBAAsB,CAAC;AAC9B,OAAO,EAAE,UAAU,EAAE,KAAK,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AACvE,OAAO,EAAE,oBAAoB,EAAE,mBAAmB,EAAE,MAAM,cAAc,CAAC;AAOzE,0DAA0D;AAC1D,MAAM,kBAAkB,GAAW,kBAAkB,CAAC;AAEtD,+DAA+D;AAC/D,MAAM,KAAK,GAAW,2BAA2B,CAAC;AAElD,yGAAyG;AACzG,MAAM,aAAa,GAAW,0BAA0B,CAAC;AAEzD;;;;;GAKG;AACH,MAAM,iBAAiB,GAA0B,CAAC,WAAW,EAAE,SAAS,EAAE,aAAa,CAAC,CAAC;AAEzF;;;;GAIG;AACH,MAAM,mBAAmB,GAAW,gCAAgC,CAAC;AA8BrE;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA+CG;AACH,MAAM,OAAO,sBAAsB;IAQjC,YAAoB,EAA0B,EAAE,KAAa,EAAE,SAA6B;QAC1F,IAAI,CAAC,GAAG,GAAG,EAAE,CAAC;QACd,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,UAAU,GAAG,SAAS,CAAC;QAC5B,IAAI,CAAC,MAAM,GAAG,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,QAAQ,EAAE,CAAC;IACtE,CAAC;IAED,yGAAyG;IACzG,IAAW,WAAW;QACpB,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,CAAC,CAAC;QACX,CAAC;QACD,8EAA8E;QAC9E,8DAA8D;QAC9D,OAAO,MAAM,CAAE,IAAI,CAAC,MAAM,CAAC,WAAW,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;IAC7E,CAAC;IAED,kGAAkG;IAClG,IAAW,aAAa;QACtB,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,CAAC,CAAC;QACX,CAAC;QACD,OAAO,MAAM,CAAE,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC,CAAC;IAC/E,CAAC;IAED;;;;;;;;;;;;OAYG;IACI,MAAM,CAAC,MAAM,CAAC,MAA2C;;QAC9D,MAAM,KAAK,GAAW,MAAA,MAAM,CAAC,SAAS,mCAAI,kBAAkB,CAAC;QAC7D,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC/B,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CAAC,0CAA0C,KAAK,kCAAkC,CAAC,CACxF,CAAC;QACJ,CAAC;QACD,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,aAAa,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAC/B,MAAM,SAAS,GAAuB,sBAAsB,CAAC,sBAAsB,CACjF,MAAM,CAAC,QAAQ,EACf,KAAK,CACN,CAAC;YACF,OAAO,IAAI,sBAAsB,CAAC,MAAM,CAAC,QAAQ,EAAE,KAAK,EAAE,SAAS,CAAC,CAAC;QACvE,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,oDAAoD,CAAC,EAAE,CAAC,CACnF,CAAC;IACJ,CAAC;IAED;;;;;;;;;;;;;;;;;;;;;;;;;;OA0BG;IACI,MAAM,CAAC,KAAK,CAAC,IAAI,CACtB,MAAyC;QAEzC,OAAO,CAAC,MAAM,mBAAmB,CAAC,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC,aAAa,CAAC,KAAK,EAAE,QAAQ,EAAE,EAAE,CACtF,CAAC,MAAM,sBAAsB,CAAC,MAAM,CAAC,EAAE,QAAQ,EAAE,SAAS,EAAE,MAAM,CAAC,SAAS,EAAE,CAAC,CAAC;aAC7E,SAAS,CAAC,CAAC,OAAO,EAAE,EAAE;QACrB,2EAA2E;QAC3E,6EAA6E;QAC7E,2EAA2E;QAC3E,2EAA2E;QAC3E,2EAA2E;QAC3E,iCAAiC;QACjC,IAAI,CAAC,gBAAgB,CAAC,OAAO,EAAE,oBAAoB,CAAC,QAAQ,EAAE,KAAK,CAAC,CAAC,CAAC,CACvE;aACA,SAAS,CAAC,CAAC,KAAK,EAAE,EAAE,CACnB,OAAO,CAAC;YACN,KAAK;YACL,KAAK,EAAE,GAAG,EAAE,CAAC,oBAAoB,CAAC,QAAQ,EAAE,KAAK,CAAC;SACnD,CAAC,CACH,CACJ,CAAC;IACJ,CAAC;IAED,sDAAsD;IAC/C,YAAY,CACjB,MAAmB,EACnB,SAA2C;QAE3C,MAAM,GAAG,GAAW,aAAa,CAAC,MAAM,CAAC,CAAC;QAC1C,gFAAgF;QAChF,kFAAkF;QAClF,mFAAmF;QACnF,gFAAgF;QAChF,4DAA4D;QAC5D,IAAI,SAAS,GAAuB,IAAI,CAAC,UAAU,CAAC;QACpD,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,IAAI,QAAQ,CAAC,MAAM,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;gBACjC,OAAO,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,+BAA+B,GAAG,0BAA0B,CAAC,CAAC,CAAC;YAC7F,CAAC;YACD,gFAAgF;YAChF,4EAA4E;YAC5E,uEAAuE;YACvE,IAAI,QAAQ,CAAC,OAAO,KAAK,SAAS,IAAI,QAAQ,CAAC,UAAU,KAAK,SAAS,EAAE,CAAC;gBACxE,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,gEAAgE,CACnG,CACF,CAAC;YACJ,CAAC;YACD,gFAAgF;YAChF,6EAA6E;YAC7E,iFAAiF;YACjF,6EAA6E;YAC7E,wEAAwE;YACxE,IACE,QAAQ,CAAC,OAAO,KAAK,SAAS;gBAC9B,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,EAC9F,CAAC;gBACD,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,eAAe,QAAQ,CAAC,OAAO,CAAC,KAAK,KAAK,QAAQ,CAAC,OAAO,CAAC,GAAG,iCAAiC,CAClI,CACF,CAAC;YACJ,CAAC;YACD,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;gBAC5B,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,MAAM,CAAC;YACrC,CAAC;iBAAM,IAAI,QAAQ,CAAC,MAAM,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAChD,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,+BAA+B,GAAG,yBAAyB,QAAQ,CAAC,MAAM,CAAC,MAAM,mCAAmC,SAAS,EAAE,CAChI,CACF,CAAC;YACJ,CAAC;QACH,CAAC;QACD,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,6EAA6E;YAC7E,0EAA0E;YAC1E,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;oBAC3B,8DAA8D;oBAC9D,OAAO,CAAC,CAAC;gBACX,CAAC;gBACD,wEAAwE;gBACxE,yEAAyE;gBACzE,uEAAuE;gBACvE,uDAAuD;gBACvD,MAAM,WAAW,GAAW,SAAS,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC;gBACvD,IAAI,CAAC,YAAY,CAAC,WAAW,CAAC,CAAC;gBAC/B,IAAI,CAAC,UAAU,GAAG,WAAW,CAAC;gBAC9B,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;YAChC,CAAC;YACD,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YACpC,OAAO,SAAS,CAAC,MAAM,CAAC;QAC1B,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,+BAA+B,GAAG,MAAM,CAAC,EAAE,CAAC,CACvE,CAAC;IACJ,CAAC;IAED,gDAAgD;IACzC,MAAM,CAAC,MAAmB;QAC/B,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,6EAA6E;YAC7E,6BAA6B;YAC7B,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,IAAI,CAAC,MAAM,CAAC,cAAc,CAAC,GAAG,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,CAAC;YACxD,CAAC;YACD,OAAO,MAAM,CAAC;QAChB,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,kCAAkC,aAAa,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,CAC5F,CAAC;IACJ,CAAC;IAED,6CAA6C;IACtC,GAAG,CAAC,MAAmB;QAC5B,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAC,GAAG,EAAE;YACjB,0EAA0E;YAC1E,gEAAgE;YAChE,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC9B,OAAO,KAAK,CAAC;YACf,CAAC;YACD,OAAO,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC,KAAK,SAAS,CAAC;QAClE,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,iCAAiC,aAAa,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,CAC3F,CAAC;IACJ,CAAC;IAED,iDAAiD;IAC1C,KAAK,CAAC,OAAO,CAClB,MAA2B,EAC3B,KAAuB,EACvB,OAA+B;;QAE/B,MAAM,OAAO,GAAY,CAAC,MAAA,OAAO,aAAP,OAAO,uBAAP,OAAO,CAAE,aAAa,mCAAI,MAAM,CAAC,KAAK,MAAM,CAAC;QACvE,6EAA6E;QAC7E,6CAA6C;QAC7C,MAAM,MAAM,GAAiC,MAAM,UAAU,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,CAAC;QACnF,IAAI,MAAM,CAAC,SAAS,EAAE,EAAE,CAAC;YACvB,6EAA6E;YAC7E,6EAA6E;YAC7E,8EAA8E;YAC9E,2CAA2C;YAC3C,OAAO,cAAc,CAAC,mDAAmD,MAAM,CAAC,OAAO,EAAE,CAAC,CAAC;QAC7F,CAAC;QACD,MAAM,OAAO,GAAiB,IAAI,CAAC,MAAM,EAAE,CAAC;QAC5C,IAAI,OAAO,CAAC,SAAS,EAAE,EAAE,CAAC;YACxB,oEAAoE;YACpE,OAAO,cAAc,CAAC,sDAAsD,OAAO,CAAC,OAAO,EAAE,CAAC,CAAC;QACjG,CAAC;QACD,MAAM,OAAO,GAAsB,IAAI,GAAG,EAAgB,CAAC;QAC3D,MAAM,SAAS,GAAsB,IAAI,GAAG,EAAgB,CAAC;QAC7D,MAAM,QAAQ,GAAsB,IAAI,GAAG,EAAgB,CAAC;QAC5D,MAAM,OAAO,GAA2B,EAAE,CAAC;QAC3C,4EAA4E;QAC5E,MAAM,MAAM,GAAG,GAAiC,EAAE,CAAC,CAAC;YAClD,OAAO;YACP,SAAS;YACT,QAAQ;YACR,QAAQ,EAAE,MAAM,CAAC,KAAK,CAAC,QAAQ;YAC/B,OAAO;SACR,CAAC,CAAC;QACH,KAAK,MAAM,MAAM,IAAI,MAAM,CAAC,KAAK,CAAC,OAAO,EAAE,CAAC;YAC1C,MAAM,IAAI,GAAS,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC;YAC/C,2EAA2E;YAC3E,yEAAyE;YACzE,2CAA2C;YAC3C,MAAM,QAAQ,GAA6C,MAAM,UAAU,CAAC,GAAG,EAAE,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;YACxG,IAAI,QAAQ,CAAC,SAAS,EAAE,EAAE,CAAC;gBACzB,MAAM,KAAK,GAAW,sCAAsC,aAAa,CAAC,MAAM,CAAC,MAAM,CAAC,aACtF,QAAQ,CAAC,OACX,EAAE,CAAC;gBACH,IAAI,CAAC,OAAO,EAAE,CAAC;oBACb,iEAAiE;oBACjE,qEAAqE;oBACrE,wBAAwB;oBACxB,OAAO,cAAc,CAAC,gBAAgB,CAAC,KAAK,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;gBAC1E,CAAC;gBACD,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC;gBAC/C,SAAS;YACX,CAAC;YACD,uEAAuE;YACvE,2DAA2D;YAC3D,MAAM,KAAK,GAAmB,MAAM,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,MAAM,EAAE,QAAQ,CAAC,KAAK,CAAC,CAAC;YACrF,IAAI,KAAK,CAAC,SAAS,EAAE,EAAE,CAAC;gBACtB,MAAM,KAAK,GAAW,2BAA2B,KAAK,CAAC,OAAO,EAAE,CAAC;gBACjE,IAAI,CAAC,OAAO,EAAE,CAAC;oBACb,OAAO,cAAc,CAAC,gBAAgB,CAAC,KAAK,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC;gBAC1E,CAAC;gBACD,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC;gBAC/C,SAAS;YACX,CAAC;YACD,IAAI,KAAK,CAAC,KAAK,KAAK,CAAC,EAAE,CAAC;gBACtB,KAAK,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC;gBACtB,SAAS;YACX,CAAC;YACD,KAAK,CAAC,OAAO,EAAE,IAAI,CAAC,CAAC;YACrB,KAAK,CAAC,SAAS,EAAE,IAAI,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC;QACtC,CAAC;QACD,OAAO,iBAAiB,CAAC,MAAM,EAAE,CAAC,CAAC;IACrC,CAAC;IAED;;;;;;;OAOG;IACK,MAAM;QACZ,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC9B,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC;QACvB,CAAC;QACD,6EAA6E;QAC7E,iFAAiF;QACjF,OAAO,aAAa,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,gBAAgB,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,SAAS,CAAC,GAAG,EAAE,CAChG,OAAO,CAAC,IAAI,CAAC,CACd,CAAC;IACJ,CAAC;IAED,+CAA+C;IACxC,KAAK,CACV,MAAoB,EACpB,IAAY,EACZ,OAA+B;QAE/B,MAAM,YAAY,GAAuB,OAAO,aAAP,OAAO,uBAAP,OAAO,CAAE,YAAY,CAAC;QAC/D,MAAM,KAAK,GAA+B,OAAO,aAAP,OAAO,uBAAP,OAAO,CAAE,KAAK,CAAC;QACzD,MAAM,EAAE,GAAyB,OAAO,aAAP,OAAO,uBAAP,OAAO,CAAE,EAAE,CAAC;QAC7C,IAAI,IAAI,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;YAC3C,OAAO,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC;QACtC,CAAC;QACD,IAAI,MAAM,CAAC,MAAM,KAAK,IAAI,CAAC,UAAU,EAAE,CAAC;YACtC,OAAO,OAAO,CAAC,OAAO,CACpB,IAAI,CACF,mCAAmC,MAAM,CAAC,MAAM,mCAAmC,IAAI,CAAC,UAAU,EAAE,CACrG,CACF,CAAC;QACJ,CAAC;QACD,MAAM,KAAK,GAAwB,IAAI,CAAC,MAAM,CAAC;QAC/C,OAAO,OAAO,CAAC,OAAO,CACpB,aAAa,CAAiC,GAAG,EAAE;;YACjD,6EAA6E;YAC7E,6EAA6E;YAC7E,2EAA2E;YAC3E,wDAAwD;YACxD,+EAA+E;YAC/E,+EAA+E;YAC/E,8EAA8E;YAC9E,6EAA6E;YAC7E,uDAAuD;YACvD,MAAM,SAAS,GACb,KAAK,KAAK,SAAS,IAAI,EAAE,KAAK,SAAS,CAAC,CAAC,CAAC,aAAa,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;YACrF,2EAA2E;YAC3E,4EAA4E;YAC5E,6EAA6E;YAC7E,sEAAsE;YACtE,2DAA2D;YAC3D,wEAAwE;YACxE,MAAM,QAAQ,GACZ,SAAS,KAAK,SAAS,IAAI,CAAC,YAAY,KAAK,SAAS,IAAI,KAAK,KAAK,SAAS,CAAC,CAAC;YACjF,MAAM,MAAM,GAAW,QAAQ;gBAC7B,CAAC,CAAC,MAAM,CAAE,KAAK,CAAC,aAAa,CAAC,GAAG,EAA6B,CAAC,CAAC,CAAC;gBACjE,CAAC,CAAC,IAAI,CAAC;YACT,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;gBAChB,OAAO,EAAE,CAAC;YACZ,CAAC;YACD,MAAM,IAAI,GAAe,sBAAsB,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;YAChE,MAAM,IAAI,GAA2B,CACnC,SAAS,KAAK,SAAS;gBACrB,CAAC,CAAC,KAAK,CAAC,mBAAmB,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,CAAC;gBACxD,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,CAAC,CACR,CAAC;YAC5B,8EAA8E;YAC9E,2EAA2E;YAC3E,YAAY;YACZ,MAAM,WAAW,GACf,KAAK,KAAK,SAAS,IAAI,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,GAAG,KAAK,IAAI,CAAC,CAAC,CAAC,SAAS,CAAC;YAC5E,0EAA0E;YAC1E,mEAAmE;YACnE,MAAM,IAAI,GAAsB,EAAE,CAAC;YACnC,MAAM,SAAS,GAAwB,IAAI,GAAG,EAAkB,CAAC;YACjE,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;gBACvB,IAAI,IAAI,CAAC,MAAM,IAAI,IAAI,EAAE,CAAC;oBACxB,MAAM;gBACR,CAAC;gBACD,IAAI,WAAW,KAAK,SAAS,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,UAAU,CAAC,WAAW,CAAC,EAAE,CAAC;oBACzE,SAAS;gBACX,CAAC;gBACD,IAAI,YAAY,KAAK,SAAS,EAAE,CAAC;oBAC/B,MAAM,IAAI,GAAW,MAAA,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,UAAU,CAAC,mCAAI,CAAC,CAAC;oBACxD,IAAI,IAAI,IAAI,YAAY,EAAE,CAAC;wBACzB,SAAS;oBACX,CAAC;oBACD,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,UAAU,EAAE,IAAI,GAAG,CAAC,CAAC,CAAC;gBAC1C,CAAC;gBACD,MAAM,GAAG,GAAW,GAAG,CAAC,UAAU,CAAC;gBACnC,IAAI,CAAC,IAAI,iBACP,MAAM,EAAE,sBAAsB,CAAC,SAAS,CAAC,GAAG,CAAC,EAC7C,KAAK,EAAE,CAAC,GAAG,GAAG,CAAC,QAAQ,IACpB,sBAAsB,CAAC,WAAW,CAAC,GAAG,EAAE,GAAG,CAAC,EAC/C,CAAC;YACL,CAAC;YACD,OAAO,IAAI,CAAC;QACd,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,iCAAiC,CAAC,EAAE,CAAC,CAChE,CAAC;IACJ,CAAC;IAED;;;;OAIG;IACK,YAAY,CAAC,SAAiB;QACpC,IAAI,CAAC,GAAG,CAAC,IAAI,CACX,uCAAuC,IAAI,CAAC,MAAM,eAAe;YAC/D,kDAAkD,SAAS,4BAA4B;YACvF,0DAA0D,CAC7D,CAAC;IACJ,CAAC;IAED,4EAA4E;IACpE,QAAQ;QACd,MAAM,GAAG,GAA4B,IAAI,CAAC,GAAG,CAAC,OAAO,CACnD,gBAAgB,IAAI,CAAC,MAAM,wBAAwB,CACpD,CAAC;QACF,MAAM,GAAG,GAA4B,IAAI,CAAC,GAAG,CAAC,OAAO,CACnD,gBAAgB,IAAI,CAAC,MAAM,4DAA4D;YACrF,wBAAwB,CAC3B,CAAC;QACF,iFAAiF;QACjF,gEAAgE;QAChE,MAAM,UAAU,GAEZ,IAAI,CAAC,GAAG,CAAC,WAAW,CAAC,CAAC,GAAW,EAAE,SAA2C,EAAE,EAAE;;YACpF,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;YACb,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;gBACjC,GAAG,CAAC,GAAG,CACL,GAAG,EACH,sBAAsB,CAAC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC;gBAC/C,yEAAyE;gBACzE,4EAA4E;gBAC5E,2EAA2E;gBAC3E,QAAQ,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,EACtE,QAAQ,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC;gBACpE,0DAA0D;gBAC1D,MAAA,QAAQ,CAAC,UAAU,mCAAI,IAAI,CAC5B,CAAC;YACJ,CAAC;QACH,CAAC,CAAC,CAAC;QACH,OAAO;YACL,cAAc,EAAE,GAAG;YACnB,OAAO,EAAE,CAAC,GAAW,EAAE,SAA2C,EAAQ,EAAE;gBAC1E,UAAU,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YAC7B,CAAC;YACD,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CACrB,sEAAsE,IAAI,CAAC,MAAM,IAAI;gBACnF,mCAAmC,CACtC;YACD,4EAA4E;YAC5E,2EAA2E;YAC3E,8EAA8E;YAC9E,kEAAkE;YAClE,mBAAmB,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CACnC,sEAAsE,IAAI,CAAC,MAAM,IAAI;gBACnF,sDAAsD,CACzD;YACD,aAAa,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,8BAA8B,IAAI,CAAC,MAAM,GAAG,CAAC;YAC7E,WAAW,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,gDAAgD,IAAI,CAAC,MAAM,GAAG,CAAC;YAC7F,0DAA0D;YAC1D,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,CAAC,kBAAkB,IAAI,CAAC,MAAM,gCAAgC,CAAC;SACrF,CAAC;IACJ,CAAC;IAED;;;;;;;;;;;OAWG;IACK,MAAM,CAAC,sBAAsB,CAAC,EAA0B,EAAE,KAAa;QAC7E,MAAM,GAAG,GAAgC,EAAE;aACxC,OAAO,CAAC,iEAAiE,CAAC;aAC1E,GAAG,CAAC,KAAK,CAAgC,CAAC;QAC7C,IAAI,GAAG,KAAK,SAAS,EAAE,CAAC;YACtB,OAAO,SAAS,CAAC;QACnB,CAAC;QACD,sBAAsB,CAAC,uBAAuB,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;QAC/D,MAAM,KAAK,GAA4B,GAAG,CAAC,GAAG,CAAC,KAAK,CAAC,gBAAgB,CAAC,CAAC;QACvE,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;YACnB,+EAA+E;YAC/E,wEAAwE;YACxE,6EAA6E;YAC7E,qEAAqE;YACrE,MAAM,IAAI,KAAK,CACb,mBAAmB,KAAK,iEAAiE;gBACvF,6EAA6E,CAChF,CAAC;QACJ,CAAC;QACD,OAAO,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1B,CAAC;IAED;;;;;;;;;OASG;IACK,MAAM,CAAC,uBAAuB,CAAC,GAAW,EAAE,KAAa;QAC/D,MAAM,KAAK,GAAa,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,mBAAmB,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACnF,MAAM,QAAQ,GAA0B,iBAAiB,CAAC;QAC1D,MAAM,OAAO,GACX,KAAK,CAAC,MAAM,KAAK,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,KAAK,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC;QACzF,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,MAAM,IAAI,KAAK,CACb,mBAAmB,KAAK,4BAA4B,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,mBAAmB;gBACrF,aAAa,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,+CAA+C;gBAC/E,4FAA4F;gBAC5F,sFAAsF;gBACtF,IAAI,KAAK,gFAAgF;gBACzF,0EAA0E,CAC7E,CAAC;QACJ,CAAC;IACH,CAAC;IAED;;;;;;;;OAQG;IACK,MAAM,CAAC,WAAW,CAAC,GAAY,EAAE,GAAW;QAClD,MAAM,OAAO,GAAiC,sBAAsB,CAAC,UAAU,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC;QAC1F,IAAI,OAAO,KAAK,SAAS,IAAI,GAAG,CAAC,WAAW,KAAK,IAAI,EAAE,CAAC;YACtD,MAAM,IAAI,KAAK,CACb,aAAa,GAAG,6EAA6E,CAC9F,CAAC;QACJ,CAAC;QACD,uCACK,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,GAC1C,CAAC,GAAG,CAAC,WAAW,KAAK,IAAI,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,GAAG,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EACpE;IACJ,CAAC;IAED;;;;;;;OAOG;IACK,MAAM,CAAC,UAAU,CAAC,GAAY,EAAE,GAAW;QACjD,MAAM,KAAK,GAA2B,GAAG,CAAC,SAAS,CAAC;QACpD,MAAM,GAAG,GAA2B,GAAG,CAAC,OAAO,CAAC;QAChD,IAAI,KAAK,KAAK,IAAI,IAAI,GAAG,KAAK,IAAI,EAAE,CAAC;YACnC,OAAO,SAAS,CAAC;QACnB,CAAC;QACD,IAAI,KAAK,KAAK,IAAI,IAAI,GAAG,KAAK,IAAI,EAAE,CAAC;YACnC,MAAM,IAAI,KAAK,CACb,aAAa,GAAG,2EAA2E,CAC5F,CAAC;QACJ,CAAC;QACD,OAAO;YACL,KAAK,EAAE,sBAAsB,CAAC,SAAS,CAAC,KAAK,EAAE,GAAG,CAAC;YACnD,GAAG,EAAE,sBAAsB,CAAC,SAAS,CAAC,GAAG,EAAE,GAAG,CAAC;SAChD,CAAC;IACJ,CAAC;IAED,sHAAsH;IAC9G,MAAM,CAAC,OAAO,CAAC,MAAoB;QACzC,OAAO,IAAI,UAAU,CAAC,YAAY,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC;IAC1D,CAAC;IAED;;;;;;OAMG;IACK,MAAM,CAAC,SAAS,CAAC,GAAW;QAClC,MAAM,GAAG,GAAW,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;QACtC,IAAI,GAAG,GAAG,CAAC,EAAE,CAAC;YACZ,MAAM,IAAI,KAAK,CAAC,yBAAyB,GAAG,wDAAwD,CAAC,CAAC;QACxG,CAAC;QACD,OAAO;YACL,KAAK,EAAE,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAA8B;YACrD,EAAE,EAAE,GAAG,CAAC,KAAK,CAAC,GAAG,GAAG,CAAC,CAAwB;SAC9C,CAAC;IACJ,CAAC;IAED;;;;;;;OAOG;IACK,MAAM,CAAC,SAAS,CAAC,KAAsB,EAAE,GAAW;QAC1D,MAAM,CAAC,GAAW,MAAM,CAAC,KAAK,CAAC,CAAC;QAChC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,CAAC;YAC7B,MAAM,IAAI,KAAK,CACb,aAAa,GAAG,qBAAqB,MAAM,CAAC,KAAK,CAAC,iDAAiD,CACpG,CAAC;QACJ,CAAC;QACD,OAAO,CAAC,CAAC;IACX,CAAC;CACF","sourcesContent":["/*\n * Copyright (c) 2026 Erik Fortune\n * SPDX-License-Identifier: MIT\n */\n\nimport type BetterSqlite3 from 'better-sqlite3';\nimport { load as loadSqliteVec } from 'sqlite-vec';\nimport {\n  DetailedResult,\n  Result,\n  captureResult,\n  fail,\n  failWithDetail,\n  succeed,\n  succeedWithDetail\n} from '@fgv/ts-utils';\nimport {\n  FragmentEmbedder,\n  IEdgeTarget,\n  IEmbeddedFragment,\n  IFragmentLocator,\n  IFragmentVectorIndex,\n  IFragmentVectorRebuildReport,\n  IMemoryRecordListing,\n  IMemoryRecordSource,\n  ISkippedVectorRecord,\n  IVectorQueryHit,\n  IVectorRebuildOptions,\n  Kind,\n  IFragmentQueryOptions,\n  MemoryId,\n  MemoryScopeKey,\n  edgeTargetKey\n} from '@fgv/ts-agent-memory';\nimport { invokeHook, tally, withRollbackNote } from './rebuildHelpers';\nimport { closeOwnedConnection, openOwnedConnection } from './connection';\nimport {\n  ISqliteVecFragmentIndexCreateParams,\n  ISqliteVecFragmentIndexHandle,\n  ISqliteVecFragmentIndexOpenParams\n} from './model';\n\n/** Default name for the fragment `vec0` virtual table. */\nconst DEFAULT_TABLE_NAME: string = 'memory_fragments';\n\n/** Package-facing prefix for this class's failure messages. */\nconst LABEL: string = 'sqlite-vec fragment index';\n\n/** A simple SQL identifier — the only shape allowed for the table name (it is interpolated into DDL). */\nconst IDENTIFIER_RE: RegExp = /^[A-Za-z_][A-Za-z0-9_]*$/;\n\n/**\n * The auxiliary (`+`-prefixed) columns this version of the index writes. A table\n * created by an earlier version carries a different set; see\n * {@link SqliteVecFragmentIndex._readExistingDimension} for why that has to be\n * detected explicitly rather than migrated.\n */\nconst AUXILIARY_COLUMNS: ReadonlyArray<string> = ['start_off', 'end_off', 'fragment_id'];\n\n/**\n * Matches one `+name` auxiliary-column declaration in a `vec0` `CREATE VIRTUAL TABLE`\n * statement. Only ever consumed via `String.matchAll`, which iterates a clone rather\n * than advancing this instance's `lastIndex`, so the shared `/g` regex is reusable.\n */\nconst AUXILIARY_COLUMN_RE: RegExp = /\\+\\s*([A-Za-z_][A-Za-z0-9_]*)/g;\n\n/**\n * One KNN row as returned by the fragment `vec0` MATCH query. The offset columns are\n * typed `number | bigint` because `better-sqlite3` returns integer columns as\n * `bigint` when a consumer enables its safe-integer mode (`defaultSafeIntegers`);\n * {@link SqliteVecFragmentIndex._toOffset} coerces them to a plain `number` (and\n * fails loudly on an out-of-safe-range value) before they reach the public locator.\n * All three identity columns are nullable: a fragment stored without a locator has\n * `NULL` offsets, and one stored without a `fragmentId` has a `NULL` `fragment_id`.\n */\ninterface IKnnRow {\n  readonly target_key: string;\n  // eslint-disable-next-line @rushstack/no-new-null -- SQLite returns NULL (not undefined) for an absent locator offset\n  readonly start_off: number | bigint | null;\n  // eslint-disable-next-line @rushstack/no-new-null -- SQLite returns NULL (not undefined) for an absent locator offset\n  readonly end_off: number | bigint | null;\n  // eslint-disable-next-line @rushstack/no-new-null -- SQLite returns NULL (not undefined) for an absent fragment id\n  readonly fragment_id: string | null;\n  readonly distance: number;\n}\n\n/**\n * The identity fields of a fragment hit, in `IVectorQueryHit` shape: a field the\n * stored fragment did not carry is *absent*, never present-but-`undefined`, so a hit\n * for a fragment stored without a `fragmentId` is structurally identical to one this\n * index produced before `fragment_id` existed.\n */\ntype FragmentIdentity = Pick<IVectorQueryHit, 'locator' | 'fragmentId'>;\n\n/**\n * A persistent, `sqlite-vec`-backed `IFragmentVectorIndex` (from\n * `@fgv/ts-agent-memory`) — the fragment-granular sibling of\n * {@link SqliteVecVectorIndex}, and the **durable** counterpart to the in-memory\n * `InMemoryFragmentCosineIndex`.\n *\n * @remarks\n * Where {@link SqliteVecVectorIndex} keys one vector per record on a\n * `target_key` primary key, this index holds **many** vectors per record — one per\n * fragment — so it keys the `vec0` table on `target_key` as a **`PARTITION KEY`**\n * (many rows may share it) and stores each fragment's identity in three auxiliary\n * columns (`+start_off`, `+end_off`, `+fragment_id`) that ride alongside the vector\n * and are returned on query but never filtered — in particular `fragment_id` is\n * stored and returned verbatim, never parsed and never part of the query path. A\n * query is a brute-force `vec0` KNN scan across all partitions returning per-fragment\n * hits, each carrying its record `target` plus whichever identity fields the stored\n * fragment was added with (a fragment must carry at least one).\n *\n * **`vec0` schema changes require a drop-and-re-index.** A\n * `CREATE VIRTUAL TABLE IF NOT EXISTS` is a no-op against an existing table (SQLite\n * does not compare schemas) and `vec0` has no `ALTER TABLE ADD COLUMN`, so a database written by an\n * earlier version of this package keeps its old auxiliary columns. `create` detects\n * that by parsing the stored `CREATE VIRTUAL TABLE` SQL and fails with an actionable\n * message naming the expected and found columns, rather than letting a widened\n * `INSERT` surface an opaque `no such column` at statement-prepare time. There are no\n * in-place migrations: drop the table (or use a fresh `tableName`) and re-index.\n * Fragment vectors are re-derivable from the records, so this costs embedding time,\n * never data.\n *\n * Semantics match `InMemoryFragmentCosineIndex` exactly: `addFragments` is\n * whole-record-replace (a single transaction deletes every prior fragment of the\n * target, then inserts the new set), `remove` drops every fragment of a target,\n * and `query` applies the optional `maxPerRecord` cap **during selection, before\n * the topK cut** — so one long document cannot crowd others out. The dimension is\n * established by the first `addFragments` (the `vec0` column is fixed-width) and\n * recovered from the table schema when a persistent file is reopened; similarity is\n * cosine (`score = 1 - cosineDistance`), byte-identical to the in-memory index.\n * Large-N ANN indexing is explicitly out of scope, same regime as the record index.\n *\n * **Connection ownership depends on which factory you use.** With\n * {@link SqliteVecFragmentIndex.create} the `Database` is consumer-owned\n * (bring-your-own): this index loads the `sqlite-vec` extension onto it and\n * reads/writes the table, but never opens or closes the connection — and that is\n * the seam for backing this index and a record index with one connection. With\n * {@link SqliteVecFragmentIndex.open} this package opens the file itself and hands\n * back a handle carrying the disposer for the connection it created.\n * @public\n */\nexport class SqliteVecFragmentIndex implements IFragmentVectorIndex {\n  private readonly _db: BetterSqlite3.Database;\n  private readonly _table: string;\n  /** The dimension of every stored fragment vector; `undefined` until the table exists. */\n  private _dimension: number | undefined;\n  /** Prepared statements; created once the table exists (established or recovered). */\n  private _stmts: IFragmentStatements | undefined;\n\n  private constructor(db: BetterSqlite3.Database, table: string, dimension: number | undefined) {\n    this._db = db;\n    this._table = table;\n    this._dimension = dimension;\n    this._stmts = dimension === undefined ? undefined : this._prepare();\n  }\n\n  /** The number of records that currently have at least one stored fragment. Zero before the first add. */\n  public get recordCount(): number {\n    if (this._stmts === undefined) {\n      return 0;\n    }\n    // `Number(...)` narrows the count in case the consumer enabled better-sqlite3\n    // safe-integer mode (which returns `count(*)` as a `bigint`).\n    return Number((this._stmts.recordCount.get() as { c: number | bigint }).c);\n  }\n\n  /** The total number of fragments currently held across all records. Zero before the first add. */\n  public get fragmentCount(): number {\n    if (this._stmts === undefined) {\n      return 0;\n    }\n    return Number((this._stmts.fragmentCount.get() as { c: number | bigint }).c);\n  }\n\n  /**\n   * Family-convention factory. Loads the `sqlite-vec` extension onto the supplied\n   * `better-sqlite3` connection and, if the fragment table already exists (a\n   * reopened persistent file), verifies its auxiliary-column set matches this\n   * version's and recovers its established dimension so no re-embedding is needed on\n   * open.\n   *\n   * @param params - See {@link ISqliteVecFragmentIndexCreateParams}.\n   * @returns `Success` with the index, or `Failure` if the table name is not a\n   * simple identifier, the extension fails to load, or the existing table was\n   * written by a version with a different auxiliary-column set (which requires a\n   * drop-and-re-index — `vec0` cannot be altered in place).\n   */\n  public static create(params: ISqliteVecFragmentIndexCreateParams): Promise<Result<SqliteVecFragmentIndex>> {\n    const table: string = params.tableName ?? DEFAULT_TABLE_NAME;\n    if (!IDENTIFIER_RE.test(table)) {\n      return Promise.resolve(\n        fail(`sqlite-vec fragment index: table name '${table}' is not a simple SQL identifier`)\n      );\n    }\n    return Promise.resolve(\n      captureResult(() => {\n        loadSqliteVec(params.database);\n        const dimension: number | undefined = SqliteVecFragmentIndex._readExistingDimension(\n          params.database,\n          table\n        );\n        return new SqliteVecFragmentIndex(params.database, table, dimension);\n      }).withErrorFormat((e) => `sqlite-vec fragment index: failed to initialize: ${e}`)\n    );\n  }\n\n  /**\n   * Path-based factory. Opens the database file itself and returns the index\n   * together with a disposer for the connection it created.\n   *\n   * @remarks\n   * The fragment-granular sibling of {@link SqliteVecVectorIndex.open}, and present\n   * for the same reason: a consumer doing sub-document retrieval only would\n   * otherwise still value-import `better-sqlite3` and hand-roll a `captureResult`\n   * around a constructor that throws.\n   *\n   * **Use `create` instead when one connection must back both a fragment index and\n   * a record index** — the intended shared-handle case. Two `open` calls on one path\n   * give two independent connections, not a shared one.\n   *\n   * If initialization fails after the file is opened, the connection is closed\n   * before returning, so a failed `open` does not leak the descriptor it created.\n   * Should that close *itself* fail — the connection is then genuinely leaked — the\n   * returned message says so rather than hiding it. That includes the\n   * auxiliary-column mismatch failure, which is reported by `create` only after the\n   * file is open.\n   *\n   * @param params - See {@link ISqliteVecFragmentIndexOpenParams}.\n   * @returns `Success` with a {@link ISqliteVecFragmentIndexHandle}, or `Failure` if\n   * the driver could not be loaded, the file could not be opened, the table name is\n   * not a simple identifier, the extension fails to load, or the existing table was\n   * written by a version with a different auxiliary-column set.\n   */\n  public static async open(\n    params: ISqliteVecFragmentIndexOpenParams\n  ): Promise<Result<ISqliteVecFragmentIndexHandle>> {\n    return (await openOwnedConnection(params.path, LABEL)).thenOnSuccess(async (database) =>\n      (await SqliteVecFragmentIndex.create({ database, tableName: params.tableName }))\n        .onFailure((message) =>\n          // This call opened the connection, so a failure to initialize on top of it\n          // must not leave the file handle behind. A close that ALSO fails is said out\n          // loud rather than swallowed — the same reasoning, and the same helper, as\n          // `withRollbackNote`: silently discarding it would make the \"a failed open\n          // leaks nothing\" guarantee untrue exactly when it stopped holding, with no\n          // way for a caller to detect it.\n          fail(withRollbackNote(message, closeOwnedConnection(database, LABEL)))\n        )\n        .onSuccess((index) =>\n          succeed({\n            index,\n            close: () => closeOwnedConnection(database, LABEL)\n          })\n        )\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.addFragments} */\n  public addFragments(\n    target: IEdgeTarget,\n    fragments: ReadonlyArray<IEmbeddedFragment>\n  ): Promise<Result<number>> {\n    const key: string = edgeTargetKey(target);\n    // Validate every fragment before touching the database, so a bad fragment never\n    // leaves the record half-replaced or the dimension half-established (whole-record\n    // replace is all-or-nothing). The effective dimension is the established one, or —\n    // on a still-dimensionless index — the first fragment's length; it is committed\n    // (via table creation) only once the whole batch validates.\n    let dimension: number | undefined = this._dimension;\n    for (const fragment of fragments) {\n      if (fragment.vector.length === 0) {\n        return Promise.resolve(fail(`fragment index: cannot add '${key}': empty fragment vector`));\n      }\n      // A fragment carrying neither identity cannot be resolved back to anything by a\n      // consumer holding the hit — the same invariant `embeddedFragmentConverter`\n      // enforces at the untyped boundary, re-checked here at the index seam.\n      if (fragment.locator === undefined && fragment.fragmentId === undefined) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': fragment requires at least one of 'locator' or 'fragmentId'`\n          )\n        );\n      }\n      // Locator offsets are persisted as SQLite integers (bound via BigInt). Reject a\n      // non-safe-integer offset up front with a clear message, rather than letting\n      // `BigInt(nonInteger)` throw cryptically inside the write transaction OR storing\n      // a value the read-side `_toOffset` guard would later reject on every query.\n      // An absent locator persists as a NULL offset pair and skips the check.\n      if (\n        fragment.locator !== undefined &&\n        (!Number.isSafeInteger(fragment.locator.start) || !Number.isSafeInteger(fragment.locator.end))\n      ) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': locator [${fragment.locator.start}, ${fragment.locator.end}) offsets must be safe integers`\n          )\n        );\n      }\n      if (dimension === undefined) {\n        dimension = fragment.vector.length;\n      } else if (fragment.vector.length !== dimension) {\n        return Promise.resolve(\n          fail(\n            `fragment index: cannot add '${key}': fragment dimension ${fragment.vector.length} does not match index dimension ${dimension}`\n          )\n        );\n      }\n    }\n    return Promise.resolve(\n      captureResult(() => {\n        // A same-target re-author (or an empty batch) still needs the table to exist\n        // to delete prior fragments; create it lazily on the first non-empty add.\n        if (this._stmts === undefined) {\n          if (fragments.length === 0) {\n            // Nothing stored yet and nothing to store: no table, no work.\n            return 0;\n          }\n          // `fragments` is non-empty here (the empty case returned above), so the\n          // validation loop proved every fragment shares `fragments[0]`'s length —\n          // which IS the dimension to establish. Read it straight from the first\n          // fragment: no cast, no invariant-dependent narrowing.\n          const established: number = fragments[0].vector.length;\n          this._createTable(established);\n          this._dimension = established;\n          this._stmts = this._prepare();\n        }\n        this._stmts.replace(key, fragments);\n        return fragments.length;\n      }).withErrorFormat((e) => `fragment index: cannot add '${key}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.remove} */\n  public remove(target: IEdgeTarget): Promise<Result<IEdgeTarget>> {\n    return Promise.resolve(\n      captureResult(() => {\n        // Idempotent: removing a target with no fragments (or before any add created\n        // the table) still succeeds.\n        if (this._stmts !== undefined) {\n          this._stmts.deleteByTarget.run(edgeTargetKey(target));\n        }\n        return target;\n      }).withErrorFormat((e) => `fragment index: cannot remove '${edgeTargetKey(target)}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.has} */\n  public has(target: IEdgeTarget): Promise<Result<boolean>> {\n    return Promise.resolve(\n      captureResult(() => {\n        // Before any add has created the table there is nothing held — a truthful\n        // `false`, matching `remove`'s idempotence and the zero counts.\n        if (this._stmts === undefined) {\n          return false;\n        }\n        return this._stmts.has.get(edgeTargetKey(target)) !== undefined;\n      }).withErrorFormat((e) => `fragment index: cannot check '${edgeTargetKey(target)}': ${e}`)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.rebuild} */\n  public async rebuild(\n    source: IMemoryRecordSource,\n    embed: FragmentEmbedder,\n    options?: IVectorRebuildOptions\n  ): Promise<DetailedResult<IFragmentVectorRebuildReport, IFragmentVectorRebuildReport>> {\n    const lenient: boolean = (options?.onRecordError ?? 'fail') === 'skip';\n    // `source` is consumer-supplied, so a throw or rejection becomes a `Failure`\n    // here rather than escaping as an exception.\n    const listed: Result<IMemoryRecordListing> = await invokeHook(() => source.list());\n    if (listed.isFailure()) {\n      // Deliberately BEFORE any clear, matching both siblings: a failed list is no\n      // evidence about the fragments already held, and clearing here would destroy\n      // a healthy PERSISTED index over a transient read error. No detail — there is\n      // nothing this call disturbed to describe.\n      return failWithDetail(`fragment index rebuild: failed to list records: ${listed.message}`);\n    }\n    const cleared: Result<true> = this._clear();\n    if (cleared.isFailure()) {\n      // Also nothing established: the table still holds whatever it held.\n      return failWithDetail(`fragment index rebuild: failed to clear the index: ${cleared.message}`);\n    }\n    const indexed: Map<Kind, number> = new Map<Kind, number>();\n    const fragments: Map<Kind, number> = new Map<Kind, number>();\n    const declined: Map<Kind, number> = new Map<Kind, number>();\n    const skipped: ISkippedVectorRecord[] = [];\n    // Absent stays absent — only the source knows whether it filtered anything.\n    const report = (): IFragmentVectorRebuildReport => ({\n      indexed,\n      fragments,\n      declined,\n      excluded: listed.value.excluded,\n      skipped\n    });\n    for (const scoped of listed.value.records) {\n      const kind: Kind = scoped.record.envelope.kind;\n      // Capture-wrapped: an embedder that throws mid-loop would otherwise escape\n      // past the `'fail'` rollback below, leaving this DURABLE table holding a\n      // partial index that survives the process.\n      const embedded: Result<ReadonlyArray<IEmbeddedFragment>> = await invokeHook(() => embed(scoped.record));\n      if (embedded.isFailure()) {\n        const error: string = `fragment index rebuild: embedding '${edgeTargetKey(scoped.target)}' failed: ${\n          embedded.message\n        }`;\n        if (!lenient) {\n          // A rollback that also fails is said out loud: the `'fail'` path\n          // promises an empty index, and on a DURABLE table a botched rollback\n          // survives the process.\n          return failWithDetail(withRollbackNote(error, this._clear()), report());\n        }\n        skipped.push({ target: scoped.target, error });\n        continue;\n      }\n      // An empty array is this lane's decline, and it is still WRITTEN — the\n      // whole-record-replace is what clears any stale fragments.\n      const added: Result<number> = await this.addFragments(scoped.target, embedded.value);\n      if (added.isFailure()) {\n        const error: string = `fragment index rebuild: ${added.message}`;\n        if (!lenient) {\n          return failWithDetail(withRollbackNote(error, this._clear()), report());\n        }\n        skipped.push({ target: scoped.target, error });\n        continue;\n      }\n      if (added.value === 0) {\n        tally(declined, kind);\n        continue;\n      }\n      tally(indexed, kind);\n      tally(fragments, kind, added.value);\n    }\n    return succeedWithDetail(report());\n  }\n\n  /**\n   * **Empties the rows; does NOT release the table's declared dimension.** That\n   * is a `vec0` constraint rather than a choice — the dimension is schema, and\n   * there is no `ALTER TABLE` for it — so a rebuild at a new dimension fails\n   * here where it would succeed on the in-memory sibling, which forgets its\n   * dimension on reset. Changing dimension needs a drop-and-re-index; see the\n   * note on `IVectorIndex.rebuild`. Tolerates a table that does not exist yet.\n   */\n  private _clear(): Result<true> {\n    if (this._stmts === undefined) {\n      return succeed(true);\n    }\n    // Capture-wrapped like every other statement path: a closed connection or an\n    // I/O error is a `Failure`, not an exception out of a `Result`-returning method.\n    return captureResult(() => this._db.prepare(`DELETE FROM \"${this._table}\"`).run()).onSuccess(() =>\n      succeed(true)\n    );\n  }\n\n  /** {@inheritDoc IFragmentVectorIndex.query} */\n  public query(\n    vector: Float32Array,\n    topK: number,\n    options?: IFragmentQueryOptions\n  ): Promise<Result<ReadonlyArray<IVectorQueryHit>>> {\n    const maxPerRecord: number | undefined = options?.maxPerRecord;\n    const scope: MemoryScopeKey | undefined = options?.scope;\n    const id: MemoryId | undefined = options?.id;\n    if (topK <= 0 || this._stmts === undefined) {\n      return Promise.resolve(succeed([]));\n    }\n    if (vector.length !== this._dimension) {\n      return Promise.resolve(\n        fail(\n          `fragment index: query dimension ${vector.length} does not match index dimension ${this._dimension}`\n        )\n      );\n    }\n    const stmts: IFragmentStatements = this._stmts;\n    return Promise.resolve(\n      captureResult<ReadonlyArray<IVectorQueryHit>>(() => {\n        // With a per-record cap the topK winners may lie past the first topK rows (a\n        // capped record's later fragments are skipped), so fetch the full ranked set\n        // and apply the cap + topK cut here — exactly as the in-memory index does.\n        // Uncapped, KNN's own `k = topK` is already the answer.\n        // A scope-only narrowing (a versioned kind's per-entity subtree) spans several\n        // records, and `target_key` equality cannot express a prefix, so it is applied\n        // over the full ranked set below. Correct either way — the caller's `topK` is\n        // applied to the NARROWED set, which is the property that matters — but only\n        // the single-record case gets the partition push-down.\n        const recordKey: string | undefined =\n          scope !== undefined && id !== undefined ? edgeTargetKey({ scope, id }) : undefined;\n        // The cap forces the full ranked set ONLY when other records can fill from\n        // behind a capped one. Under a single-record narrowing every row belongs to\n        // that record, so the result is exactly `min(topK, maxPerRecord, fragments)`\n        // and those are the first rows KNN returns — `k = topK` suffices, and\n        // expanding to the table-wide `fragmentCount` would ask an\n        // already-partition-restricted query for far more rows than it can use.\n        const wholeSet: boolean =\n          recordKey === undefined && (maxPerRecord !== undefined || scope !== undefined);\n        const fetchK: number = wholeSet\n          ? Number((stmts.fragmentCount.get() as { c: number | bigint }).c)\n          : topK;\n        if (fetchK <= 0) {\n          return [];\n        }\n        const blob: Uint8Array = SqliteVecFragmentIndex._toBlob(vector);\n        const rows: ReadonlyArray<IKnnRow> = (\n          recordKey !== undefined\n            ? stmts.queryScopedToRecord.all(blob, fetchK, recordKey)\n            : stmts.query.all(blob, fetchK)\n        ) as ReadonlyArray<IKnnRow>;\n        // The scope prefix every record in `scope` shares. `edgeTargetKey` joins with\n        // a NUL, so this cannot collide with a longer scope that merely starts the\n        // same way.\n        const scopePrefix: string | undefined =\n          scope !== undefined && recordKey === undefined ? `${scope}\\0` : undefined;\n        // sqlite-vec returns rows ascending by distance (nearest first); score is\n        // `1 - cosineDistance`, so this order is already descending score.\n        const hits: IVectorQueryHit[] = [];\n        const perRecord: Map<string, number> = new Map<string, number>();\n        for (const row of rows) {\n          if (hits.length >= topK) {\n            break;\n          }\n          if (scopePrefix !== undefined && !row.target_key.startsWith(scopePrefix)) {\n            continue;\n          }\n          if (maxPerRecord !== undefined) {\n            const used: number = perRecord.get(row.target_key) ?? 0;\n            if (used >= maxPerRecord) {\n              continue;\n            }\n            perRecord.set(row.target_key, used + 1);\n          }\n          const key: string = row.target_key;\n          hits.push({\n            target: SqliteVecFragmentIndex._parseKey(key),\n            score: 1 - row.distance,\n            ...SqliteVecFragmentIndex._toIdentity(row, key)\n          });\n        }\n        return hits;\n      }).withErrorFormat((e) => `fragment index: query failed: ${e}`)\n    );\n  }\n\n  /**\n   * Create the fragment `vec0` virtual table with the established dimension. The\n   * auxiliary columns must stay in sync with `AUXILIARY_COLUMNS`, which\n   * `create` compares against an existing table's stored DDL.\n   */\n  private _createTable(dimension: number): void {\n    this._db.exec(\n      `CREATE VIRTUAL TABLE IF NOT EXISTS \"${this._table}\" USING vec0(` +\n        `target_key TEXT PARTITION KEY, embedding float[${dimension}] distance_metric=cosine, ` +\n        `+start_off integer, +end_off integer, +fragment_id text)`\n    );\n  }\n\n  /** Prepare the statements the index reuses. Requires the table to exist. */\n  private _prepare(): IFragmentStatements {\n    const del: BetterSqlite3.Statement = this._db.prepare(\n      `DELETE FROM \"${this._table}\" WHERE target_key = ?`\n    );\n    const ins: BetterSqlite3.Statement = this._db.prepare(\n      `INSERT INTO \"${this._table}\"(target_key, embedding, start_off, end_off, fragment_id) ` +\n        `VALUES (?, ?, ?, ?, ?)`\n    );\n    // Whole-record replace: drop every prior fragment of the target, then insert the\n    // new set, atomically. An empty set collapses to a pure delete.\n    const replaceTxn: BetterSqlite3.Transaction<\n      (key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => void\n    > = this._db.transaction((key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => {\n      del.run(key);\n      for (const fragment of fragments) {\n        ins.run(\n          key,\n          SqliteVecFragmentIndex._toBlob(fragment.vector),\n          // vec0 typed columns reject a JS float; bind the offsets as integers. An\n          // absent locator binds the pair as NULL — never a partial pair, so the read\n          // side can treat a half-NULL pair as corruption rather than a legal shape.\n          fragment.locator === undefined ? null : BigInt(fragment.locator.start),\n          fragment.locator === undefined ? null : BigInt(fragment.locator.end),\n          // Stored verbatim and never parsed; absent binds as NULL.\n          fragment.fragmentId ?? null\n        );\n      }\n    });\n    return {\n      deleteByTarget: del,\n      replace: (key: string, fragments: ReadonlyArray<IEmbeddedFragment>): void => {\n        replaceTxn(key, fragments);\n      },\n      query: this._db.prepare(\n        `SELECT target_key, start_off, end_off, fragment_id, distance FROM \"${this._table}\" ` +\n          `WHERE embedding MATCH ? AND k = ?`\n      ),\n      // The single-record narrowing constrains `target_key`, which is the table's\n      // PARTITION KEY — so this is a partition-restricted KNN rather than a scan\n      // plus a filter. That is the performance reason this narrowing belongs in the\n      // library instead of in a bigger over-fetch on the caller's side.\n      queryScopedToRecord: this._db.prepare(\n        `SELECT target_key, start_off, end_off, fragment_id, distance FROM \"${this._table}\" ` +\n          `WHERE embedding MATCH ? AND k = ? AND target_key = ?`\n      ),\n      fragmentCount: this._db.prepare(`SELECT count(*) AS c FROM \"${this._table}\"`),\n      recordCount: this._db.prepare(`SELECT count(DISTINCT target_key) AS c FROM \"${this._table}\"`),\n      // `LIMIT 1`: membership needs existence, not cardinality.\n      has: this._db.prepare(`SELECT 1 FROM \"${this._table}\" WHERE target_key = ? LIMIT 1`)\n    };\n  }\n\n  /**\n   * Recover the established dimension of an existing fragment `vec0` table from its\n   * stored `CREATE VIRTUAL TABLE` SQL (`float[<n>]`), after checking that the table's\n   * auxiliary columns match `AUXILIARY_COLUMNS`. Returns `undefined` when the\n   * table does not exist yet (a fresh database — dimension is set by the first add).\n   *\n   * Throws when a table of that name exists but is not a usable fragment index (a\n   * mismatched auxiliary-column set, or no `vec0` embedding column); the caller runs\n   * this inside `captureResult`, so it surfaces as a loud `Failure` from `create`.\n   * The same stored DDL answers every one of those questions, so the checks cost\n   * nothing extra.\n   */\n  private static _readExistingDimension(db: BetterSqlite3.Database, table: string): number | undefined {\n    const row: { sql: string } | undefined = db\n      .prepare(\"SELECT sql FROM sqlite_master WHERE type = 'table' AND name = ?\")\n      .get(table) as { sql: string } | undefined;\n    if (row === undefined) {\n      return undefined;\n    }\n    SqliteVecFragmentIndex._verifyAuxiliaryColumns(row.sql, table);\n    const match: RegExpMatchArray | null = row.sql.match(/float\\[(\\d+)\\]/);\n    if (match === null) {\n      // The auxiliary columns matched but there is no `float[<n>]` embedding column,\n      // so this is not a usable fragment index table. Same remedy as a column\n      // mismatch — and failing here beats handing back a dimensionless index whose\n      // first add would `CREATE VIRTUAL TABLE IF NOT EXISTS` into a no-op.\n      throw new Error(\n        `existing table '${table}' has no vec0 embedding column, so it is not a usable fragment ` +\n          `index table. Drop it (or pass a fresh tableName) and re-add every fragment.`\n      );\n    }\n    return Number(match[1]);\n  }\n\n  /**\n   * Compare an existing table's auxiliary columns against `AUXILIARY_COLUMNS`.\n   *\n   * `CREATE VIRTUAL TABLE IF NOT EXISTS` is a no-op against an existing table (SQLite\n   * never compares schemas) and `vec0` has no `ALTER TABLE ADD COLUMN`, so a table\n   * written by an earlier version of this package silently keeps its old columns and\n   * only fails later — as an opaque `no such column` when the widened `INSERT` is\n   * prepared. Detect it here instead and say what to do about it. Order is not\n   * compared: every statement names its columns explicitly, so only the set matters.\n   */\n  private static _verifyAuxiliaryColumns(sql: string, table: string): void {\n    const found: string[] = Array.from(sql.matchAll(AUXILIARY_COLUMN_RE), (m) => m[1]);\n    const expected: ReadonlyArray<string> = AUXILIARY_COLUMNS;\n    const matches: boolean =\n      found.length === expected.length && expected.every((column) => found.includes(column));\n    if (!matches) {\n      throw new Error(\n        `existing table '${table}' has auxiliary columns [${found.join(', ')}] but this index ` +\n          `requires [${expected.join(', ')}] — it was written by a different version of ` +\n          `@fgv/ts-agent-memory-sqlite-vec, or it is not a fragment index table at all. vec0 virtual ` +\n          `tables cannot be altered in place, so this requires a drop-and-re-index: DROP TABLE ` +\n          `\"${table}\" (or pass a fresh tableName) and re-add every fragment. Fragment vectors are ` +\n          `re-derivable from the records, so this costs embedding time, never data.`\n      );\n    }\n  }\n\n  /**\n   * Rebuild the identity fields of a hit from a persisted row, omitting each field\n   * the stored fragment did not carry (so a hit is structurally identical to one this\n   * index produced before `fragment_id` existed).\n   *\n   * A row carrying neither identity violates the write-side invariant and could not\n   * be resolved by the caller, so it fails loudly instead of yielding an anonymous\n   * hit.\n   */\n  private static _toIdentity(row: IKnnRow, key: string): FragmentIdentity {\n    const locator: IFragmentLocator | undefined = SqliteVecFragmentIndex._toLocator(row, key);\n    if (locator === undefined && row.fragment_id === null) {\n      throw new Error(\n        `fragment '${key}': row carries neither a locator nor a fragment id (corrupt persisted data)`\n      );\n    }\n    return {\n      ...(locator !== undefined ? { locator } : {}),\n      ...(row.fragment_id !== null ? { fragmentId: row.fragment_id } : {})\n    };\n  }\n\n  /**\n   * Rebuild a fragment's locator from its persisted offsets, or `undefined` when the\n   * fragment was stored without one (both offsets `NULL`).\n   *\n   * The pair is written all-or-nothing, so a half-`NULL` pair can only come from\n   * corrupt / externally-edited data. Throw rather than coerce — `Number(null)` is\n   * `0`, which would silently fabricate a span starting at the top of the body.\n   */\n  private static _toLocator(row: IKnnRow, key: string): IFragmentLocator | undefined {\n    const start: number | bigint | null = row.start_off;\n    const end: number | bigint | null = row.end_off;\n    if (start === null && end === null) {\n      return undefined;\n    }\n    if (start === null || end === null) {\n      throw new Error(\n        `fragment '${key}': locator has only one of its start/end offsets (corrupt persisted data)`\n      );\n    }\n    return {\n      start: SqliteVecFragmentIndex._toOffset(start, key),\n      end: SqliteVecFragmentIndex._toOffset(end, key)\n    };\n  }\n\n  /** Pack a `Float32Array` as the little-endian byte blob `vec0` stores. Copies, so the caller may reuse its buffer. */\n  private static _toBlob(vector: Float32Array): Uint8Array {\n    return new Uint8Array(Float32Array.from(vector).buffer);\n  }\n\n  /**\n   * Reverse `edgeTargetKey` — the canonical key is `scope\\0id` with NUL excluded\n   * from both components, so the first NUL splits it unambiguously. A key with no\n   * NUL cannot have been written by `edgeTargetKey`; rather than fabricate a wrong\n   * `(scope, id)` from corrupt / externally-edited table data, throw so the query\n   * surfaces it as a loud `Failure`.\n   */\n  private static _parseKey(key: string): IEdgeTarget {\n    const nul: number = key.indexOf('\\0');\n    if (nul < 0) {\n      throw new Error(`malformed target key '${key}': missing scope/id separator (corrupt persisted data)`);\n    }\n    return {\n      scope: key.slice(0, nul) as unknown as MemoryScopeKey,\n      id: key.slice(nul + 1) as unknown as MemoryId\n    };\n  }\n\n  /**\n   * Coerce a persisted locator offset to a plain `number`. `better-sqlite3` returns\n   * integer columns as `bigint` under safe-integer mode, so an offset can arrive as\n   * either; both narrow to `number` here. A value outside the safe-integer range\n   * (only reachable via corrupt / externally-edited data — the index only ever\n   * writes in-document offsets) throws rather than silently losing precision, so the\n   * query surfaces it as a loud `Failure`.\n   */\n  private static _toOffset(value: number | bigint, key: string): number {\n    const n: number = Number(value);\n    if (!Number.isSafeInteger(n)) {\n      throw new Error(\n        `fragment '${key}': locator offset ${String(value)} is not a safe integer (corrupt persisted data)`\n      );\n    }\n    return n;\n  }\n}\n\n/** The prepared statements / helpers the fragment index reuses once its table exists. */\ninterface IFragmentStatements {\n  /** KNN restricted to one record's partition; see `queryScopedToRecord` above. */\n  readonly queryScopedToRecord: BetterSqlite3.Statement;\n  readonly deleteByTarget: BetterSqlite3.Statement;\n  readonly replace: (key: string, fragments: ReadonlyArray<IEmbeddedFragment>) => void;\n  readonly query: BetterSqlite3.Statement;\n  readonly fragmentCount: BetterSqlite3.Statement;\n  readonly recordCount: BetterSqlite3.Statement;\n  readonly has: BetterSqlite3.Statement;\n}\n"]}

package/dist/packlets/sqlite-vec-index/sqliteVecVectorIndex.js

@@ -6,8 +6,11 @@ import { load as loadSqliteVec } from 'sqlite-vec';
 import { captureResult, fail, failWithDetail, succeed, succeedWithDetail } from '@fgv/ts-utils';
 import { edgeTargetKey } from '@fgv/ts-agent-memory';
 import { invokeHook, tally, withRollbackNote } from './rebuildHelpers';
+import { closeOwnedConnection, openOwnedConnection } from './connection';
 /** Default name for the `vec0` virtual table. */
 const DEFAULT_TABLE_NAME = 'memory_vectors';
+/** Package-facing prefix for this class's failure messages. */
+const LABEL = 'sqlite-vec index';
 /** A simple SQL identifier — the only shape allowed for the table name (it is interpolated into DDL). */
 const IDENTIFIER_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
 /**
@@ -36,9 +39,13 @@ const IDENTIFIER_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
  * durable, appropriate for the same "thousands of records" regime the in-memory
  * index targets. Large-N ANN indexing is explicitly out of scope — see the README.
  *
- * The `better-sqlite3` `Database` is consumer-owned (bring-your-own): this index
- * loads the `sqlite-vec` extension onto it and reads/writes the table, but never
- * opens or closes the connection.
+ * **Connection ownership depends on which factory you use.** With
+ * {@link SqliteVecVectorIndex.create} the `Database` is consumer-owned
+ * (bring-your-own): this index loads the `sqlite-vec` extension onto it and
+ * reads/writes the table, but never opens or closes the connection — and that is
+ * the seam for backing a record index and a fragment index with one connection.
+ * With {@link SqliteVecVectorIndex.open} this package opens the file itself and
+ * hands back a handle carrying the disposer for the connection it created.
  * @public
  */
 export class SqliteVecVectorIndex {
@@ -84,6 +91,46 @@ export class SqliteVecVectorIndex {
             return new SqliteVecVectorIndex(params.database, table, dimension);
         }).withErrorFormat((e) => `sqlite-vec index: failed to initialize: ${e}`));
     }
+    /**
+     * Path-based factory. Opens the database file itself and returns the index
+     * together with a disposer for the connection it created.
+     *
+     * @remarks
+     * The convenience over {@link SqliteVecVectorIndex.create} is that the consumer
+     * neither value-imports `better-sqlite3` nor re-establishes `Result` discipline
+     * around a constructor that throws — this is the one place the package leaked its
+     * own dependency into consumer source.
+     *
+     * **Use `create` instead when one connection must back more than one index** (a
+     * record index and a fragment index in the same file, the intended shared-handle
+     * case). Two `open` calls on one path give two independent connections, not a
+     * shared one.
+     *
+     * If initialization fails after the file is opened, the connection is closed
+     * before returning, so a failed `open` does not leak the descriptor it created.
+     * Should that close *itself* fail — the connection is then genuinely leaked — the
+     * returned message says so rather than hiding it.
+     *
+     * @param params - See {@link ISqliteVecVectorIndexOpenParams}.
+     * @returns `Success` with a {@link ISqliteVecVectorIndexHandle}, or `Failure` if
+     * the driver could not be loaded, the file could not be opened, the table name is
+     * not a simple identifier, or the extension fails to load.
+     */
+    static async open(params) {
+        return (await openOwnedConnection(params.path, LABEL)).thenOnSuccess(async (database) => (await SqliteVecVectorIndex.create({ database, tableName: params.tableName }))
+            .onFailure((message) => 
+        // This call opened the connection, so a failure to initialize on top of it
+        // must not leave the file handle behind. A close that ALSO fails is said out
+        // loud rather than swallowed — the same reasoning, and the same helper, as
+        // `withRollbackNote`: silently discarding it would make the "a failed open
+        // leaks nothing" guarantee untrue exactly when it stopped holding, with no
+        // way for a caller to detect it.
+        fail(withRollbackNote(message, closeOwnedConnection(database, LABEL))))
+            .onSuccess((index) => succeed({
+            index,
+            close: () => closeOwnedConnection(database, LABEL)
+        })));
+    }
     /** {@inheritDoc IVectorIndex.add} */
     add(target, vector) {
         const key = edgeTargetKey(target);