libsql-search 0.8.0 → 0.9.1

package/README.md

@@ -26,11 +26,21 @@ npm install libsql-search @libsql/client
 ```
 
 ```bash
-deno add jsr:@logan/libsql-search npm:@libsql/client
+deno add jsr:@logan/libsql-search npm:@libsql/client@^0.17.0
 ```
 
 For npm usage, the package requires Node `>=22.12.0`.
 
+**On npm/pnpm**, the peer range is `@libsql/client ^0.15.0 || ^0.17.0`. Both lines are supported: every behavior this package depends on — `vector_top_k()`'s result shape, the vector index error wording that `search()` matches on, and transactional `batch()` rollback — is identical across them, so an existing `0.15.x` install does not have to move. There is no `0.16.x` line upstream, which is why the range is a disjunction rather than a span. The packaged build is smoke-tested against both arms on every release, at the newest release each arm admits (currently `0.15.15` and `0.17.4`).
+
+**On JSR/Deno the range does not apply to you.** `deno.json` declares no dependency on `@libsql/client` — this package imports only its *types* — so the client you `deno add` separately is constrained by nothing on our side, and a plain `deno add npm:@libsql/client` will silently take whatever is newest, including a future major we have never tested. Deno also cannot express our range: `npm:@libsql/client@^0.15.0 || ^0.17.0` is a parse error, as is any `>=`/`<` span. Pin an arm yourself instead:
+
+```bash
+deno add jsr:@logan/libsql-search npm:@libsql/client@^0.17.0
+```
+
+Note for `0.17.x`: the client no longer exports `./package.json`, so `require("@libsql/client/package.json")` throws `ERR_PACKAGE_PATH_NOT_EXPORTED`. Nothing in this package reads it, but tooling of yours that inspected the client manifest by specifier needs a direct `node_modules` path instead. See [`@libsql/client` version differences](./docs/TROUBLESHOOTING.md#libsqlclient-version-differences) for the other upgrade-visible change.
+
 ## Quick Start
 
 This example uses the default local provider. Local embeddings run in-process after the initial model download and cache warmup; they are not automatically air-gapped.
@@ -74,13 +84,33 @@ console.log(results.map((result) => ({
 
 Important behavior:
 
-- Call `createTable()` before indexing or searching.
+- Call `createTable()` before indexing or searching. It creates the `<tableName>_embedding_idx` vector index that `search()` needs.
 - Keep table width, provider, and dimensions aligned across create/index/query.
 - `indexContent()` embeds every document before it touches the database, then replaces the table in one transaction, so a failed rebuild leaves the previous index intact.
 - `indexContent()` throws `IndexingError` when a file fails; pass `failurePolicy: "skip"` to rebuild from the remaining files.
 - `indexContent()` throws `IndexingError` when no source files are found; pass `allowEmptyIndex: true` to intentionally empty the index.
 - Hosted providers send indexed and queried text to external services and may incur provider charges.
 
+## Search Accuracy And Performance
+
+`search()` queries the `<tableName>_embedding_idx` vector index through libSQL's `vector_top_k()`. It does not score every row, so query cost no longer grows linearly with the size of the index.
+
+That index is an approximate-nearest-neighbor structure, so **the default search path is approximate and can miss a true nearest neighbor.** To limit the loss, `search()` over-fetches candidates from the index, recomputes the true cosine distance for each, and orders exactly by `(distance, id)` before trimming to `limit`. Distances on returned rows are always exact, and result ordering is fully deterministic — including when two rows tie — even though the index's own candidate order is not.
+
+Two options control the trade-off:
+
+```ts
+// Widen the index probe to raise recall (default: max(limit * 4, 32))
+await search({ client, query, limit: 10, candidates: 200 });
+
+// Bypass the index entirely: exact, but linear in table size
+await search({ client, query, exact: true });
+```
+
+`exact: true` is the only way to guarantee exactness. Use it for small corpora, for correctness checks against the index path, and for tables that have no vector index.
+
+Requirements: `vector_top_k()` and `libsql_vector_idx()` need a libSQL build with native vector support. The peer dependency is `@libsql/client ^0.15.0 || ^0.17.0`, verified against `0.15.15` and `0.17.4`; remote Turso/libSQL servers must support vector indexes as well. See the [API reference](./docs/API.md#searchoptions) for full semantics, and [Indexing and operations](./docs/INDEXING.md) for tables created before the index existed.
+
 ## Providers
 
 Built-in providers:

package/dist/index.cjs

@@ -788,6 +788,9 @@ function prepareTextForEmbedding(fields) {
 const SQL_IDENTIFIER_PATTERN = /^[A-Za-z_][A-Za-z0-9_]*$/;
 const DEFAULT_SEARCH_LIMIT = 10;
 const MAX_SEARCH_LIMIT = 100;
+const DEFAULT_SEARCH_CANDIDATE_MULTIPLIER = 4;
+const MIN_SEARCH_CANDIDATES = 32;
+const MAX_SEARCH_CANDIDATES = MAX_SEARCH_LIMIT * 10;
 function validateSqlIdentifier(identifier, name = "identifier") {
   if (typeof identifier !== "string" || !SQL_IDENTIFIER_PATTERN.test(identifier)) {
     throw new Error(
@@ -806,6 +809,23 @@ function normalizeSearchLimit(limit = DEFAULT_SEARCH_LIMIT) {
   }
   return limit;
 }
+function defaultSearchCandidates(limit) {
+  return Math.min(
+    Math.max(limit * DEFAULT_SEARCH_CANDIDATE_MULTIPLIER, MIN_SEARCH_CANDIDATES),
+    MAX_SEARCH_CANDIDATES
+  );
+}
+function normalizeSearchCandidates(candidates, limit) {
+  if (candidates === void 0) {
+    return defaultSearchCandidates(limit);
+  }
+  if (typeof candidates !== "number" || !Number.isFinite(candidates) || !Number.isInteger(candidates) || candidates < limit || candidates > MAX_SEARCH_CANDIDATES) {
+    throw new Error(
+      `Invalid search candidates: expected an integer from the search limit (${limit}) to ${MAX_SEARCH_CANDIDATES}`
+    );
+  }
+  return candidates;
+}
 function normalizeVectorDimensions(dimensions) {
   if (typeof dimensions !== "number" || !Number.isFinite(dimensions) || !Number.isInteger(dimensions) || dimensions < 1) {
     throw new Error("Invalid vector dimensions: expected a positive integer");
@@ -1096,39 +1116,102 @@ async function createTable(client, tableName = "articles", dimensions = 384) {
   `);
 }
 
+const RESULT_COLUMNS = "a.id, a.slug, a.title, a.content, a.folder, a.tags, a.created_at";
+const MISSING_VECTOR_INDEX_PATTERNS = [
+  /failed to parse vector index parameters/i
+];
+const NO_VECTOR_SUPPORT_PATTERNS = [
+  /no such table:\s*vector_top_k/i
+];
 async function search(options) {
   const {
     client,
     query,
     limit = 10,
     tableName = "articles",
-    embeddingOptions = {}
+    embeddingOptions = {},
+    candidates,
+    exact = false
   } = options;
   const quotedTableName = quoteSqlIdentifier(tableName, "tableName");
   const resultLimit = normalizeSearchLimit(limit);
+  const embeddingIndexName = validateSqlIdentifier(
+    `${tableName}_embedding_idx`,
+    "embedding index name"
+  );
+  const candidateCount = normalizeSearchCandidates(candidates, resultLimit);
   const queryEmbedding = await generateEmbedding(query, {
     ...embeddingOptions,
     intent: embeddingOptions.intent ?? "query"
   });
-  const results = await client.execute({
+  const queryVector = JSON.stringify(queryEmbedding);
+  const results = exact ? await executeExactSearch(client, quotedTableName, queryVector, resultLimit) : await executeIndexSearch(
+    client,
+    quotedTableName,
+    embeddingIndexName,
+    queryVector,
+    candidateCount,
+    resultLimit
+  );
+  return results.rows.map(toSearchResult);
+}
+async function executeIndexSearch(client, quotedTableName, embeddingIndexName, queryVector, candidateCount, resultLimit) {
+  try {
+    return await client.execute({
+      sql: `
+        SELECT
+          ${RESULT_COLUMNS},
+          vector_distance_cos(a.embedding, vector(:queryVector)) as distance
+        FROM vector_top_k(:indexName, vector(:queryVector), :candidates) AS v
+        JOIN ${quotedTableName} a ON a.rowid = v.id
+        ORDER BY distance, a.id
+        LIMIT :resultLimit
+      `,
+      args: {
+        queryVector,
+        indexName: embeddingIndexName,
+        candidates: candidateCount,
+        resultLimit
+      }
+    });
+  } catch (error) {
+    throw wrapIndexPathError(error, embeddingIndexName);
+  }
+}
+async function executeExactSearch(client, quotedTableName, queryVector, resultLimit) {
+  return client.execute({
     sql: `
       SELECT
-        id,
-        slug,
-        title,
-        content,
-        folder,
-        tags,
-        created_at,
-        vector_distance_cos(embedding, vector(?)) as distance
-      FROM ${quotedTableName}
-      WHERE embedding IS NOT NULL
-      ORDER BY distance
-      LIMIT ?
+        ${RESULT_COLUMNS},
+        vector_distance_cos(a.embedding, vector(:queryVector)) as distance
+      FROM ${quotedTableName} a
+      WHERE a.embedding IS NOT NULL
+      ORDER BY distance, a.id
+      LIMIT :resultLimit
     `,
-    args: [JSON.stringify(queryEmbedding), resultLimit]
+    args: { queryVector, resultLimit }
   });
-  return results.rows.map((row) => ({
+}
+function wrapIndexPathError(error, embeddingIndexName) {
+  if (!(error instanceof Error)) {
+    return error;
+  }
+  if (MISSING_VECTOR_INDEX_PATTERNS.some((pattern) => pattern.test(error.message))) {
+    return new Error(
+      `Vector index "${embeddingIndexName}" could not be used for search. createTable() creates this index; a table created before it existed, or created by hand, will not have it. Create it with CREATE INDEX IF NOT EXISTS "${embeddingIndexName}" ON <table>(libsql_vector_idx(embedding)), or pass exact: true to search without the index.`,
+      { cause: error }
+    );
+  }
+  if (NO_VECTOR_SUPPORT_PATTERNS.some((pattern) => pattern.test(error.message))) {
+    return new Error(
+      `This libSQL deployment has no vector index support: vector_top_k() is unavailable. Pass exact: true to search on the full-scan path.`,
+      { cause: error }
+    );
+  }
+  return error;
+}
+function toSearchResult(row) {
+  return {
     id: row.id,
     slug: row.slug,
     title: row.title,
@@ -1137,7 +1220,7 @@ async function search(options) {
     tags: JSON.parse(row.tags || "[]"),
     distance: row.distance,
     created_at: row.created_at
-  }));
+  };
 }
 async function getAllArticles(client, tableName = "articles") {
   const quotedTableName = quoteSqlIdentifier(tableName, "tableName");
@@ -1211,7 +1294,10 @@ async function getFolders(client, tableName = "articles") {
   return results.rows.map((row) => row.folder);
 }
 
+exports.DEFAULT_SEARCH_CANDIDATE_MULTIPLIER = DEFAULT_SEARCH_CANDIDATE_MULTIPLIER;
 exports.IndexingError = IndexingError;
+exports.MAX_SEARCH_CANDIDATES = MAX_SEARCH_CANDIDATES;
+exports.MIN_SEARCH_CANDIDATES = MIN_SEARCH_CANDIDATES;
 exports.createEmbeddingProvider = createEmbeddingProvider;
 exports.createTable = createTable;
 exports.generateEmbedding = generateEmbedding;

package/dist/index.d.ts

@@ -161,6 +161,27 @@ declare function indexContent(options: IndexerOptions): Promise<IndexResult>;
  */
 declare function createTable(client: Client, tableName?: string, dimensions?: number): Promise<void>;
 
+/**
+ * Multiplier applied to `limit` when deriving the default candidate count.
+ *
+ * The vector index is approximate, so the default search over-fetches
+ * candidates and re-ranks them exactly. Four candidates per requested result
+ * recovers most of the recall an ANN probe loses without materially changing
+ * the cost of the join.
+ */
+declare const DEFAULT_SEARCH_CANDIDATE_MULTIPLIER = 4;
+/**
+ * Floor for the default candidate count. Small limits would otherwise probe
+ * the index too shallowly for the exact re-rank to have anything to work with.
+ */
+declare const MIN_SEARCH_CANDIDATES = 32;
+/**
+ * Ceiling for an explicit `candidates` value. Mirrors {@link MAX_SEARCH_LIMIT},
+ * scaled by an order of magnitude, so the widest legal probe still stays well
+ * short of a full table scan.
+ */
+declare const MAX_SEARCH_CANDIDATES: number;
+
 /**
  * Vector search functionality
  */
@@ -171,6 +192,22 @@ interface SearchOptions {
     limit?: number;
     tableName?: string;
     embeddingOptions?: EmbeddingOptions;
+    /**
+     * How many candidates to pull from the vector index before the exact
+     * re-rank. Must be an integer from `limit` through {@link MAX_SEARCH_CANDIDATES}.
+     *
+     * Defaults to `max(limit * 4, 32)`.
+     *
+     * Has no effect when `exact` is `true`, but is still validated: an
+     * out-of-range value throws on both paths rather than being quietly accepted
+     * on one of them.
+     */
+    candidates?: number;
+    /**
+     * Bypass the vector index and score every row instead. Exact but linear in
+     * table size. Defaults to `false`.
+     */
+    exact?: boolean;
 }
 interface SearchResult {
     id: number;
@@ -183,7 +220,17 @@ interface SearchResult {
     created_at: string;
 }
 /**
- * Perform semantic search using vector similarity
+ * Perform semantic search using vector similarity.
+ *
+ * By default this queries the `<tableName>_embedding_idx` vector index, which
+ * is an approximate-nearest-neighbor structure: the candidate set it returns is
+ * not guaranteed to contain the true nearest rows, and is not stable across
+ * runs when distances tie. To limit both effects the query over-fetches
+ * `candidates` rows, recomputes the true cosine distance for each, and orders
+ * exactly by `(distance, id)`. The returned ordering is therefore fully
+ * deterministic even though the candidate set is not.
+ *
+ * Pass `exact: true` for a guaranteed-exact full scan.
  */
 declare function search(options: SearchOptions): Promise<SearchResult[]>;
 /**
@@ -226,5 +273,5 @@ declare function getArticlesByFolder(client: Client, folder: string, tableName?:
  */
 declare function getFolders(client: Client, tableName?: string): Promise<string[]>;
 
-export { IndexingError, createEmbeddingProvider, createTable, generateEmbedding, generateEmbeddings, getAllArticles, getArticleBySlug, getArticlesByFolder, getEmbeddingProviderMetadata, getFolders, indexContent, padEmbedding, prepareTextForEmbedding, search, validateEmbeddingBatch };
+export { DEFAULT_SEARCH_CANDIDATE_MULTIPLIER, IndexingError, MAX_SEARCH_CANDIDATES, MIN_SEARCH_CANDIDATES, createEmbeddingProvider, createTable, generateEmbedding, generateEmbeddings, getAllArticles, getArticleBySlug, getArticlesByFolder, getEmbeddingProviderMetadata, getFolders, indexContent, padEmbedding, prepareTextForEmbedding, search, validateEmbeddingBatch };
 export type { EmbeddingBatchBehavior, EmbeddingBatchItem, EmbeddingBatchItemResult, EmbeddingBatchMode, EmbeddingBatchResult, EmbeddingIntent, EmbeddingOptions, EmbeddingProvider, EmbeddingProviderClient, EmbeddingProviderMetadata, EmbeddingRequestOptions, IndexFailure, IndexFailurePolicy, IndexFailureStage, IndexResult, IndexedDocument, IndexerOptions, IndexingErrorPhase, SearchOptions, SearchResult };

package/dist/index.esm.js

@@ -786,6 +786,9 @@ function prepareTextForEmbedding(fields) {
 const SQL_IDENTIFIER_PATTERN = /^[A-Za-z_][A-Za-z0-9_]*$/;
 const DEFAULT_SEARCH_LIMIT = 10;
 const MAX_SEARCH_LIMIT = 100;
+const DEFAULT_SEARCH_CANDIDATE_MULTIPLIER = 4;
+const MIN_SEARCH_CANDIDATES = 32;
+const MAX_SEARCH_CANDIDATES = MAX_SEARCH_LIMIT * 10;
 function validateSqlIdentifier(identifier, name = "identifier") {
   if (typeof identifier !== "string" || !SQL_IDENTIFIER_PATTERN.test(identifier)) {
     throw new Error(
@@ -804,6 +807,23 @@ function normalizeSearchLimit(limit = DEFAULT_SEARCH_LIMIT) {
   }
   return limit;
 }
+function defaultSearchCandidates(limit) {
+  return Math.min(
+    Math.max(limit * DEFAULT_SEARCH_CANDIDATE_MULTIPLIER, MIN_SEARCH_CANDIDATES),
+    MAX_SEARCH_CANDIDATES
+  );
+}
+function normalizeSearchCandidates(candidates, limit) {
+  if (candidates === void 0) {
+    return defaultSearchCandidates(limit);
+  }
+  if (typeof candidates !== "number" || !Number.isFinite(candidates) || !Number.isInteger(candidates) || candidates < limit || candidates > MAX_SEARCH_CANDIDATES) {
+    throw new Error(
+      `Invalid search candidates: expected an integer from the search limit (${limit}) to ${MAX_SEARCH_CANDIDATES}`
+    );
+  }
+  return candidates;
+}
 function normalizeVectorDimensions(dimensions) {
   if (typeof dimensions !== "number" || !Number.isFinite(dimensions) || !Number.isInteger(dimensions) || dimensions < 1) {
     throw new Error("Invalid vector dimensions: expected a positive integer");
@@ -1094,39 +1114,102 @@ async function createTable(client, tableName = "articles", dimensions = 384) {
   `);
 }
 
+const RESULT_COLUMNS = "a.id, a.slug, a.title, a.content, a.folder, a.tags, a.created_at";
+const MISSING_VECTOR_INDEX_PATTERNS = [
+  /failed to parse vector index parameters/i
+];
+const NO_VECTOR_SUPPORT_PATTERNS = [
+  /no such table:\s*vector_top_k/i
+];
 async function search(options) {
   const {
     client,
     query,
     limit = 10,
     tableName = "articles",
-    embeddingOptions = {}
+    embeddingOptions = {},
+    candidates,
+    exact = false
   } = options;
   const quotedTableName = quoteSqlIdentifier(tableName, "tableName");
   const resultLimit = normalizeSearchLimit(limit);
+  const embeddingIndexName = validateSqlIdentifier(
+    `${tableName}_embedding_idx`,
+    "embedding index name"
+  );
+  const candidateCount = normalizeSearchCandidates(candidates, resultLimit);
   const queryEmbedding = await generateEmbedding(query, {
     ...embeddingOptions,
     intent: embeddingOptions.intent ?? "query"
   });
-  const results = await client.execute({
+  const queryVector = JSON.stringify(queryEmbedding);
+  const results = exact ? await executeExactSearch(client, quotedTableName, queryVector, resultLimit) : await executeIndexSearch(
+    client,
+    quotedTableName,
+    embeddingIndexName,
+    queryVector,
+    candidateCount,
+    resultLimit
+  );
+  return results.rows.map(toSearchResult);
+}
+async function executeIndexSearch(client, quotedTableName, embeddingIndexName, queryVector, candidateCount, resultLimit) {
+  try {
+    return await client.execute({
+      sql: `
+        SELECT
+          ${RESULT_COLUMNS},
+          vector_distance_cos(a.embedding, vector(:queryVector)) as distance
+        FROM vector_top_k(:indexName, vector(:queryVector), :candidates) AS v
+        JOIN ${quotedTableName} a ON a.rowid = v.id
+        ORDER BY distance, a.id
+        LIMIT :resultLimit
+      `,
+      args: {
+        queryVector,
+        indexName: embeddingIndexName,
+        candidates: candidateCount,
+        resultLimit
+      }
+    });
+  } catch (error) {
+    throw wrapIndexPathError(error, embeddingIndexName);
+  }
+}
+async function executeExactSearch(client, quotedTableName, queryVector, resultLimit) {
+  return client.execute({
     sql: `
       SELECT
-        id,
-        slug,
-        title,
-        content,
-        folder,
-        tags,
-        created_at,
-        vector_distance_cos(embedding, vector(?)) as distance
-      FROM ${quotedTableName}
-      WHERE embedding IS NOT NULL
-      ORDER BY distance
-      LIMIT ?
+        ${RESULT_COLUMNS},
+        vector_distance_cos(a.embedding, vector(:queryVector)) as distance
+      FROM ${quotedTableName} a
+      WHERE a.embedding IS NOT NULL
+      ORDER BY distance, a.id
+      LIMIT :resultLimit
     `,
-    args: [JSON.stringify(queryEmbedding), resultLimit]
+    args: { queryVector, resultLimit }
   });
-  return results.rows.map((row) => ({
+}
+function wrapIndexPathError(error, embeddingIndexName) {
+  if (!(error instanceof Error)) {
+    return error;
+  }
+  if (MISSING_VECTOR_INDEX_PATTERNS.some((pattern) => pattern.test(error.message))) {
+    return new Error(
+      `Vector index "${embeddingIndexName}" could not be used for search. createTable() creates this index; a table created before it existed, or created by hand, will not have it. Create it with CREATE INDEX IF NOT EXISTS "${embeddingIndexName}" ON <table>(libsql_vector_idx(embedding)), or pass exact: true to search without the index.`,
+      { cause: error }
+    );
+  }
+  if (NO_VECTOR_SUPPORT_PATTERNS.some((pattern) => pattern.test(error.message))) {
+    return new Error(
+      `This libSQL deployment has no vector index support: vector_top_k() is unavailable. Pass exact: true to search on the full-scan path.`,
+      { cause: error }
+    );
+  }
+  return error;
+}
+function toSearchResult(row) {
+  return {
     id: row.id,
     slug: row.slug,
     title: row.title,
@@ -1135,7 +1218,7 @@ async function search(options) {
     tags: JSON.parse(row.tags || "[]"),
     distance: row.distance,
     created_at: row.created_at
-  }));
+  };
 }
 async function getAllArticles(client, tableName = "articles") {
   const quotedTableName = quoteSqlIdentifier(tableName, "tableName");
@@ -1209,4 +1292,4 @@ async function getFolders(client, tableName = "articles") {
   return results.rows.map((row) => row.folder);
 }
 
-export { IndexingError, createEmbeddingProvider, createTable, generateEmbedding, generateEmbeddings, getAllArticles, getArticleBySlug, getArticlesByFolder, getEmbeddingProviderMetadata, getFolders, indexContent, padEmbedding, prepareTextForEmbedding, search, validateEmbeddingBatch };
+export { DEFAULT_SEARCH_CANDIDATE_MULTIPLIER, IndexingError, MAX_SEARCH_CANDIDATES, MIN_SEARCH_CANDIDATES, createEmbeddingProvider, createTable, generateEmbedding, generateEmbeddings, getAllArticles, getArticleBySlug, getArticlesByFolder, getEmbeddingProviderMetadata, getFolders, indexContent, padEmbedding, prepareTextForEmbedding, search, validateEmbeddingBatch };

package/docs/API.md

@@ -19,6 +19,9 @@
 - `padEmbedding`
 - `prepareTextForEmbedding`
 - `IndexingError`
+- `DEFAULT_SEARCH_CANDIDATE_MULTIPLIER`
+- `MIN_SEARCH_CANDIDATES`
+- `MAX_SEARCH_CANDIDATES`
 
 It also exports these types:
 
@@ -187,6 +190,8 @@ interface SearchOptions {
   limit?: number;
   tableName?: string;
   embeddingOptions?: EmbeddingOptions;
+  candidates?: number;
+  exact?: boolean;
 }
 ```
 
@@ -194,9 +199,67 @@ Defaults:
 
 - `limit`: `10`
 - `tableName`: `"articles"`
+- `candidates`: `Math.max(limit * 4, 32)`
+- `exact`: `false`
 
 `limit` must be an integer from `1` through `100`; invalid values are rejected before query embedding generation.
 
+### Search Is Approximate By Default
+
+The default path queries the `<tableName>_embedding_idx` vector index through libSQL's `vector_top_k()`. **That index is an approximate-nearest-neighbor structure. It can miss a true nearest neighbor.** There is no configuration that makes the index path exact; only `exact: true` guarantees exactness.
+
+To limit the accuracy loss, the default path over-fetches: it pulls `candidates` rows from the index, recomputes the true `vector_distance_cos` for each, and orders exactly by `(distance, id)` before trimming to `limit`. So:
+
+- **Recall is approximate.** A row the index does not return as a candidate cannot appear in the results, no matter its true distance. Raising `candidates` raises recall.
+- **Ranking within the candidate set is exact.** Distances on returned rows are always the true cosine distances, not index approximations.
+- **Ordering is fully deterministic.** The `id` tiebreaker means two rows at an identical distance always come back in the same order, even though the index's own candidate order is not stable across runs.
+- **Rows with a `NULL` embedding are never returned.** The vector index excludes them on its own.
+
+**Both paths order by `(distance, id)`.** The determinism guarantee covers `exact: true` as well as the default, so the two paths return identical orderings for identical inputs and can be compared directly. This is a change in tie ordering for the exact path, which previously sorted by distance alone and left tied rows in whatever order the scan produced.
+
+### `candidates`
+
+Controls how many rows the index returns for the exact re-rank. It must be an integer from `limit` through `MAX_SEARCH_CANDIDATES`; a value below `limit` is rejected rather than clamped, because it would silently truncate the result set. Invalid values throw before query embedding generation and before any database call.
+
+```ts
+// Trade query cost for recall on a large corpus
+const results = await search({ client, query, limit: 10, candidates: 200 });
+```
+
+`candidates` has no effect when `exact` is `true` — that path scans every row — but it is **still validated**. `search({ exact: true, limit: 10, candidates: 5 })` throws, exactly as it would on the index path. Validity does not depend on which path a call happens to take.
+
+Related exported constants:
+
+- `DEFAULT_SEARCH_CANDIDATE_MULTIPLIER` (`4`): multiplier applied to `limit`
+- `MIN_SEARCH_CANDIDATES` (`32`): floor for the derived default
+- `MAX_SEARCH_CANDIDATES` (`1000`): ceiling for any explicit value. The derived default cannot reach it today, since `limit` tops out at `100`; the cap constrains explicit values.
+
+### `exact`
+
+Set `exact: true` to bypass the index and score every row in the table.
+
+```ts
+const results = await search({ client, query, exact: true });
+```
+
+This is the guaranteed-exact path: it computes `vector_distance_cos` for every row with a non-`NULL` embedding, sorts by `(distance, id)`, and trims to `limit`. Cost grows linearly with table size, so it is intended for small corpora, correctness checks against the index path, and tables that have no vector index.
+
+### Missing Vector Index
+
+If the target table has no `<tableName>_embedding_idx`, the default path throws an error naming the missing index and pointing at `createTable()` and `exact: true`. libSQL's own message for this case ("failed to parse vector index parameters") says nothing about a missing index, so it is preserved as the thrown error's `cause` rather than surfaced directly.
+
+A deployment with no vector support at all is a separate case with its own message. `vector_top_k()` does not exist there, so no `CREATE INDEX` can help and the error points only at `exact: true`.
+
+**Those two messages are the only ones rewritten.** Every other failure from the index path reaches the caller with its original message intact. In particular, a width mismatch between the query embedding and the `embedding` column surfaces as libSQL's own `vector index(search): dimensions are different: 384 != 4`, which names both widths and is the useful diagnostic for the dimension drift described in the [Migration and reindexing guide](./MIGRATIONS.md).
+
+Search never falls back to the exact scan on its own. A silent fallback would turn a one-line schema fix into an invisible, permanent full-table scan.
+
+### Requirements
+
+`vector_top_k()` and `libsql_vector_idx()` require a libSQL build with native vector support. The peer dependency is `@libsql/client ^0.15.0 || ^0.17.0`; this behavior is verified against `@libsql/client` `0.15.15` and `0.17.4` with a local `:memory:` database. On both, `vector_top_k()` returns the matched rowids in an `id` column, libSQL's missing-index wording ("failed to parse vector index parameters") is byte-identical, and so is the dimension-mismatch wording that is passed through unchanged — so the index path and the diagnostics above behave the same on either line. The two lines share one embedded engine (`libsql` `0.5.29`), which is why the on-disk format and index semantics do not differ between them.
+
+Coverage differs by layer, and is worth stating plainly: the packaged build is smoke-tested against **both** peer arms on every release, which covers table and vector-index creation. The full test suite — including the byte-exact assertions on the messages above — runs against the dev-pinned client, currently `0.17.4`. The no-vector-support case is the one message not reproducible against a local build on either version; it is asserted from a synthesized error rather than a measured one. Remote Turso/libSQL servers must also support vector indexes — that is a property of the server, not the client, and no minimum server version is claimed here beyond that requirement. A deployment without it fails the default path with an error saying so and naming `exact: true` as the remedy. Use `exact: true` against any deployment where vector index support is unavailable or unverified.
+
 Result shape:
 
 ```ts

package/docs/INDEXING.md

@@ -111,6 +111,40 @@ await indexContent({
 
 Both behaviors changed in a breaking way: partial failures used to be counted and reported, and an empty directory used to return zeros without clearing the table.
 
+## The Embedding Vector Index
+
+`createTable()` creates `<tableName>_embedding_idx` alongside the table:
+
+```sql
+CREATE INDEX IF NOT EXISTS "<tableName>_embedding_idx"
+ON "<tableName>"(libsql_vector_idx(embedding))
+```
+
+`search()` requires that index by default. It queries the index through `vector_top_k()` instead of scoring every row, so query cost no longer grows linearly with the size of the index.
+
+That index is approximate. `search()` compensates by over-fetching candidates and re-ranking them exactly; see [`search(options)`](./API.md#searchoptions) for the recall and ordering semantics and for the `candidates` and `exact` options.
+
+### Tables Built Before The Index Existed
+
+A table created by hand, or by a version of this package that predated the embedding index, has no `<tableName>_embedding_idx`. The default search path fails on such a table with an error naming the missing index — libSQL's own message for the case ("failed to parse vector index parameters") does not mention it.
+
+`indexContent()` does not create the index; it only replaces rows. Two ways forward:
+
+```ts
+// Preferred: createTable() is idempotent and adds only what is missing
+await createTable(client, "articles", 384);
+```
+
+```sql
+-- Or create the index directly against the existing table
+CREATE INDEX IF NOT EXISTS "articles_embedding_idx"
+ON "articles"(libsql_vector_idx(embedding));
+```
+
+`createTable()` uses `CREATE TABLE IF NOT EXISTS` and `CREATE INDEX IF NOT EXISTS`, so calling it against an existing table adds the missing index without touching rows. It still does not resize an existing vector column.
+
+Until the index exists, pass `exact: true` to `search()` to keep queries working on the full-scan path.
+
 ## Quality Guidelines
 
 - include descriptive frontmatter titles
@@ -118,6 +152,7 @@ Both behaviors changed in a breaking way: partial failures used to be counted an
 - use the same provider and dimensions at index and query time
 - keep `maxLength` intentional if your content is very large
 - start with a smaller search `limit` and tune from real query behavior
+- raise `candidates` if the approximate index path misses results the exact path finds; compare the two with `exact: true` on a fixed set of queries
 
 ## Build Integration
 
@@ -139,6 +174,6 @@ Many projects wire indexing into a dedicated script and call it before their sit
 ## Runtime Notes
 
 - local embeddings may download and cache a model on the first run
-- Node users need `@libsql/client` installed alongside the package
+- Node users need `@libsql/client` installed alongside the package, at `^0.15.0 || ^0.17.0`; the packaged build is smoke-tested against both arms (`0.15.15` and `0.17.4`), which covers table and vector-index creation. `batch()` rollback behaves identically on both at the contract level, though its error text differs — see [Version differences](./TROUBLESHOOTING.md#libsqlclient-version-differences). Upgrading the client is not a prerequisite for upgrading this package. Deno/JSR users are not covered by that range and should pin the client themselves — see [Install](../README.md#install)
 - hosted providers send indexed or queried text to external services
 - the repository validates package build and `deno check`, but indexing still depends on filesystem access

package/docs/MIGRATIONS.md

@@ -23,6 +23,7 @@ References:
 - if dimensions stay the same but provider, model, endpoint, model revision, pooling, normalization, or input formatting changes, fully reindex anyway
 - never mix two embedding spaces in one table
 - prefer a parallel table migration because `indexContent()` replaces the whole target table, so an in-place rebuild leaves no way back to the old vectors
+- upgrading `@libsql/client` is not one of these migrations. The client moves bytes; it does not define the embedding space. Moving between the supported `^0.15.0` and `^0.17.0` lines leaves stored vectors, table widths, and the embedding index untouched and needs no reindex. This is checkable rather than merely inferred: `libsql`, the embedded native engine that owns the on-disk `F32_BLOB` format and the vector index, resolves to `0.5.29` under both client lines — the client bump does not move it. See [`@libsql/client` version differences](./TROUBLESHOOTING.md#libsqlclient-version-differences) for the two client-side behaviors that do change.
 
 In practice, this means:
 
@@ -75,6 +76,26 @@ const results = await search({
 });
 ```
 
+## Tables Without The Embedding Vector Index
+
+`search()` queries the `<tableName>_embedding_idx` vector index by default rather than scanning the whole table. A table created by hand, or by a version of this package that predated that index, does not have it, and the default search path fails against such a table.
+
+Re-running `createTable()` with the table's existing width is the fix. It is idempotent — `CREATE TABLE IF NOT EXISTS` and `CREATE INDEX IF NOT EXISTS` — so it adds the missing index without touching rows and without resizing the vector column:
+
+```ts
+// Same name and same width as the existing table
+await createTable(client, "articles_local_384", 384);
+```
+
+Equivalently, in SQL:
+
+```sql
+CREATE INDEX IF NOT EXISTS "articles_local_384_embedding_idx"
+ON "articles_local_384"(libsql_vector_idx(embedding));
+```
+
+Reindexing does not create the index; `indexContent()` only replaces rows. Any new table created by `createTable()` as part of a migration already has it, so this applies only to pre-existing tables you are carrying forward. Until the index exists, `search({ ..., exact: true })` keeps queries working on the exact full-scan path.
+
 ## Common Migration Paths
 
 | From | To | Why a rebuild is required | Recommended table move |

package/docs/TROUBLESHOOTING.md

@@ -17,3 +17,62 @@ Common operational checks:
   or use a new table name before rebuilding
 - after upgrading an existing local 768-dimensional padded index, create or
   recreate a 384-dimensional table and fully re-index before querying it
+- if `search()` reports that the `<tableName>_embedding_idx` vector index could
+  not be used, the table has no embedding vector index: re-run `createTable()`
+  with the table's existing name and width to add it without touching rows, or
+  pass `exact: true` to search on the full-scan path meanwhile. See
+  [Tables without the embedding vector index](./MIGRATIONS.md#tables-without-the-embedding-vector-index).
+  The underlying libSQL message, preserved as the error's `cause`, reads
+  "failed to parse vector index parameters" and does not mention the index
+- if `search()` reports "no vector index support: vector_top_k() is
+  unavailable", the libSQL build or server itself has no vector support, so
+  creating an index will not help. Pass `exact: true` to search on the
+  full-scan path, or move to a deployment with vector support
+- a libSQL error reading "dimensions are different: 384 != 4" is a different
+  problem and is passed through unchanged: the index exists, but the query
+  embedding's width does not match the `embedding` column's width. Creating an
+  index will not help and neither will `exact: true` — the exact path reports
+  the same root cause in different words, as
+  "vector_distance: vectors must have the same length: 4 != 384", with the
+  operands in the opposite order. Align the widths across
+  `createTable()`, `indexContent()`, and `search()`, and re-index if the stored
+  vectors are in the wrong space; see the
+  [Migration and reindexing guide](./MIGRATIONS.md)
+
+## `@libsql/client` Version Differences
+
+The peer range is `^0.15.0 || ^0.17.0`, verified against `0.15.15` and `0.17.4`.
+There is no `0.16.x` line upstream. Everything this package depends on behaves
+the same on both — `vector_top_k()` returns rowids in an `id` column, the
+missing-index message this library rewrites ("failed to parse vector index
+parameters") and the dimension-mismatch message it passes through unchanged are
+both byte-identical, and `batch(..., "write")` still rolls the whole rebuild back
+on failure — so upgrading the client is optional and neither direction requires
+re-indexing. (The third message, the no-vector-support case, is not reproducible
+against a local build on either version; see
+[Requirements](./API.md#requirements).)
+
+Two client-side differences are visible to callers on `0.17.x`:
+
+- the client no longer exports `./package.json`. Reading it by specifier, as in
+  `require("@libsql/client/package.json")`, throws
+  `ERR_PACKAGE_PATH_NOT_EXPORTED`. Nothing in this package does that; if your own
+  tooling did, read the file through a direct `node_modules` path instead
+- **constraint error codes lost the `_UNIQUE` suffix.** A duplicate slug that
+  reported `SQLITE_CONSTRAINT_UNIQUE` on `0.15.x` reports the broader
+  `SQLITE_CONSTRAINT` on `0.17.x`. This affects every caller on both query paths:
+
+  | | `0.15.15` | `0.17.4` |
+  | --- | --- | --- |
+  | `execute()` | `SQLITE_CONSTRAINT_UNIQUE: UNIQUE constraint failed: …` | `SQLITE_CONSTRAINT: UNIQUE constraint failed: …` |
+  | `batch(…, "write")` | `SQLITE_CONSTRAINT_UNIQUE: UNIQUE constraint failed: …` | `SQLITE_CONSTRAINT: SQLITE_CONSTRAINT: UNIQUE constraint failed: …` |
+
+  Note that the prefix is additionally **doubled on the `batch()` path only** —
+  that is the path `indexContent()` uses, so it is what surfaces as the `cause`
+  of an `IndexingError` with `phase: "replace"`. Your own `execute()` calls show
+  the single prefix. Both are cosmetic: the rollback and the `IndexingError`
+  contract are unchanged.
+
+  A log matcher or alert rule keyed on `SQLITE_CONSTRAINT_UNIQUE` will stop
+  matching after the upgrade, on either path. Match on `UNIQUE constraint failed`
+  instead — it is the one substring stable across all four cells above.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "libsql-search",
-  "version": "0.8.0",
+  "version": "0.9.1",
   "description": "Semantic search for static sites using libSQL/Turso with multi-provider embeddings",
   "type": "module",
   "packageManager": "pnpm@10.34.5",
@@ -60,14 +60,14 @@
     "node": ">=22.12.0"
   },
   "peerDependencies": {
-    "@libsql/client": "^0.15.0"
+    "@libsql/client": "^0.15.0 || ^0.17.0"
   },
   "dependencies": {
     "@huggingface/transformers": "4.2.0",
     "gray-matter": "^4.0.3"
   },
   "devDependencies": {
-    "@libsql/client": "^0.15.15",
+    "@libsql/client": "^0.17.4",
     "@rollup/plugin-commonjs": "^29.0.3",
     "@rollup/plugin-node-resolve": "^16.0.3",
     "@types/node": "^24.13.3",