libsql-search 0.7.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/PROVIDERS.md CHANGED
@@ -1,6 +1,8 @@
1
1
  # Embedding Providers
2
2
 
3
- `libsql-search` currently supports six embedding providers:
3
+ Use this page to choose an embedding provider, confirm the table width it needs, and understand what crosses a network boundary.
4
+
5
+ `libsql-search` supports these provider values:
4
6
 
5
7
  - `local`
6
8
  - `cloudflare`
@@ -9,11 +11,9 @@
9
11
  - `openai`
10
12
  - `openai-compatible`
11
13
 
12
- Use the same provider and dimensions for both indexing and querying. A mismatch
13
- between stored vectors and query vectors will break search quality or fail at
14
- query time.
14
+ ## Shared Behavior
15
15
 
16
- ## Shared Options
16
+ All providers share the same `EmbeddingOptions` surface:
17
17
 
18
18
  ```ts
19
19
  interface EmbeddingOptions {
@@ -38,88 +38,37 @@ interface EmbeddingOptions {
38
38
  }
39
39
  ```
40
40
 
41
- - `provider` defaults to `"local"`
42
- - `dimensions` defaults to `384` for the library's default local provider.
43
- Provider-specific defaults can differ; Cloudflare and Mistral use `1024`,
44
- and Gemini defaults to `3072`.
45
- - `maxLength` defaults to `8000`
46
- - `intent` can be `"document"` or `"query"`; indexing defaults to
47
- `"document"` and search defaults to `"query"` unless explicitly set
48
- - `timeoutMs` defaults to `30000`
49
- - `apiKey` is used by Mistral, Gemini, and OpenAI and falls back to
50
- `MISTRAL_API_KEY`, `GEMINI_API_KEY`, or `OPENAI_API_KEY`. For
51
- `openai-compatible`, `apiKey` is optional and never falls back to
52
- `OPENAI_API_KEY`.
53
- - `accountId` and `apiToken` are used by Cloudflare and fall back to
54
- `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN`
55
- - `baseUrl`, `model`, and `batchSize` are used by `openai-compatible`
56
-
57
- ## Provider Contract
41
+ Shared defaults and rules:
58
42
 
59
- Each provider exposes immutable metadata:
60
-
61
- ```ts
62
- interface EmbeddingProviderMetadata {
63
- name:
64
- | "local"
65
- | "cloudflare"
66
- | "mistral"
67
- | "gemini"
68
- | "openai"
69
- | "openai-compatible";
70
- model: string;
71
- dimensions: number;
72
- batch: {
73
- mode: "native" | "sequential";
74
- maxSize?: number;
75
- };
76
- }
77
- ```
78
-
79
- Use `getEmbeddingProviderMetadata(options)` or
80
- `createEmbeddingProvider(options).metadata` to inspect the effective model,
81
- dimensions, and batch behavior. Metadata inspection does not require hosted
82
- provider credentials.
83
-
84
- Batch modes:
85
-
86
- - `"native"` means the upstream provider accepts the batch in one request
87
- - `"sequential"` means the library accepts a batch and processes items one at a
88
- time
89
- - when `maxSize` is present, it is a hard maximum enforced before provider or
90
- network work
91
-
92
- `generateEmbeddings(texts, options)` returns vectors in the same order as the
93
- input texts. Provider responses are validated before database writes:
94
-
95
- - result count must match input count
96
- - each vector must match the provider's effective dimensions
97
- - every vector value must be a finite number
98
- - indexed batch responses must contain unique contiguous indices and are
99
- reordered before being returned
43
+ - `provider` defaults to `local`
44
+ - `maxLength` defaults to `8000` code units
45
+ - `timeoutMs` defaults to `30000`
46
+ - `indexContent()` defaults to `intent: "document"`
47
+ - `search()` defaults to `intent: "query"`
48
+ - `getEmbeddingProviderMetadata()` reports the effective provider, model, dimensions, and batch mode without making a hosted call
49
+ - use one embedding space per table: if provider, dimensions, model selection, endpoint, or formatting contract changes, build a new table and reindex
100
50
 
101
- Empty batches return `[]` without loading a local model, creating hosted clients,
102
- or making network calls.
51
+ Intent behavior is intentionally narrow today:
103
52
 
104
- Lower-level provider clients return an `EmbeddingBatchResult` with the validated
105
- vectors plus provider, model, dimensions, and intent. The compatibility helpers
106
- `generateEmbedding()` and `generateEmbeddings()` return only arrays.
53
+ - only the current Gemini adapter changes the formatted payload for `"document"` versus `"query"`
54
+ - all other providers still carry `intent` metadata through the API, but they embed the same text string either way
107
55
 
108
- Cloudflare, Mistral, Gemini, and OpenAI clients are scoped to their current
109
- options. They are not cached globally across different credentials or
110
- configurations. The local Hugging Face Transformers pipeline is loaded lazily
111
- and cached by model name.
56
+ The `model` option is only used by `openai-compatible`.
112
57
 
113
- Hosted provider failures are reported with bounded provider/status/request-id
114
- context and without raw upstream bodies, credentials, Authorization headers, or
115
- full URLs with query strings.
58
+ ## Provider Matrix
116
59
 
117
- ## Local
60
+ | Provider | Literal | Upstream model used by this adapter | Dimensions | Credentials | Batching | Network and privacy boundary | Cost and table planning |
61
+ | --- | --- | --- | --- | --- | --- | --- | --- |
62
+ | Local | `local` | `Xenova/all-MiniLM-L6-v2` | Fixed `384` | None | Sequential in-process | No hosted API call. First use may download model artifacts and cache them locally. | No hosted API bill. Table must be `F32_BLOB(384)`. |
63
+ | Cloudflare Workers AI | `cloudflare` | `@cf/baai/bge-m3` | Fixed `1024` | `accountId` and `apiToken`, or `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN` | Native batch in one request | Indexed and queried text is sent to Cloudflare. | Check Cloudflare pricing before large rebuilds. Table must be `F32_BLOB(1024)`. |
64
+ | Mistral | `mistral` | `mistral-embed` | Fixed `1024` | `apiKey`, or `MISTRAL_API_KEY` | Native batch in one request | Indexed and queried text is sent to Mistral. | Check Mistral pricing before rebuilds. Table must be `F32_BLOB(1024)`. |
65
+ | Gemini | `gemini` | `gemini-embedding-2` | Default `3072`; allowed integers `128-3072` | `apiKey`, or `GEMINI_API_KEY` | Sequential SDK request per input | Indexed and queried text is sent to Google. The adapter currently rewrites payload text by intent. | Check Gemini pricing before rebuilds. Table width must match the chosen dimension count exactly. |
66
+ | OpenAI | `openai` | `text-embedding-3-small` when `dimensions <= 1536`, otherwise `text-embedding-3-large` | Default `768`; any positive integer accepted locally and forwarded as `dimensions` | `apiKey`, or `OPENAI_API_KEY` | Native batch in one request, max `2048` inputs | Indexed and queried text is sent to OpenAI. | Check OpenAI pricing before rebuilds. Table width must match the chosen dimension count exactly. |
67
+ | OpenAI-compatible | `openai-compatible` | Your configured `model` | Required positive integer; no default | `baseUrl`, `model`, and `dimensions` are required. `apiKey` is optional and never falls back to env. | Metadata reports `native`; outbound requests are chunked sequentially at `batchSize`, default `32` | Boundary depends on the operator behind `baseUrl`. Treat `baseUrl` as a trusted server-side setting and review HTTPS, SSRF, logging, and retention controls yourself. | Table width must match the configured `dimensions`. Any endpoint or model change should use a new table plus full reindex. |
118
68
 
119
- Provider value: `local`
69
+ ## Provider Notes
120
70
 
121
- The local provider loads `Xenova/all-MiniLM-L6-v2` through
122
- `@huggingface/transformers`.
71
+ ### Local
123
72
 
124
73
  ```ts
125
74
  embeddingOptions: {
@@ -127,26 +76,16 @@ embeddingOptions: {
127
76
  }
128
77
  ```
129
78
 
130
- Notes:
79
+ - fixed at `384` dimensions
80
+ - rejects any other `dimensions` value before loading the runtime
81
+ - uses `@huggingface/transformers` lazily and caches the local pipeline by model name
131
82
 
132
- - the model emits 384 dimensions, and local vectors are validated at exactly
133
- 384 finite numbers
134
- - `dimensions: 384` is accepted explicitly; any other local dimension is
135
- rejected before the runtime is imported or loaded
136
- - metadata reports 384 dimensions
137
- - batch metadata is `{ mode: "sequential" }`
138
- - the first run downloads and caches the model and can take longer on a fresh
139
- machine
140
- - no API key is required
141
- - this remains the default provider for offline use
83
+ References:
142
84
 
143
- ## Cloudflare Workers AI
85
+ - [Transformers.js in Node.js](https://huggingface.co/docs/transformers.js/en/tutorials/node)
86
+ - [Transformers.js environment and cache controls](https://huggingface.co/docs/transformers.js/en/api/env)
144
87
 
145
- Provider value: `cloudflare`
146
-
147
- Cloudflare is the recommended hosted provider for low-cost Markdown search.
148
- It uses Workers AI `@cf/baai/bge-m3` through Cloudflare's OpenAI-compatible
149
- embeddings endpoint.
88
+ ### Cloudflare Workers AI
150
89
 
151
90
  ```ts
152
91
  embeddingOptions: {
@@ -156,25 +95,16 @@ embeddingOptions: {
156
95
  }
157
96
  ```
158
97
 
159
- Behavior:
160
-
161
- - if `accountId` is omitted, the library reads `CLOUDFLARE_ACCOUNT_ID`
162
- - if `apiToken` is omitted, the library reads `CLOUDFLARE_API_TOKEN`
163
- - blank Cloudflare credentials are treated as missing
164
- - `@cf/baai/bge-m3` returns 1024 dimensions
165
- - metadata reports `@cf/baai/bge-m3` and 1024 dimensions without requiring
166
- credentials
167
- - batch metadata is `{ mode: "native" }`
168
- - response items are reordered by provider-supplied index before being returned
169
- - Cloudflare does not accept custom dimensions in this provider; use
170
- `createTable(client, "articles", 1024)` for Cloudflare-backed indexes
98
+ - fixed at `1024` dimensions
99
+ - uses the account-scoped Workers AI embeddings endpoint
100
+ - blank credentials are treated as missing
171
101
 
172
- ## Mistral
102
+ References:
173
103
 
174
- Provider value: `mistral`
104
+ - [Cloudflare `@cf/baai/bge-m3`](https://developers.cloudflare.com/workers-ai/models/bge-m3/)
105
+ - [Cloudflare Workers AI pricing](https://developers.cloudflare.com/workers-ai/platform/pricing/)
175
106
 
176
- Mistral uses the hosted `mistral-embed` model through
177
- `https://api.mistral.ai/v1/embeddings`.
107
+ ### Mistral
178
108
 
179
109
  ```ts
180
110
  embeddingOptions: {
@@ -183,24 +113,17 @@ embeddingOptions: {
183
113
  }
184
114
  ```
185
115
 
186
- Behavior:
187
-
188
- - if `apiKey` is omitted, the library reads `MISTRAL_API_KEY`
189
- - blank Mistral credentials are treated as missing
190
- - `mistral-embed` returns 1024 dimensions
191
- - metadata reports `mistral-embed` and 1024 dimensions without requiring
192
- credentials
193
- - batch metadata is `{ mode: "native" }`
194
- - request bodies send `encoding_format: "float"`
195
- - response items are reordered by provider-supplied index before being returned
196
- - Mistral does not accept custom dimensions in this provider; use
197
- `createTable(client, "articles", 1024)` for Mistral-backed indexes
116
+ - fixed at `1024` dimensions
117
+ - sends `encoding_format: "float"`
118
+ - expects indexed upstream responses
198
119
 
199
- ## Gemini
120
+ References:
200
121
 
201
- Provider value: `gemini`
122
+ - [Mistral embeddings guide](https://docs.mistral.ai/studio/knowledge-rag/embeddings/text_embeddings)
123
+ - [Mistral embeddings API reference](https://docs.mistral.ai/api/endpoint/embeddings)
124
+ - [Mistral model overview for `mistral-embed`](https://docs.mistral.ai/models/mistral-embed-23-12)
202
125
 
203
- Gemini uses Google `gemini-embedding-2` through `@google/genai`.
126
+ ### Gemini
204
127
 
205
128
  ```ts
206
129
  embeddingOptions: {
@@ -210,26 +133,18 @@ embeddingOptions: {
210
133
  }
211
134
  ```
212
135
 
213
- Behavior:
136
+ - defaults to `3072` dimensions
137
+ - accepts only integer dimensions from `128` through `3072`
138
+ - currently formats document inputs as `title: none | text: ...`
139
+ - currently formats query inputs as `task: search result | query: ...`
140
+ - sends one SDK request per input, even when you call `generateEmbeddings()`
214
141
 
215
- - if `apiKey` is omitted, the library reads `GEMINI_API_KEY`
216
- - blank Gemini credentials are treated as missing
217
- - Gemini defaults to 3072 dimensions
218
- - explicit Gemini dimensions must be integers from 128 through 3072
219
- - 768, 1536, and 3072 are recommended practical sizes
220
- - metadata reports `gemini-embedding-2` and the effective dimensions
221
- - batch metadata is `{ mode: "sequential" }`
222
- - the library sends one SDK request per input and verifies one vector per input
223
- - document inputs are formatted as `title: none | text: ...`
224
- - query inputs are formatted as `task: search result | query: ...`
225
- - the current implementation does not expose custom model selection
142
+ References:
226
143
 
227
- ## OpenAI
144
+ - [Gemini embeddings guide](https://ai.google.dev/gemini-api/docs/embeddings)
145
+ - [Gemini pricing](https://ai.google.dev/gemini-api/docs/pricing)
228
146
 
229
- Provider value: `openai`
230
-
231
- OpenAI uses `text-embedding-3-small` when `dimensions <= 1536` and
232
- `text-embedding-3-large` when `dimensions > 1536`.
147
+ ### OpenAI
233
148
 
234
149
  ```ts
235
150
  embeddingOptions: {
@@ -239,119 +154,45 @@ embeddingOptions: {
239
154
  }
240
155
  ```
241
156
 
242
- Behavior:
243
-
244
- - if `apiKey` is omitted, the library reads `OPENAI_API_KEY`
245
- - the request sends the `dimensions` value to the OpenAI embeddings API
246
- - metadata reports `text-embedding-3-small` when `dimensions <= 1536` and
247
- `text-embedding-3-large` when `dimensions > 1536`
248
- - batch metadata is `{ mode: "native", maxSize: 2048 }`
249
- - use the same dimension count in `createTable()`
157
+ - defaults to `768` dimensions
158
+ - uses `text-embedding-3-small` through `1536`
159
+ - uses `text-embedding-3-large` above `1536`
160
+ - rejects batches above `2048` inputs before any network request
250
161
 
251
- ## OpenAI-Compatible Endpoints
162
+ References:
252
163
 
253
- Provider value: `openai-compatible`
164
+ - [OpenAI embeddings guide](https://developers.openai.com/api/docs/guides/embeddings)
165
+ - [OpenAI embeddings API reference](https://developers.openai.com/api/reference/resources/embeddings/methods/create/)
166
+ - [OpenAI data controls](https://developers.openai.com/api/docs/guides/your-data#default-usage-policies-by-endpoint)
167
+ - [OpenAI models overview](https://developers.openai.com/api/docs/models)
168
+ - [OpenAI API pricing](https://openai.com/api/pricing/)
254
169
 
255
- Use this provider for trusted OpenAI-compatible embedding services such as
256
- Hugging Face Text Embeddings Inference (TEI) or an internal gateway. This is an
257
- optional escape hatch; `local` remains the default offline provider, and the
258
- named hosted providers above are still preferred when their fixed adapters fit.
170
+ ### OpenAI-compatible
259
171
 
260
172
  ```ts
261
173
  embeddingOptions: {
262
174
  provider: "openai-compatible",
263
- baseUrl: "http://localhost:8080/v1",
264
- model: "BAAI/bge-large-en-v1.5",
265
- dimensions: 1024,
175
+ baseUrl: process.env.EMBEDDING_BASE_URL,
176
+ model: process.env.EMBEDDING_MODEL,
177
+ dimensions: Number(process.env.EMBEDDING_DIMENSIONS),
178
+ apiKey: process.env.EMBEDDING_API_KEY,
266
179
  batchSize: 32,
267
180
  }
268
181
  ```
269
182
 
270
- If your endpoint requires bearer auth, pass `apiKey` explicitly:
271
-
272
- ```ts
273
- embeddingOptions: {
274
- provider: "openai-compatible",
275
- baseUrl: "https://embeddings.example.com/v1",
276
- apiKey: process.env.EMBEDDINGS_API_KEY,
277
- model: "BAAI/bge-large-en-v1.5",
278
- dimensions: 1024,
279
- }
280
- ```
183
+ - `baseUrl`, `model`, and `dimensions` are required
184
+ - `baseUrl` must be an absolute `http` or `https` URL without URL credentials, query strings, or fragments
185
+ - the library normalizes `baseUrl` to an `/embeddings` endpoint
186
+ - `batchSize` defaults to `32` and controls outbound chunking
187
+ - `apiKey` is optional and never falls back to `OPENAI_API_KEY`
281
188
 
282
- Behavior:
189
+ References:
283
190
 
284
- - `baseUrl`, `model`, and `dimensions` are required
285
- - `baseUrl` is the API base, such as `http://localhost:8080/v1`; the library
286
- sends requests to `/embeddings` below that base
287
- - a base URL that already ends in `/embeddings` is used as-is
288
- - only absolute `http` and `https` URLs are accepted
289
- - URL usernames, passwords, query strings, and fragments are rejected
290
- - `apiKey` is optional; blank keys are ignored and no `Authorization` header is
291
- sent
292
- - `OPENAI_API_KEY` is never read for this provider
293
- - `batchSize` defaults to `32`, matching TEI's conservative client batch size;
294
- larger input arrays are split into sequential outbound requests and returned
295
- in the original input order
296
- - requests send `{ input, model, dimensions, encoding_format: "float" }`
297
- - responses must use the standard OpenAI embeddings shape with indexed
298
- `data[]` items; each response chunk must include unique contiguous indices
299
- - batch metadata is `{ mode: "native" }`; the internal outbound chunk size is
300
- not reported as `batch.maxSize`
301
-
302
- Security boundary:
303
-
304
- - treat `baseUrl` as trusted server-side configuration only
305
- - never pass user-supplied request values directly into `baseUrl`
306
- - use HTTPS for remote endpoints
307
- - credentials are not sent across redirects
308
- - non-2xx response bodies are not read, and errors avoid echoing API keys or
309
- configured endpoint URLs
310
-
311
- TEI exposes an OpenAI-compatible base at `/v1`, so a local TEI server usually
312
- uses:
191
+ - [Text Embeddings Inference quick tour](https://huggingface.co/docs/text-embeddings-inference/quick_tour)
192
+ - [Text Embeddings Inference CLI arguments](https://huggingface.co/docs/text-embeddings-inference/cli_arguments)
313
193
 
314
- ```ts
315
- embeddingOptions: {
316
- provider: "openai-compatible",
317
- baseUrl: "http://localhost:8080/v1",
318
- model: "BAAI/bge-large-en-v1.5",
319
- dimensions: 1024,
320
- }
321
- ```
194
+ ## Next Steps
322
195
 
323
- This library does not call TEI's native `/embed` endpoint.
324
-
325
- ## Dimension Guidelines
326
-
327
- - `local` is fixed at `384`
328
- - `cloudflare` is fixed at `1024`
329
- - `mistral` is fixed at `1024`
330
- - Gemini defaults to `3072` and accepts explicit dimensions from `128` through
331
- `3072`; use `768`, `1536`, or `3072` unless you have a specific reason
332
- - OpenAI can be used at 1536 or 3072, or another supported OpenAI dimension
333
- value you explicitly set
334
- - `openai-compatible` requires you to set the dimension count that your
335
- endpoint/model actually returns
336
-
337
- If you switch provider, endpoint, model, or dimensions for an existing table,
338
- recreate the table or rebuild the index into a separate table so stored vectors
339
- stay consistent.
340
-
341
- Existing Gemini indexes created with `text-embedding-004` must be fully
342
- re-embedded for `gemini-embedding-2`, even if you keep `dimensions: 768`,
343
- because both the model and query/document input formatting changed. If you move
344
- to the new 3072-dimensional default, create a new table or recreate the vector
345
- table first; `indexContent()` clears rows but does not change the `F32_BLOB`
346
- width. A separate table is safer because rebuilds are not transactional.
347
-
348
- Existing local indexes created with the older padded-local behavior usually have
349
- `F32_BLOB(768)` rows containing the 384 model values followed by zero padding.
350
- The current local contract stores the native 384-dimensional model output. To
351
- migrate, create or recreate a `F32_BLOB(384)` table and run a full re-index
352
- before querying it. Using the same model ID avoids an intentional model-space
353
- change, but bit-identical vectors are not promised across runtime, model
354
- revision, dtype, pooling, or normalization changes; validate search quality and
355
- re-index when those details change.
356
-
357
- Routine unit tests mock the local runtime and do not download the model.
196
+ - [Integration examples](./INTEGRATIONS.md) for end-to-end configuration flows
197
+ - [Migration guide](./MIGRATIONS.md) before changing widths, models, or endpoints
198
+ - [Testing guide](./TESTING.md) for CI-safe provider coverage
package/docs/README.md CHANGED
@@ -1,16 +1,14 @@
1
1
  # Documentation
2
2
 
3
3
  This directory holds the longer-form reference material for `libsql-search`.
4
- Start with the page that matches the job you are doing:
5
4
 
6
- - [Provider guide](./PROVIDERS.md): local, hosted, and OpenAI-compatible
7
- embedding options, dimensions, and API key behavior
5
+ - [Provider selection and configuration](./PROVIDERS.md): compare local, hosted, and custom embedding providers before you build an index
6
+ - [Integration examples](./INTEGRATIONS.md): reusable provider flow plus Astro and Next.js examples
7
+ - [Migration and reindexing guide](./MIGRATIONS.md): table-width changes, provider/model swaps, and safe cutovers
8
8
  - [API reference](./API.md): exported functions, option shapes, and result data
9
- - [Integration examples](./INTEGRATIONS.md): Astro and Next.js server-side usage
10
- - [Indexing and operations](./INDEXING.md): content layout, rebuild scripts,
11
- search quality tips, and indexing gotchas
12
- - [Troubleshooting](./TROUBLESHOOTING.md): known install/runtime issues
9
+ - [Indexing and operations](./INDEXING.md): content layout, rebuild behavior, and search quality notes
10
+ - [Testing guidance](./TESTING.md): CI-safe mocks and no-live-call policy
11
+ - [Troubleshooting](./TROUBLESHOOTING.md): known install and runtime issues
13
12
  - [Releasing](./RELEASING.md): maintainer release workflow
14
13
 
15
- For the shortest first-use path, go back to the repository
16
- [README](../README.md).
14
+ For the shortest first-use path, go back to the repository [README](../README.md).
@@ -0,0 +1,138 @@
1
+ # Testing Guidance
2
+
3
+ Routine unit tests and CI should not make live embedding-provider calls and should not require real provider credentials.
4
+
5
+ ## Repository Policy
6
+
7
+ - no live provider keys in unit tests or CI
8
+ - no routine network calls to hosted embedding providers in CI
9
+ - validate option handling and response parsing with mocks first
10
+ - assert failures happen before network calls when configuration is invalid
11
+
12
+ The current test suite follows that pattern in `tests/embeddings.test.ts` and [`tests/huggingface-transformers.mock.ts`](../tests/huggingface-transformers.mock.ts).
13
+
14
+ ## Local Provider Mocks
15
+
16
+ The local provider should use a lightweight Transformers.js mock instead of downloading the real model during routine tests.
17
+
18
+ ```ts
19
+ import {
20
+ huggingFaceTransformersMock,
21
+ resetHuggingFaceTransformersMock,
22
+ } from "./huggingface-transformers.mock.js";
23
+
24
+ beforeEach(() => {
25
+ resetHuggingFaceTransformersMock();
26
+ });
27
+ ```
28
+
29
+ The repository source file is `huggingface-transformers.mock.ts`. The example keeps the `.js` import suffix because this repo's ESM TypeScript source uses explicit `.js` relative imports that resolve after compilation.
30
+
31
+ Test the contract you care about:
32
+
33
+ - the library requests `Xenova/all-MiniLM-L6-v2`
34
+ - the call uses `pooling: "mean"` and `normalize: true`
35
+ - non-`384` local dimensions fail before runtime loading
36
+
37
+ ## HTTP Provider Mocks
38
+
39
+ Cloudflare, Mistral, OpenAI, and `openai-compatible` should use `fetch` mocks.
40
+
41
+ ```ts
42
+ import { vi } from "vitest";
43
+
44
+ const fetchMock = vi.fn().mockResolvedValue({
45
+ ok: true,
46
+ headers: new Headers(),
47
+ json: async () => ({
48
+ data: [
49
+ { index: 0, embedding: [1, 2] },
50
+ { index: 1, embedding: [3, 4] },
51
+ ],
52
+ }),
53
+ });
54
+
55
+ vi.stubGlobal("fetch", fetchMock);
56
+ ```
57
+
58
+ Useful assertions:
59
+
60
+ - request body includes the expected `model`
61
+ - OpenAI includes `dimensions`
62
+ - Mistral and `openai-compatible` include `encoding_format: "float"`
63
+ - `openai-compatible` chunking honors `batchSize`
64
+ - Cloudflare and Mistral reorder indexed responses correctly
65
+ - invalid config fails before `fetch` is called
66
+
67
+ ## Gemini SDK Mocks
68
+
69
+ Gemini uses the `@google/genai` SDK, so routine tests should mock the SDK client rather than calling Google.
70
+
71
+ ```ts
72
+ vi.mock("@google/genai", () => ({
73
+ GoogleGenAI: class {
74
+ readonly models = {
75
+ embedContent: vi.fn(async () => ({
76
+ embeddings: [{ values: [1, 2, 3] }],
77
+ })),
78
+ };
79
+ },
80
+ }));
81
+ ```
82
+
83
+ Useful assertions:
84
+
85
+ - the adapter sends `gemini-embedding-2`
86
+ - document intent formats text as `title: none | text: ...`
87
+ - query intent formats text as `task: search result | query: ...`
88
+ - explicit dimensions are forwarded as `outputDimensionality`
89
+ - invalid dimensions fail before SDK work
90
+
91
+ ## Environment Cleanup
92
+
93
+ Tests that touch provider credentials should save and restore environment variables so one case cannot leak into another:
94
+
95
+ ```ts
96
+ let originalOpenAIKey: string | undefined;
97
+
98
+ beforeEach(() => {
99
+ originalOpenAIKey = process.env.OPENAI_API_KEY;
100
+ delete process.env.OPENAI_API_KEY;
101
+ });
102
+
103
+ afterEach(() => {
104
+ if (originalOpenAIKey === undefined) {
105
+ delete process.env.OPENAI_API_KEY;
106
+ } else {
107
+ process.env.OPENAI_API_KEY = originalOpenAIKey;
108
+ }
109
+ });
110
+ ```
111
+
112
+ Apply the same pattern to `GEMINI_API_KEY`, `MISTRAL_API_KEY`, `CLOUDFLARE_ACCOUNT_ID`, and `CLOUDFLARE_API_TOKEN`.
113
+
114
+ ## Validation-before-network Coverage
115
+
116
+ Prefer tests that prove bad inputs fail locally:
117
+
118
+ - unknown provider
119
+ - missing provider credentials
120
+ - blank credentials where trimming is expected
121
+ - invalid local dimensions
122
+ - invalid Gemini dimensions
123
+ - invalid `openai-compatible` `baseUrl`
124
+ - invalid `openai-compatible` `batchSize`
125
+ - OpenAI batches above `2048`
126
+
127
+ These checks keep CI fast and prove the library rejects bad inputs before it ships them to a provider.
128
+
129
+ ## Optional Live Smoke Tests
130
+
131
+ If you want live provider smoke coverage, keep it outside routine CI:
132
+
133
+ - run it only when a developer explicitly opts in
134
+ - use dedicated throwaway credentials and test content
135
+ - isolate it from unit-test jobs
136
+ - expect provider cost and external data transfer
137
+
138
+ This repository does not require or expect live provider smoke tests for normal pull-request validation.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "libsql-search",
3
- "version": "0.7.0",
3
+ "version": "0.7.1",
4
4
  "description": "Semantic search for static sites using libSQL/Turso with multi-provider embeddings",
5
5
  "type": "module",
6
6
  "packageManager": "pnpm@10.34.5",