@4fu/arbor-search-zvec 0.0.0-stage → 2026.1107.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 4fuu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,135 @@
-# Temporary Holding Version
+# @4fu/arbor-search-zvec
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The optional semantic search channel: text embedded and searched by
+meaning, with [zvec](https://github.com/alibaba/zvec), an embedded vector
+database.
+
+`@4fu/arbor-journal-sqlite` and `@4fu/arbor-notes-sqlite` take this package as
+an optional peer. Installed, each indexes the text it searches and fuses what it
+finds by meaning with its lexical indexes. Absent - or when zvec's prebuilt
+binary does not load on the platform, or when no embedding model is available -
+search is exactly what it is without it, and nothing else changes.
+
+## Installing it
+
+```bash
+npm install @4fu/arbor-search-zvec
+```
+
+zvec is a Node-API addon published per platform as optional dependencies
+(`@zvec/bindings-linux-x64`, `@zvec/bindings-darwin-arm64`, ...), so installing
+it needs no compiler and no build step. Linux (x86_64, arm64), macOS (x86_64,
+arm64) and Windows (x86_64, arm64) are supported. Its collection lives in a
+directory of its own beside the journal's database - `<journal>.vectors` - and
+persists through a write-ahead log exactly as the database does.
+
+**One host per data directory.** zvec holds a `LOCK` in the collection
+directory and refuses a second open, so one host owns one journal directory. A
+collection another process holds is reported as no channel at all: the journal
+warns once and searches lexically.
+
+## The default embedding model
+
+The default embedder runs a small bilingual retrieval model in-process through
+[transformers.js](https://github.com/huggingface/transformers.js), which is an
+optional peer of this package:
+
+```bash
+npm install @4fu/arbor-search-zvec @huggingface/transformers
+```
+
+The model is `Xenova/bge-small-zh-v1.5` (512 dimensions, 24 MB quantized). It
+runs locally and needs no key. Its files are fetched from the Hugging Face Hub
+on first use and cached from then on; **nothing a host holds leaves it** - only
+the text being embedded is handed to the model, and the fetch is of the model,
+never of the text.
+
+The fetch never blocks anything. It happens in the background on the first
+append or search; until the model arrives the channel answers `ready()` with
+`false`, an empty `search`, and holds the text it was given so it can be indexed
+once the model is there. A fetch that fails - `huggingface.co` is unreachable
+from mainland China, for instance - degrades to lexical search with one warning
+naming what is missing, and is retried later rather than on every call.
+
+### Pointing the model somewhere else
+
+Both settings are process-global, because transformers.js keeps them on its
+`env` singleton. There is no environment variable for either.
+
+| Option | transformers.js setting | Default |
+| --- | --- | --- |
+| `endpoint` | `env.remoteHost` | `https://huggingface.co/` |
+| `cacheDir` | `env.cacheDir` | the library's own `.cache` |
+
+A mirror, for a network where `huggingface.co` is blocked:
+
+```ts
+createZvecIndex({ path: "/var/lib/arbor/journal.db.vectors", endpoint: "https://hf-mirror.com" });
+```
+
+An offline install, where the model files are already on disk, names the
+directory that holds them. transformers.js reads a model id or a local path the
+same way, so nothing else changes:
+
+```ts
+createZvecIndex({ path: "/var/lib/arbor/journal.db.vectors", model: "/opt/models/bge-small-zh-v1.5" });
+```
+
+`dtype` (default `q8`) and `pooling` (default `cls`) choose the build and the
+pooling. `retryAfterMs` (default 300000) is how long a failed fetch waits before
+it is tried again, and `warn` (default `console.warn`) is where the one warning
+goes.
+
+## Bringing your own embedder
+
+A deployment replaces the model by implementing `Embedder` and passing it. This
+is also the only way text reaches anything outside the host, so it is opt-in by
+construction:
+
+```ts
+import { createZvecIndex } from "@4fu/arbor-search-zvec";
+import type { Embedder } from "@4fu/arbor-interfaces";
+
+const embed: Embedder = {
+  dimension: 1024,
+  async embed(texts) {
+    // A remote model, a different local one, anything at all.
+    return callMyModel(texts);
+  },
+};
+
+const index = createZvecIndex({ path: "/var/lib/arbor/journal.db.vectors", embed });
+```
+
+With `embed` given, no model is fetched and transformers.js is never loaded.
+
+## What a store does with it
+
+- the journal indexes each event's text when the event is appended, and the note
+  store indexes each note's title, summary and body as one text when its
+  revision is written;
+- `catchUpSemantic()` on either store indexes what the channel does not hold -
+  after a restart over an existing database, after the model arrives, and after
+  an indexing attempt that failed;
+- a redaction erases the event's vector with its content, and a note's vector is
+  erased when the note is deleted and replaced when its revision is replaced;
+- the channel's results merge with the lexical ones through the shared
+  `mergeHits`: exact substring hits first, then word hits, then the hits only
+  the semantic channel found. A hit no lexical index found is reported with
+  `match: "semantic"`. Each store applies its own filters to what the channel
+  returns, so a channel that filters less than the store does still answers the
+  read it was asked.
+
+See [docs/journal.md#search](../../docs/journal.md#search) and
+[docs/memory.md#search](../../docs/memory.md#search).
+
+## Tests
+
+```bash
+pnpm --filter @4fu/arbor-search-zvec test
+```
+
+The tests that need zvec's prebuilt binary, and the ones that need the default
+embedding model, skip with a message naming what is missing. Running them all
+needs `@huggingface/transformers` installed and network access to the model's
+host; the rest need only the platform binding.

package/embedder.d.ts

@@ -0,0 +1,45 @@
+/**
+ * The default embedder: a small bilingual retrieval model that runs in-process
+ * through transformers.js and needs no key (docs/journal.md#search).
+ *
+ * The model files are fetched from the Hugging Face Hub on first use and cached
+ * from then on. Nothing a host holds leaves it: only the text being embedded is
+ * handed to the model, and the fetch is of the model, never of the text. A
+ * deployment that must not fetch anything names a local directory with `model`,
+ * or passes its own embed function - the only way text reaches anything outside
+ * the host.
+ *
+ * transformers.js reads its model source from `env`, a process-global singleton,
+ * so `cacheDir` and `endpoint` are written once here and apply to every pipeline
+ * in the process. Neither has an environment variable: the setting is
+ * `env.remoteHost`, which is what `endpoint` writes, and `env.cacheDir`, which
+ * is what `cacheDir` writes.
+ */
+import type { Embedder } from "@4fu/arbor-interfaces";
+/** The model the channel embeds with when the caller names none. */
+export declare const DEFAULT_MODEL = "Xenova/bge-small-zh-v1.5";
+export interface LocalEmbedderOptions {
+    /** A Hugging Face repo id, or a path to a directory that holds a model. */
+    model?: string;
+    /** Where fetched model files are cached. */
+    cacheDir?: string;
+    /** Where models are fetched from, for a mirror. Defaults to huggingface.co. */
+    endpoint?: string;
+    /** The build to load. Defaults to `q8`. */
+    dtype?: string;
+    /** How the token vectors become one vector. Defaults to `cls`. */
+    pooling?: string;
+}
+/** The default embedder, which has to load its model before it can embed. */
+export interface LocalEmbedder extends Embedder {
+    /** Load the model. Resolves once the channel can embed, rejects when it cannot. */
+    load(): Promise<void>;
+}
+/** Whether transformers.js is installed, without loading it. */
+export declare function transformersInstalled(): boolean;
+/**
+ * The default embedder, lazy: nothing is loaded and nothing is fetched until
+ * `load` is called, so constructing a semantic channel costs nothing and a
+ * journal that never searches never downloads a model.
+ */
+export declare function localEmbedder(options?: LocalEmbedderOptions): LocalEmbedder;

package/embedder.js

@@ -0,0 +1,108 @@
+/**
+ * The default embedder: a small bilingual retrieval model that runs in-process
+ * through transformers.js and needs no key (docs/journal.md#search).
+ *
+ * The model files are fetched from the Hugging Face Hub on first use and cached
+ * from then on. Nothing a host holds leaves it: only the text being embedded is
+ * handed to the model, and the fetch is of the model, never of the text. A
+ * deployment that must not fetch anything names a local directory with `model`,
+ * or passes its own embed function - the only way text reaches anything outside
+ * the host.
+ *
+ * transformers.js reads its model source from `env`, a process-global singleton,
+ * so `cacheDir` and `endpoint` are written once here and apply to every pipeline
+ * in the process. Neither has an environment variable: the setting is
+ * `env.remoteHost`, which is what `endpoint` writes, and `env.cacheDir`, which
+ * is what `cacheDir` writes.
+ */
+var __rewriteRelativeImportExtension = (this && this.__rewriteRelativeImportExtension) || function (path, preserveJsx) {
+    if (typeof path === "string" && /^\.\.?\//.test(path)) {
+        return path.replace(/\.(tsx)$|((?:\.d)?)((?:\.[^./]+?)?)\.([cm]?)ts$/i, function (m, tsx, d, ext, cm) {
+            return tsx ? preserveJsx ? ".jsx" : ".js" : d && (!ext || !cm) ? m : (d + ext + "." + cm.toLowerCase() + "js");
+        });
+    }
+    return path;
+};
+import { createRequire } from "node:module";
+/** The package that runs the model in-process. */
+const TRANSFORMERS_PACKAGE = "@huggingface/transformers";
+/** The model the channel embeds with when the caller names none. */
+export const DEFAULT_MODEL = "Xenova/bge-small-zh-v1.5";
+/** The quantized build of the default model: 24 MB instead of 94 MB. */
+const DEFAULT_DTYPE = "q8";
+/** The pooling the default model was trained with. */
+const DEFAULT_POOLING = "cls";
+/** One text, embedded only to learn the model's dimension. */
+const PROBE = "arbor";
+/** Whether transformers.js is installed, without loading it. */
+export function transformersInstalled() {
+    try {
+        createRequire(import.meta.url).resolve(TRANSFORMERS_PACKAGE);
+        return true;
+    }
+    catch {
+        return false;
+    }
+}
+/**
+ * The default embedder, lazy: nothing is loaded and nothing is fetched until
+ * `load` is called, so constructing a semantic channel costs nothing and a
+ * journal that never searches never downloads a model.
+ */
+export function localEmbedder(options = {}) {
+    const model = options.model ?? DEFAULT_MODEL;
+    const pooling = options.pooling ?? DEFAULT_POOLING;
+    let extractor = null;
+    let dimension = 0;
+    const load = async () => {
+        const loaded = await import(__rewriteRelativeImportExtension(TRANSFORMERS_PACKAGE));
+        const pipeline = loaded.pipeline;
+        if (typeof pipeline !== "function") {
+            throw new Error(`${TRANSFORMERS_PACKAGE} exports no pipeline`);
+        }
+        const env = loaded.env;
+        if (env !== undefined && typeof env === "object") {
+            if (options.cacheDir !== undefined)
+                env.cacheDir = options.cacheDir;
+            // A mirror, for a network where huggingface.co is unreachable.
+            if (options.endpoint !== undefined)
+                env.remoteHost = options.endpoint;
+        }
+        extractor = (await pipeline("feature-extraction", model, {
+            dtype: options.dtype ?? DEFAULT_DTYPE,
+        }));
+        // One short text, so the dimension is known before anything is stored: the
+        // vector store is created with it.
+        const [probe] = await embedWith(extractor, [PROBE], pooling);
+        if (probe === undefined || probe.length === 0) {
+            throw new Error(`${model} returned no embedding dimension`);
+        }
+        dimension = probe.length;
+    };
+    return {
+        get dimension() {
+            return dimension;
+        },
+        load,
+        async embed(texts) {
+            if (extractor === null)
+                await load();
+            if (extractor === null)
+                return [];
+            return embedWith(extractor, texts, pooling);
+        },
+    };
+}
+async function embedWith(extractor, texts, pooling) {
+    if (texts.length === 0)
+        return [];
+    const output = await extractor(texts, { pooling, normalize: true });
+    const width = output.dims[output.dims.length - 1] ?? 0;
+    if (width <= 0)
+        throw new Error("the embedding model returned no dimension");
+    const vectors = [];
+    for (let index = 0; index < texts.length; index += 1) {
+        vectors.push(Array.from(output.data.slice(index * width, (index + 1) * width)));
+    }
+    return vectors;
+}

package/index.d.ts

@@ -0,0 +1,14 @@
+/**
+ * @4fu/arbor-search-zvec - the optional semantic search channel.
+ *
+ * `@4fu/arbor-journal-sqlite` and `@4fu/arbor-notes-sqlite` take this package as
+ * an optional peer: installed, text is embedded and searched by meaning beside
+ * the lexical indexes; absent, or when zvec's native binary does not load on the
+ * platform, or when no embedding model is available, search is exactly what it
+ * is without it (docs/journal.md#search, docs/memory.md#search). See README.md
+ * for what a deployment configures.
+ */
+export { createZvecIndex, filterExpression } from "./zvec-index.ts";
+export type { ZvecIndexOptions } from "./zvec-index.ts";
+export { localEmbedder, transformersInstalled, DEFAULT_MODEL } from "./embedder.ts";
+export type { LocalEmbedder, LocalEmbedderOptions } from "./embedder.ts";

package/index.js

@@ -0,0 +1,12 @@
+/**
+ * @4fu/arbor-search-zvec - the optional semantic search channel.
+ *
+ * `@4fu/arbor-journal-sqlite` and `@4fu/arbor-notes-sqlite` take this package as
+ * an optional peer: installed, text is embedded and searched by meaning beside
+ * the lexical indexes; absent, or when zvec's native binary does not load on the
+ * platform, or when no embedding model is available, search is exactly what it
+ * is without it (docs/journal.md#search, docs/memory.md#search). See README.md
+ * for what a deployment configures.
+ */
+export { createZvecIndex, filterExpression } from "./zvec-index.js";
+export { localEmbedder, transformersInstalled, DEFAULT_MODEL } from "./embedder.js";

package/package.json

@@ -1,6 +1,46 @@
 {
   "name": "@4fu/arbor-search-zvec",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "2026.1107.1",
+  "type": "module",
+  "description": "The optional semantic search channel: event text embedded and searched by meaning with zvec, an embedded vector database.",
+  "exports": {
+    ".": {
+      "types": "./index.d.ts",
+      "default": "./index.js"
+    }
+  },
+  "types": "./index.d.ts",
+  "main": "./index.js",
+  "files": [
+    "embedder.d.ts",
+    "embedder.js",
+    "index.d.ts",
+    "index.js",
+    "zvec-index.d.ts",
+    "zvec-index.js"
+  ],
+  "engines": {
+    "node": ">=24"
+  },
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/4fuu/arbor.git",
+    "directory": "packages/search-zvec"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "dependencies": {
+    "@4fu/arbor-interfaces": "2026.1107.1",
+    "@zvec/zvec": "^0.7.0"
+  },
+  "peerDependencies": {
+    "@huggingface/transformers": "^4.3.2"
+  },
+  "peerDependenciesMeta": {
+    "@huggingface/transformers": {
+      "optional": true
+    }
+  }
+}

package/zvec-index.d.ts

@@ -0,0 +1,57 @@
+/**
+ * The semantic channel over zvec (docs/journal.md#search, docs/memory.md#search):
+ * text embedded and searched by meaning, beside a store's lexical indexes.
+ *
+ * zvec is an embedded vector database: a Node-API addon, published per platform
+ * as optional dependencies, so installing it needs no compiler and no server.
+ * A collection is a directory of its own beside the store's database, written
+ * through a write-ahead log, so it survives a restart exactly as the database
+ * does. Three limits shape this file:
+ *
+ * - **One writer per collection.** zvec holds a `LOCK` in the collection
+ *   directory and refuses a second open, so one host owns one data directory.
+ *   A collection another process holds is reported as no channel at all, and
+ *   search stays lexical.
+ * - **The dimension is the store's, and it is fixed at creation.** The default
+ *   model's dimension is only known once the model is loaded, so the collection
+ *   is created on the first successful embed rather than at construction.
+ * - **A filter needs the data it filters on.** `kind` and `window` are stored as
+ *   scalar fields beside each vector, so a filtered search is answered by the
+ *   store rather than by discarding candidates afterwards.
+ *
+ * The ref a store hands over is its own - an event ref for the journal, a note
+ * id for the note store - and the channel keeps one vector per ref, replacing
+ * it when the store hands the same ref over again. The journal indexes an
+ * event's body; the note store indexes the title, the summary and the body of
+ * the note's current revision as one text.
+ */
+import type { Embedder, SemanticFilters, SemanticIndex } from "@4fu/arbor-interfaces";
+import type { LocalEmbedderOptions } from "./embedder.ts";
+/**
+ * The filters a semantic search applies, as a zvec filter expression. Targets
+ * and an external id are not fields of this collection, so the journal applies
+ * those to what this returns. Neither is a note's status: a note's status
+ * changes at every snapshot epoch without a revision, so the note store applies
+ * that one to what this returns as well (docs/memory.md#search).
+ */
+export declare function filterExpression(filters: SemanticFilters): string | undefined;
+export interface ZvecIndexOptions extends LocalEmbedderOptions {
+    /** Where the collection lives. A store passes `<its own path>.vectors`. */
+    path: string;
+    /** The embedder. Omitted, the default local model is used. */
+    embed?: Embedder;
+    /** Where a warning goes. Defaults to `console.warn`. */
+    warn?: (message: string) => void;
+    /** How long a failed model load waits before it is tried again. */
+    retryAfterMs?: number;
+    /** The clock the retry window reads. Tests pass a fixed one. */
+    now?: () => number;
+}
+/**
+ * Open the semantic channel, or answer `null` when there is none: no embedding
+ * model is available, or the collection is held by another host.
+ *
+ * Throws when zvec itself will not load on this platform, which is the caller's
+ * signal that the channel is absent.
+ */
+export declare function createZvecIndex(options: ZvecIndexOptions): SemanticIndex | null;

package/zvec-index.js

@@ -0,0 +1,329 @@
+/**
+ * The semantic channel over zvec (docs/journal.md#search, docs/memory.md#search):
+ * text embedded and searched by meaning, beside a store's lexical indexes.
+ *
+ * zvec is an embedded vector database: a Node-API addon, published per platform
+ * as optional dependencies, so installing it needs no compiler and no server.
+ * A collection is a directory of its own beside the store's database, written
+ * through a write-ahead log, so it survives a restart exactly as the database
+ * does. Three limits shape this file:
+ *
+ * - **One writer per collection.** zvec holds a `LOCK` in the collection
+ *   directory and refuses a second open, so one host owns one data directory.
+ *   A collection another process holds is reported as no channel at all, and
+ *   search stays lexical.
+ * - **The dimension is the store's, and it is fixed at creation.** The default
+ *   model's dimension is only known once the model is loaded, so the collection
+ *   is created on the first successful embed rather than at construction.
+ * - **A filter needs the data it filters on.** `kind` and `window` are stored as
+ *   scalar fields beside each vector, so a filtered search is answered by the
+ *   store rather than by discarding candidates afterwards.
+ *
+ * The ref a store hands over is its own - an event ref for the journal, a note
+ * id for the note store - and the channel keeps one vector per ref, replacing
+ * it when the store hands the same ref over again. The journal indexes an
+ * event's body; the note store indexes the title, the summary and the body of
+ * the note's current revision as one text.
+ */
+import { statSync } from "node:fs";
+import { ZVecCollectionSchema, ZVecCreateAndOpen, ZVecDataType, ZVecOpen } from "@zvec/zvec";
+import { localEmbedder, transformersInstalled } from "./embedder.js";
+/** The collection's name, which is also the directory zvec writes beside it. */
+const COLLECTION = "events";
+/** The one vector field, and the scalar fields a filter reads. */
+const VECTOR_FIELD = "embedding";
+/**
+ * The store's own order, which is the ref's number and only grows: an event's
+ * `seq` for the journal, a note's id for the note store. It is what an
+ * `afterSeq`/`beforeSeq` filter reads, and it orders the collection for a
+ * reader that wants the newest first.
+ */
+const SEQ_FIELD = "seq";
+const KIND_FIELD = "kind";
+const WINDOW_FIELD = "window";
+/** zvec refuses a write batch larger than this. */
+const MAX_WRITE_BATCH = 1024;
+/** How many texts one embedding call takes. */
+const EMBED_BATCH = 32;
+/** How long a failed model load waits before it is tried again. */
+const RETRY_AFTER_MS = 300_000;
+/** How many items wait in memory while the model loads. */
+const MAX_PENDING = 5_000;
+/** One value in a filter expression: a quoted string, or a bare number. */
+function quote(value) {
+    return `'${value.replace(/\\/g, "\\\\").replace(/'/g, "\\'")}'`;
+}
+/**
+ * The filters a semantic search applies, as a zvec filter expression. Targets
+ * and an external id are not fields of this collection, so the journal applies
+ * those to what this returns. Neither is a note's status: a note's status
+ * changes at every snapshot epoch without a revision, so the note store applies
+ * that one to what this returns as well (docs/memory.md#search).
+ */
+export function filterExpression(filters) {
+    const clauses = [];
+    if (filters.afterSeq !== undefined)
+        clauses.push(`${SEQ_FIELD} > ${filters.afterSeq}`);
+    if (filters.beforeSeq !== undefined)
+        clauses.push(`${SEQ_FIELD} < ${filters.beforeSeq}`);
+    const kinds = filters.kinds ?? [];
+    if (kinds.length === 1) {
+        clauses.push(`${KIND_FIELD} = ${quote(kinds[0])}`);
+    }
+    else if (kinds.length > 1) {
+        clauses.push(`${KIND_FIELD} IN (${kinds.map(quote).join(", ")})`);
+    }
+    const window = filters.window;
+    if (window !== undefined && window !== null)
+        clauses.push(`${WINDOW_FIELD} = ${quote(window)}`);
+    return clauses.length === 0 ? undefined : clauses.join(" AND ");
+}
+/** A directory that is already there. */
+function exists(path) {
+    try {
+        statSync(path);
+        return true;
+    }
+    catch {
+        return false;
+    }
+}
+function messageOf(error) {
+    return error instanceof Error ? error.message : String(error);
+}
+/**
+ * Open the semantic channel, or answer `null` when there is none: no embedding
+ * model is available, or the collection is held by another host.
+ *
+ * Throws when zvec itself will not load on this platform, which is the caller's
+ * signal that the channel is absent.
+ */
+export function createZvecIndex(options) {
+    const embedder = options.embed ?? (transformersInstalled() ? localEmbedder(options) : null);
+    if (embedder === null)
+        return null;
+    if (!exists(options.path))
+        return new ZvecSemanticIndex(options, embedder, null);
+    try {
+        return new ZvecSemanticIndex(options, embedder, ZVecOpen(options.path));
+    }
+    catch (error) {
+        // One writer per collection: another host owns this data directory.
+        options.warn?.(`zvec: the collection at ${options.path} is held by another host; search stays lexical (${messageOf(error)})`);
+        return null;
+    }
+}
+class ZvecSemanticIndex {
+    path;
+    embedder;
+    loadModel;
+    warn;
+    retryAfterMs;
+    now;
+    collection;
+    pending = new Map();
+    loading = null;
+    loaded = false;
+    failedAt = null;
+    warned = new Set();
+    constructor(options, embedder, collection) {
+        this.path = options.path;
+        this.embedder = embedder;
+        this.loadModel = embedder.load;
+        this.warn = options.warn ?? ((message) => console.warn(message));
+        this.retryAfterMs = options.retryAfterMs ?? RETRY_AFTER_MS;
+        this.now = options.now ?? (() => Date.now());
+        this.collection = collection;
+    }
+    async add(ref, text, fields) {
+        this.kick();
+        if (!this.loaded) {
+            // The model has not arrived: the item stays lexical, and the text waits
+            // here, bounded, until the channel is ready.
+            if (this.pending.size < MAX_PENDING)
+                this.pending.set(ref, { text, fields });
+            return;
+        }
+        await this.guard("add", () => this.store([[ref, text, fields]]));
+    }
+    async remove(ref) {
+        this.pending.delete(ref);
+        if (this.collection === null)
+            return;
+        await this.guard("remove", () => {
+            this.collection?.deleteSync(ref);
+        });
+    }
+    async search(query, limit, filters) {
+        this.kick();
+        if (!this.loaded || this.collection === null)
+            return [];
+        const [vector] = (await this.embed([query])) ?? [];
+        if (vector === undefined || vector.length === 0)
+            return [];
+        const filter = filterExpression(filters);
+        const hits = await this.guard("search", () => this.collection?.querySync({
+            fieldName: VECTOR_FIELD,
+            vector,
+            topk: Math.max(1, limit),
+            includeVector: false,
+            ...(filter === undefined ? {} : { filter }),
+        }));
+        return (hits ?? []).map((hit) => hit.id);
+    }
+    async ready() {
+        // A channel that is only asked about still loads its model: a host that
+        // restarts over an existing store calls this before it searches.
+        this.kick();
+        return this.loaded;
+    }
+    async held() {
+        if (this.collection === null)
+            return new Set();
+        const held = new Set();
+        for (const doc of this.collection.iterDocsSync({ outputFields: [], includeVector: false })) {
+            held.add(doc.id);
+        }
+        return held;
+    }
+    async close() {
+        this.pending.clear();
+        const collection = this.collection;
+        if (collection === null)
+            return;
+        this.collection = null;
+        this.loaded = false;
+        try {
+            collection.closeSync();
+        }
+        catch (error) {
+            this.warnOnce("close", `zvec: closing the collection at ${this.path} failed (${messageOf(error)})`);
+        }
+    }
+    // --- internals ---------------------------------------------------------
+    /** Embed texts, and turn a refusal into nothing rather than a failed search. */
+    async embed(texts) {
+        return this.guard("embed", () => this.embedder.embed(texts));
+    }
+    /** One warning per kind of failure, so a broken channel is not a log flood. */
+    warnOnce(key, message) {
+        if (this.warned.has(key))
+            return;
+        this.warned.add(key);
+        this.warn(message);
+    }
+    /** Run one step, and turn a refusal into the channel's own answer. */
+    async guard(key, step) {
+        try {
+            return await step();
+        }
+        catch (error) {
+            this.warnOnce(key, `zvec: the semantic channel refused (${messageOf(error)}); search stays lexical`);
+            return undefined;
+        }
+    }
+    /**
+     * Start loading the model unless something already is, and unless a load that
+     * failed is still inside its retry window. Never awaited by a caller, so a
+     * fetch is never on the path of an append or a search.
+     */
+    kick() {
+        if (this.loaded || this.loading !== null)
+            return;
+        if (this.failedAt !== null && this.now() - this.failedAt < this.retryAfterMs)
+            return;
+        this.loading = this.load().finally(() => {
+            this.loading = null;
+        });
+    }
+    async load() {
+        try {
+            if (this.loadModel !== undefined)
+                await this.loadModel();
+            await this.open();
+            // Index what arrived while the model was loading, and anything that
+            // arrived while that was happening: `add` holds its text until the
+            // channel is ready, so draining until nothing is left is exact.
+            while (this.pending.size > 0)
+                await this.flush();
+            this.loaded = true;
+        }
+        catch (error) {
+            this.loaded = false;
+            this.failedAt = this.now();
+            // One warning per attempt, and the attempt is not retried until the
+            // window passes: an unreachable mirror costs one line, not one per call.
+            this.warn(`zvec: the embedding model is not available (${messageOf(error)}); search stays lexical and the model is retried in ${Math.round(this.retryAfterMs / 1000)}s`);
+            // The text that arrived while the model was being fetched stays here: the
+            // next attempt indexes it, and a catch-up indexes whatever the bound let
+            // go of.
+        }
+    }
+    /** Open an existing collection, or create one now that the dimension is known. */
+    async open() {
+        if (this.collection !== null)
+            return;
+        const dimension = this.embedder.dimension;
+        if (dimension <= 0)
+            throw new Error("the embedder reported no dimension");
+        this.collection = ZVecCreateAndOpen(this.path, new ZVecCollectionSchema({
+            name: COLLECTION,
+            vectors: { name: VECTOR_FIELD, dataType: ZVecDataType.VECTOR_FP32, dimension },
+            fields: [
+                { name: SEQ_FIELD, dataType: ZVecDataType.INT64 },
+                { name: KIND_FIELD, dataType: ZVecDataType.STRING },
+                { name: WINDOW_FIELD, dataType: ZVecDataType.STRING },
+            ],
+        }));
+    }
+    /** Index what arrived while the model was loading. */
+    async flush() {
+        if (this.pending.size === 0)
+            return;
+        const entries = [...this.pending.entries()].map(([ref, held]) => [
+            ref,
+            held.text,
+            held.fields,
+        ]);
+        this.pending.clear();
+        try {
+            await this.store(entries);
+        }
+        catch (error) {
+            // The store refused: put the text back so a later flush, or a catch-up,
+            // tries again.
+            this.warnOnce("add", `zvec: the semantic channel refused (${messageOf(error)}); search stays lexical`);
+            for (const [ref, text, fields] of entries)
+                this.pending.set(ref, { text, fields });
+        }
+    }
+    /** Embed and store entries, in the batches the model and zvec take. */
+    async store(entries) {
+        if (entries.length === 0)
+            return;
+        for (let start = 0; start < entries.length; start += EMBED_BATCH) {
+            const batch = entries.slice(start, start + EMBED_BATCH);
+            const vectors = await this.embedder.embed(batch.map(([, text]) => text));
+            const docs = [];
+            batch.forEach(([ref, , fields], index) => {
+                const vector = vectors[index];
+                if (vector === undefined || vector.length === 0)
+                    return;
+                docs.push({
+                    id: ref,
+                    vectors: { [VECTOR_FIELD]: vector },
+                    fields: {
+                        // The ref's number: the event's seq for the journal, the note's id
+                        // for the note store. Both only grow, so both order the collection.
+                        [SEQ_FIELD]: Number(ref.slice(1)),
+                        [KIND_FIELD]: fields?.kind ?? "",
+                        [WINDOW_FIELD]: fields?.window ?? "",
+                    },
+                });
+            });
+            for (let offset = 0; offset < docs.length; offset += MAX_WRITE_BATCH) {
+                this.collection?.upsertSync(docs.slice(offset, offset + MAX_WRITE_BATCH));
+            }
+        }
+    }
+}