@oomkapwn/enquire-mcp 1.11.1 → 2.0.0-beta.0

package/CHANGELOG.md

@@ -2,6 +2,161 @@
 
 All notable changes to this project will be documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.0.0-beta.0] — 2026-05-06
+
+**Theme: Hybrid RRF retrieval.** v2.0.0-alpha.0 shipped ML embeddings as a standalone tool. v2.0.0-beta.0 ships the integration step: a single `obsidian_search` umbrella tool that fuses every available retrieval signal — BM25 (FTS5) + TF-IDF cosine + ML embeddings — via Reciprocal Rank Fusion (Cormack et al, 2009).
+
+This is the v2.0 user-facing thesis: instead of agents picking between four nearly-identical search tools and getting different recall depending on the choice, they call one tool that automatically picks the best evidence available. Result: better recall on paraphrase / synonym / cross-language queries without configuration.
+
+### Added — `obsidian_search` MCP tool (the new default)
+
+Single umbrella tool that auto-detects available signals and fuses them via RRF with k=60. Gracefully degrades:
+
+- **TF-IDF only** (no `--persistent-index`, no embeddings) → produces TF-IDF ranking.
+- **BM25 + TF-IDF** (add `--persistent-index`) → keyword-augmented retrieval, sub-100ms.
+- **BM25 + TF-IDF + embeddings** (add `enquire-mcp build-embeddings`) → matches Smart Connections-quality retrieval, free / offline / open-source.
+
+Returns per-signal observability so agents can see WHY each hit ranked:
+
+```json
+{
+  "query": "OAuth flows",
+  "method": "rrf",
+  "k": 60,
+  "signals_used": ["bm25", "tfidf", "embeddings"],
+  "total_candidates": 47,
+  "matches": [
+    {
+      "path": "Auth/OAuth Flows.md",
+      "score": 0.0492,
+      "snippet": "OAuth authentication flow with JWT tokens...",
+      "chunk_index": 0,
+      "line_start": 1,
+      "line_end": 1,
+      "per_signal": {
+        "bm25": { "rank": 1, "score": 8.5 },
+        "tfidf": { "rank": 2, "score": 0.7 },
+        "embeddings": { "rank": 1, "score": 0.92 }
+      }
+    }
+  ]
+}
+```
+
+`min_signals` parameter lets agents request consensus search — e.g. `min_signals: 2` returns only hits that ranked in two-or-more rankers.
+
+### Added — `src/rrf.ts` (isolated RRF math)
+
+Standalone module implementing Reciprocal Rank Fusion. Pure function over named ranked lists; doesn't know about vaults, SQLite, or embeddings — testable in isolation. 13 unit tests cover the formula, union-safety (missing signals don't penalize), per-signal observability, rank-validation, and graceful degradation.
+
+### Architecture decisions (locked)
+
+- **Note-level fusion** (not chunk-level). BM25 + embeddings return chunks; we collapse to the best chunk per note before fusing. Chunk identity comparison would require all rankers to share a chunk space, which TF-IDF (note-level) doesn't. Note-level wins on simplicity and matches what most agents actually want ("which notes are relevant"). Chunk-level fusion is v2.1 backlog.
+- **Hardcoded equal weights, k=60.** Per Cormack et al, RRF without per-signal weights outperforms most learned alternatives on heterogeneous rankers. We resist the urge to add `--rrf-weights` — sensible defaults serve >80% of users; advanced users can fork. Add the flag in v2.1 if real issues come in.
+- **Graceful degradation, not feature gate.** `obsidian_search` works the moment the server starts (TF-IDF only). Adding `--persistent-index` enables BM25. Running `build-embeddings` enables ML retrieval. Each layer adds quality without changing the API or surface.
+- **Per-signal observability is required, not optional.** The `per_signal` field on every hit is the foundation for v2.1 features (UI explainability, weights tuning, ranker disagreement detection). Hidden by default would be cheap; explicit is the right primitive.
+
+### Tests
+
+384 unit tests pass (was 364, +20). New: `tests/rrf.test.ts` (13 tests covering RRF math, union-safety, observability, rank validation, custom k, topK truncation, all-empty input). `tests/search-hybrid.test.ts` (7 tests covering graceful-degradation paths, min_signals filter, folder filter, empty query rejection, total_candidates accounting).
+
+End-to-end ML smoke remains out-of-band; the v2.0 alpha smoke + hybrid path verifies the wiring against synthetic + real vaults.
+
+### Migration from v2.0.0-alpha.0
+
+**No breaking changes.** `obsidian_search` is purely additive; the existing `obsidian_search_text`, `obsidian_full_text_search`, `obsidian_semantic_search`, `obsidian_embeddings_search` tools continue to work for diagnostics. The v2.0 RC will likely move them all behind a `--diagnostic-tools` opt-in to declutter the default tool list, but that's not yet decided — feedback welcome on the alpha/beta channel.
+
+### Try it
+
+```bash
+npm install -g @oomkapwn/enquire-mcp@beta
+# Or upgrade from alpha:
+# npm install -g @oomkapwn/enquire-mcp@beta
+
+# Tier 1: TF-IDF only (zero setup)
+enquire-mcp serve --vault ~/Documents/Obsidian\ Vault
+# → obsidian_search { query: "OAuth flows" }  ← signals_used: ["tfidf"]
+
+# Tier 2: + BM25
+enquire-mcp serve --vault ~/Documents/Obsidian\ Vault --persistent-index
+# → obsidian_search { query: "OAuth flows" }  ← signals_used: ["bm25","tfidf"]
+
+# Tier 3: + ML embeddings
+enquire-mcp install-model multilingual
+enquire-mcp build-embeddings --vault ~/Documents/Obsidian\ Vault
+enquire-mcp serve --vault ~/Documents/Obsidian\ Vault --persistent-index
+# → obsidian_search { query: "OAuth flows" }  ← signals_used: ["bm25","tfidf","embeddings"]
+```
+
+## [2.0.0-alpha.0] — 2026-05-06
+
+**Theme: ML-embedding retrieval.** v1.8 shipped TF-IDF cosine as the no-deps semantic-search floor. v2.0 raises the ceiling with real transformer embeddings — closer to Smart Connections quality, but free, offline-capable, multilingual, and (uniquely) chunk-aligned with the FTS5 BM25 index so the v2.0 beta hybrid RRF can score across both surfaces using the same identifier space.
+
+### Added — `obsidian_embeddings_search`
+
+ML-embedding retrieval via [@huggingface/transformers](https://github.com/huggingface/transformers.js) + `paraphrase-multilingual-MiniLM-L12-v2` (50+ languages, 384-dim, runs on CPU). Persistent SQLite vector index next to the FTS5 db. Brute-force cosine top-K (sub-100ms on 50K chunks; HNSW ladder is v2.1 if real users hit that ceiling).
+
+Higher-quality than `obsidian_semantic_search` for paraphrases, synonyms, and cross-language queries — but requires a one-time setup (see below). The TF-IDF path remains the no-deps default.
+
+### Added — `enquire-mcp install-model [alias]` subcommand
+
+Pre-downloads an embedding model so the first MCP call doesn't block on a ~120MB HuggingFace download. Aliases:
+
+- `multilingual` (default) — `Xenova/paraphrase-multilingual-MiniLM-L12-v2`, 384-dim, ~120MB, 50+ languages
+- `bge` — `Xenova/bge-small-en-v1.5`, 384-dim, ~33MB, English-only (better recall on technical content)
+
+Models are cached under `~/.cache/huggingface/transformers.js/` and reused across vaults. Subsequent `install-model` calls are no-ops if the cache is warm.
+
+### Added — `enquire-mcp build-embeddings --vault <path>` subcommand
+
+Cold-build (or refresh) the persistent embedding index for a vault. Same paragraph-level chunking as the FTS5 index (`fts5.chunkContent`) so chunk identity matches across BM25 and embeddings — foundation for the v2.0 beta hybrid RRF.
+
+Incremental rebuilds via `source_state` mtime tracking — only re-embeds notes whose mtime changed since the last `build-embeddings`. ~5-30ms per chunk on M1 CPU.
+
+Supports `--embedding-model <alias>`, `--exclude-glob`, `--read-paths`, `--embed-file <path>`.
+
+### Added — `enquire-mcp clear-embeddings --vault <path>` subcommand
+
+Removes the `.embed.db` + WAL/SHM sidecars. Mirrors `clear-cache` and `clear-index`.
+
+### Added — `@huggingface/transformers ^4.2.0` as `optionalDependencies`
+
+Mirrors the `better-sqlite3` pattern: the heavy ONNX runtime + tokenizer transitive deps install only if the user's npm policy allows optional deps (default). Read-only / TF-IDF / FTS5 paths stay zero-cost — no model load, no runtime cost. Tarball stays under 200KB.
+
+If optional deps are skipped (`npm install --omit=optional`), the embedding tools and subcommands surface a clean error pointing the user at `npm install @huggingface/transformers` rather than an opaque module-not-found.
+
+### Architecture decisions (locked for v2.0)
+
+- **Default model = multilingual.** The user's dogfood vault is bilingual Russian + English; v2.0 covers >80% of real Obsidian users (most personal vaults are not pure English).
+- **Models download on subcommand, not on first MCP call.** Predictable for CI; air-gap-friendly; explicit consent for networked operations. Pattern follows Stripe / Cloudflare CLI conventions.
+- **Hardcoded RRF in v2.0 beta.** No `--rrf-weights` flag (yet). Sensible defaults work in 80% of cases per Cormack et al. Add the flag in v2.1 if real issues come in.
+- **CJK is v2.0 backlog.** The Unicode tokenizer in v1.11.1 catches Cyrillic / Greek / Hebrew / Arabic. Chinese / Japanese / Thai need an `Intl.Segmenter` pass first; out-of-scope for alpha.
+- **Brute-force cosine, not HNSW.** ~50ms top-10 on 50K × 384 floats — fine for >99% of personal vaults. HNSW ladder when the ceiling is hit.
+
+### Tests
+
+364 unit tests pass (was 341, +23). New: `tests/embed-db.test.ts` (synthetic-vector schema + upsert/delete/search semantics, cross-vault contamination guard, dim mismatch, folder filter, minScore threshold). `tests/embeddings.test.ts` (catalog + cosine math, no model load).
+
+End-to-end ML smoke is out-of-band — CI doesn't download the model. Manual verification:
+```bash
+enquire-mcp install-model multilingual
+enquire-mcp build-embeddings --vault ~/Documents/Obsidian\ Vault
+# then via MCP: obsidian_embeddings_search { query: "OAuth flows" }
+```
+
+### Migration from v1.x
+
+**No breaking changes for read-only / TF-IDF / FTS5 users.** All v1.x tools and CLI flags continue to work exactly as before. Embedding features are pure additions, gated behind explicit subcommand invocations.
+
+The next prerelease (v2.0.0-beta.0) will add hybrid RRF scoring (`obsidian_search` umbrella tool over BM25 + TF-IDF + embeddings) — additive, not breaking.
+
+### Excluded from this alpha (deferred to v2.0 beta / RC)
+
+- Hybrid RRF tool (`obsidian_search`) — needs alpha shipping first to validate the embedding plumbing in real vaults
+- HNSW vector index — only matters past 50K chunks, which no current user has
+- `--persistent-embeddings` server flag (auto-build on serve startup) — pulls model load into hot path; alpha users prefer explicit subcommand
+- CJK segmenter — needs `Intl.Segmenter` v18+ feature gating; v2.1 backlog
+
 ## [1.11.1] — 2026-05-05
 
 Audit-driven patch. Five-agent audit of the v1.10 → v1.11 surface flagged two real P1 code bugs and one CI/process gap; this release fixes all three plus the doc drift the audit found.

package/README.md

@@ -6,7 +6,11 @@
 
 ## Stop your AI guessing at vault paths.
 
-**enquire** is the MCP server that gives **Claude Code, Cursor, OpenClaw 🦞, and Codex** native, structured access to your **Obsidian** vault — wikilinks resolved, backlinks ranked, frontmatter typed, Dataview queries first-class. Read-only by default. No plugin. One command to install.
+**enquire** turns your **Obsidian** vault into structured, searchable memory for **Claude Code, Cursor, OpenClaw 🦞, Codex**, and any MCP-compatible agent.
+
+Wikilinks resolved with aliases and sections. Backlinks ranked with snippets. Frontmatter typed. Dataview queries first-class. **Hybrid retrieval** (BM25 + TF-IDF + ML embeddings, fused via Reciprocal Rank Fusion) that catches paraphrases and synonyms the way Smart Connections does — but free, offline-capable, and open-source.
+
+Read-only by default. No Obsidian plugin required. One command to install.
 
 [![CI](https://github.com/oomkapwn/enquire-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/oomkapwn/enquire-mcp/actions/workflows/ci.yml)
 [![npm version](https://img.shields.io/npm/v/@oomkapwn/enquire-mcp.svg)](https://www.npmjs.com/package/@oomkapwn/enquire-mcp)
@@ -86,9 +90,10 @@ There are several Obsidian-MCP servers out there. enquire differentiates on thre
 | **Strict path allowlist** (`--read-paths '01_Projects/**'` — only paths matching one of these globs are visible; complement to `--exclude-glob` denylist) | ❌ | ✅ |
 | **Canvas (`.canvas`) read tools** (`obsidian_list_canvases` + `obsidian_read_canvas` — typed nodes + edges, broken-ref detection) | ❌ rare / partial | ✅ first-class |
 | **Semantic search** (`obsidian_semantic_search` — TF-IDF cosine, free / offline / no model download) | ❌ usually paywalled (Smart Connections) | ✅ in-tree |
-| TypeScript strict + Biome lint + 341+ unit tests | varies | ✅ |
+| **ML embeddings search** (`obsidian_embeddings_search` — paraphrase-multilingual-MiniLM-L12-v2, 50+ languages, persistent SQLite vector index) | ❌ usually paywalled (Smart Connections) | ✅ free + offline-capable (v2.0 alpha) |
+| TypeScript strict + Biome lint + 364+ unit tests | varies | ✅ |
 
-That's the gap. enquire closes it in ~3000 lines of TypeScript with five mandatory runtime dependencies (`@modelcontextprotocol/sdk`, `chokidar`, `commander`, `gray-matter`, `zod`) plus one optional (`better-sqlite3`, only loaded when `--persistent-index` is passed).
+That's the gap. enquire closes it in ~3500 lines of TypeScript with five mandatory runtime dependencies (`@modelcontextprotocol/sdk`, `chokidar`, `commander`, `gray-matter`, `zod`) plus two optional (`better-sqlite3` for `--persistent-index` and `--build-embeddings`; `@huggingface/transformers` for ML embeddings — both are no-ops when not invoked).
 
 > **Not affiliated with Obsidian.md.** Obsidian and the Obsidian logo are trademarks of Dynalist Inc. enquire-mcp is an independent open-source project that reads Obsidian-format vaults. The name «enquire» is a tribute to Tim Berners-Lee's 1980 hypertext system, not a trademark claim against any party.
 
@@ -185,7 +190,9 @@ Restart your client. The server logs `enquire <version> ready (read-only, vault=
 | `obsidian_open_in_ui` | Returns an `obsidian://open?vault=<v>&file=<f>` URI for hand-off to the running Obsidian desktop app. No filesystem or network side effect — just URI emission. `new_pane=true` opens the note in a split. |
 | `obsidian_list_canvases` | Lists `.canvas` files (Obsidian's whiteboard / mind-map format) with each canvas's node and edge counts. Honors `--exclude-glob` and `--read-paths`. |
 | `obsidian_read_canvas` | Parses one `.canvas` file into typed nodes (text / file / link / group / unknown) + edges (with from/to node IDs, sides, labels, colors). Each `file` node carries a `file_resolved` field (vault-relative path or `null` if broken). Returns a node-kind summary + `broken_file_refs` array. |
+| `obsidian_search` | **The default search tool from v2.0.** Hybrid retrieval via Reciprocal Rank Fusion (Cormack et al, 2009) over BM25 (FTS5) + TF-IDF + ML embeddings. Auto-detects available signals and gracefully degrades — TF-IDF only out of the box, BM25+TF-IDF with `--persistent-index`, full hybrid with `build-embeddings`. Returns per-signal observability so you can see which rankers contributed each hit. Use this instead of the per-ranker tools unless you specifically need single-ranker output for diagnostics. |
 | `obsidian_semantic_search` | **TF-IDF cosine retrieval.** Free / offline / no model download. Tokenizes + TF-IDFs + L2-normalizes every note's body once per session, then ranks notes by cosine similarity to the query. Catches synonym + related-term matches that `obsidian_search_text` (substring) and `obsidian_full_text_search` (BM25) miss. |
+| `obsidian_embeddings_search` | _Opt-in via `enquire-mcp install-model` + `enquire-mcp build-embeddings --vault <path>`._ **ML-embedding retrieval** via @huggingface/transformers + paraphrase-multilingual-MiniLM-L12-v2 (50+ languages, 384-dim, runs on CPU). Higher-quality than `obsidian_semantic_search` for paraphrases / synonyms / cross-language queries. Persistent SQLite vector index next to the FTS5 db. Chunks match the FTS5 chunker so v2.0 beta can do hybrid RRF over both. |
 | `obsidian_full_text_search` | _Opt-in via `--persistent-index`._ BM25-ranked full-text search backed by SQLite FTS5 inverted index. Sub-100ms on multi-thousand-note vaults. Hyphenated tokens (`claude-telegram`) auto-quoted. Returns chunk-level hits with `«…»`-bracketed snippets. |
 
 ### 5 write tools (opt-in via `--enable-write`)

package/SECURITY.md

@@ -132,6 +132,30 @@ Posture:
 - **Write-tool gating composes with `--enable-write`.** Disabling `obsidian_create_note` while leaving `obsidian_replace_in_notes` enabled is a valid configuration; the gate is independent of the global write flag.
 - **Posture is "fail closed".** Tools blocked at registration time never appear in `tools/list` and a `tools/call` against a gated name returns a clean MCP-protocol error from the SDK — there's no codepath where a disabled tool can still execute.
 
+## ML embeddings (v2.0 alpha): networked-download + cache posture
+
+The `obsidian_embeddings_search` tool plus the `install-model` and `build-embeddings` subcommands (added v2.0.0-alpha.0) introduce two new surfaces with networked / on-disk implications:
+
+### Model download (`install-model`)
+
+- **Explicit, opt-in.** The `enquire-mcp install-model [alias]` subcommand is the ONLY codepath that hits the network. Serving / read-only / TF-IDF / FTS5 paths never make outbound HTTP. Air-gap-safe by default.
+- **Source: HuggingFace Hub.** Model weights ship as ONNX from the `Xenova/*` org. `@huggingface/transformers` handles the download, hash verification, and caching to `~/.cache/huggingface/transformers.js/`.
+- **Reusable across vaults.** The cache is per-machine, not per-vault. Multiple `enquire-mcp` instances on different vaults share the same model files.
+- **Manual purge.** Delete `~/.cache/huggingface/transformers.js/` to remove cached models.
+
+### Persistent embedding index (`build-embeddings`)
+
+- **0600 chmod** on `<vault-hash>.embed.db` + WAL + SHM sidecar files, parent directory mode 0700 — same as the FTS5 index posture.
+- **Cross-vault contamination guard.** `meta` table stores `vault_root`, `model_alias`, `dim`, and `schema_version`; if any change between runs, the embedding tables are dropped and rebuilt with a stderr warning. Prevents a stale index from leaking content into a different vault.
+- **Caveat — embedding values can leak content via cosine.** Float32 vectors stored in the index are reversible-ish: with the same model loaded, an attacker with read access to the .embed.db file can run nearest-neighbor searches against arbitrary queries to recover note content topics. Treat the .embed.db as having the same sensitivity as the .fts5.db (which already stores raw chunk content). If your threat model includes other local users on the same machine, do not use `--persistent-cache` / `--persistent-index` / build-embeddings.
+- **Manual purge.** `enquire-mcp clear-embeddings --vault <path>` removes the `.embed.db`, `.embed.db-wal`, and `.embed.db-shm` files.
+- **`--exclude-glob` / `--read-paths` honored.** The `build-embeddings` subcommand accepts both flags — excluded notes are never embedded, never appear in results.
+
+### Optional-dep failure mode
+
+- If `@huggingface/transformers` failed to install (e.g., user ran `npm install --omit=optional`, or the platform lacks ONNX runtime binaries), the embedding tools and subcommands surface a clean error message pointing the user at `npm install @huggingface/transformers` — never a cryptic module-not-found stack trace.
+- Read-only / TF-IDF / FTS5 surfaces are unaffected. The server starts and serves all v1.x tools normally.
+
 ## Persistent FTS5 index: privacy posture
 
 When `--persistent-index` is enabled, the search-index file at `<vault-hash>.fts5.db` (alongside the parse cache) stores **chunked note content** (paragraph-level, ~4 KB each), the **comma-serialized tag list** of each note, and the **list of wikilink targets** as part of the FTS5 enrichment for recall.

package/dist/embed-db.d.ts

@@ -0,0 +1,73 @@
+export interface EmbedSearchHit {
+    rel_path: string;
+    chunk_index: number;
+    line_start: number;
+    line_end: number;
+    /** Raw chunk text — caller can render snippets. */
+    text_preview: string;
+    /** Cosine similarity (since vectors are L2-normalized at insert time). */
+    score: number;
+}
+export interface EmbedSyncReport {
+    added: number;
+    updated: number;
+    deleted: number;
+    unchanged: number;
+    total_chunks: number;
+}
+interface SourceStateRow {
+    rel_path: string;
+    mtime_ms: number;
+}
+export interface EmbedDbOptions {
+    /** Absolute path to the .embed.db file. */
+    file: string;
+    /** Vault root for cross-vault contamination guard. */
+    vaultRoot: string;
+    /** Model alias the user built this index with (e.g. "multilingual"). */
+    modelAlias: string;
+    /** Vector dimensionality (must match the model). */
+    dim: number;
+}
+export declare class EmbedDb {
+    private db;
+    private readonly file;
+    private readonly vaultRoot;
+    private readonly modelAlias;
+    private readonly dim;
+    constructor(opts: EmbedDbOptions);
+    open(): Promise<void>;
+    /** Remove the embed db + WAL/SHM sidecars. Idempotent. */
+    clearOnDisk(): Promise<boolean>;
+    close(): void;
+    private bootstrapSchema;
+    private readMeta;
+    private writeMeta;
+    private requireDb;
+    /** Replace all embeddings for a single note. Caller computes vectors. */
+    upsertNote(relPath: string, mtimeMs: number, chunks: ReadonlyArray<{
+        chunkIndex: number;
+        lineStart: number;
+        lineEnd: number;
+        textPreview: string;
+        vector: Float32Array;
+    }>): void;
+    /** Drop a note's embeddings entirely (used on file deletion). */
+    deleteNote(relPath: string): void;
+    /** Read the source-state table — caller compares mtimes to decide what to
+     *  re-embed. */
+    getSourceStates(): SourceStateRow[];
+    /** Brute-force cosine top-K. Vectors are L2-normalized at insert time so
+     *  cosine == dot product. Acceptable up to ~50K chunks; v2.1 will swap to
+     *  HNSW if real vaults hit that ceiling. */
+    search(queryVec: Float32Array, k: number, opts?: {
+        folder?: string;
+        minScore?: number;
+    }): EmbedSearchHit[];
+    /** Total embedded chunks — used by stats / UI. */
+    totalChunks(): number;
+}
+/** Default location for the embed db, alongside the FTS5 db + parse cache. */
+export declare function defaultEmbedDbFile(vaultHashPrefix: string): string;
+export {};
+//# sourceMappingURL=embed-db.d.ts.map

package/dist/embed-db.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"embed-db.d.ts","sourceRoot":"","sources":["../src/embed-db.ts"],"names":[],"mappings":"AAmBA,MAAM,WAAW,cAAc;IAC7B,QAAQ,EAAE,MAAM,CAAC;IACjB,WAAW,EAAE,MAAM,CAAC;IACpB,UAAU,EAAE,MAAM,CAAC;IACnB,QAAQ,EAAE,MAAM,CAAC;IACjB,mDAAmD;IACnD,YAAY,EAAE,MAAM,CAAC;IACrB,0EAA0E;IAC1E,KAAK,EAAE,MAAM,CAAC;CACf;AAED,MAAM,WAAW,eAAe;IAC9B,KAAK,EAAE,MAAM,CAAC;IACd,OAAO,EAAE,MAAM,CAAC;IAChB,OAAO,EAAE,MAAM,CAAC;IAChB,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;CACtB;AAED,UAAU,cAAc;IACtB,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;CAClB;AAoCD,MAAM,WAAW,cAAc;IAC7B,2CAA2C;IAC3C,IAAI,EAAE,MAAM,CAAC;IACb,sDAAsD;IACtD,SAAS,EAAE,MAAM,CAAC;IAClB,wEAAwE;IACxE,UAAU,EAAE,MAAM,CAAC;IACnB,oDAAoD;IACpD,GAAG,EAAE,MAAM,CAAC;CACb;AAED,qBAAa,OAAO;IAClB,OAAO,CAAC,EAAE,CAAmB;IAC7B,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAS;IAC9B,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IACnC,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAS;IACpC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAS;gBAEjB,IAAI,EAAE,cAAc;IAO1B,IAAI,IAAI,OAAO,CAAC,IAAI,CAAC;IAc3B,0DAA0D;IACpD,WAAW,IAAI,OAAO,CAAC,OAAO,CAAC;IAcrC,KAAK,IAAI,IAAI;IAOb,OAAO,CAAC,eAAe;IAqDvB,OAAO,CAAC,QAAQ;IAQhB,OAAO,CAAC,SAAS;IAMjB,OAAO,CAAC,SAAS;IAKjB,yEAAyE;IACzE,UAAU,CACR,OAAO,EAAE,MAAM,EACf,OAAO,EAAE,MAAM,EACf,MAAM,EAAE,aAAa,CAAC;QACpB,UAAU,EAAE,MAAM,CAAC;QACnB,SAAS,EAAE,MAAM,CAAC;QAClB,OAAO,EAAE,MAAM,CAAC;QAChB,WAAW,EAAE,MAAM,CAAC;QACpB,MAAM,EAAE,YAAY,CAAC;KACtB,CAAC,GACD,IAAI;IAiCP,iEAAiE;IACjE,UAAU,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI;IAMjC;oBACgB;IAChB,eAAe,IAAI,cAAc,EAAE;IAKnC;;gDAE4C;IAC5C,MAAM,CAAC,QAAQ,EAAE,YAAY,EAAE,CAAC,EAAE,MAAM,EAAE,IAAI,GAAE;QAAE,MAAM,CAAC,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,EAAE,MAAM,CAAA;KAAO,GAAG,cAAc,EAAE;IA6C9G,kDAAkD;IAClD,WAAW,IAAI,MAAM;CAKtB;AAED,8EAA8E;AAC9E,wBAAgB,kBAAkB,CAAC,eAAe,EAAE,MAAM,GAAG,MAAM,CAIlE"}

package/dist/embed-db.js

@@ -0,0 +1,231 @@
+// Persistent embedding store (v2.0 alpha). SQLite-backed Float32 vectors,
+// brute-force cosine top-K retrieval. Same chunking as FTS5 (paragraph-level
+// via fts5.chunkContent) so chunk identity matches across BM25 and embeddings —
+// foundation for the v2.0 beta hybrid RRF scorer.
+//
+// Architecture mirrors fts5.ts:
+//   - Lazy-loaded better-sqlite3 (optional dep)
+//   - 0600 chmod on db + WAL/SHM sidecars
+//   - meta-table cross-vault contamination guard (vault_root, model alias, dim)
+//   - source_state mtime tracking for incremental rebuilds
+//
+// Brute-force cosine is fast enough for vaults up to ~50K chunks (~50ms top-10
+// on 50K × 384 floats). HNSW comes in v2.1 if real users hit that ceiling.
+import { promises as fs } from "node:fs";
+import * as path from "node:path";
+const SCHEMA_VERSION = 1;
+let BetterSqliteCtor = null;
+async function loadBetterSqlite() {
+    if (BetterSqliteCtor)
+        return BetterSqliteCtor;
+    try {
+        const mod = (await import("better-sqlite3"));
+        const ctor = mod.default;
+        if (!ctor)
+            throw new Error("better-sqlite3 has no default export");
+        BetterSqliteCtor = ctor;
+        return ctor;
+    }
+    catch (err) {
+        throw new Error(`Persistent embeddings require the optional 'better-sqlite3' dependency; install failed or the binding could not be loaded. ${err instanceof Error ? err.message : String(err)}`);
+    }
+}
+export class EmbedDb {
+    db = null;
+    file;
+    vaultRoot;
+    modelAlias;
+    dim;
+    constructor(opts) {
+        this.file = opts.file;
+        this.vaultRoot = opts.vaultRoot;
+        this.modelAlias = opts.modelAlias;
+        this.dim = opts.dim;
+    }
+    async open() {
+        if (this.db)
+            return;
+        const Ctor = await loadBetterSqlite();
+        await fs.mkdir(path.dirname(this.file), { recursive: true, mode: 0o700 });
+        await fs.chmod(path.dirname(this.file), 0o700).catch(() => { });
+        this.db = new Ctor(this.file);
+        this.db.pragma("journal_mode = WAL");
+        this.db.pragma("synchronous = NORMAL");
+        this.bootstrapSchema();
+        await Promise.all([this.file, `${this.file}-wal`, `${this.file}-shm`].map((p) => fs.chmod(p, 0o600).catch(() => { })));
+    }
+    /** Remove the embed db + WAL/SHM sidecars. Idempotent. */
+    async clearOnDisk() {
+        this.close();
+        let removed = false;
+        for (const p of [this.file, `${this.file}-wal`, `${this.file}-shm`]) {
+            try {
+                await fs.unlink(p);
+                removed = true;
+            }
+            catch {
+                // missing is fine
+            }
+        }
+        return removed;
+    }
+    close() {
+        if (this.db) {
+            this.db.close();
+            this.db = null;
+        }
+    }
+    bootstrapSchema() {
+        const db = this.requireDb();
+        db.exec(`
+      CREATE TABLE IF NOT EXISTS meta (
+        key TEXT PRIMARY KEY,
+        value TEXT NOT NULL
+      );
+    `);
+        const meta = this.readMeta();
+        const versionMatch = meta.schema_version === undefined || meta.schema_version === String(SCHEMA_VERSION);
+        const rootMatch = meta.vault_root === undefined || meta.vault_root === this.vaultRoot;
+        const modelMatch = meta.model_alias === undefined || meta.model_alias === this.modelAlias;
+        const dimMatch = meta.dim === undefined || meta.dim === String(this.dim);
+        if (!versionMatch || !rootMatch || !modelMatch || !dimMatch) {
+            const reason = [];
+            if (!versionMatch)
+                reason.push(`schema_version ${meta.schema_version} → ${SCHEMA_VERSION}`);
+            if (!rootMatch)
+                reason.push(`vault_root ${meta.vault_root} → ${this.vaultRoot}`);
+            if (!modelMatch)
+                reason.push(`model ${meta.model_alias} → ${this.modelAlias}`);
+            if (!dimMatch)
+                reason.push(`dim ${meta.dim} → ${this.dim}`);
+            process.stderr.write(`enquire: rebuilding embed index (${reason.join("; ")})\n`);
+            db.exec("DROP TABLE IF EXISTS embeddings; DROP TABLE IF EXISTS source_state;");
+        }
+        db.exec(`
+      CREATE TABLE IF NOT EXISTS embeddings (
+        id INTEGER PRIMARY KEY AUTOINCREMENT,
+        rel_path TEXT NOT NULL,
+        chunk_index INTEGER NOT NULL,
+        line_start INTEGER NOT NULL,
+        line_end INTEGER NOT NULL,
+        text_preview TEXT NOT NULL,
+        vector BLOB NOT NULL,
+        UNIQUE(rel_path, chunk_index)
+      );
+      CREATE INDEX IF NOT EXISTS embeddings_rel_path ON embeddings(rel_path);
+      CREATE TABLE IF NOT EXISTS source_state (
+        rel_path TEXT PRIMARY KEY,
+        mtime_ms INTEGER NOT NULL,
+        n_chunks INTEGER NOT NULL,
+        indexed_at TEXT NOT NULL
+      );
+    `);
+        this.writeMeta({
+            schema_version: String(SCHEMA_VERSION),
+            vault_root: this.vaultRoot,
+            model_alias: this.modelAlias,
+            dim: String(this.dim)
+        });
+    }
+    readMeta() {
+        const db = this.requireDb();
+        const rows = db.prepare("SELECT key, value FROM meta").all();
+        const out = {};
+        for (const r of rows)
+            out[r.key] = r.value;
+        return out;
+    }
+    writeMeta(kv) {
+        const db = this.requireDb();
+        const stmt = db.prepare("INSERT OR REPLACE INTO meta (key, value) VALUES (?, ?)");
+        for (const [k, v] of Object.entries(kv))
+            stmt.run(k, v);
+    }
+    requireDb() {
+        if (!this.db)
+            throw new Error("EmbedDb is not open — call .open() first");
+        return this.db;
+    }
+    /** Replace all embeddings for a single note. Caller computes vectors. */
+    upsertNote(relPath, mtimeMs, chunks) {
+        const db = this.requireDb();
+        const dim = this.dim;
+        const tx = db.transaction((...args) => {
+            const rows = args[0];
+            db.prepare("DELETE FROM embeddings WHERE rel_path = ?").run(relPath);
+            const insert = db.prepare(`INSERT INTO embeddings (rel_path, chunk_index, line_start, line_end, text_preview, vector)
+         VALUES (?, ?, ?, ?, ?, ?)`);
+            for (const c of rows) {
+                if (c.vector.length !== dim) {
+                    throw new Error(`vector dim mismatch for ${relPath} chunk ${c.chunkIndex}: got ${c.vector.length}, expected ${dim}`);
+                }
+                insert.run(relPath, c.chunkIndex, c.lineStart, c.lineEnd, c.textPreview, Buffer.from(c.vector.buffer, c.vector.byteOffset, c.vector.byteLength));
+            }
+            db.prepare(`INSERT OR REPLACE INTO source_state (rel_path, mtime_ms, n_chunks, indexed_at)
+         VALUES (?, ?, ?, datetime('now'))`).run(relPath, mtimeMs, rows.length);
+        });
+        tx(chunks);
+    }
+    /** Drop a note's embeddings entirely (used on file deletion). */
+    deleteNote(relPath) {
+        const db = this.requireDb();
+        db.prepare("DELETE FROM embeddings WHERE rel_path = ?").run(relPath);
+        db.prepare("DELETE FROM source_state WHERE rel_path = ?").run(relPath);
+    }
+    /** Read the source-state table — caller compares mtimes to decide what to
+     *  re-embed. */
+    getSourceStates() {
+        const db = this.requireDb();
+        return db.prepare("SELECT rel_path, mtime_ms FROM source_state").all();
+    }
+    /** Brute-force cosine top-K. Vectors are L2-normalized at insert time so
+     *  cosine == dot product. Acceptable up to ~50K chunks; v2.1 will swap to
+     *  HNSW if real vaults hit that ceiling. */
+    search(queryVec, k, opts = {}) {
+        const db = this.requireDb();
+        if (queryVec.length !== this.dim) {
+            throw new Error(`query vector dim mismatch: got ${queryVec.length}, expected ${this.dim}`);
+        }
+        const minScore = opts.minScore ?? -Infinity;
+        const folderPrefix = opts.folder ? `${opts.folder.replace(/\/+$/, "")}/` : null;
+        const rows = db
+            .prepare(folderPrefix
+            ? `SELECT rel_path, chunk_index, line_start, line_end, text_preview, vector
+             FROM embeddings WHERE rel_path LIKE ? || '%'`
+            : `SELECT rel_path, chunk_index, line_start, line_end, text_preview, vector FROM embeddings`)
+            .all(...(folderPrefix ? [folderPrefix] : []));
+        const heap = [];
+        for (const r of rows) {
+            const vec = new Float32Array(r.vector.buffer, r.vector.byteOffset, this.dim);
+            let score = 0;
+            for (let i = 0; i < this.dim; i++) {
+                score += (queryVec[i] ?? 0) * (vec[i] ?? 0);
+            }
+            if (score < minScore)
+                continue;
+            heap.push({
+                rel_path: r.rel_path,
+                chunk_index: r.chunk_index,
+                line_start: r.line_start,
+                line_end: r.line_end,
+                text_preview: r.text_preview,
+                score
+            });
+        }
+        heap.sort((a, b) => b.score - a.score);
+        return heap.slice(0, k);
+    }
+    /** Total embedded chunks — used by stats / UI. */
+    totalChunks() {
+        const db = this.requireDb();
+        const row = db.prepare("SELECT COUNT(*) AS n FROM embeddings").get();
+        return row?.n ?? 0;
+    }
+}
+/** Default location for the embed db, alongside the FTS5 db + parse cache. */
+export function defaultEmbedDbFile(vaultHashPrefix) {
+    // Caller is expected to compose the prefix with `~/.cache/enquire/<hash>` —
+    // we just append the .embed.db extension for consistency with .fts5.db.
+    return `${vaultHashPrefix}.embed.db`;
+}
+//# sourceMappingURL=embed-db.js.map

package/dist/embed-db.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"embed-db.js","sourceRoot":"","sources":["../src/embed-db.ts"],"names":[],"mappings":"AAAA,0EAA0E;AAC1E,6EAA6E;AAC7E,gFAAgF;AAChF,kDAAkD;AAClD,EAAE;AACF,gCAAgC;AAChC,gDAAgD;AAChD,0CAA0C;AAC1C,gFAAgF;AAChF,2DAA2D;AAC3D,EAAE;AACF,+EAA+E;AAC/E,2EAA2E;AAE3E,OAAO,EAAE,QAAQ,IAAI,EAAE,EAAE,MAAM,SAAS,CAAC;AACzC,OAAO,KAAK,IAAI,MAAM,WAAW,CAAC;AAElC,MAAM,cAAc,GAAG,CAAC,CAAC;AA0BzB,IAAI,gBAAgB,GAA2C,IAAI,CAAC;AACpE,KAAK,UAAU,gBAAgB;IAC7B,IAAI,gBAAgB;QAAE,OAAO,gBAAgB,CAAC;IAC9C,IAAI,CAAC;QACH,MAAM,GAAG,GAAG,CAAC,MAAM,MAAM,CAAC,gBAAgB,CAAC,CAAgD,CAAC;QAC5F,MAAM,IAAI,GAAG,GAAG,CAAC,OAAO,CAAC;QACzB,IAAI,CAAC,IAAI;YAAE,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC;QACnE,gBAAgB,GAAG,IAAI,CAAC;QACxB,OAAO,IAAI,CAAC;IACd,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,MAAM,IAAI,KAAK,CACb,8HACE,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CACjD,EAAE,CACH,CAAC;IACJ,CAAC;AACH,CAAC;AA6BD,MAAM,OAAO,OAAO;IACV,EAAE,GAAc,IAAI,CAAC;IACZ,IAAI,CAAS;IACb,SAAS,CAAS;IAClB,UAAU,CAAS;IACnB,GAAG,CAAS;IAE7B,YAAY,IAAoB;QAC9B,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC;QACtB,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC;QAChC,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,UAAU,CAAC;QAClC,IAAI,CAAC,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC;IACtB,CAAC;IAED,KAAK,CAAC,IAAI;QACR,IAAI,IAAI,CAAC,EAAE;YAAE,OAAO;QACpB,MAAM,IAAI,GAAG,MAAM,gBAAgB,EAAE,CAAC;QACtC,MAAM,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;QAC1E,MAAM,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,KAAK,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC;QAC/D,IAAI,CAAC,EAAE,GAAG,IAAI,IAAI,CAAC,IAAI,CAAC,IAAI,CAAO,CAAC;QACpC,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,oBAAoB,CAAC,CAAC;QACrC,IAAI,CAAC,EAAE,CAAC,MAAM,CAAC,sBAAsB,CAAC,CAAC;QACvC,IAAI,CAAC,eAAe,EAAE,CAAC;QACvB,MAAM,OAAO,CAAC,GAAG,CACf,CAAC,IAAI,CAAC,IAAI,EAAE,GAAG,IAAI,CAAC,IAAI,MAAM,EAAE,GAAG,IAAI,CAAC,IAAI,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,EAAE,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,GAAE,CAAC,CAAC,CAAC,CACnG,CAAC;IACJ,CAAC;IAED,0DAA0D;IAC1D,KAAK,CAAC,WAAW;QACf,IAAI,CAAC,KAAK,EAAE,CAAC;QACb,IAAI,OAAO,GAAG,KAAK,CAAC;QACpB,KAAK,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,EAAE,GAAG,IAAI,CAAC,IAAI,MAAM,EAAE,GAAG,IAAI,CAAC,IAAI,MAAM,CAAC,EAAE,CAAC;YACpE,IAAI,CAAC;gBACH,MAAM,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;gBACnB,OAAO,GAAG,IAAI,CAAC;YACjB,CAAC;YAAC,MAAM,CAAC;gBACP,kBAAkB;YACpB,CAAC;QACH,CAAC;QACD,OAAO,OAAO,CAAC;IACjB,CAAC;IAED,KAAK;QACH,IAAI,IAAI,CAAC,EAAE,EAAE,CAAC;YACZ,IAAI,CAAC,EAAE,CAAC,KAAK,EAAE,CAAC;YAChB,IAAI,CAAC,EAAE,GAAG,IAAI,CAAC;QACjB,CAAC;IACH,CAAC;IAEO,eAAe;QACrB,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAE5B,EAAE,CAAC,IAAI,CAAC;;;;;KAKP,CAAC,CAAC;QAEH,MAAM,IAAI,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;QAC7B,MAAM,YAAY,GAAG,IAAI,CAAC,cAAc,KAAK,SAAS,IAAI,IAAI,CAAC,cAAc,KAAK,MAAM,CAAC,cAAc,CAAC,CAAC;QACzG,MAAM,SAAS,GAAG,IAAI,CAAC,UAAU,KAAK,SAAS,IAAI,IAAI,CAAC,UAAU,KAAK,IAAI,CAAC,SAAS,CAAC;QACtF,MAAM,UAAU,GAAG,IAAI,CAAC,WAAW,KAAK,SAAS,IAAI,IAAI,CAAC,WAAW,KAAK,IAAI,CAAC,UAAU,CAAC;QAC1F,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,KAAK,SAAS,IAAI,IAAI,CAAC,GAAG,KAAK,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QACzE,IAAI,CAAC,YAAY,IAAI,CAAC,SAAS,IAAI,CAAC,UAAU,IAAI,CAAC,QAAQ,EAAE,CAAC;YAC5D,MAAM,MAAM,GAAa,EAAE,CAAC;YAC5B,IAAI,CAAC,YAAY;gBAAE,MAAM,CAAC,IAAI,CAAC,kBAAkB,IAAI,CAAC,cAAc,MAAM,cAAc,EAAE,CAAC,CAAC;YAC5F,IAAI,CAAC,SAAS;gBAAE,MAAM,CAAC,IAAI,CAAC,cAAc,IAAI,CAAC,UAAU,MAAM,IAAI,CAAC,SAAS,EAAE,CAAC,CAAC;YACjF,IAAI,CAAC,UAAU;gBAAE,MAAM,CAAC,IAAI,CAAC,SAAS,IAAI,CAAC,WAAW,MAAM,IAAI,CAAC,UAAU,EAAE,CAAC,CAAC;YAC/E,IAAI,CAAC,QAAQ;gBAAE,MAAM,CAAC,IAAI,CAAC,OAAO,IAAI,CAAC,GAAG,MAAM,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;YAC5D,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC,oCAAoC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YACjF,EAAE,CAAC,IAAI,CAAC,qEAAqE,CAAC,CAAC;QACjF,CAAC;QAED,EAAE,CAAC,IAAI,CAAC;;;;;;;;;;;;;;;;;;KAkBP,CAAC,CAAC;QAEH,IAAI,CAAC,SAAS,CAAC;YACb,cAAc,EAAE,MAAM,CAAC,cAAc,CAAC;YACtC,UAAU,EAAE,IAAI,CAAC,SAAS;YAC1B,WAAW,EAAE,IAAI,CAAC,UAAU;YAC5B,GAAG,EAAE,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC;SACtB,CAAC,CAAC;IACL,CAAC;IAEO,QAAQ;QACd,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC5B,MAAM,IAAI,GAAG,EAAE,CAAC,OAAO,CAAC,6BAA6B,CAAC,CAAC,GAAG,EAAkC,CAAC;QAC7F,MAAM,GAAG,GAA2B,EAAE,CAAC;QACvC,KAAK,MAAM,CAAC,IAAI,IAAI;YAAE,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC;QAC3C,OAAO,GAAG,CAAC;IACb,CAAC;IAEO,SAAS,CAAC,EAA0B;QAC1C,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC5B,MAAM,IAAI,GAAG,EAAE,CAAC,OAAO,CAAC,wDAAwD,CAAC,CAAC;QAClF,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,EAAE,CAAC;YAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAC1D,CAAC;IAEO,SAAS;QACf,IAAI,CAAC,IAAI,CAAC,EAAE;YAAE,MAAM,IAAI,KAAK,CAAC,0CAA0C,CAAC,CAAC;QAC1E,OAAO,IAAI,CAAC,EAAE,CAAC;IACjB,CAAC;IAED,yEAAyE;IACzE,UAAU,CACR,OAAe,EACf,OAAe,EACf,MAME;QAEF,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC5B,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC;QACrB,MAAM,EAAE,GAAG,EAAE,CAAC,WAAW,CAAC,CAAC,GAAG,IAAe,EAAE,EAAE;YAC/C,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,CAAkB,CAAC;YACtC,EAAE,CAAC,OAAO,CAAC,2CAA2C,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YACrE,MAAM,MAAM,GAAG,EAAE,CAAC,OAAO,CACvB;mCAC2B,CAC5B,CAAC;YACF,KAAK,MAAM,CAAC,IAAI,IAAI,EAAE,CAAC;gBACrB,IAAI,CAAC,CAAC,MAAM,CAAC,MAAM,KAAK,GAAG,EAAE,CAAC;oBAC5B,MAAM,IAAI,KAAK,CACb,2BAA2B,OAAO,UAAU,CAAC,CAAC,UAAU,SAAS,CAAC,CAAC,MAAM,CAAC,MAAM,cAAc,GAAG,EAAE,CACpG,CAAC;gBACJ,CAAC;gBACD,MAAM,CAAC,GAAG,CACR,OAAO,EACP,CAAC,CAAC,UAAU,EACZ,CAAC,CAAC,SAAS,EACX,CAAC,CAAC,OAAO,EACT,CAAC,CAAC,WAAW,EACb,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC,MAAM,CAAC,UAAU,EAAE,CAAC,CAAC,MAAM,CAAC,UAAU,CAAC,CACvE,CAAC;YACJ,CAAC;YACD,EAAE,CAAC,OAAO,CACR;2CACmC,CACpC,CAAC,GAAG,CAAC,OAAO,EAAE,OAAO,EAAE,IAAI,CAAC,MAAM,CAAC,CAAC;QACvC,CAAC,CAAC,CAAC;QACH,EAAE,CAAC,MAAM,CAAC,CAAC;IACb,CAAC;IAED,iEAAiE;IACjE,UAAU,CAAC,OAAe;QACxB,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC5B,EAAE,CAAC,OAAO,CAAC,2CAA2C,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;QACrE,EAAE,CAAC,OAAO,CAAC,6CAA6C,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;IACzE,CAAC;IAED;oBACgB;IAChB,eAAe;QACb,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC5B,OAAO,EAAE,CAAC,OAAO,CAAC,6CAA6C,CAAC,CAAC,GAAG,EAAkB,CAAC;IACzF,CAAC;IAED;;gDAE4C;IAC5C,MAAM,CAAC,QAAsB,EAAE,CAAS,EAAE,OAA+C,EAAE;QACzF,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC5B,IAAI,QAAQ,CAAC,MAAM,KAAK,IAAI,CAAC,GAAG,EAAE,CAAC;YACjC,MAAM,IAAI,KAAK,CAAC,kCAAkC,QAAQ,CAAC,MAAM,cAAc,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC;QAC7F,CAAC;QACD,MAAM,QAAQ,GAAG,IAAI,CAAC,QAAQ,IAAI,CAAC,QAAQ,CAAC;QAC5C,MAAM,YAAY,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC;QAEhF,MAAM,IAAI,GAAG,EAAE;aACZ,OAAO,CACN,YAAY;YACV,CAAC,CAAC;0DAC8C;YAChD,CAAC,CAAC,0FAA0F,CAC/F;aACA,GAAG,CAOD,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;QAE9C,MAAM,IAAI,GAAqB,EAAE,CAAC;QAClC,KAAK,MAAM,CAAC,IAAI,IAAI,EAAE,CAAC;YACrB,MAAM,GAAG,GAAG,IAAI,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC,MAAM,CAAC,UAAU,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC;YAC7E,IAAI,KAAK,GAAG,CAAC,CAAC;YACd,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC;gBAClC,KAAK,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;YAC9C,CAAC;YACD,IAAI,KAAK,GAAG,QAAQ;gBAAE,SAAS;YAC/B,IAAI,CAAC,IAAI,CAAC;gBACR,QAAQ,EAAE,CAAC,CAAC,QAAQ;gBACpB,WAAW,EAAE,CAAC,CAAC,WAAW;gBAC1B,UAAU,EAAE,CAAC,CAAC,UAAU;gBACxB,QAAQ,EAAE,CAAC,CAAC,QAAQ;gBACpB,YAAY,EAAE,CAAC,CAAC,YAAY;gBAC5B,KAAK;aACN,CAAC,CAAC;QACL,CAAC;QACD,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;QACvC,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAC1B,CAAC;IAED,kDAAkD;IAClD,WAAW;QACT,MAAM,EAAE,GAAG,IAAI,CAAC,SAAS,EAAE,CAAC;QAC5B,MAAM,GAAG,GAAG,EAAE,CAAC,OAAO,CAAC,sCAAsC,CAAC,CAAC,GAAG,EAAiB,CAAC;QACpF,OAAO,GAAG,EAAE,CAAC,IAAI,CAAC,CAAC;IACrB,CAAC;CACF;AAED,8EAA8E;AAC9E,MAAM,UAAU,kBAAkB,CAAC,eAAuB;IACxD,4EAA4E;IAC5E,wEAAwE;IACxE,OAAO,GAAG,eAAe,WAAW,CAAC;AACvC,CAAC"}

package/dist/embeddings.d.ts

@@ -0,0 +1,38 @@
+/** Catalog of embedding models supported by enquire. Add new entries by
+ *  pinning the Xenova-converted ONNX model id + the dim count + a friendly
+ *  alias users pass on the CLI. */
+export interface EmbeddingModel {
+    /** CLI-friendly alias passed via `--embedding-model <alias>`. */
+    alias: string;
+    /** HuggingFace model id (Xenova-converted to ONNX). */
+    hfId: string;
+    /** Output vector dimensionality (384 for MiniLM family). */
+    dim: number;
+    /** Approximate disk footprint in MB after download, for progress messages. */
+    approxSizeMB: number;
+    /** True if this model has been trained on multilingual data. */
+    multilingual: boolean;
+    /** Maximum input tokens before transformers.js truncates. */
+    maxTokens: number;
+}
+export declare const EMBEDDING_MODELS: Readonly<Record<string, EmbeddingModel>>;
+/** Default model alias when the user doesn't pass `--embedding-model`. */
+export declare const DEFAULT_MODEL_ALIAS = "multilingual";
+export declare function resolveModel(alias: string | undefined): EmbeddingModel;
+/** Opaque handle for a loaded embedder. Constructed via `loadEmbedder()`. */
+export interface Embedder {
+    readonly model: EmbeddingModel;
+    /** Embed a batch of texts. Each text is L2-normalized; output is one
+     *  Float32Array per input, length === model.dim. */
+    embed(texts: readonly string[]): Promise<Float32Array[]>;
+}
+/** Load an embedder for the given model alias. First call may block on
+ *  model download from HuggingFace (~120MB for multilingual). Subsequent
+ *  calls reuse the cached weights under `~/.cache/huggingface/`.
+ *
+ *  @param alias - Model alias from EMBEDDING_MODELS (default: "multilingual").
+ */
+export declare function loadEmbedder(alias?: string): Promise<Embedder>;
+/** Cosine similarity between two L2-normalized vectors (= dot product). */
+export declare function cosineSim(a: Float32Array, b: Float32Array): number;
+//# sourceMappingURL=embeddings.d.ts.map

package/dist/embeddings.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"embeddings.d.ts","sourceRoot":"","sources":["../src/embeddings.ts"],"names":[],"mappings":"AAcA;;mCAEmC;AACnC,MAAM,WAAW,cAAc;IAC7B,iEAAiE;IACjE,KAAK,EAAE,MAAM,CAAC;IACd,uDAAuD;IACvD,IAAI,EAAE,MAAM,CAAC;IACb,4DAA4D;IAC5D,GAAG,EAAE,MAAM,CAAC;IACZ,8EAA8E;IAC9E,YAAY,EAAE,MAAM,CAAC;IACrB,gEAAgE;IAChE,YAAY,EAAE,OAAO,CAAC;IACtB,6DAA6D;IAC7D,SAAS,EAAE,MAAM,CAAC;CACnB;AAED,eAAO,MAAM,gBAAgB,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,cAAc,CAAC,CAiBpE,CAAC;AAEH,0EAA0E;AAC1E,eAAO,MAAM,mBAAmB,iBAAiB,CAAC;AAElD,wBAAgB,YAAY,CAAC,KAAK,EAAE,MAAM,GAAG,SAAS,GAAG,cAAc,CAQtE;AAED,6EAA6E;AAC7E,MAAM,WAAW,QAAQ;IACvB,QAAQ,CAAC,KAAK,EAAE,cAAc,CAAC;IAC/B;wDACoD;IACpD,KAAK,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,GAAG,OAAO,CAAC,YAAY,EAAE,CAAC,CAAC;CAC1D;AA0BD;;;;;GAKG;AACH,wBAAsB,YAAY,CAAC,KAAK,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,QAAQ,CAAC,CAgCpE;AAED,2EAA2E;AAC3E,wBAAgB,SAAS,CAAC,CAAC,EAAE,YAAY,EAAE,CAAC,EAAE,YAAY,GAAG,MAAM,CASlE"}