npm - @oomkapwn/enquire-mcp - Versions diffs - 2.8.0 → 2.10.0 - Mend

@oomkapwn/enquire-mcp 2.8.0 → 2.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (20) hide show

package/CHANGELOG.md CHANGED Viewed

@@ -2,6 +2,108 @@
 All notable changes to this project will be documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [2.10.0] — 2026-05-08
+**Sprint 10 — OCR for image-only / scanned PDFs.** Closes the v2.7-v2.8-v2.9 PDF retrieval story. v2.7.0 added text-extraction tools; v2.8.0 blended PDF chunks into hybrid search; v2.9.0 added cross-encoder reranking. v2.10.0 makes the **scanned / camera-captured** PDFs in your vault searchable too — Tesseract.js OCR over each page bitmap.
+### Added — `obsidian_ocr_pdf`
+Runs Tesseract OCR over each page of an image-only / scanned PDF and returns the same shape as `obsidian_read_pdf` plus a per-page `confidence` score (0-100) and a doc-level `mean_confidence`. Use this when `obsidian_read_pdf` returns `has_text: false` (typical for scans, photographed paper, image-only PDFs).
+- **Multilingual** via `lang` (default `'eng'`; multi-lang via `'+'`, e.g. `'eng+rus'` for English+Russian mixed scans). Trained-data files for each language download on first use into Tesseract's local cache (~10 MB per language).
+- **Optional `pages` range** for partial OCR of long docs — OCR is the slowest step in the pipeline (~1-2s per page on M1 CPU), so a 100-page paper takes minutes.
+- **Optional `scale`** (DPI multiplier, default 2 ~ 150 DPI, capped at 4 server-side to prevent adversarial-PDF OOM).
+- **Per-page failure isolation** — one bad page doesn't sink the document.
+- **Tesseract worker terminated after each call** so HTTP transport doesn't accumulate per-request state.
+### Added — two new optional dependencies
+`tesseract.js@^7.0.0` (~1.4 MB unpacked, pure WebAssembly OCR engine) and `@napi-rs/canvas@^1.0.0` (~125 KB unpacked, native PDF→bitmap rendering with platform-specific binaries downloading conditionally) — both `optionalDependencies` so the markdown-only path stays zero-cost.
+Lazy-imported via the same pattern as `pdfjs-dist` (v2.7.0), `better-sqlite3` (v1.x), and `@huggingface/transformers` (v2.0.0). Missing-deps surface a clean install-hint error rather than a cryptic module-not-found stack.
+### Server-side hardening
+- `isEvalSupported: false`, `useSystemFonts: false`, `verbosity: 0` on pdfjs's `loadingTask` (matches v2.7.0 PDF read path).
+- Render scale clamped to `[0.5, 4]` so adversarial PDFs claiming 100-DPI multipliers don't OOM the server.
+- Tesseract worker terminated in a `finally` block so WebAssembly state never leaks even if a render or recognize call throws mid-page.
+- Same path-safety + privacy filter (`--exclude-glob` / `--read-paths` / `vault.stat`) as `obsidian_read_note` and `obsidian_read_pdf`. Audit-tested at every read boundary.
+### Tests
+507 unit tests pass (was 502 in v2.9.0, +5 new):
+- **ocrPdf path + privacy contract (+5):** rejects missing path arg, rejects non-existent file, refuses paths excluded by `--exclude-glob`, refuses paths outside `--read-paths` allowlist, accepts both `.pdf` and bare-stem paths consistently.
+End-to-end OCR validation (loading a real Tesseract worker against a synthetic image-only PDF) is deferred to manual smoke — Tesseract.js + @napi-rs/canvas startup is heavy (~2s) and a real synthetic image-PDF fixture would inflate the test repo.
+### Surface delta vs v2.9.0
+- **+1 read tool** (`obsidian_ocr_pdf`)
+- **+2 optional deps** (`tesseract.js`, `@napi-rs/canvas`) — both lazy-loaded, markdown-only path zero-cost
+- **Total surface:** 39 tools (28 always-on read + 1 opt-in `--persistent-index` + 3 opt-in diagnostic + 7 opt-in write) + 17 prompts
+### Migration
+**No-op for default users.** OCR runs only when an agent explicitly calls `obsidian_ocr_pdf`. Existing `obsidian_read_pdf` behavior unchanged — it still returns `has_text: false` for scanned PDFs and now points at `obsidian_ocr_pdf` in its tool description.
+Users on `--omit=optional` who try to call `obsidian_ocr_pdf` get a clean error message naming exactly what to install (`npm install tesseract.js @napi-rs/canvas`).
+### Strategic position
+The PDF retrieval story is now complete:
+- v2.7.0 — extraction tools (`obsidian_list_pdfs` / `obsidian_read_pdf`)
+- v2.8.0 — blended into hybrid search (`obsidian_search` returns PDF chunks with `kind: "pdf"` + page citations)
+- v2.9.0 — cross-encoder reranking on the blended candidate set
+- **v2.10.0 — OCR for the image-only / scanned PDFs** the v2.8.0 pipeline previously skipped
+**No other Obsidian-MCP currently does OCR for scanned PDFs.** Combined with the v2.0-v2.9 retrieval moats, enquire is now the only Obsidian-MCP that gives an agent searchable access to **every** PDF in your vault — text-PDFs, scanned-PDFs, multilingual content — with hybrid retrieval + cross-encoder reranking on top.
+## [2.9.0] — 2026-05-08
+**Sprint 9 — BGE cross-encoder reranking on top of RRF.** Cross-encoder reranking is the SOTA technique in IR for boosting retrieval quality over bi-encoder candidates: after RRF fusion, the top-N hits are re-scored by a model that sees query+document interaction directly (instead of comparing pre-computed embeddings). Typical wins: +5-10 NDCG@10 on real-world retrieval. **No other Obsidian-MCP currently does cross-encoder reranking** — this extends our retrieval quality leadership claim.
+### Added — `--enable-reranker` CLI flag
+Off by default — opt-in because the cross-encoder model is downloaded from HuggingFace on first call (~25-110 MB depending on alias) and adds ~30-50ms per query at top-50 candidates on M1 CPU. When enabled:
+- `enquire-mcp serve --vault <path> --persistent-index --enable-reranker` → boots; reranker model lazy-loads on first search call.
+- After RRF fusion + graph-boost, top-N candidates (default 50; tunable via `--reranker-top-n <n>`) are re-scored by a cross-encoder, then re-sorted before the response is truncated to `limit`.
+- Each reranked hit carries a `reranker_score` field in `[0, 1]` (sigmoid of the model's relevance logit) so agents see the cross-encoder's relevance estimate alongside RRF observability.
+### Added — `RERANKER_MODELS` catalog
+Two models ship out of the box, both via the existing `@huggingface/transformers` `optionalDependency`:
+- **`rerank-multilingual`** (default) — `Xenova/mxbai-rerank-xsmall-v1`, ~25 MB, multilingual. Best balance of speed × quality × language coverage.
+- **`rerank-bge`** — `Xenova/bge-reranker-base`, ~110 MB, English-only. Higher peak quality on English content; recommended only when you don't need multilingual support.
+Choose via `--reranker-model <alias>`. Same lazy-load pattern as embedding models — first call downloads weights into `~/.cache/huggingface/transformers.js/`; subsequent queries hit the warm cache.
+### Wiring
+- `searchHybrid(vault, args, ctx)` accepts an optional `ctx.reranker?: { alias?, topN? }`. When set, the reranker runs after RRF + graph-boost; failures surface via `signal_errors.reranker` (matching the existing per-signal failure-reporting pattern from v2.0.0-beta.2). The fused order is preserved if reranking fails, so a model load problem doesn't break search.
+- A `ctx.rerankerOverride` injection point lets unit tests validate the rerank-and-resort plumbing without pulling in the real ML model.
+- Reranker passages are derived from each candidate's best snippet (BM25 > embeddings > TF-IDF preference), with FTS5 highlight markers stripped and length capped at 600 chars to fit safely under the 512-token model budget.
+### Tests
+502 unit tests pass (was 493 in v2.8.0, +9 new):
+- **RERANKER_MODELS catalog (+5):** rerank-multilingual is the multilingual default, rerank-bge is English-only, defaults to rerank-multilingual on undefined alias, throws on unknown alias with helpful list, every entry has sensible approxSizeMB.
+- **searchHybrid + reranker plumbing (+4):** reranker invoked when override is set, top-N re-orders by reranker score (high.md beats mid.md beats low.md by synthetic scores), errors surface via `signal_errors.reranker` with original RRF order preserved, `topN` caps how many candidates carry `reranker_score`.
+### Surface delta vs v2.8.0
+- **No new tools.** Reranking is a property of `obsidian_search`, not a new tool surface.
+- **+3 CLI flags** (`--enable-reranker`, `--reranker-model <alias>`, `--reranker-top-n <n>`) on `serve` (and via the same options shape, on `serve-http`).
+### Migration
+**No-op for default users.** Reranking is opt-in via `--enable-reranker`. Existing users keep working unchanged. Once you opt in, the first search call downloads the reranker model (~25 MB for default `rerank-multilingual`); subsequent queries reuse the cached weights.
+### Strategic position
+Combined with v2.0-v2.8's hybrid RRF + wikilink graph-boost + breadcrumb chunking + multilingual embeddings + remote MCP transport + PDF retrieval, **enquire-mcp is now the only Obsidian-MCP that runs cross-encoder reranking on top of hybrid retrieval over markdown + PDFs**. Smart Connections (paid) doesn't rerank. Khoj doesn't either. The retrieval-quality moat widens.
 ## [2.8.0] — 2026-05-08
 **Sprint 8 — PDF retrieval integration.** v2.7.0 added PDF text-extraction tools (`obsidian_list_pdfs` / `obsidian_read_pdf`); v2.8.0 makes PDFs **first-class citizens of `obsidian_search`**. Index PDFs into the same FTS5 + embedding stores as markdown, blend them in hybrid retrieval (BM25 + TF-IDF + embeddings → RRF fusion), and surface a `kind: "md" | "pdf"` flag on every hit so agents can distinguish content sources at a glance.

package/README.md CHANGED Viewed

@@ -1,283 +1,265 @@
 <div align="center">
-<a href="https://github.com/oomkapwn/enquire-mcp"><img src="./assets/social-preview.png" alt="enquire — MCP server for Obsidian vaults. Hybrid retrieval (BM25 + TF-IDF + ML embeddings via RRF). Wikilinks, frontmatter, backlinks, Dataview, multilingual semantic search. For Claude Code, Cursor, Codex." width="100%"></a>
+<a href="https://github.com/oomkapwn/enquire-mcp"><img src="./assets/social-preview.png" alt="enquire — MCP server for Obsidian vaults. Hybrid retrieval (BM25 + TF-IDF + ML embeddings via RRF + cross-encoder reranking). Wikilinks, frontmatter, backlinks, Dataview, multilingual semantic search, PDFs, remote MCP. For Claude Code, Cursor, Codex." width="100%"></a>
-# enquire — MCP server for Obsidian
+# enquire — give your AI a search engine for your Obsidian vault
-**Hybrid retrieval (BM25 + TF-IDF + ML embeddings, RRF-fused) for your Obsidian vault.** Drop-in for Claude Code, Cursor, OpenClaw 🦞, Codex, Devin, and any MCP-compatible agent. Free. Multilingual (50+ languages). Offline-capable. No Obsidian plugin required.
+**Hybrid retrieval. Cross-encoder reranking. PDFs. Multilingual. Remote MCP. Free.**
+The most advanced Obsidian-MCP you can run today — drop into Claude Code, Claude.ai web, Cursor, ChatGPT, or any MCP client and your agent gets a single `obsidian_search` tool that fuses BM25 + TF-IDF + ML embeddings, reranks with a BGE cross-encoder, and surfaces blended markdown + PDF hits with page citations.
 [![CI](https://github.com/oomkapwn/enquire-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/oomkapwn/enquire-mcp/actions/workflows/ci.yml)
-[![npm](https://img.shields.io/npm/v/@oomkapwn/enquire-mcp/latest.svg?label=npm)](https://www.npmjs.com/package/@oomkapwn/enquire-mcp)
-[![beta](https://img.shields.io/npm/v/@oomkapwn/enquire-mcp/beta.svg?label=beta)](https://www.npmjs.com/package/@oomkapwn/enquire-mcp/v/beta)
-[![Tests](https://img.shields.io/badge/tests-408_passing-brightgreen.svg)](#engineering)
-[![License](https://img.shields.io/badge/license-MIT-yellow.svg)](./LICENSE)
-[![MCP](https://img.shields.io/badge/MCP-1.29-8A2BE2.svg)](https://modelcontextprotocol.io/)
+[![npm](https://img.shields.io/npm/v/@oomkapwn/enquire-mcp/latest.svg?label=npm%20%40latest&color=cb3837)](https://www.npmjs.com/package/@oomkapwn/enquire-mcp)
+[![tests](https://img.shields.io/badge/tests-502%20passing-brightgreen.svg)](#trust)
 [![SLSA-3](https://img.shields.io/badge/SLSA-3-blue.svg)](https://slsa.dev/spec/v1.0/levels#build-l3)
+[![MCP](https://img.shields.io/badge/MCP-1.29-8A2BE2.svg)](https://modelcontextprotocol.io/)
+[![License](https://img.shields.io/badge/license-MIT-yellow.svg)](./LICENSE)
+[![Node](https://img.shields.io/badge/node-%E2%89%A520-3c873a.svg)](https://nodejs.org)
 </div>
+---
+## ⚡ 30-second quick start
 ```bash
 npm install -g @oomkapwn/enquire-mcp
 enquire-mcp serve --vault ~/Documents/Obsidian\ Vault
 ```
-That's it. Your AI now has structured access to wikilinks, backlinks, frontmatter, Dataview queries, and **`obsidian_search`** — a single hybrid-retrieval tool that auto-fuses BM25 + TF-IDF + ML embeddings.
+That's it. Your AI now has structured access to wikilinks, backlinks, frontmatter, Dataview, and **`obsidian_search`** — the umbrella retrieval tool.
+**For Claude Code / Cursor / Codex / any MCP client:**
+```json
+{
+  "mcpServers": {
+    "obsidian": {
+      "command": "npx",
+      "args": ["-y", "@oomkapwn/enquire-mcp", "serve", "--vault", "/path/to/vault"]
+    }
+  }
+}
+```
+**Want hybrid retrieval at full power?** One-time setup, ~10 min for a 100-note vault:
+```bash
+enquire-mcp install-model multilingual          # ~120MB, 50+ languages
+enquire-mcp build-embeddings --vault <path>     # ~30ms/chunk on M1
+# then: serve --persistent-index for BM25 + --enable-reranker for cross-encoder
+```
 ---
-## Why enquire (vs alternatives)
+## 🎯 The only Obsidian-MCP with…
-| | Other Obsidian-MCPs | Smart Connections (paid plugin) | **enquire** |
-|---|:---:|:---:|:---:|
-| Read-only by default | varies | n/a | ✅ |
-| Resolves wikilinks (alias / section / block) | partial | n/a | ✅ full |
-| Backlinks ranked + snippeted | rare | n/a | ✅ |
-| Dataview-style queries | needs plugin | n/a | ✅ first-class |
-| Canvas (`.canvas`) read | rare | n/a | ✅ typed nodes + edges |
-| BM25 full-text search | rare | ❌ | ✅ FTS5 SQLite |
-| TF-IDF semantic search | ❌ | ❌ | ✅ |
-| **ML embeddings (multilingual)** | ❌ | ✅ paid | ✅ **free** |
-| **Hybrid (BM25+TF-IDF+embeddings, RRF)** | ❌ | ❌ | ✅ **only here** |
-| **PDFs blended into hybrid search** | ❌ | ❌ | ✅ **only here** (v2.8.0) |
-| Per-signal observability on each hit | ❌ | ❌ | ✅ |
-| Privacy filter (`--exclude-glob` / `--read-paths`) | ❌ | n/a | ✅ verified at search + write paths |
-| Standalone (no Obsidian plugin) | varies | ❌ requires Obsidian | ✅ direct vault read |
-| MCP-native (any agent) | varies | ❌ Obsidian-only | ✅ stdio JSON-RPC |
-| **Remote MCP (HTTP transport, bearer auth)** | ❌ | ❌ | ✅ **only here** (v2.6.0) |
-| SLSA-3 provenance | ❌ | n/a | ✅ |
-| Test suite | rare | n/a | ✅ 493 unit tests |
+- ✅ **Hybrid retrieval** (BM25 + TF-IDF + ML embeddings, RRF-fused)
+- ✅ **Cross-encoder reranking** on top of RRF (+5-10 NDCG@10) — `v2.9.0`
+- ✅ **PDFs blended into hybrid search** with `[page: N]` citation markers — `v2.8.0`
+- ✅ **OCR for scanned / image-only PDFs** (Tesseract.js, multilingual) — `v2.10.0`
+- ✅ **Wikilink graph-boost** as a retrieval signal (1-step personalised PageRank seeded by RRF top-K)
+- ✅ **Remote MCP** over HTTP with bearer auth + rate-limit + CORS — `v2.6.0`
+- ✅ **Multilingual** semantic search (50+ languages, runs on CPU, free)
+- ✅ **Note-tethered AI chat threads** persisted as markdown — Smart Connections' #1 paid feature, free here
+**Read-only by default.** All 7 write tools gated behind `--enable-write`. Privacy filter (`--exclude-glob` / `--read-paths`) verified at every search + write path. SLSA-3 release provenance.
 ---
-## Architecture
+## 🏗️ How retrieval works
 ```mermaid
 graph LR
     Q[Query]
     Q --> S[obsidian_search]
-    S --> BM25[BM25 / FTS5<br/>opt-in: --persistent-index]
+    S --> BM25[BM25 / FTS5<br/>--persistent-index]
     S --> TFIDF[TF-IDF<br/>always on]
-    S --> EMB[ML embeddings<br/>opt-in: build-embeddings]
+    S --> EMB[ML embeddings<br/>build-embeddings]
     BM25 --> RRF{RRF fusion<br/>k=60}
     TFIDF --> RRF
     EMB --> RRF
-    RRF --> R[Ranked hits<br/>per_signal observability]
+    RRF --> GB[Graph boost<br/>α × in-degree]
+    GB --> RR[Cross-encoder<br/>reranker<br/>--enable-reranker]
+    RR --> R[Ranked hits<br/>per_signal observability]
 ```
-`obsidian_search` auto-detects available signals and fuses them via Reciprocal Rank Fusion (Cormack et al, 2009). Returns per-signal contributions on every hit so agents can see WHY each result ranked.
+`obsidian_search` auto-detects available signals and fuses them via Reciprocal Rank Fusion (Cormack et al, 2009). Wikilink graph-boost reranks top-K by 1-step personalised PageRank. Optional cross-encoder reranking (BGE) re-scores top-N for +5-10 NDCG@10. Every hit returns `per_signal` observability so you see WHY each result ranked.
-```
-Tier 1: serve --vault <path>                      → TF-IDF (zero setup, instant)
-Tier 2: serve --vault <path> --persistent-index   → + BM25 (sub-100ms top-10)
-Tier 3: + install-model + build-embeddings        → + ML embeddings (multilingual)
-Tier 4: serve-http --bearer-token <token>         → remote MCP (v2.6.0)
-        same retrieval stack, exposed over HTTP for Claude.ai web,
-        ChatGPT, Cursor HTTP, mobile. Tailscale Funnel / Cloudflare
-        Tunnel for HTTPS. See docs/http-transport.md.
-```
+| Tier | Setup | What you get |
+|---|---|---|
+| **1** | `serve --vault <path>` | TF-IDF (zero setup, instant) |
+| **2** | + `--persistent-index` | + BM25 (sub-100ms top-10) |
+| **3** | + `install-model` + `build-embeddings` | + multilingual ML embeddings |
+| **4** | + `--enable-reranker` | + BGE cross-encoder reranking |
+| **5** | + `--include-pdfs` | + PDFs blended into all of the above |
+| **6** | `serve-http --bearer-token …` | + remote MCP for Claude.ai web, ChatGPT, Cursor HTTP, mobile |
 ---
-## Quick start
-> **Two channels:** `npm install @oomkapwn/enquire-mcp` → **v2.0** (stable, hybrid retrieval). `@beta` → preview track. `@1` → legacy v1.x line.
+## 🆚 vs alternatives
-**Claude Desktop / Claude Code / Cursor / Codex / any MCP client**:
+| | Other Obsidian-MCPs | Smart Connections (paid) | **enquire** |
+|---|:---:|:---:|:---:|
+| Wikilinks (alias / section / block) | partial | n/a | ✅ full |
+| Backlinks ranked + snippeted | rare | n/a | ✅ |
+| Dataview-style queries | needs plugin | n/a | ✅ first-class |
+| Canvas (`.canvas`) read | rare | n/a | ✅ typed nodes + edges |
+| BM25 full-text | rare | ❌ | ✅ FTS5 SQLite |
+| TF-IDF semantic | ❌ | ❌ | ✅ |
+| ML embeddings (multilingual) | ❌ | 💰 paid | ✅ **free** |
+| **Hybrid (BM25+TF-IDF+embeddings, RRF)** | ❌ | ❌ | ✅ **only here** |
+| **Wikilink graph-boost retrieval signal** | ❌ | ❌ | ✅ **only here** |
+| **PDFs blended into hybrid search** | ❌ | ❌ | ✅ **only here** |
+| **OCR for scanned / image-only PDFs** | ❌ | ❌ | ✅ **only here** |
+| **Cross-encoder reranking** | ❌ | ❌ | ✅ **only here** |
+| **Remote MCP (HTTP + bearer auth)** | ❌ | ❌ | ✅ **only here** |
+| Per-signal observability per hit | ❌ | ❌ | ✅ |
+| Privacy filter (exclude/allow globs) | ❌ | n/a | ✅ verified at search + write paths |
+| Standalone (no Obsidian plugin) | varies | ❌ requires Obsidian | ✅ direct vault read |
+| MCP-native (any agent) | varies | ❌ Obsidian-only | ✅ stdio + HTTP |
+| SLSA-3 release provenance | ❌ | n/a | ✅ |
+| Test suite | rare | n/a | ✅ 507 unit tests |
-```json
-{
-  "mcpServers": {
-    "obsidian": {
-      "command": "npx",
-      "args": ["-y", "@oomkapwn/enquire-mcp", "serve", "--vault", "/path/to/vault"]
-    }
-  }
-}
-```
+> **Strategic claim:** enquire is the open-source backend for [Karpathy-style LLM Wikis](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) on top of your existing Obsidian vault. The `vault_synth` / `vault_wiki_compile` / `vault_lint_extended` prompts implement the ingest → query → lint → compile workflow natively over `.md` + `[[wikilinks]]`. Knowledge that compounds, traceable to sources.
-| Client | Config file |
-|---|---|
-| Claude Desktop | macOS `~/Library/Application Support/Claude/claude_desktop_config.json` · Windows `%APPDATA%\Claude\claude_desktop_config.json` |
-| Claude Code (CLI) | `~/.claude.json` (global) or `.mcp.json` (per-project) |
-| Cursor | `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (per-project) |
-| Codex / OpenClaw / Devin | per-tool MCP config |
+---
-**Enable hybrid retrieval** (one-time setup, ~10 min for a 100-note vault):
+## 🛠️ All 39 tools at a glance
-```bash
-enquire-mcp install-model multilingual          # ~120MB, 50+ languages
-enquire-mcp build-embeddings --vault <path>     # ~30ms/chunk on M1
-# Add --persistent-index to your serve invocation for BM25.
-```
+The umbrella `obsidian_search` plus 38 specialized tools for wikilinks, backlinks, Dataview, frontmatter, canvas, PDFs, OCR, vault stats, graph navigation, and writes.
-**Remote MCP** (v2.6.0 — Claude.ai web, ChatGPT, Cursor HTTP, mobile):
+<details>
+<summary><b>28 always-on read tools</b> — click to expand</summary>
-```bash
-enquire-mcp gen-token > ~/.enquire/token        # 256-bit bearer
-enquire-mcp serve-http \
-  --vault ~/Obsidian \
-  --bearer-token "$(cat ~/.enquire/token)" \
-  --persistent-index
-# Front with Tailscale Funnel / Cloudflare Tunnel for HTTPS — see
-# docs/http-transport.md.
-```
+`obsidian_search` · `obsidian_context_pack` · `obsidian_chat_thread_read` · `obsidian_frontmatter_get` · `obsidian_frontmatter_search` · `obsidian_read_note` · `obsidian_list_notes` · `obsidian_resolve_wikilink` · `obsidian_get_backlinks` · `obsidian_get_outbound_links` · `obsidian_get_unresolved_wikilinks` · `obsidian_get_recent_edits` · `obsidian_list_tags` · `obsidian_dataview_query` · `obsidian_find_path` · `obsidian_find_similar` · `obsidian_get_note_neighbors` · `obsidian_stats` · `obsidian_lint_wiki` · `obsidian_open_questions` · `obsidian_paper_audit` · `obsidian_validate_note_proposal` · `obsidian_list_canvases` · `obsidian_read_canvas` · `obsidian_list_pdfs` · `obsidian_read_pdf` · `obsidian_ocr_pdf` · `obsidian_open_in_ui`
-No other Obsidian-MCP currently ships a remote-HTTP transport. Same vault, same tools, same hybrid retrieval — just over HTTPS instead of stdio.
+</details>
----
+<details>
+<summary><b>4 opt-in read tools</b> (diagnostic single-rankers) — click to expand</summary>
-## Tools (38 total)
+`obsidian_full_text_search` (`--persistent-index`) · `obsidian_search_text` · `obsidian_semantic_search` · `obsidian_embeddings_search` (all 3 require `--diagnostic-search-tools`)
-### 27 always-on read tools
+</details>
-| Tool | What it does |
-|---|---|
-| `obsidian_search` | **Hybrid retrieval** — fuses BM25 + TF-IDF + ML embeddings via RRF. The default search tool. Auto-detects available signals. v2.2.0: `granularity: "block"` arg returns chunks instead of notes. **v2.8.0:** with `--include-pdfs`, PDF chunks blend in alongside markdown — every hit carries `kind: "md" \| "pdf"` and PDF snippets include `[page: N]` markers for citation. |
-| `obsidian_context_pack` | **v2.2.0.** Token-budgeted context bundling: takes a question, runs hybrid search, gathers note bodies + backlinks + optionally recent dailies, returns one ready-to-paste markdown bundle. Saves ~5 tool calls. |
-| `obsidian_chat_thread_read` | **v2.2.0.** Parse a note's `## Chat: <title>` block into structured messages (role/timestamp/content/line-range). Pair with `_append` (write) for note-tethered AI conversations. |
-| `obsidian_frontmatter_get` | **v2.3.0.** Read parsed YAML frontmatter for a note. With `key`, returns just that field. |
-| `obsidian_frontmatter_search` | **v2.3.0.** Find notes by frontmatter predicate (`equals` / `exists` / `contains`). Useful as a precursor to bulk `_set`. |
-| `obsidian_read_note` | Full content + frontmatter + wikilinks + embeds + tags. Also accepts periodic-note aliases (`title: "today"` / `"weekly"` / `"monthly"`). |
-| `obsidian_list_notes` | Vault-wide or folder-scoped. Includes title + tags + mtime + counts. |
-| `obsidian_resolve_wikilink` | Resolves `[[Note]]`, `[[Note\|Alias]]`, `[[Note#Section]]`, `[[Note#^block]]`, with did-you-mean on near-miss. |
-| `obsidian_get_backlinks` | Every note linking to X, ranked + snippeted. |
-| `obsidian_get_outbound_links` | Outbound `[[wikilinks]]` from one note, with resolution status. |
-| `obsidian_get_unresolved_wikilinks` | Vault-wide broken-link audit. |
-| `obsidian_get_recent_edits` | Notes modified in the last N minutes. |
-| `obsidian_list_tags` | Tag census across vault (frontmatter + inline). |
-| `obsidian_dataview_query` | First-class Dataview-style queries (`LIST` / `TABLE`, `WHERE`, `AND` / `OR` / `LIKE` / `contains`). |
-| `obsidian_find_path` | Multi-hop graph BFS between two notes (with alternatives). |
-| `obsidian_find_similar` | Note-to-note similarity (tag + folder + content signals). |
-| `obsidian_get_note_neighbors` | Outbound + inbound + tag-sibling for one note. |
-| `obsidian_stats` | Vault dashboard: note count, tag count, broken links. |
-| `obsidian_lint_wiki` | Karpathy LLM-Wiki `/lint` workflow (orphans / broken / stubs / stale). |
-| `obsidian_open_questions` | Find open `?` questions across the vault. |
-| `obsidian_paper_audit` | Track arXiv references and read-status. |
-| `obsidian_validate_note_proposal` | Lint a draft note before writing (closes the #1 LLM-write pain). |
-| `obsidian_list_canvases` | List `.canvas` files with node + edge counts. |
-| `obsidian_read_canvas` | Parse `.canvas` into typed nodes (text/file/link/group) + edges. |
-| `obsidian_list_pdfs` | **v2.7.0.** List `.pdf` files with size + mtime. PDFs are the #1 non-markdown content kind in real vaults; no other Obsidian-MCP indexes them. |
-| `obsidian_read_pdf` | **v2.7.0.** Extract page-by-page text + doc metadata (title/author/etc) from a PDF. Optional 1-indexed page-range slice. Image-only / scanned PDFs surface `has_text: false`. Powered by Mozilla PDF.js (Apache-2.0, pinned to `optionalDependencies` so the markdown-only path stays zero-cost). |
-| `obsidian_open_in_ui` | Emit `obsidian://open?vault=...` URI. |
-### 4 opt-in read tools
-| Tool | Flag | Notes |
-|---|---|---|
-| `obsidian_full_text_search` | `--persistent-index` + `--diagnostic-search-tools` | BM25 ranking, sub-100ms. |
-| `obsidian_search_text` | `--diagnostic-search-tools` | Token search (all/any/phrase modes). |
-| `obsidian_semantic_search` | `--diagnostic-search-tools` | TF-IDF cosine standalone. |
-| `obsidian_embeddings_search` | `--diagnostic-search-tools` | ML embeddings standalone. |
+<details>
+<summary><b>7 opt-in write tools</b> (require <code>--enable-write</code>) — click to expand</summary>
-### 7 opt-in write tools (`--enable-write`)
+`obsidian_create_note` · `obsidian_append_to_note` · `obsidian_rename_note` (rewrites every wikilink across vault, code-fence-aware) · `obsidian_replace_in_notes` (bulk find/replace) · `obsidian_archive_note` · `obsidian_chat_thread_append` · `obsidian_frontmatter_set` (atomic YAML manipulation, `dry_run` supported)
-| Tool | Notes |
-|---|---|
-| `obsidian_create_note` | Refuses overwrite without `overwrite=true`. Empty path rejected. |
-| `obsidian_append_to_note` | Symlink-safe; respects `--max-file-bytes`. |
-| `obsidian_rename_note` | Rewrites every wikilink across vault. Code-fence-aware. `dry_run` available. |
-| `obsidian_replace_in_notes` | Bulk find/replace. Per-file errors collected; `partial: true` on mid-loop fail. |
-| `obsidian_archive_note` | Wraps rename to `Archive/`. Preserves backlinks. |
-| `obsidian_chat_thread_append` | **v2.2.0.** Append a user/assistant/system message to a note's `## Chat:` block. Creates note + heading if absent. Threads stored as markdown — searchable, version-controllable, survive sessions. |
-| `obsidian_frontmatter_set` | **v2.3.0.** Surgical YAML manipulation — set/unset keys on a note atomically. Pass `null` as value to delete. Round-trips through gray-matter so YAML formatting stays consistent. `dry_run` supported. |
+</details>
-Plus **2 + 1 opt-in MCP resources** (`obsidian://note/...`, `obsidian://vault-info`, `obsidian://chunk/...`) and **17 MCP prompts** (`summarize_recent_edits`, `weekly_review`, `monthly_review`, `find_orphans`, `extract_todos`, `process_inbox`, `review_tag`, `consolidate_tags`, `find_duplicates`, `lint_wiki`, `search_with_query_expansion`, `vault_synth`, `vault_wiki_compile`, `vault_lint_extended`, `vault_capture`, `vault_persona_search`, `vault_automation_setup`).
+**Plus:** 2 + 1 opt-in MCP resources, and **17 MCP prompts** (`summarize_recent_edits`, `weekly_review`, `monthly_review`, `find_orphans`, `extract_todos`, `process_inbox`, `review_tag`, `consolidate_tags`, `find_duplicates`, `lint_wiki`, `search_with_query_expansion`, `vault_synth`, `vault_wiki_compile`, `vault_lint_extended`, `vault_capture`, `vault_persona_search`, `vault_automation_setup`).
-> **v2.4.0 strategic position:** enquire-mcp is the open-source backend for **Karpathy-style LLM Wikis** on top of your existing Obsidian vault. The `vault_synth` + `vault_wiki_compile` + `vault_lint_extended` prompts implement the [Karpathy LLM-Wiki workflow](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) (ingest → query → lint → compile) natively over Obsidian's `.md` + `[[wikilinks]]` substrate. Knowledge that compounds, traceable to sources.
+📖 Full reference: **[docs/api.md](./docs/api.md)** · Remote-MCP deployment guide: **[docs/http-transport.md](./docs/http-transport.md)**
 ---
-## Configuration
+## ⚙️ Configuration
+The flags you'll actually use:
-| Flag | Default | Notes |
+| Flag | Default | What it does |
 |---|---|---|
-| `--vault <path>` | required | Path to Obsidian vault root. |
-| `--enable-write` | off | Register the 5 write tools. |
-| `--exclude-glob <pat...>` | none | Privacy denylist. Repeatable. Example: `'02_Personal/**'`. |
-| `--read-paths <pat...>` | none | Privacy allowlist (only matching paths visible). Repeatable. |
-| `--persistent-index` | off | SQLite FTS5 BM25. Enables `obsidian_full_text_search` (with `--diagnostic-search-tools`). |
-| `--persistent-cache` | off | Persist parsed-note cache across restarts. |
-| `--watch` | off | Live invalidation on `.md` add/change/unlink. |
-| `--diagnostic-search-tools` | off | Register 4 single-ranker search tools (defaults: hybrid `obsidian_search` only). |
-| `--enabled-tools <name...>` | all | Strict allowlist (gate to a subset). |
-| `--disabled-tools <name...>` | none | Denylist. |
-| `--max-file-bytes <n>` | 5 MB | Per-file read/write cap. |
-| `--cache-size <n>` | 1024 | LRU cap for parsed-note cache. |
-Subcommands: `serve` · `serve-http` (v2.6.0 — remote MCP) · `gen-token` (v2.6.0) · `clear-cache` · `clear-index` · `index` (cold-build FTS5) · `install-model` · `build-embeddings` · `clear-embeddings`.
-Full reference: [docs/api.md](./docs/api.md). Remote-MCP deployment guide: [docs/http-transport.md](./docs/http-transport.md).
+| `--vault <path>` | required | Path to Obsidian vault root |
+| `--persistent-index` | off | SQLite FTS5 BM25, sub-100ms top-10 |
+| `--include-pdfs` | off | Index PDFs into FTS5 + embeddings |
+| `--enable-reranker` | off | BGE cross-encoder reranking on RRF top-N |
+| `--enable-write` | off | Register the 7 write tools |
+| `--exclude-glob <pat...>` | none | Privacy denylist (e.g. `'02_Personal/**'`) |
+| `--read-paths <pat...>` | none | Privacy allowlist (only matching paths visible) |
+| `--watch` | off | Live invalidation on `.md` add/change/unlink |
+| `--persistent-cache` | off | Survive cold starts |
+Subcommands: `serve` · `serve-http` · `gen-token` · `clear-cache` · `clear-index` · `clear-embeddings` · `index` · `install-model` · `build-embeddings`.
+**Remote MCP** for Claude.ai web / ChatGPT / Cursor HTTP / mobile:
+```bash
+enquire-mcp gen-token > ~/.enquire/token        # one-time
+enquire-mcp serve-http \
+  --vault ~/Obsidian \
+  --bearer-token "$(cat ~/.enquire/token)" \
+  --persistent-index --include-pdfs --enable-reranker
+# Front with Tailscale Funnel / Cloudflare Tunnel for HTTPS.
+```
 ---
-## Security
+## 🛡️ Trust
-- **Read-only by default.** Write tools require `--enable-write`.
-- **Path traversal blocked.** Realpath check on every read+write target. Symlinks inside the vault that resolve outside are rejected.
-- **Privacy boundary verified across all paths**, including persistent indexes (FTS5 / embed-db search-time filter) and the `obsidian://chunk/...` resource. Privacy fail-closed: empty `--read-paths` / `--exclude-glob` patterns refuse to start.
+- **Read-only by default.** Every write tool requires `--enable-write`.
+- **Path traversal blocked.** Realpath check on every read+write target. Symlinks resolving outside the vault are rejected.
+- **Privacy boundary verified across all paths** including persistent FTS5 + embed indexes and the `obsidian://chunk/...` resource. Privacy fail-closed: empty `--read-paths` / `--exclude-glob` patterns refuse to start.
+- **HTTP transport hardened.** Bearer auth (constant-time SHA-256 + `timingSafeEqual`), per-token sliding rate-limit, strict CORS allowlist with credential-leak guard.
 - **`gray-matter` (`js-yaml` safeLoad)** — no code execution via frontmatter.
-- **DQL parser** — no shell, no `eval`, no template expansion.
 - **Cache + index files** — chmod 0600, parent dir 0700.
 - **SLSA-3 provenance** on every npm release.
-- **Branch protection ruleset** with `bypass_mode: pull_request` — every change goes through PR with audit trail. Release pipeline verifies SHA-on-main + 8 required CI checks before publishing.
-Full posture: [SECURITY.md](./SECURITY.md). Report vulnerabilities to `oomkapwn@gmail.com`.
----
-## Engineering
+- **Branch protection** with `bypass_mode: pull_request` — every change goes through PR review. Release pipeline verifies tagged SHA is on `main` AND all 8 required CI checks reported `success` on it.
 | Surface | Posture |
 |---|---|
-| Language | TypeScript strict + `noUncheckedIndexedAccess` |
-| Lint | Biome 2 (zero-warning policy) |
-| Tests | 493 unit tests across 23 files |
-| CI | ubuntu × {Node 20, 22, 24} required + macOS advisory job |
+| Tests | 507 unit tests across 25 files, 8 required CI gates per PR |
 | Coverage | Lines ≥86%, statements ≥82%, functions ≥75%, branches ≥73% (gated) |
 | Audit | `npm audit --audit-level=moderate` for prod; high for dev |
-| Runtime deps | 5 mandatory (`@modelcontextprotocol/sdk`, `chokidar`, `commander`, `gray-matter`, `zod`) + 2 optional (`better-sqlite3` for FTS5 / embed-db; `@huggingface/transformers` for ML embeddings) |
-| Releases | npm + GitHub release per tag · semver · provenance attached · channels: `latest` (stable), `beta`, `alpha` |
+| CI | Ubuntu × {Node 20, 22, 24} required + macOS advisory job |
+| Lint | Biome 2 (zero-warning policy) |
+| Language | TypeScript strict + `noUncheckedIndexedAccess` |
+| Runtime deps | 5 mandatory, 3 optional (FTS5 + ML embeddings + PDF parser — markdown-only path stays zero-cost) |
+| Releases | npm + GitHub release per tag · semver · SLSA-3 provenance |
-```bash
-git clone https://github.com/oomkapwn/enquire-mcp.git
-cd enquire-mcp && npm install
-npm test          # full suite
-npm run lint      # zero warnings
-npm run build     # tsc → dist/
-```
+Full posture: **[SECURITY.md](./SECURITY.md)**. Report vulnerabilities to `oomkapwn@gmail.com`.
 ---
-## FAQ
+## ❓ FAQ
+**Do I need Obsidian installed?**
+No. enquire reads `.md` + `.canvas` + `.pdf` files directly. Works against any Obsidian-format vault.
+**Will this write to my vault?**
+Not unless you start with `--enable-write`. Even then, all 7 write tools are gated by privacy filters and refuse to overwrite without `overwrite: true`. `dry_run` modes available on the destructive ones.
+**Is my data sent anywhere?**
+Only on `enquire-mcp install-model` (downloads ONNX weights from HuggingFace, one-time). Serve mode itself never makes outbound HTTP. Embeddings + reranker run on CPU locally.
+**How is this different from Smart Connections?**
+Smart Connections is a paid Obsidian plugin that runs ML embeddings inside Obsidian. enquire is a standalone MCP server: free, MCP-native (works with Claude / Cursor / Codex / any agent), and fuses 3 retrieval signals + cross-encoder reranking for higher recall + precision than embeddings alone. PDFs index too.
+**Performance on large vaults?**
+Cold-build of FTS5 on a 1k-note vault: ~5s. Warm BM25 top-10: sub-100ms. Embedding build: ~30ms/chunk on M1 (~8min for 8k chunks). Hybrid query latency: <200ms typical. Reranker adds ~30-50ms at top-50. Maintainer dogfoods on a 128-note bilingual vault with all of the above on.
-**Q: Do I need Obsidian installed?**
-No. enquire reads `.md` + `.canvas` files directly. Works against any Obsidian-format vault.
+**Languages?**
+Default embedding model is `paraphrase-multilingual-MiniLM-L12-v2` (50+ languages). Multilingual cross-encoder reranker (`mxbai-rerank-xsmall-v1`) is the default too. Validated end-to-end on Russian + English bilingual vaults. CJK / Thai / Khmer / Lao tokenization via `Intl.Segmenter` (Node 16+ ICU).
-**Q: Will this write to my vault?**
-No, unless you explicitly start with `--enable-write`. Even then, all 5 write tools are gated by privacy filters and refuse to overwrite without `overwrite: true`.
+**Can I run it remotely?**
+Yes. `serve-http` exposes the same server over [Streamable HTTP](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#streamable-http) with bearer auth. Front with Tailscale Funnel or Cloudflare Tunnel for HTTPS — works with claude.ai web, ChatGPT custom GPT, Cursor HTTP mode, mobile MCP clients. See [docs/http-transport.md](./docs/http-transport.md).
-**Q: Is my data sent anywhere?**
-Only on `enquire-mcp install-model` (downloads ONNX weights from HuggingFace). The serve mode itself never makes outbound HTTP. Embeddings run on CPU locally.
+---
-**Q: How is this different from Smart Connections?**
-Smart Connections is a paid Obsidian plugin that does ML embeddings inside Obsidian. enquire-mcp is a standalone MCP server: free, MCP-native (works with Claude / Cursor / Codex / any agent), and adds hybrid retrieval (BM25 + TF-IDF + embeddings via RRF) for higher recall than embeddings alone.
+## 🚀 Releases
-**Q: Performance on large vaults?**
-Cold-build of FTS5 on a 1k-note vault: ~5s. Warm BM25 top-10: sub-100ms. Embedding build: ~30ms/chunk on M1 (8s for 256 chunks, ~8min for 8k chunks). Hybrid query latency: <200ms typical.
+`v2.0.0` (stable) · `v2.5.0` (5-sprint roadmap consolidated) · `v2.6.0` (remote MCP) · `v2.7.0` (PDF read tools) · `v2.8.0` (PDFs blended into hybrid search) · `v2.9.0` (BGE cross-encoder reranking)
-**Q: Does it support languages other than English?**
-Yes. Default embedding model is `paraphrase-multilingual-MiniLM-L12-v2` (50+ languages). Validated end-to-end on Russian + English bilingual vaults. CJK requires the upcoming Intl.Segmenter pre-pass (v2.1 backlog).
+Channel: `npm install @oomkapwn/enquire-mcp` → latest stable. Full changelog: [CHANGELOG.md](./CHANGELOG.md).
 ---
-## Contributing
+## 🤝 Contributing
-Issues and PRs welcome. Read [CONTRIBUTING.md](./CONTRIBUTING.md). All changes go through PR review per branch protection ruleset.
+```bash
+git clone https://github.com/oomkapwn/enquire-mcp.git
+cd enquire-mcp && npm install
+npm test       # full suite (502 tests, ~5s)
+npm run lint   # zero warnings
+npm run build  # tsc → dist/
+```
-## License & credits
+Issues, PRs, and ideas welcome. Branch protection requires PR review on `main`.
-[MIT](./LICENSE). Built by Alex — [GitHub `@oomkapwn`](https://github.com/oomkapwn) · [X `@OomkaBear`](https://x.com/OomkaBear).
+---
-Named after [ENQUIRE](https://en.wikipedia.org/wiki/ENQUIRE) — Tim Berners-Lee's 1980 hypertext prototype of the World Wide Web.
+## 📜 License & credits
-> **Not affiliated with Obsidian.md.** Obsidian and the Obsidian logo are trademarks of Dynalist Inc. enquire-mcp is an independent open-source project that reads Obsidian-format vaults.
+MIT. Built by [Alex (@OomkaBear)](https://github.com/oomkapwn). Named after [Tim Berners-Lee's 1980 prototype of the WWW](https://en.wikipedia.org/wiki/ENQUIRE) — the original hypertext system, before the web. The original spec was that you could ask the system anything; this brings that to your vault.