@oomkapwn/enquire-mcp 3.5.4 → 3.5.6

package/CHANGELOG.md

@@ -2,6 +2,80 @@
 
 All notable changes to this project will be documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.5.6] — 2026-05-10
+
+**Patch — root-cause fixes for two issue classes surfaced by external reviews.** Closes the systemic gaps, not just the individual symptoms. No behavior changes.
+
+### Root-cause #1 — cold-import test flakes (class fix)
+
+v3.5.5 fixed ONE symptom (`tests/doctor.test.ts` timeout) with a per-test 30s timeout. Audit of the wider codebase found the same pattern in **3 more test files**:
+
+- `pdf.test.ts` (~20 tests) — `extractPdfText()` triggers `pdfjs-dist` cold load
+- `ocr.test.ts` (`isOcrAvailable` + `extractPdfWithOcr`) — loads tesseract.js + canvas + pdfjs (combined ~30 MB WASM/native)
+- `hnsw.test.ts` (24 tests) — `buildHnsw()` loads hnswlib-node native binding
+
+Per-test timeout bumps don't address the class. The systemic fix: **`tests/setup.ts`** that warms every native / heavy optional dep ONCE per Vitest process via `setupFiles` in `vitest.config.ts`. The first test process startup pays the cumulative cold-import cost (~10 s in our environment); every subsequent test in every file sees a fully cached module.
+
+Cost: +10 s to the first process; saved ad-hoc timeout bumps for every future test that touches optional deps. Net win.
+
+### Root-cause #2 — security-doc drift (selective fix)
+
+v3.5.5 fixed ONE symptom (SECURITY.md stale on stateful HTTP from v2.14.0). Audit of doc-vs-feature drift found that v3.2.0+ tools (`obsidian_list_bases` / `read_base` / `query_base` / `get_communities` / `hyde_search`) introduced **new attack surfaces** that SECURITY.md didn't cover:
+
+- `.base` file parsing — malformed YAML, DSL predicate ReDoS risk, path traversal via base file path, filter-against-private-paths concern
+- GraphRAG-light community detection — vault-wide read amplification, memory bounds on dense vaults, Louvain compute cap (50 passes), no LLM call surface
+
+Added two new SECURITY.md sections covering these. Out-of-scope items called out explicitly (formula evaluation deferred; adversarial graph construction = user-owns-vault non-threat).
+
+### Known remaining gap (tracking for v3.6+)
+
+`docs/api.md` is missing entries for 12 tools (the 5 v3.x ones plus 7 that pre-date v3 — `chat_thread_read/append`, `context_pack`, `frontmatter_get/search/set`, `list_pdfs`, `read_pdf`). This drift pre-dates v3.5 and would need a substantial backfill + a new docs-consistency invariant to prevent recurrence. Tracked for the v3.6 docs-completion sprint, not blocking this release. The existing `tests/docs-consistency.test.ts` invariant catches drift in **README** only (where every tool IS mentioned).
+
+### Tests
+
+664 unit tests pass (unchanged). Setup time +10 s once per process, individual tests faster (no cold-load cost).
+
+### Migration
+
+**No-op for default users.** Pure test stability + docs additions.
+
+## [3.5.5] — 2026-05-10
+
+**Patch — fixes from external review #2.** Two issues: test flakiness on cold I/O + a documentation drift between SECURITY.md and the v2.14.0 stateful-HTTP code path. No behavior changes.
+
+### Fixed — `tests/doctor.test.ts` cold-import flake
+
+The first test in `runDoctor (v2.11.0)` calls `runDoctor()`, which probes optional deps via `await import(...)` — including `@huggingface/transformers` (~100 MB + ONNX runtime). On a slow disk / cold module cache, the first import in a fresh Vitest process can take 5-30 seconds, tripping the default 5 s test timeout. Subsequent tests in the same describe block reuse Node's module cache and finish in <100 ms each — the flake only ever hits the first test.
+
+Fix: per-test `30_000 ms` timeout on the offending case, with a comment explaining why. Lighter than mocking the import (which would hide real "transformers actually loads" regressions), heavier than wishing-it-away.
+
+### Fixed — SECURITY.md: stateful-HTTP posture (v2.14.0+) now documented
+
+Pre-v3.5.5 `SECURITY.md` said:
+
+> v2.6.0 ships **stateless** mode only [...] Stateful sessions with `Mcp-Session-Id` + persistent SSE streams are tracked for v2.7+ if there's demand.
+
+That document hadn't been updated since v2.6 — but v2.14.0 (2026-05-09) shipped the stateful path with `Mcp-Session-Id` + persistent SSE via `GET /mcp` + `DELETE /mcp` termination + idle eviction + max-sessions cap. The security posture was real in code but missing from SECURITY.md, leaving consumers without the threat model for a path they could already enable via `--stateful`.
+
+Fix: rewrote the `### Stateful sessions` section to cover the actual v2.14 surface:
+- Off by default (`--stateful` is opt-in)
+- Session ID = 128-bit random hex, allocated at `initialize`
+- Max concurrent sessions cap via `--max-sessions <n>` (default 100); overflow → 503 + Retry-After
+- Idle eviction via `--session-idle-timeout-ms <n>` (default 30 min)
+- Explicit termination via `DELETE /mcp` (idempotent — 404 on unknown ID)
+- Persistent SSE via `GET /mcp` with the same auth + rate-limit predicates
+- Privacy filter parity with stateless
+- Graceful shutdown drains the session map
+- Out-of-scope: session takeover on bearer-token leak, cross-session shared-state leakage (single-tenant tool by design)
+
+### Tests
+
+664 unit tests pass (unchanged count). The flake is now ABSENT on slow-I/O machines — the test gets up to 30 s headroom for the cold transformers.js import.
+
+### Migration
+
+**No-op for default users.** Pure test stability + docs sync.
+
 ## [3.5.4] — 2026-05-10
 
 **Patch — quick wins from external review.** Two single-line config tightenings, no behavior changes.

package/SECURITY.md

@@ -190,9 +190,23 @@ The `serve-http` subcommand (added v2.6.0) exposes the same MCP server over [Str
 - **Multi-tenant cross-token attacks.** This is a single-tenant tool. A small team should run **one process per user** (e.g. systemd template unit) and not share tokens. We don't do tenant isolation in-process beyond the per-token rate-limit.
 - **OAuth.** No OAuth flow, no token minting, no refresh logic. Static long-lived bearer is by design — generated with `enquire-mcp gen-token`, rotated manually. OAuth is tracked for v2.7+ if a user explicitly needs it.
 
-### Stateful sessions
+### Stateful sessions (v2.14.0+)
 
-v2.6.0 ships **stateless** mode only (`sessionIdGenerator: undefined`) — fresh `McpServer` per request over the SHARED vault + FTS5 + embedding handles. No long-lived session map; no SSE persistent streams. This is the right default for our short-running tools (search, read, frontmatter ops) and avoids the persistence-aware shutdown complexity. Stateful sessions with `Mcp-Session-Id` + persistent SSE streams are tracked for v2.7+ if there's demand.
+v2.6.0 initially shipped **stateless** mode only (fresh `McpServer` per request over the SHARED vault + FTS5 + embedding handles). v2.14.0 added an **opt-in stateful** mode via `--stateful` for clients that need persistent state across requests (notably ChatGPT custom GPT actions). Stateful posture:
+
+- **Off by default.** `--stateful` is explicit opt-in; stateless remains the default for minimum attack surface. Short-running tools (search, read, frontmatter ops) work fine stateless and don't need the persistence-aware shutdown complexity.
+- **Session ID generation.** `Mcp-Session-Id` is `randomBytes(16).toString("hex")` — 128 bits, allocated at `initialize` time, returned in response header. Clients must echo it on subsequent requests; unknown IDs return 404 (no info leak about whether the ID was ever valid).
+- **Per-token + per-session rate limit.** The bearer-token rate limit still applies. Sessions are anchored to a bearer token; one token holding multiple sessions is allowed, but each session is bound to the token that initialized it.
+- **Max concurrent sessions cap.** `--max-sessions <n>` (default **100**). New sessions beyond the cap return **503 + `Retry-After`**. Prevents memory exhaustion via session-spam.
+- **Idle eviction.** `--session-idle-timeout-ms <n>` (default **1,800,000 ms = 30 min**). A periodic sweep terminates transports idle longer than the timeout. Memory bounded.
+- **Explicit termination.** `DELETE /mcp` with a valid `Mcp-Session-Id` tears down the transport immediately. Idempotent — repeat DELETE on a no-longer-existing ID returns 404, not 500.
+- **GET /mcp for persistent SSE.** A `GET /mcp` with a valid `Mcp-Session-Id` opens a server-sent-events stream for server-initiated notifications. Same auth + rate-limit predicates as POST. Stream closes on DELETE or idle eviction.
+- **Privacy filter parity.** `--exclude-glob` / `--read-paths` apply identically to stateful and stateless paths. There is no codepath where a stateful session bypasses the privacy filter.
+- **Graceful shutdown.** SIGINT / SIGTERM / `beforeExit` trigger session-map drain — all transports are closed before the process exits. No leaked SSE streams.
+
+Out of scope (stateful mode specifically):
+- **Session takeover** if a bearer token leaks. The session-id is in a response header, not a secret — possession of the bearer token is sufficient to initialize new sessions OR (if the attacker captured a previous `Mcp-Session-Id`) re-attach to an existing one. Treat the bearer token as the trust boundary; don't share it.
+- **Cross-session leakage.** Each session has its own `McpServer` instance but shares the vault + FTS5 + embedding handles. A misbehaving tool that mutates shared state could affect other sessions. Write tools (`--enable-write`) are still atomic per-file; read tools don't mutate. We don't run per-session sandboxing — single-tenant tool, see "Multi-tenant cross-token attacks" above.
 
 ### Observability
 
@@ -200,3 +214,35 @@ v2.6.0 ships **stateless** mode only (`sessionIdGenerator: undefined`) — fresh
 - Transport errors written to stderr with no token / no credential leakage.
 - `/health` endpoint (`GET /health → 200 ok`) is **unauthenticated** and exists specifically for tunnel/uptime monitors. It returns the literal string `ok` — no version info, no vault path, no operational metadata. Health probes can't be used to fingerprint the deployment.
 - `OPTIONS` preflight requests are unauthenticated (per CORS spec) but only emit CORS headers when the request's `Origin` is in the allowlist.
+
+## Obsidian Bases (`.base`) execution (v3.2.0+): parser + DSL posture
+
+`obsidian_list_bases` / `obsidian_read_base` / `obsidian_query_base` parse user-authored YAML files and evaluate a filter-DSL subset against the vault's markdown notes. New attack surfaces vs the markdown-only v1.x baseline:
+
+**Threat model:**
+- **Malformed YAML / YAML bombs.** Parsed via `js-yaml`'s `SAFE_SCHEMA` (the same engine and schema `gray-matter` uses for frontmatter). No anchor-expansion, no `!!js/function` tag, no code execution path. A YAML bomb (deeply nested anchors) is rejected at parse time before our zod schema validation runs.
+- **ReDoS in DSL predicate regexes.** Each predicate is matched against a small set of fixed, non-backtracking regexes (`^tag\s*(==|!=)\s*..." literal "$"` style). No user-controlled regex compilation. Predicate strings that don't match any pattern fall into `unevaluated_predicates` and are treated as `true` (permissive) — they don't cause regex evaluation against user content.
+- **Path traversal via `.base` file path.** `obsidian_read_base({ path })` and `obsidian_query_base({ path })` resolve through `vault.readBinaryFile` → `vault.resolveSafePath` — the same realpath + `--exclude-glob` + `--read-paths` chain as `readNote`. Symlinks-out-of-vault rejected; excluded paths refuse to load.
+- **Filter against private paths.** `queryBase`'s vault walk goes through `vault.listFilesByExtension(".md", folder)`, which respects `--exclude-glob` / `--read-paths`. A `.base` filter cannot surface content that the privacy filter would block from `readNote`.
+- **Outbound wikilink-set materialization.** v3.5.0 added `linksTo()` predicate evaluation; the per-note outbound set is computed from `extractWikilinks(body)` — same parser as the read-only `obsidian_get_outbound_links` tool. No new file reads or path resolution beyond what's already exposed.
+
+**Out of scope (deferred):**
+- **Formula evaluation** (`formulas:` section). Our DSL is filters-only; formulas are surfaced as metadata via `obsidian_read_base` but never evaluated. Until a formula evaluator ships (separate sprint), there is no code execution path through `.base` formulas — they're inert strings.
+- **Summaries / aggregations.** Same — surfaced as metadata, not evaluated. No SQL-injection-class concern since there's no executable backend.
+- **Date arithmetic** (`inDate` etc). Falls into `unevaluated_predicates`, permissive. No date-parser surface yet.
+
+When formula evaluation lands, this section gets an "Expression engine sandbox" subsection covering the threat model for that.
+
+## GraphRAG-light: wikilink community detection (v3.4.0+): graph-build posture
+
+`obsidian_get_communities` builds an in-memory undirected graph from every resolved `[[wikilink]]` in the vault, then runs single-phase Louvain modularity optimization. New attack surfaces:
+
+**Threat model:**
+- **Vault-wide read amplification.** Where a single `obsidian_read_note` call reads one file, `obsidian_get_communities` reads ALL markdown files in the vault (or under `--folder` if specified) to extract their wikilinks. Privacy filter is honored: excluded/disallowed paths are never in `listFilesByExtension(".md")`'s output, so they don't contribute nodes or edges to the graph. The graph is also bounded by the vault: nodes are vault-relative paths only, no off-vault leakage.
+- **Memory bounds on huge vaults.** The adjacency map is `Map<path, Map<path, weight>>` — O(|V| + |E|). For a vault with 50K notes and average degree 10, that's ~250 KB of node strings plus ~3 MB of edge weights — comfortably bounded. Pathologically dense vaults (every note links every other note) hit O(|V|²) memory; this is acceptable since the dense case is implausible and the user controls the vault.
+- **Modularity-optimization compute bounds.** Louvain is capped at 50 passes per `detectCommunities` call; each pass is O(|E|). On a 50K-node vault with 500K edges this is ~25M ops × 50 = ~1.3B ops, ~5-10 s wall time. The tool is read-only and per-call (we don't cache), so cost is paid only when the agent explicitly invokes the tool. No persistent background work.
+- **No LLM call surface.** The server stays LLM-free for this tool — the agent is expected to summarize communities itself with the member list we return. There is no code path where `obsidian_get_communities` makes outbound HTTP or invokes an embedding model.
+
+**Out of scope:**
+- **Multi-phase Louvain refinement.** Single-phase is "good enough up to ~50K notes" by design; the trade-off is documented in `src/communities.ts`. Vault > 50K notes may see lower modularity quality, but never an unbounded compute spike (the 50-pass cap holds).
+- **Adversarial graph construction.** A vault author could construct a graph designed to be slow to partition (e.g. specific dense bipartite structures). Acceptable — the user owns the vault; there is no "attacker writes a vault" threat model.

package/dist/index.js

@@ -12,7 +12,7 @@ import { chunkContent, defaultIndexFile, FtsIndex } from "./fts5.js";
 import { appendToNote, archiveNote, chatThreadAppend, chatThreadRead, contextPack, createNote, dataviewQuery, embeddingsSearch, findPath, findSimilar, frontmatterGet, frontmatterSearch, frontmatterSet, getBacklinks, getNoteNeighbors, getOpenQuestions, getOutboundLinks, getRecentEdits, getUnresolvedWikilinks, getVaultStats, lintWiki, listCanvases, listNotes, listPdfs, listTags, ocrPdf, openInUi, paperAudit, readCanvas, readNote, readPdf, renameNote, replaceInNotes, resolveWikilink, searchHybrid, searchText, semanticSearch, validateNoteProposal } from "./tools.js";
 import { Vault } from "./vault.js";
 import { VaultWatcher } from "./watcher.js";
-const VERSION = "3.5.4";
+const VERSION = "3.5.6";
 /** Default location for the persistent embedding index, alongside .fts5.db. */
 function embedDbPath(vaultRoot) {
     // Match the FTS5 location convention by stripping the .fts5.db extension

package/package.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://json.schemastore.org/package.json",
   "name": "@oomkapwn/enquire-mcp",
-  "version": "3.5.4",
+  "version": "3.5.6",
   "description": "The most advanced MCP server for Obsidian vaults. Hybrid retrieval (BM25 + TF-IDF + multilingual ML embeddings, RRF-fused) with BGE cross-encoder reranking, HNSW vector index, int8 quantization, late-chunking, HyDE-augmented retrieval, sub-question decomposition, PDFs (with OCR), Bases (.base query execution, standalone — no Obsidian needed), GraphRAG-light (Louvain wikilink community detection), wikilinks, backlinks, Dataview, frontmatter, canvas. 44 tools, 19 MCP prompts, 5 cross-encoder reranker models, 664 tests, SLSA-3, semver-bound. Works with Claude Code, Claude Desktop, Cursor, ChatGPT custom GPT, Codex, and any MCP client.",
   "type": "module",
   "bin": {