@oomkapwn/enquire-mcp 3.10.0-rc.8 → 3.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +1653 -0
- package/README.md +63 -43
- package/README.zh.md +206 -0
- package/SECURITY.md +6 -5
- package/STABILITY.md +3 -3
- package/dist/bases.d.ts.map +1 -1
- package/dist/bases.js +58 -30
- package/dist/bases.js.map +1 -1
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +147 -7
- package/dist/cli.js.map +1 -1
- package/dist/communities.d.ts +13 -0
- package/dist/communities.d.ts.map +1 -1
- package/dist/communities.js +29 -11
- package/dist/communities.js.map +1 -1
- package/dist/doctor.d.ts +18 -1
- package/dist/doctor.d.ts.map +1 -1
- package/dist/doctor.js +16 -7
- package/dist/doctor.js.map +1 -1
- package/dist/dql.d.ts +34 -7
- package/dist/dql.d.ts.map +1 -1
- package/dist/dql.js +72 -70
- package/dist/dql.js.map +1 -1
- package/dist/embed-db.d.ts +14 -0
- package/dist/embed-db.d.ts.map +1 -1
- package/dist/embed-db.js +64 -17
- package/dist/embed-db.js.map +1 -1
- package/dist/embed-pipeline.d.ts.map +1 -1
- package/dist/embed-pipeline.js +8 -2
- package/dist/embed-pipeline.js.map +1 -1
- package/dist/embeddings.d.ts +43 -24
- package/dist/embeddings.d.ts.map +1 -1
- package/dist/embeddings.js +158 -9
- package/dist/embeddings.js.map +1 -1
- package/dist/eval.d.ts +47 -0
- package/dist/eval.d.ts.map +1 -1
- package/dist/eval.js +66 -11
- package/dist/eval.js.map +1 -1
- package/dist/frontmatter.d.ts +37 -0
- package/dist/frontmatter.d.ts.map +1 -0
- package/dist/frontmatter.js +191 -0
- package/dist/frontmatter.js.map +1 -0
- package/dist/fts5.d.ts +15 -0
- package/dist/fts5.d.ts.map +1 -1
- package/dist/fts5.js +86 -11
- package/dist/fts5.js.map +1 -1
- package/dist/hnsw.d.ts.map +1 -1
- package/dist/hnsw.js +21 -6
- package/dist/hnsw.js.map +1 -1
- package/dist/http-transport.d.ts +39 -0
- package/dist/http-transport.d.ts.map +1 -1
- package/dist/http-transport.js +228 -137
- package/dist/http-transport.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/name-fold.d.ts +33 -0
- package/dist/name-fold.d.ts.map +1 -0
- package/dist/name-fold.js +35 -0
- package/dist/name-fold.js.map +1 -0
- package/dist/ocr.d.ts +7 -3
- package/dist/ocr.d.ts.map +1 -1
- package/dist/ocr.js +73 -48
- package/dist/ocr.js.map +1 -1
- package/dist/optional-dep.d.ts +9 -0
- package/dist/optional-dep.d.ts.map +1 -0
- package/dist/optional-dep.js +27 -0
- package/dist/optional-dep.js.map +1 -0
- package/dist/parser.d.ts +7 -1
- package/dist/parser.d.ts.map +1 -1
- package/dist/parser.js +14 -3
- package/dist/parser.js.map +1 -1
- package/dist/pdf.d.ts.map +1 -1
- package/dist/pdf.js +94 -83
- package/dist/pdf.js.map +1 -1
- package/dist/periodic.js +8 -0
- package/dist/periodic.js.map +1 -1
- package/dist/retrieval-opts.d.ts +34 -0
- package/dist/retrieval-opts.d.ts.map +1 -0
- package/dist/retrieval-opts.js +43 -0
- package/dist/retrieval-opts.js.map +1 -0
- package/dist/server.d.ts +6 -3
- package/dist/server.d.ts.map +1 -1
- package/dist/server.js +63 -61
- package/dist/server.js.map +1 -1
- package/dist/shutdown.d.ts +41 -0
- package/dist/shutdown.d.ts.map +1 -0
- package/dist/shutdown.js +60 -0
- package/dist/shutdown.js.map +1 -0
- package/dist/tool-registry.d.ts.map +1 -1
- package/dist/tool-registry.js +26 -10
- package/dist/tool-registry.js.map +1 -1
- package/dist/tools/media.d.ts.map +1 -1
- package/dist/tools/media.js +22 -3
- package/dist/tools/media.js.map +1 -1
- package/dist/tools/meta.d.ts +42 -16
- package/dist/tools/meta.d.ts.map +1 -1
- package/dist/tools/meta.js +392 -48
- package/dist/tools/meta.js.map +1 -1
- package/dist/tools/read.d.ts +11 -2
- package/dist/tools/read.d.ts.map +1 -1
- package/dist/tools/read.js +50 -24
- package/dist/tools/read.js.map +1 -1
- package/dist/tools/search.d.ts +76 -2
- package/dist/tools/search.d.ts.map +1 -1
- package/dist/tools/search.js +153 -13
- package/dist/tools/search.js.map +1 -1
- package/dist/tools/write.d.ts +12 -5
- package/dist/tools/write.d.ts.map +1 -1
- package/dist/tools/write.js +71 -15
- package/dist/tools/write.js.map +1 -1
- package/dist/vault.d.ts +52 -7
- package/dist/vault.d.ts.map +1 -1
- package/dist/vault.js +274 -108
- package/dist/vault.js.map +1 -1
- package/dist/watcher.d.ts.map +1 -1
- package/dist/watcher.js +52 -14
- package/dist/watcher.js.map +1 -1
- package/dist/wildcard-match.d.ts +81 -0
- package/dist/wildcard-match.d.ts.map +1 -0
- package/dist/wildcard-match.js +183 -0
- package/dist/wildcard-match.js.map +1 -0
- package/docs/COMPARISON.md +18 -4
- package/docs/QUICKSTART.md +4 -4
- package/docs/api.md +19 -7
- package/docs/benchmarks.md +1 -1
- package/examples/README.md +1 -0
- package/examples/tweetclaw-openclaw.md +115 -0
- package/package.json +17 -12
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,1659 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project will be documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
4
4
|
|
|
5
|
+
## [3.10.0] — 2026-06-22
|
|
6
|
+
|
|
7
|
+
> **TL;DR:** **v3.10.0 STABLE — promoted `@rc` → `@latest` after 78 RCs.** The forgetting-aware retrieval line + a deep multi-round security/correctness hardening cascade. New: **`obsidian_stale_notes`** (45th tool), `age_days`/`stale` freshness signals on every hit, opt-in recency re-ranking (`--recency-weight`, default off), and **frontmatter-aware `obsidian_search`** (`filter_frontmatter`). Plus the gray-matter→js-yaml@4 frontmatter migration, pdfjs-dist 5→6, and the full closure of the ReDoS / abs-path-leak / NFC-name-resolution / reserve-before-try / resource-DoS-amplifier classes (1 HIGH + many MED/LOW across rc.1→rc.78). **No API breaks** (additive minor; the v3.x tool/argument surface is unchanged). **45 tools · 19 MCP prompts · 1311 unit tests · 11 required + advisory CI gates.**
|
|
8
|
+
|
|
9
|
+
**Promotes the entire v3.10.0-rc.1 → rc.78 pre-release line to `@latest`** (per-RC detail in the entries below). Maintainer call on the ≥2-independent-external-auditor promotion gate (the rc.32 deep-audit + rc.34/rc.35 from-scratch Mavis passes + the round-1/2/3 + state-driven internal multi-lens Workflow audits constitute the evidence base; the fresh-on-HEAD external pass was waived as for v3.9.0). CI publishes `@latest` with signed npm build provenance (SLSA Build L2) and syncs the canonical MCP Registry via OIDC.
|
|
10
|
+
|
|
11
|
+
### Highlights (delivered across the rc line)
|
|
12
|
+
|
|
13
|
+
- **Forgetting-aware staleness** (rc.1→rc.10): live-mtime `age_days` + `stale` on every retrieval hit; the `obsidian_stale_notes` tool; opt-in recency re-ranking (provable no-op at weight 0); frontmatter-aware `obsidian_search` (`filter_frontmatter`).
|
|
14
|
+
- **Frontmatter engine migration** (rc.53→rc.56): dropped gray-matter → in-repo `src/frontmatter.ts` on js-yaml@4, resolving GHSA-h67p-54hq-rp68 at the root (allowlist empty); scalar-resolution contract documented + pinned.
|
|
15
|
+
- **Dependency majors** (rc.52): pdfjs-dist 5→6; protobufjs/hono advisory overrides; scoped `check-audit.mjs` gate.
|
|
16
|
+
- **Security/correctness class closures** (rc.16→rc.78, multi-round adversarial Workflow audits + post-merge re-sweeps): ReDoS (non-backtracking DP matcher for DQL `LIKE` + globs, worker sink-bound for `obsidian_open_questions`); abs-path-leak (Vault `sanitizeFsError` at the source + inventory invariant); NFC name-resolution (`src/name-fold.ts` + inventory invariant); reserve-before-try handle leaks (self-cleaning `open()` + pdfjs try/finally); resource-DoS amplifiers (read_canvas/validateNoteProposal O(1) indices, caps); HNSW right-to-erasure; truncate-before-sort in the list tools.
|
|
17
|
+
- **Discoverability** (rc.13→rc.31): official MCP Registry (auto-published on stable via OIDC), Docker/Glama introspection, Schema.org JSON-LD, bilingual `README.zh.md`, `llms.txt` agent contract.
|
|
18
|
+
|
|
19
|
+
### Quality bar
|
|
20
|
+
|
|
21
|
+
All 11 required CI gates green; 1311 unit tests; lint pristine (biome 2.5.0); coverage thresholds met; OIA (12 checks) + scope-completeness + docs-consistency + version-consistency clean; `npm audit` allowlist empty.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## [3.10.0-rc.78] — 2026-06-22
|
|
26
|
+
|
|
27
|
+
> **TL;DR:** **Config hygiene — `biome.json` migrated to 2.5.0.** The devDep is `@biomejs/biome@^2.5.0` (installed 2.5.0) but `biome.json` still pinned `$schema` 2.4.16 and used the deprecated `recommended: true` linter field — both non-blocking lint `infos` that slipped through several RCs (the α-class stale-version claim, in a tooling config). `biome migrate` updated the schema + renamed `recommended: true` → `preset: "recommended"` (same rule set; no repo-wide cascade), and a pre-existing `useTemplate` nit was cleared so lint output is now pristine. Config + 1-line test tidy only; no runtime change. **1311 source tests unchanged.**
|
|
28
|
+
|
|
29
|
+
**Pre-release (v3.10 line) — config hygiene (biome 2.5.0 migration; zero runtime change).**
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- **`biome.json` migrated to the installed biome 2.5.0** (`biome.json`). The `@biomejs/biome` devDep is `^2.5.0` (resolves to 2.5.0), but `biome.json` declared `"$schema": ".../2.4.16/schema.json"` AND used `"recommended": true` in `linter.rules` — which biome 2.5.0 deprecated (to be removed in the next major). Both surfaced as non-blocking `i` / DEPRECATED warnings on every `npm run lint`, so they slipped through rc.71→rc.77 as the lint "infos" until a commit (rc.77) touched a file the newer biome reformatted. Ran `biome migrate --write`: `$schema` → `2.5.0`, `recommended: true` → `preset: "recommended"` (a field rename to the same recommended rule set — verified by a full-repo `npm run lint`: 135 files, 0 errors, no format/rule cascade). Also cleared the one remaining lint info — a pre-existing `useTemplate` nit in `tests/docs-consistency.test.ts:67` (`"\`" + p + "\`"` → a template literal) — so the lint output is now 0 findings.
|
|
34
|
+
|
|
35
|
+
### Tests (1311)
|
|
36
|
+
|
|
37
|
+
- Unchanged (config + a 1-line `useTemplate` tidy; no `it()` added/removed). Full suite green; lint pristine.
|
|
38
|
+
|
|
39
|
+
> **Lesson:** a `^`-ranged dev-tool (biome) silently minor-bumped past its config's pinned `$schema`, and the resulting version-mismatch + field-deprecation surfaced only as NON-blocking lint `infos` — so the drift survived several RCs. `biome migrate` is the clean close; the broader watch is to keep a `^`-ranged tool's config in lockstep with the installed version.
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## [3.10.0-rc.77] — 2026-06-22
|
|
44
|
+
|
|
45
|
+
> **TL;DR:** **Full state-driven audit close — LOW: STABILITY.md `obsidian_full_text_search` enabling-flag drift + structural guard.** The packaged semver-contract doc said the FTS tool is opt-in via `--persistent-index` alone, but the code requires `--persistent-index` AND `--diagnostic-search-tools`. Corrected + added a docs-consistency invariant that derives each opt-in/gated tool's flag-set from `TOOL_MANIFEST.gating` and pins STABILITY.md's breakdown headings to it. Docs+test only. **1309 → 1311 source tests. Closes the full state-driven audit (rc.76 MED + rc.77 LOW).**
|
|
46
|
+
|
|
47
|
+
**Pre-release (v3.10 line) — full state-driven audit close (1 LOW; docs+test, zero runtime change).**
|
|
48
|
+
|
|
49
|
+
### Fixed
|
|
50
|
+
|
|
51
|
+
- **LOW — STABILITY.md misstated the enabling flags for `obsidian_full_text_search`** (`STABILITY.md` lines 13 + 19). The packaged stability-contract doc attributed the tool to `--persistent-index` ALONE, but `server.ts:691` registers it under `if (deps.ftsIndex && opts.diagnosticSearchTools)` — i.e. it requires BOTH `--persistent-index` (for `deps.ftsIndex`) AND `--diagnostic-search-tools`. Every other surface is correct (`tool-manifest.ts` gating = `"--persistent-index + --diagnostic-search-tools"`, the tool description, and `docs/api.md` in four places); STABILITY.md was the lone outlier, drifted since v3.5.1. A user who followed STABILITY.md and started `serve --persistent-index` WITHOUT `--diagnostic-search-tools` would not get the tool registered. Same α-class as the rc.22 STABILITY reranker-default drift; LOW because it's a packaged-doc accuracy gap (no security/data impact) and the umbrella `obsidian_search` already exposes BM25/FTS5 with `--persistent-index` alone. **Fix: corrected both lines + closed the untested gap with a structural guard** — the docs-consistency STABILITY invariants pinned tool/prompt COUNTS but nothing pinned the per-flag GATING breakdown prose.
|
|
52
|
+
|
|
53
|
+
### Tests (1311)
|
|
54
|
+
|
|
55
|
+
- +2 (`tests/docs-consistency.test.ts`): a new invariant `stabilityGatingMismatches` (pure fn) that DERIVES each non-`"always"` tool's flag-set from `TOOL_MANIFEST.gating` and asserts STABILITY.md's "opt-in via / gated by `<flags>`" breakdown headings name exactly that set — for every opt-in/gated tool, not just FTS. Plus a NEGATIVE control feeding the exact rc.77-drift string (`--persistent-index` alone) and asserting it's caught, with a POSITIVE control on the corrected string. **1309 → 1311.**
|
|
56
|
+
|
|
57
|
+
> **Lesson:** count-pinning a docs surface (tools/prompts) does not pin its PROSE breakdown (per-flag gating) — a separate claim dimension drifts independently (here since v3.5.1, surviving ~30 audit rounds). Derive the breakdown from the machine-readable source (`TOOL_MANIFEST.gating`) and pin it. **This CLOSES the full state-driven audit (rc.76 MED truncate-before-sort + rc.77 LOW gating drift) — and the entire rc.63→rc.77 cascade (round-3 12-lens audit + 2 post-merge re-sweeps + this full state-driven audit): 1 HIGH + 7 MED + 6 LOW across 15 RCs, every confirmed finding shipped or reasoned-accepted.**
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## [3.10.0-rc.76] — 2026-06-21
|
|
62
|
+
|
|
63
|
+
> **TL;DR:** **Full state-driven audit — MEDIUM: `listPdfs`/`listCanvases`/`listBases` truncated to `limit` in walk order BEFORE sorting by mtime.** A fresh whole-project 6-lens audit (0 CRIT / 0 HIGH / 1 MED / 1 LOW) caught a latent bug the change-driven sweeps missed: all three always-on list tools broke their build loop at `out.length >= limit` over the raw readdir-order entries, then sorted only that already-cut subset — so on a vault with > `limit` (default 100) files of that type the result was an arbitrary, not-newest set, violating the documented "newest first" contract (reproduced: 4 PDFs, limit=2 → the 2 oldest). Fixed by sorting by mtime DESC before truncating at all 3 sites. **1306 → 1309 source tests.**
|
|
64
|
+
|
|
65
|
+
**Pre-release (v3.10 line) — full state-driven audit, MEDIUM code fix (truncate-before-sort).**
|
|
66
|
+
|
|
67
|
+
### Fixed
|
|
68
|
+
|
|
69
|
+
- **MEDIUM — `obsidian_list_pdfs` / `obsidian_list_canvases` / `obsidian_list_bases` returned a not-newest subset on vaults with more than `limit` files** (`src/tools/media.ts` listCanvases + listPdfs, `src/bases.ts` listBases). Each iterated `vault.listFilesByExtension(...)` (which returns filesystem readdir/walk order — NO mtime sort), broke the loop the moment `out.length >= limit`, and only THEN sorted the already-truncated subset by mtime desc. So the `limit` truncation happened in walk order and the mtime sort applied only within that arbitrary subset — the response was NOT the `limit` most-recently-modified files, contradicting each tool's documented "newest first" / "sorted by mtime desc" contract. Empirically reproduced (4 PDFs `a_old < b_mid < y_new < z_newest`, limit=2 → `[b_mid, a_old]` instead of `[z_newest, y_new]`). The correct sort-THEN-truncate pattern is what `read.ts` listNotes/getRecentEdits/staleNotes/listTags already use. **Fix: `all.sort((a, b) => b.mtimeMs - a.mtimeMs)` BEFORE the truncation loop at all three sites**, so the first `limit` walked are genuinely the newest. This list logic had been untouched for many RCs (recent media.ts edits were rc.65 resource-bound + rc.44 canvas-OOM, not this loop), which is exactly why the change-driven sweeps missed it — a full STATE-DRIVEN audit (read every module as-is) is what surfaced it.
|
|
70
|
+
|
|
71
|
+
### Tests (1309)
|
|
72
|
+
|
|
73
|
+
- +3 (`tests/pdf.test.ts`, `tests/bases.test.ts`, `tests/canvas.test.ts` — one per fixed site): create 5 files with explicit ascending `fs.utimes` mtimes and assert `limit: 2` returns the 2 NEWEST (by name), not a walk-order subset. The pre-existing "honors limit" (length-only) + "sorts by mtime" (2 files, under the limit) tests provably never overlapped this >limit case. **Revert-verified: all 3 deterministically FAIL with the pre-sort removed.** **1306 → 1309.**
|
|
74
|
+
|
|
75
|
+
> **Lesson:** a full STATE-DRIVEN audit (read every module as it exists, verify each contract) finds latent bugs in long-untouched code that both change-driven sweeps and behavioral re-sweeps are structurally blind to — this truncate-before-sort defect survived ~30 audit rounds because nothing recently touched those three list loops. The always-on read tools deserve the same sort-then-truncate discipline `read.ts` already had. (rc.77 closes the audit's 1 LOW: a STABILITY.md enabling-flag drift + a structural guard.)
|
|
76
|
+
|
|
77
|
+
---
|
|
78
|
+
|
|
79
|
+
## [3.10.0-rc.75] — 2026-06-21
|
|
80
|
+
|
|
81
|
+
> **TL;DR:** **Post-rc.74 re-sweep close — 1 LOW: DQL `LIKE` Unicode case-fold contract.** The mandatory post-rc.74 re-sweep (6-lens Workflow) returned 0 CRIT / 0 HIGH / 0 MED / 1 LOW (the first re-sweep this session that left no behavioral sibling). The LOW: rc.71's DP matcher folds via `toLowerCase()`, not the pre-rc.71 regex `i`+`u` canonical fold, so DQL `LIKE` UNDER-matches for ~22 exotic codepoints (`µ`/`ſ`/`ς`/Greek-symbol-variants/Cyrillic-small-caps); the glob privacy path is unaffected (case-sensitive). Closed accept-and-document (the rc.54 playbook): documented the contract + pinned it with a case-fold-contract test that proves the divergence via a NEGATIVE control. Docs+test only. **1304 → 1306 source tests.**
|
|
82
|
+
|
|
83
|
+
**Pre-release (v3.10 line) — post-rc.74 re-sweep close (1 LOW; docs+test, zero runtime change).**
|
|
84
|
+
|
|
85
|
+
### Fixed
|
|
86
|
+
|
|
87
|
+
- **LOW — DQL `LIKE` case-folding diverges from the pre-rc.71 `iu` regex for ~22 exotic Unicode codepoints (differential-corpus gap)** (`src/wildcard-match.ts`, `tests/wildcard-match.test.ts`). rc.71 replaced the backtracking `^…$/iu` regex with a non-backtracking DP matcher that case-folds via `String.prototype.toLowerCase()`. ECMAScript `RegExp` `i`+`u` uses CANONICAL case-folding, which agrees with `toLowerCase` for ASCII + ordinary accented letters but diverges for ~22 BMP codepoints whose canonical fold differs (micro-sign `µ` U+00B5, long-s `ſ` U+017F, final-sigma `ς` U+03C2, the Greek symbol variants `ϐϑϕϖϰϱϵ`, the U+1C80–U+1C88 Cyrillic small-caps block, `ẛ` U+1E9B, `ι` U+1FBE). For these, `field LIKE "µ"` no longer matches a value of `"Μ"`. **Direction is UNDER-match** (fewer rows) → no privacy over-exposure, no DoS, no crash; and the **privacy glob filter is unaffected** (`compileGlob` is case-SENSITIVE and never folds — verified byte-faithful over an 80,180-pair old-regex-vs-new differential). The rc.71 differential test corpus was ASCII + `café` only, so its "0 mismatches on every corpus pair" assertion was in reality scoped to ASCII — it structurally could not produce a folding-divergent codepoint (the rc.54 lesson: a differential corpus is only as strong as the shapes it can produce). **Fix (accept-and-document, the rc.54 playbook — a custom Unicode-canonical folder is its own bug surface, not worth it for these characters): documented the divergence as a deliberate CONTRACT in the `matchWildcardTokens` header (naming the codepoint classes + the under-match direction + the case-sensitive-glob carve-out) and re-scoped the differential describe's comment to "ASCII + ordinary-accented".**
|
|
88
|
+
|
|
89
|
+
### Tests (1306)
|
|
90
|
+
|
|
91
|
+
- +2 source `it()` (`tests/wildcard-match.test.ts`, new "LIKE Unicode case-fold contract" describe): a POSITIVE control (ASCII `FOO`/`foo` + ordinary accented `É`/`é` still fold) + a data-driven contract test over `µ`/`ſ`/`ς` that pins the DP `toLowerCase` semantics (these do NOT match their canonical-fold partner) AND a NEGATIVE control asserting the inlined pre-rc.71 `oldLikeToRegex` WOULD have matched — proving the divergence is real and the corpus row is non-vacuous (the dimension the ASCII differential corpus is blind to). **1304 → 1306.**
|
|
92
|
+
|
|
93
|
+
> **Lesson:** the rc.54 differential-corpus principle applied to itself — when a migration's differential test asserts "byte-identical on every corpus pair", that claim is only true for the DIMENSIONS the corpus can produce; a folding/scalar/encoding divergence needs its own contract test (pinning the new semantics + a NEGATIVE control vs the old) or the "every pair" message is an overclaim. **This CLOSES the round-3 fresh 12-lens audit + both its post-merge re-sweeps (rc.63→rc.75): 1 HIGH + 6 MED + 5 LOW across 13 RCs, every confirmed finding shipped or reasoned-accepted.**
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## [3.10.0-rc.74] — 2026-06-19
|
|
98
|
+
|
|
99
|
+
> **TL;DR:** **Post-rc.70 re-sweep, batch 4/4 — CLOSES the re-sweep — pdfjs document/loadingTask reserve-before-try leak in BOTH `extractPdfText` + `extractPdfWithOcr` (rc.70 sibling).** Both pdfjs callers acquired `doc = await loadingTask.promise` before guards that throw post-acquisition (`pdf.ts` had NO try/finally at all; `ocr.ts` had the range/maxPages guards outside its try), so a crafted `obsidian_read_pdf` / `obsidian_ocr_pdf` over-span request leaked a pdfjs document + worker port per call on serve-http. Fixed by wrapping the full lifecycle in a try/finally that always destroys. **1302 → 1304 source tests. Closes the post-rc.70 re-sweep (6 confirmed across rc.71→rc.74).**
|
|
100
|
+
|
|
101
|
+
**Pre-release (v3.10 line) — post-rc.70 re-sweep, batch 4/4 (reserve-before-try pdfjs-leak class); closes the re-sweep.**
|
|
102
|
+
|
|
103
|
+
### Fixed
|
|
104
|
+
|
|
105
|
+
- **MED/LOW resource leak — pdfjs `document` + `loadingTask` leaked on a post-acquisition throw in both PDF callers** (`src/pdf.ts`, `src/ocr.ts`). Both `extractPdfText` and `extractPdfWithOcr` did `doc = await loadingTask.promise` BEFORE guards that throw: **(pdf.ts)** the page-range validation + the `requestedSpan > maxPages` guard threw, and the `doc.cleanup()` + `loadingTask.destroy()` calls were PLAIN TRAILING CODE — there was no `try/finally` at all, so any throw (or any error in the page loop / metadata block) leaked the document. **(ocr.ts)** `resolveOcrPageRange` + the maxPages guard sat OUTSIDE the existing `try`, whose `finally` only covered the page loop, so those throws leaked `doc`/`loadingTask` (the worker isn't created yet). Reachable via the always-registered, read-only `obsidian_read_pdf` (`pages:[1,600]` on a ≥600-page PDF passes zod `600≥1` then exceeds `DEFAULT_PDF_MAX_PAGES=500`) and `obsidian_ocr_pdf` (`pages:[1,250]` exceeds `DEFAULT_OCR_MAX_PAGES=200`, or a post-clamp-inverted range); each crafted call leaked one pdfjs document + worker port on serve-http, accumulating over the serve lifetime. **Fix (per the rc.70 self-cleaning lesson): wrap the FULL lifecycle — from `doc = await loadingTask.promise` through metadata extraction — in a `try` whose `finally` always runs guarded `await doc.cleanup().catch(()=>{})` + `await loadingTask.destroy().catch(()=>{})`.** `extractPdfText` gains the finally it never had; `extractPdfWithOcr` moves `resolveOcrPageRange` + the maxPages guard + worker creation inside the try, declares `worker` before it, and the finally does `if (worker) await worker.terminate().catch(…)` (the worker is undefined when a pre-worker guard throws) then destroys doc/loadingTask — each guarded so a cleanup error never masks the original throw.
|
|
106
|
+
|
|
107
|
+
### Tests (1304)
|
|
108
|
+
|
|
109
|
+
- +1 (`tests/pdf.test.ts`, CI-running): `extractPdfText` survives 30× post-acquisition throws (maxPages + inverted-range) on a real `makePdf` fixture, then a NORMAL extraction still succeeds — a leaked doc/worker would have exhausted handles or hung by then (the behavioral proof the finally releases each).
|
|
110
|
+
- +1 (`tests/ocr.test.ts`, deps-gated): `extractPdfWithOcr` cleanup-on-throw, probing the maxPages guard once and skipping VISIBLY (`ctx.skip()`, not a silent return) when lang packs are absent (CI), since `assertOcrLangsInstalled` gates before `getDocument`. **1302 → 1304.**
|
|
111
|
+
|
|
112
|
+
> **Lesson:** the rc.70 reserve-before-try fix made the SQLite `open()` primitive self-cleaning but did NOT enumerate the pdfjs-document dimension of the same class — every resource-acquiring sink (SQLite handle, fd, HNSW index, pdfjs document, Tesseract worker) must be checked for "is the cleanup wired to a finally that covers EVERY post-acquisition throw." **This CLOSES the post-rc.70 re-sweep** (6 confirmed across rc.71→rc.74: rc.71 ReDoS class [HIGH+MED], rc.72 findBestMatch amplifier [MED], rc.73 bases NFC [MED], rc.74 pdfjs leak [MED/LOW]) — every one a sibling a named prior fix left open, caught only by a fresh adversarial multi-lens read of the shipped commit, the session's signature pattern.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## [3.10.0-rc.73] — 2026-06-19
|
|
117
|
+
|
|
118
|
+
> **TL;DR:** **Post-rc.70 re-sweep, batch 3/4 — MED NFC-blind `path`/`file.path` predicate in `.base` queries (rc.69 NFC sibling).** `obsidian_query_base` compared `ctx.path` (raw relPath, NFD on macOS APFS) against the NFC `.base` filter literal in the `path`/`file.path` startsWith/contains branch with NO normalization, so `path startsWith "Café/"` silently returned zero matches for an accented folder. rc.69 fixed the DQL twin; this bases.ts branch was the missed sibling (the `file.name ==` branch 10 lines below already folds). Fixed by NFC-normalizing both operands. **1301 → 1302 source tests.**
|
|
119
|
+
|
|
120
|
+
**Pre-release (v3.10 line) — post-rc.70 re-sweep, batch 3/4 (NFC correctness, rc.69 sibling).**
|
|
121
|
+
|
|
122
|
+
### Fixed
|
|
123
|
+
|
|
124
|
+
- **MED — `.base` `path`/`file.path` startsWith/contains was NFC-blind (silent zero-match for accented paths on macOS)** (`src/bases.ts`). The `path startsWith "X"` / `path contains "X"` / `file.path startsWith "X"` / `file.path contains "X"` branch compared `ctx.path` (= `e.relPath` with `\`→`/`, NFD-decomposed on APFS) against the unnormalized NFC literal extracted from the `.base` filter — neither operand normalized. So a `.base` with `filters: 'path startsWith "Café/"'` silently returned ZERO rows for notes under a `Café/` folder whose on-disk path is NFD, though they exist and match user intent. This is the same FS(NFD)-vs-user(NFC) mismatch rc.69 closed for the DQL engine's `file.name`/`file.path`; the bases.ts `file.name ==` twin was folded in rc.46, but this path/file.path branch one block above did neither (the rc.46/rc.69 name-fold detector can't see it — the strip and the raw `startsWith`/`includes` are in different places, no `toLowerCase` signature to grep). **Fix: NFC-normalize `ctx.path` once at its assignment + the predicate literal — NFC-only, NOT case-fold, since `path`/`file.path` is case-SENSITIVE in Obsidian/Dataview. The result-row projection keeps the raw relPath verbatim.** `obsidian_query_base` is always-registered and bearer-reachable on serve-http, so any authed client got silent under-matching.
|
|
125
|
+
|
|
126
|
+
### Tests (1302)
|
|
127
|
+
|
|
128
|
+
- +1 (`tests/bases.test.ts`): an NFD-on-disk `Café/note.md` resolves an NFC `path startsWith "Café/"` (POSITIVE) + `file.path startsWith` + `file.path contains`, with a non-matching accented prefix as the NEGATIVE control — fails if the normalize is dropped. Mirrors the rc.69 DQL NFC test. **1301 → 1302.**
|
|
129
|
+
|
|
130
|
+
> **Lesson:** an inventory/name-fold detector is only as complete as the signatures it greps — a comparison where the `.md`-strip and the `startsWith`/`includes` live in different functions (and that does NO `toLowerCase`, since the field is case-sensitive) escapes the rc.46/rc.69 strip-then-lowercase signature. The durable close for such a detector-missed site is a behavioral test that fails on a dropped normalize (rc.66/rc.69 lesson), not another detector signature. Batch 4/4 (rc.74 — pdfjs document/loadingTask reserve-before-try leak, rc.70 sibling) follows.
|
|
131
|
+
|
|
132
|
+
---
|
|
133
|
+
|
|
134
|
+
## [3.10.0-rc.72] — 2026-06-19
|
|
135
|
+
|
|
136
|
+
> **TL;DR:** **Post-rc.70 re-sweep, batch 2/4 — MED remote-DoS amplifier: `validateNoteProposal` `findBestMatch` O(K×N) (rc.67 sibling).** rc.67 closed the `suggestSimilar` re-walk in the wikilink loop, but `findBestMatch` (called per link in the SAME loop) still fell into an O(N) `endsWith` scan for the path-qualified MISS case — a 1 MB body of K distinct path-qualified broken `[[a/X]]` links → O(K×N) (measured 8+ min at K=150k/N=20k) on the always-on, bearer-reachable `obsidian_validate_note_proposal`. Fixed by adding a `bySuffix` index to the cached `EntryIndex` so the path-qualified miss is O(1). **1300 → 1301 source tests.**
|
|
137
|
+
|
|
138
|
+
**Pre-release (v3.10 line) — post-rc.70 re-sweep, batch 2/4 (resource-DoS amplifier, rc.67 sibling).**
|
|
139
|
+
|
|
140
|
+
### Fixed
|
|
141
|
+
|
|
142
|
+
- **MED (remote DoS) — `findBestMatch` path-qualified resolution was O(N) per call, amplified per wikilink in `validateNoteProposal`** (`src/tools/meta.ts`). rc.67 made `validateNoteProposal` pass its single `listMarkdown()` listing into `suggestSimilar` + memoize per target — but the SAME loop calls `findBestMatch(all, target, …)` for EVERY wikilink, and `findBestMatch`'s path-qualified MISS branch (`target.includes("/")` with no exact relPath match) fell into a `for (const e of entries) if (foldKey(e.relPath).endsWith("/" + lower)) …` linear scan that the `indexFor` WeakMap did not cover. A 1 MB body packed with K distinct path-qualified broken `[[a/X]]` links therefore costs O(K × N) (measured 8+ min at K=150k / N=20k of pure event-loop blocking). The rc.67 test used basename-only `[[NoSuchNote{i}]]` targets that hit the O(1) `byBasename` miss and NEVER reached the `endsWith` branch — the test's input generator could not produce the failing shape (the rc.25/rc.36 generator-blindspot pattern, here at the TEST level). **Root fix: `EntryIndex` gains a `bySuffix` map — every `/`-aligned tail of `foldKey(relPath)` at segment index ≥ 1, first-wins in `entries` order (byte-identical to the old scan's result) — so the path-qualified miss is an O(1) lookup. This closes the class at the `findBestMatch` HELPER, benefiting every caller (find_similar, get_note_neighbors, rename_note, validate_note_proposal).**
|
|
143
|
+
|
|
144
|
+
### Tests (1301)
|
|
145
|
+
|
|
146
|
+
- +1 (`tests/tools.test.ts`): `findBestMatch` resolves a PATH-QUALIFIED target (POSITIVE) and 5000 distinct path-qualified MISSES against an N=20,000 entry vault complete in O(1) per call (<1500 ms; the old O(K × N) `endsWith` scan was minutes — the audit measured 8+ min at K=150k/N=20k). Sits beside the rc.67 `listMarkdown`-count-constant test that the basename-only generator left blind to this branch. **1300 → 1301.**
|
|
147
|
+
|
|
148
|
+
> **Lesson:** rc.67 fixed the `suggestSimilar` re-walk in the loop but left the `findBestMatch` `endsWith` scan in the SAME iteration — a fix that addressed the helper the auditor named while leaving an adjacent helper of the same amplifier class open. The durable close is the index (`bySuffix`), which fixes EVERY `findBestMatch` caller at once, plus a test whose generator can actually produce the path-qualified shape the rc.67 test couldn't. Batches 3/4 (rc.73 — bases.ts path/file.path NFC-blind, rc.69 sibling) and 4/4 (rc.74 — pdfjs reserve-before-try leak, rc.70 sibling) follow.
|
|
149
|
+
|
|
150
|
+
---
|
|
151
|
+
|
|
152
|
+
## [3.10.0-rc.71] — 2026-06-19
|
|
153
|
+
|
|
154
|
+
> **TL;DR:** **Post-rc.70 re-sweep, batch 1/4 — ReDoS class: literal-separated unbounded quantifiers in BOTH `likeToRegex` (DQL `like`, HIGH, remote) + `globToRegex` (privacy filter, MED).** A fresh 6-lens re-sweep of the shipped rc.70 commit found 6 confirmed siblings (1 HIGH / 5 MED, 0 dropped) of the rc.67→rc.70 fixes. rc.63 collapsed only ADJACENT `*` runs and rc.68 only adjacent globstar runs; a pattern with wildcards SEPARATED BY LITERALS (`*a*a*…` → `^.*a.*a…$`, `**a**a…`) was untouched and catastrophic — empirically **110 s** for `*a`×14 vs a 41-char subject via the always-on, bearer-reachable `obsidian_dataview_query`. The catastrophe scales with the SUBJECT length, so a wildcard count cap is not structurally safe. Fixed by replacing BOTH backtracking regexes with a shared NON-backtracking O(tokens×len) DP matcher (`src/wildcard-match.ts`). **1288 → 1300 source tests.**
|
|
155
|
+
|
|
156
|
+
**Pre-release (v3.10 line) — post-rc.70 re-sweep, batch 1/4 (ReDoS class, rc.63/rc.68 siblings).**
|
|
157
|
+
|
|
158
|
+
### Fixed
|
|
159
|
+
|
|
160
|
+
- **HIGH (remote DoS) + MED — catastrophic backtracking from literal-separated unbounded quantifiers in `likeToRegex` (DQL `like`) and `globToRegex` (path privacy filter)** (`src/dql.ts`, `src/vault.ts`, new `src/wildcard-match.ts`). rc.63 fixed `likeToRegex` by collapsing a RUN of adjacent `*` and rc.68 fixed `globToRegex`'s adjacent globstar runs — but neither touched wildcards SEPARATED BY LITERALS, which compile to `^.*a.*a…$` / `^[^/]*a[^/]*a…$`: against a NON-matching subject the engine tries ≈ C(len, k) partitions. Empirically reproduced: `likeToRegex("*a"×14)` hung V8 **~110 s** on a 41-char subject (a 28-char LIKE value, well under the 512 cap), reachable remotely via the always-registered, read-only `obsidian_dataview_query` on bearer-auth serve-http and AMPLIFIED per-note in the scan loop; `globToRegex("*a"×15)` likewise hung (operator-controlled `--exclude-glob`/`--read-paths`, `.test()`'d on every path of every scan). Both prior TSDocs falsely claimed "linear-time" (overclaim #22). **Because the catastrophe scales with the matched SUBJECT length (a long path or field value blows up at a handful of wildcards), a wildcard COUNT cap is NOT structurally safe; and the JS atomic-group emulation `(?=(.*))\1` was empirically rejected — it stops the backtracking but CHANGES matching semantics (it cannot yield characters back to a required following literal). Root fix (per the rc.39 "bound the SINK, don't chase shapes" lesson): a shared dependency-free leaf `src/wildcard-match.ts` whose `matchWildcardTokens` is a tabular O(tokens × len) DP — both sinks now match WITHOUT a backtracking regex.** `likeToRegex`→`compileLike`, `globToRegex`→`compileGlob` (each returns a `{ test(value): boolean }` matcher so the DQL caller + all 9 privacy-filter call sites stay byte-identical; the Vault `excludeRegexes`/`readPathRegexes` fields are renamed `excludeMatchers`/`readPathMatchers`). The `MAX_LIKE_PATTERN_LEN`/`MAX_GLOB_PATTERN_LEN` caps remain as cheap secondary bounds.
|
|
161
|
+
- **Incidental fix — a LIKE pattern containing `?` no longer crashes.** The pre-rc.71 `likeToRegex` omitted `?` from its regex-specials set, so any LIKE value with `?` compiled to an invalid `RegExp` ("Nothing to repeat") and threw. The matcher has no regex, so `?` is just a literal (LIKE's only wildcard is `*`).
|
|
162
|
+
|
|
163
|
+
### Tests (1300)
|
|
164
|
+
|
|
165
|
+
- New `tests/wildcard-match.test.ts`: matcher unit + tokenizer coverage, a **DIFFERENTIAL** regression guard (the new matcher vs INLINED copies of the pre-rc.71 `likeToRegex`/`globToRegex` builders over a broad LIKE+glob corpus — 0 mismatches proves behavior preservation, the rc.53 differential-corpus method), and a **LINEAR-budget** guard running the exact `*a*a…`/`**a**a…` shapes that hung V8 pre-rc.71 and asserting <500 ms. `tests/dql.test.ts` + `tests/security.test.ts` reworked from `likeToRegex`/`globToRegex` (`.source` assertions, which no longer apply) to `compileLike`/`compileGlob` sink-level smokes + linearity checks. **1288 → 1300.**
|
|
166
|
+
|
|
167
|
+
> **Lesson:** a ReDoS class is NOT closed by collapsing the shape the LAST fix saw (adjacent runs) — the SAME class re-manifests as literal-separated quantifiers in EVERY regex-compiling sink. The durable close is to bound the sink STRUCTURALLY (a non-backtracking matcher that cannot exceed a linear budget for ANY input), proven by a differential-vs-old + linear-budget test pair, not by extending a shape-collapse the next sibling shape escapes. Batches 2/4 (rc.72 — `validateNoteProposal` findBestMatch O(K×N) amplifier, rc.67 sibling), 3/4 (rc.73 — bases.ts path/file.path NFC-blind, rc.69 sibling), and 4/4 (rc.74 — pdfjs document/loadingTask reserve-before-try leak, rc.70 sibling) follow.
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
## [3.10.0-rc.70] — 2026-06-19
|
|
172
|
+
|
|
173
|
+
> **TL;DR:** **Post-rc.66 re-sweep, batch 4 — CLOSES the re-sweep — reserve-before-try handle-leak class.** `EmbedDb.open()`/`FtsIndex.open()` assigned the SQLite handle before `pragma`+`bootstrapSchema`, so a throw on a corrupt db leaked it (the worst instance: `server.ts:435 db.open()` outside its try/finally → leaked for the serve lifetime). Fixed by making both `open()` methods self-cleaning (close-on-throw) + wrapping `startHttpServer`'s `listen()` reject to release prepared deps. **1286 → 1288 source tests. Closes the post-rc.66 re-sweep (6 confirmed across rc.67→rc.70).**
|
|
174
|
+
|
|
175
|
+
**Pre-release (v3.10 line) — post-rc.66 re-sweep, batch 4/4 (reserve-before-try class); closes the re-sweep.**
|
|
176
|
+
|
|
177
|
+
### Fixed
|
|
178
|
+
|
|
179
|
+
- **MED/LOW resource leak — `open()` was not close-on-throw; one caller leaked the handle for the serve lifetime** (`src/embed-db.ts`, `src/fts5.ts`, `src/server.ts`, `src/http-transport.ts`). Three findings of one class (rc.65 `runWithPendingInit` reserve-before-try sibling): **(1)** `EmbedDb.open()` and `FtsIndex.open()` did `this.db = new Ctor(file)` and THEN ran `pragma` + `bootstrapSchema` with no try/finally — on a corrupt/legacy/locked db those throw AFTER the handle is assigned, and cleanup was left to each caller's discretion (an undocumented contract). **(2)** `server.ts`'s `--use-hnsw` path did `await db.open()` OUTSIDE its `try { … } finally { db.close() }`, and the outer catch only nulled `hnswContext` and let the server run on — so a corrupt `.embed.db` (the `existsSync`-gated path) leaked the SQLite handle + its WAL/SHM locks for the WHOLE serve session. **(3)** `startHttpServer` leaked `prepareServerDeps`' fts5 + watcher handles when `httpServer.listen()` rejected (EADDRINUSE / EACCES) — bounded (OS reclaims on boot abort) but flagged. **Fix (root-class, per the rc.45/rc.49 "fix the source every caller funnels through" lesson):** both `open()` methods are now SELF-CLEANING — `try { pragma; bootstrapSchema } catch (e) { this.close(); throw e }` after the handle assignment — so a post-construction throw releases the handle for EVERY caller regardless of its own discipline (this closes (1) + (2) at the root). Plus `startHttpServer`'s `listen()` promise now `await shutdownHttpServer(httpServer)` on reject before re-throwing (3).
|
|
180
|
+
|
|
181
|
+
### Tests (1288)
|
|
182
|
+
|
|
183
|
+
- +2 (close-on-throw, both primitives): a corrupt-db `EmbedDb.open()` / `FtsIndex.open()` throws, and a SECOND `open()` RE-THROWS — the precise behavioral proof the handle was released (pre-rc.70 the `if (this.db) return` guard made the second call a silent no-op; the re-throw is only possible if `this.db` was reset to null). 1286 → 1288.
|
|
184
|
+
|
|
185
|
+
> **Lesson:** the rc.65 `runWithPendingInit` fix was an INSTANCE fix for one reserve-before-try site; the durable close for the CLASS is making the resource-acquiring primitive itself self-cleaning (close-on-throw inside `open()`), so no call site can leak. The "2nd open() re-throws" assertion is the behavioral proof a handle was released without reaching into private state. **This CLOSES the post-rc.66 re-sweep** (6 confirmed: 4 MED + 2 LOW across rc.67→rc.70) — the re-sweep found that the round-3 fixes had each left a sibling, the session's signature pattern, caught only by a fresh adversarial multi-lens read of the shipped commit.
|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
## [3.10.0-rc.69] — 2026-06-19
|
|
190
|
+
|
|
191
|
+
> **TL;DR:** **Post-rc.66 re-sweep, batch 3 — MED NFC-blind DQL `file.name`/`file.path` (rc.46/rc.66 NFC sibling).** `obsidian_dataview_query` compared `file.name` (filesystem-derived, NFD on macOS) against NFC predicate literals with no normalization, so `WHERE file.name = "Café"` silently returned zero rows for an accented note. `bases.ts`'s twin was folded in rc.46; this DQL sink was missed. Fixed by NFC-normalizing both the projection and the literal. **1285 → 1286 source tests.**
|
|
192
|
+
|
|
193
|
+
**Pre-release (v3.10 line) — post-rc.66 re-sweep, batch 3/4 (MED NFC correctness).**
|
|
194
|
+
|
|
195
|
+
### Fixed
|
|
196
|
+
|
|
197
|
+
- **MED NFC-blind comparison — `obsidian_dataview_query` `WHERE file.name`/`file.path` never matched accented names on macOS** (`src/dql.ts`). `resolveField` (and the row projection) returned `file.name` as `stripMd(entry.basename)` and `file.path` as `entry.relPath` — RAW, which on macOS APFS is NFD (`Café` = `Cafe` + combining accent). A predicate literal like `"Café"` is user-authored NFC. The comparators normalize neither: `looseEq` does `a.toLowerCase() === b.toLowerCase()`, `contains` does `value.toLowerCase().includes(...)`, and `like` compiles with `iu` flags (Unicode-aware but NO normalization). So NFC literal !== NFD on-disk name even after lowercasing → `WHERE file.name = "Café"` (and `contains`/`like`) returned zero rows; `!=` returned the inverse. The `bases.ts` `file.name ==` twin was folded through `foldName` in rc.46 — this DQL sink was the missed sibling (the rc.46 `name-fold-invariant` detector can't see it: the `stripMd` is in `resolveField` and flows as a variable into the comparators, so neither detector signature appears at the comparison site). **Fix:** `.normalize("NFC")` the `file.name`/`file.path` projection (in `resolveField` and the row `out`) AND the string predicate literal (in `parseValue`) — both sides NFC, so they compare equal; the comparators continue to handle case. (Only Unicode form needs normalizing here; case is already handled.)
|
|
198
|
+
|
|
199
|
+
### Tests (1286)
|
|
200
|
+
|
|
201
|
+
- +1 in `tests/dql.test.ts`: an NFD-on-disk `Café.md` resolves a NFC `LIST WHERE file.name = "Café"` (and `contains`), with a non-matching literal NEGATIVE control + a `nfc !== nfd` non-vacuity assertion. The test fails if the `.normalize("NFC")` is dropped — it is the regression guard. 1285 → 1286.
|
|
202
|
+
|
|
203
|
+
> **Lesson:** the rc.46 NFC inventory invariant (signature: extension-strip-then-`.toLowerCase()`) structurally cannot see a comparison where the strip and the compare live in DIFFERENT functions — DQL is the 2nd such blind spot after rc.66's graph-boost. For these the durable close is a behavioral test that fails on a dropped normalize, not another detector signature. rc.70 (reserve-before-try handle leaks) closes the re-sweep.
|
|
204
|
+
|
|
205
|
+
---
|
|
206
|
+
|
|
207
|
+
## [3.10.0-rc.68] — 2026-06-19
|
|
208
|
+
|
|
209
|
+
> **TL;DR:** **Post-rc.66 re-sweep, batch 2 — MED ReDoS in `globToRegex` (the rc.63 `likeToRegex` sibling).** The `--exclude-glob`/`--read-paths` privacy-filter glob compiler had the same un-collapsed-wildcard defect rc.63 fixed: a globstar run emitted adjacent unbounded quantifiers (`****`→`^.*.*$`) that backtrack catastrophically against a non-matching path (>15s V8 hangs), run on every path of every scan. Operator-controlled (not remote). Fixed by collapsing consecutive `*` runs + a total post-process collapse on the emitted source + a length cap. **1282 → 1285 source tests.**
|
|
210
|
+
|
|
211
|
+
**Pre-release (v3.10 line) — post-rc.66 re-sweep, batch 2/4 (MED ReDoS, operator-controlled).**
|
|
212
|
+
|
|
213
|
+
### Fixed
|
|
214
|
+
|
|
215
|
+
- **MED ReDoS — `globToRegex` emitted adjacent unbounded quantifiers on globstar runs** (`src/vault.ts`). `globToRegex` compiles operator-supplied glob patterns (`--exclude-glob` / `--read-paths`) and runs the result via `.test()` in `isExcluded`/`exclusionReason` for EVERY path on EVERY vault scan (list/read/watch). Its `**` branch consumed only TWO stars + a trailing `/`, so a run of consecutive globstars produced adjacent unbounded quantifiers — `****` → `^.*.*$`, `***` → `^.*[^/]*$` — which backtrack catastrophically against a non-matching path (no nesting needed; measured >15s V8 hangs at ~8–12 globstar runs). The IDENTICAL un-collapsed-wildcard defect rc.63 fixed in `likeToRegex`, and exactly the un-enumerated sink rc.63's own TSDoc warned about ("a class is not closed until EVERY RegExp-compiling sink is enumerated"). **Severity MED, not HIGH:** the glob is operator-controlled at server boot, NOT caller-influenceable by a remote bearer-auth client — a self-inflicted hang (a fat-fingered `**foo**` freezes the operator's own scans), not a remote DoS. **Fix:** (a) consume the entire `*` run inline (→ one `.*`); (b) a FINAL total-collapse pass on the emitted source, `out.replace(/(?:\.\*|\[\^\/\]\*){2,}/g, ".*")` — the inline `while` handles a single run (`***`), but REDUNDANT globstars split by a slash (`a/**/**/b`) still emit `.*.*` because the first `**` eats its trailing `/` so the next is adjacent; the post-process makes the compiled source provably free of adjacent quantifiers however the globstars were spelled (any-chars-incl-slash is idempotent); (c) a `MAX_GLOB_PATTERN_LEN`=1024 cap (fail-fast on an absurd glob). The match semantics are unchanged (verified: `02_Personal/**` still excludes its subtree, `a/**/b` still matches `a/b`).
|
|
216
|
+
|
|
217
|
+
### Tests (1285)
|
|
218
|
+
|
|
219
|
+
- +3 in `tests/security.test.ts`: the compiled source contains no adjacent unbounded quantifiers for an adversarial globstar corpus (incl. `a/**/**/b`, `**foo**`) with semantics preserved + a NEGATIVE control proving the adjacency detector fires on `^.*.*$`/`^.*[^/]*$`; a wall-clock linearity check (adversarial patterns complete in <500 ms, were >15 s); the `MAX_GLOB_PATTERN_LEN` cap throw (+ boundary NEGATIVE control). 1282 → 1285.
|
|
220
|
+
|
|
221
|
+
> **Lesson:** my first inline-only fix (consume one run) was INSUFFICIENT — the structural adjacency test caught `a/**/**/b` (two globstar runs split by a consumed slash) on the very first build, so the durable close is a TOTAL post-process collapse on the source, not a per-run consume. And `globToRegex` is precisely the "enumerate EVERY RegExp-compiling sink" the rc.63 lesson named — missed by the rc.63 sweep, found by the post-merge re-sweep. rc.69 (DQL NFC) + rc.70 (reserve-before-try handle leaks) close the re-sweep.
|
|
222
|
+
|
|
223
|
+
---
|
|
224
|
+
|
|
225
|
+
## [3.10.0-rc.67] — 2026-06-19
|
|
226
|
+
|
|
227
|
+
> **TL;DR:** **Post-rc.66 re-sweep, batch 1 — MED remote DoS in `obsidian_validate_note_proposal`.** The always-on, bearer-reachable tool called `suggestSimilar` per broken `[[wikilink]]`, each doing a FRESH whole-vault `listMarkdown()` walk, with `content` uncapped → a body of thousands of broken targets = thousands of back-to-back directory walks on the event loop (serve-http DoS). The rc.65 `readCanvas` resource-bound-escape sibling. Fixed by sharing one listing + per-target memoization + a 1 MB content cap. **1281 → 1282 source tests.**
|
|
228
|
+
|
|
229
|
+
**Pre-release (v3.10 line) — post-rc.66 re-sweep, batch 1/4 (MED remote DoS).**
|
|
230
|
+
|
|
231
|
+
### Fixed
|
|
232
|
+
|
|
233
|
+
- **MED remote DoS — `validateNoteProposal` re-walked the whole vault per broken wikilink** (`src/tools/meta.ts` + `src/tools/write.ts` + `src/tool-registry.ts`). `obsidian_validate_note_proposal` is always-registered (does NOT require `--enable-write`) and bearer-reachable on serve-http. For every BROKEN `[[wikilink]]` in the caller-supplied `content` (which was `z.string()` with no `.max()`), it called `suggestSimilar(vault, target)`, and `suggestSimilar` did its OWN fresh `vault.listMarkdown()` — a full filesystem directory walk, uncached. So a single request with B distinct broken targets cost B whole-vault walks: O(B × N) disk I/O on the single event loop, B fully attacker-controlled, starving all other serve-http clients. It escaped `tests/resource-bound-invariant.test.ts` for the same structural reason `readCanvas` did (rc.65): it calls `listMarkdown` but not `readNote`, so `discoverScanners` never sees it, and the whole-vault cost lives in the `suggestSimilar` helper, not a visible per-entry loop. **Fix:** `suggestSimilar` gains an optional `entries?: FileEntry[]` param; `validateNoteProposal` passes its single already-fetched `all` listing into it AND memoizes suggestions per target (a `Map`), so the `listMarkdown` count is INDEPENDENT of broken-link count (constant, not one-per-link). Plus a `.max(1_000_000)` cap on the `content` schema as defense-in-depth (generous for a real note draft; bounds the abuse input). The rc.65 fix (bound the per-item resolution) is now applied to this sibling.
|
|
234
|
+
|
|
235
|
+
### Tests (1282)
|
|
236
|
+
|
|
237
|
+
- +1 in `tests/tools.test.ts`: asserts `vault.listMarkdown` is called a CONSTANT number of times as the broken-wikilink count grows 3 → 60 (pre-rc.67 it grew one-per-link). Robust to internal `listTags` calls (both runs share the same constant overhead). 1281 → 1282.
|
|
238
|
+
|
|
239
|
+
> **Lesson:** the round-3 audit's own fixes left siblings — exactly the session's signature pattern, and exactly why the post-merge re-sweep is non-negotiable. This is the rc.65 `readCanvas` class recurring in a tool the rc.65 sweep didn't reach: an always-on, bearer-reachable whole-vault scanner whose cost hides in a helper (so the auto-detector is blind) is a per-request DoS amplifier. The re-sweep found 6 such siblings (4 MED / 2 LOW); rc.68–rc.70 ship the rest (globToRegex ReDoS, DQL NFC, reserve-before-try handle leaks).
|
|
240
|
+
|
|
241
|
+
---
|
|
242
|
+
|
|
243
|
+
## [3.10.0-rc.66] — 2026-06-19
|
|
244
|
+
|
|
245
|
+
> **TL;DR:** **Round-3 audit, batch 4 — CLOSES the round-3 12-lens audit (2 LOW correctness).** **NFC graph-boost residual**: the searchHybrid in-degree tie-break was the one name-comparison site the rc.46 NFC sweep missed (raw `stripMd` without case-fold) → accented notes silently lost their graph-boost on macOS; now folded through `foldName`. **UTC-midnight Date fidelity**: an explicit midnight-UTC timestamp resolves to the same Date as a bare date and is demoted to `YYYY-MM-DD` — documented as a deliberate, pinned tradeoff. **1279 → 1281 source tests.**
|
|
246
|
+
|
|
247
|
+
**Pre-release (v3.10 line) — round-3 audit, batch 4/4 (2 LOW correctness); closes the round-3 audit.**
|
|
248
|
+
|
|
249
|
+
### Fixed
|
|
250
|
+
|
|
251
|
+
- **LOW NFC name-resolution residual — graph-boost in-degree mis-resolved accented note names** (`src/tools/search.ts`). The `searchHybrid` wikilink graph-boost built its in-degree membership set from `stripMd(wl.target)` and tested candidate paths with `targets.has(stripMd(fPath))` — WITHOUT NFC normalization or case-fold. It was the one name-comparison site the rc.46 sweep missed because its signature (raw `stripMd`, no `.toLowerCase()`) didn't match the name-fold detector's strip+lowercase pattern. On macOS (APFS returns NFD filenames; wikilinks/titles are NFC) an accented note name never matched, so it silently lost its `α=0.005` in-degree tie-break (and never contributed to others'). **Fix:** fold both sides through `foldName` (NFC + case-fold) — the same canonical key the other 14 rc.46 sites + `findBestMatch` use. Pinned with a separate `name-fold-invariant` assertion (the generic detector can't reach the no-`toLowerCase` shape).
|
|
252
|
+
- **LOW Date-fidelity — explicit UTC-midnight timestamp demoted to date-only (documented, deliberate)** (`src/frontmatter.ts`). rc.58's `normalizeDateOnly` renders a date-only (midnight-UTC) `Date` as `YYYY-MM-DD` to stop `frontmatter_set` appending a spurious time to bare dates. But an EXPLICIT timestamp on midnight UTC (`2026-01-15T00:00:00Z`, or an offset form netting to 00:00:00Z) resolves — post js-yaml `load` — to the **byte-identical** `Date` as a bare `2026-01-15`, so the two are indistinguishable at stringify time and BOTH render as `YYYY-MM-DD`. The time-of-day is irrecoverable without preserving the raw scalar (a custom js-yaml type — real risk on the freshly-hardened parser, for the same calendar date). **Accepted as a deliberate tradeoff** (far less harmful than the rc.58 bug it descends from), documented in the `frontmatter.ts` header and pinned as a contract test so it is explicit, not a silent surprise.
|
|
253
|
+
|
|
254
|
+
### Tests (1281)
|
|
255
|
+
|
|
256
|
+
- +1 in `tests/name-fold-invariant.test.ts` pinning the graph-boost membership routes through `foldName` (POSITIVE) with the pre-rc.66 unfolded shapes asserted gone (NEGATIVE); +1 in `tests/frontmatter.test.ts` pinning the midnight-UTC → date-only contract (with the byte-identical-Date sanity check showing the root collision; the rc.58 non-midnight NEGATIVE control still holds). 1279 → 1281.
|
|
257
|
+
|
|
258
|
+
> **Lesson:** the rc.46 NFC inventory invariant had a blind spot — a name comparison that does NOT lowercase (a case-SENSITIVE raw `stripMd`) escaped its strip+lowercase signature entirely. An inventory invariant is only as complete as the signatures it enumerates; the durable close for a detector-missed site is a separate pinned assertion. **This CLOSES the round-3 fresh 12-lens audit** (6 confirmed: 1 HIGH rc.63 + 2 MED rc.64/rc.65 + 3 LOW rc.65/rc.66, 0 dropped) — found by a from-scratch Workflow audit with 3-skeptic adversarial verification on the shipped rc.62 commit.
|
|
259
|
+
|
|
260
|
+
---
|
|
261
|
+
|
|
262
|
+
## [3.10.0-rc.65] — 2026-06-19
|
|
263
|
+
|
|
264
|
+
> **TL;DR:** **Round-3 audit, batch 3 — two serve-http per-request amplifiers.** **read_canvas [MED]**: the always-on `obsidian_read_canvas` did a per-file-node O(N) linear scan → O(K×N) event-loop stall (K canvas nodes × N vault notes); fixed with a one-pass `byRelPath` Map → O(1) per node. **pendingInits leak [LOW]**: a `buildMcpServer`/transport constructor throw leaked the stateful session reservation permanently → eventual 503; fixed by wrapping the init body in a `runWithPendingInit` try/finally helper. **1275 → 1279 source tests.**
|
|
265
|
+
|
|
266
|
+
**Pre-release (v3.10 line) — round-3 audit, batch 3/4 (MED resource-DoS + LOW accounting-leak).**
|
|
267
|
+
|
|
268
|
+
### Fixed
|
|
269
|
+
|
|
270
|
+
- **MED resource-DoS — `obsidian_read_canvas` per-node O(N) linear scan → O(K×N)** (`src/tools/media.ts`). `readCanvas` (always-on, read-only, bearer-reachable) loaded the whole markdown index then, for EACH `file:` node, ran a fresh `allMarkdown.find(...)` linear scan — O(K×N) where K = canvas file-nodes (bounded only by the 5 MB file cap → tens of thousands) and N = vault notes, blocking the single event loop for all serve-http clients. It escaped `tests/resource-bound-invariant.test.ts` entirely because `media.ts` is outside `SCANNER_SOURCES` and `readCanvas` uses `listMarkdown` WITHOUT `readNote`, so the auto-detector never saw it. **Fix:** build a `byRelPath` Map once (O(N)) and resolve each node via an O(1) lookup (the `findBestMatch` basename fallback already has its own cached index). Total cost drops from O(K×N) to O(N+K). Added a SEPARATE resource-bound assertion (mirrors the `queryBase`/`buildWikilinkGraph` pattern) pinning the O(1) index present AND no per-node linear scan, so the class is now structurally covered.
|
|
271
|
+
- **LOW resource-DoS/accounting — `pendingInits` leaked on a constructor throw → permanent 503** (`src/http-transport.ts`). In stateful serve-http, the fresh-initialize path did `registry.pendingInits += 1` and constructed `buildMcpServer(...)` + `new StreamableHTTPServerTransport(...)` OUTSIDE the `try { … } finally { pendingInits -= 1 }`. If either constructor threw, control jumped to the outer catch and the reservation was never released — permanently lowering the effective `maxSessions` cap by one each time, until every `initialize` returned 503 "max sessions reached" with zero live sessions (violating the documented "always returns to 0 after init" invariant). **Fix:** extracted an exported `runWithPendingInit(registry, fn)` helper (increment + `try/finally` decrement) and wrapped the WHOLE build+connect+initialize body in it (the constructors are now inside the try; the inner catch is undefined-guarded so a partial allocation cleans up). The decrement now runs on every exit path including a constructor throw.
|
|
272
|
+
|
|
273
|
+
### Tests (1279)
|
|
274
|
+
|
|
275
|
+
- +1 in `tests/resource-bound-invariant.test.ts` (readCanvas resolves via the O(1) `byRelPath` index, NOT a per-node `allMarkdown.find`); +3 in `tests/http-transport.test.ts` (`runWithPendingInit` balances on success, **NEGATIVE control** — a throwing init body leaves `pendingInits===0`, and a 50-iteration erosion check). The existing `canvas.test.ts` resolution tests (exact-path + broken-ref) already cover the Map fix's correctness. 1275 → 1279.
|
|
276
|
+
|
|
277
|
+
> **Lesson:** both are serve-http per-request amplifiers the drift/claim gates are structurally blind to. `read_canvas` slipped the resource-bound inventory because its signature (`listMarkdown` without `readNote`) didn't match the auto-detector — closed with an explicit separate assertion (the same escape hatch `queryBase` needed). The `pendingInits` leak is the classic "reserve before the try" shape — closed with a pure `try/finally` helper that's unit-testable with an injected throwing body. rc.66 ships the final 2 LOW correctness items (NFC graph-boost + UTC-midnight Date).
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
281
|
+
## [3.10.0-rc.64] — 2026-06-19
|
|
282
|
+
|
|
283
|
+
> **TL;DR:** **Round-3 audit, batch 2 — MED silent write-path data-loss (the rc.61 WRITE-2 sibling).** `frontmatter_set` on a note whose existing frontmatter is a valid-YAML NON-mapping (bare scalar / sequence) silently REPLACED and destroyed it, reporting a phantom success. The rc.61 WRITE-2 guard only caught the *throws-on-parse* case; this is the *parses-but-coerced-to-`{}`* sibling. Fixed by exposing a `coerced` flag from `parseFrontmatter` and refusing fail-closed. **1273 → 1275 source tests.**
|
|
284
|
+
|
|
285
|
+
**Pre-release (v3.10 line) — round-3 audit, batch 2/4 (MED silent write-path data-loss).**
|
|
286
|
+
|
|
287
|
+
### Fixed
|
|
288
|
+
|
|
289
|
+
- **MED silent data-loss — `frontmatter_set` destroyed valid-YAML NON-mapping frontmatter** (`src/frontmatter.ts` + `src/tools/write.ts`). When a note's frontmatter block is valid YAML but NOT a mapping — a bare scalar (`---\nhello\n---`) or a sequence (`---\n- a\n- b\n---`) — `parseFrontmatter` deliberately coerces it to `data:{}` (the rc.54/rc.55 `isPlainObject` guard, so a non-mapping is never spread char-indexed). `frontmatterSet` therefore computed `before:{}`, built `after={...args.set}`, and `stringifyFrontmatter(body, after)` REPLACED the original block with a fresh mapping — silently destroying the scalar/sequence while the response reported a phantom success (`before:{}`, `changed_keys:['+key']`). Empirically: `---\n- item 1\n- item 2\n---\nBody.\n` after `frontmatter_set({set:{status:'done'}})` became `---\nstatus: done\n---\nBody.\n`. The rc.61 WRITE-2 guard only refused when the re-parse THREW (malformed YAML); this valid-but-non-mapping case parses cleanly, so it slipped through. **Fix:** `parseFrontmatter` now returns a `coerced` flag (it already computed the `isPlainObject` branch — the flag is free), true iff a non-empty block was coerced away from a mapping; `frontmatterSet` refuses fail-closed (`"its existing frontmatter is not a YAML mapping … editing it would replace and destroy that block"`) when `coerced` is set. This generalizes the rc.61 guard from "throws-on-parse" to "throws-OR-coerced", closing both halves of the class at one shared signal. A note with NO frontmatter parses cleanly (`coerced:false`) → the legitimate add path stays open.
|
|
290
|
+
|
|
291
|
+
### Tests (1275)
|
|
292
|
+
|
|
293
|
+
- +1 in `tests/frontmatter.test.ts` pinning the `coerced` contract (sequence/scalar/bare-Date → `true`; mapping/empty-fence/comment-only/absent → `false`, the POSITIVE control); +1 in `tests/frontmatter-ops.test.ts` asserting `frontmatter_set` REFUSES on a sequence AND a scalar frontmatter with the file left BYTE-unchanged (the existing rc.61 no-frontmatter test is the NEGATIVE control — adding still works). 1273 → 1275.
|
|
294
|
+
|
|
295
|
+
> **Lesson:** the rc.61 WRITE-2 fix closed only the *throws-on-parse* half of "frontmatter_set must not destroy what it can't represent as a mapping" — the valid-YAML-non-mapping half stayed open one RC later, the project's signature "audit-driven fix leaves a sibling" pattern. The `coerced` flag closes the whole class at the parse layer (a single signal both guard surfaces consume), and the destroy-path test asserts BYTE-equality of the file, not merely that an error was thrown.
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
## [3.10.0-rc.63] — 2026-06-19
|
|
300
|
+
|
|
301
|
+
> **TL;DR:** **Round-3 fresh 12-lens audit, batch 1 — HIGH ReDoS in DQL `like`.** `obsidian_dataview_query`'s `likeToRegex` translated each `*` to `.*`, so N adjacent `*` compiled to `^.*.*…$` — adjacent unbounded quantifiers that backtrack catastrophically against a non-matching subject (no nesting needed). Empirically reproduced + independently re-confirmed: an 11-char `**********Q` hangs V8 >5s, a remote event-loop DoS for ALL serve-http clients via the always-registered, read-only tool. The TSDoc falsely claimed "catastrophic-backtracking-SAFE by construction" (overclaim #21). Fixed by collapsing consecutive `*` into a single `.*`; corrected the TSDoc; added a static + an empirical (worker-timed) structural guard. **1270 → 1273 source tests.**
|
|
302
|
+
|
|
303
|
+
**Pre-release (v3.10 line) — round-3 audit, batch 1/4 (HIGH ReDoS, shipped first + alone).**
|
|
304
|
+
|
|
305
|
+
### Fixed
|
|
306
|
+
|
|
307
|
+
- **HIGH ReDoS — `likeToRegex` emitted adjacent `.*` runs → catastrophic backtracking, remote DoS via `obsidian_dataview_query` `LIKE`** (`src/dql.ts`). The translator output one `.*` per `*` (line 486), so a pattern of N adjacent `*` produced `^.*.*…$` (N adjacent unbounded quantifiers). Against a non-matching subject the engine must try every partition of the subject across the runs — superlinear/exponential — WITHOUT any nesting, which the TSDoc on `MAX_LIKE_PATTERN_LEN` (lines 433-436) wrongly claimed was impossible ("catastrophic-backtracking-SAFE by construction"). `obsidian_dataview_query` is always-registered, read-only, and bearer-reachable on serve-http with no write/CLI gate; the `MAX_LIKE_PATTERN_LEN=512` cap permits ~255 adjacent stars, far past the danger threshold. **Empirically reproduced + independently re-confirmed** (not taken on the auditor's word): an 11-char `LIKE` pattern `**********Q` vs a 40-char non-matching subject hangs V8 >5s in a worker. The rc.21/24/25/36 detector and the rc.39 worker sink-bound both protect `obsidian_open_questions`, but DQL `like` compiles its OWN RegExp and was never covered — a second RegExp sink of the same class. **Fix:** collapse a run of consecutive (unescaped) `*` into a SINGLE `.*` (a run of SQL-LIKE wildcards is semantically identical to one `*`), so the compiled source contains only non-adjacent `.*` separated by required literals → linear-time. An escaped `\*` stays a literal (handled by the backslash branch, not collapsed). Verified post-fix: the 255-star and 10-star patterns complete in 0ms and the compiled source has no `.*.*` adjacency. The false "catastrophic-backtracking-SAFE by construction" TSDoc was corrected to point at the real guard (overclaim #21, the claimed-guarantee-vs-code-guard class).
|
|
308
|
+
|
|
309
|
+
### Tests (1273)
|
|
310
|
+
|
|
311
|
+
- +3 source `it()` in `tests/dql.test.ts`: (1) STATIC — `likeToRegex` collapses `**`/`***`/255-star runs to a single `.*` with no `.*.*` adjacency, and a run-in-the-middle (`a**b`) and separated runs (`*x*y*`) each stay one `.*`; (2) SQL-LIKE semantics preserved after the collapse (`****` matches anything incl. empty; `*foo*` matches/doesn't; an escaped `\*\*` stays literal); (3) EMPIRICAL — the COMPILED `likeToRegex` output for a corpus of adversarial adjacent-star patterns is run against non-matching subjects in a worker with a wall-clock budget (re-confirmed to avoid load-flake), so the next adjacency regression fails CI on behavior, not just a hand-checked source string. 1270 → 1273.
|
|
312
|
+
|
|
313
|
+
> **Lesson:** the rc.39 sink-bound ended the ReDoS class FOR `obsidian_open_questions`, but a SECOND RegExp-compiling sink (DQL `like` → `likeToRegex`) existed and was never enumerated — a class is not closed until EVERY sink of that class is found, and the durable close is an evaluation-time guard (does the compiled regex actually hang?), not a source inspection or a length cap. This is the project's signature recurrence (an audit-closed class re-surfaces at a sibling sink), exactly what a fresh from-scratch multi-lens audit on the shipped commit is for. Round-3 found 6 (1 HIGH + 2 MED + 3 LOW); rc.64–rc.66 ship the remainder.
|
|
314
|
+
|
|
315
|
+
---
|
|
316
|
+
|
|
317
|
+
## [3.10.0-rc.62] — 2026-06-19
|
|
318
|
+
|
|
319
|
+
> **TL;DR:** **Round-2 audit tail, batch 3 — CLOSES the round-2 fresh 8-lens audit.** **HTTP-CORS-EXPOSE-SESSION-ID [MED]**: serve-http now sends `Access-Control-Expose-Headers: Mcp-Session-Id` so a browser MCP client can read the session id the server returns on `initialize`. **CLI-SERVEHTTP-RECENCY-FAILLATE [LOW]**: `--recency-weight` / `--stale-days` / `--reranker-top-n` are now validated FAST at serve-http boot (was: only on the first search request). **PERIODIC-WW-LOCALE-CONFLATION [LOW]**: documented the deliberate ISO-8601 resolution of lowercase week tokens (ww/wo/gggg) + a pinning test. **1255 → 1270 source tests.**
|
|
320
|
+
|
|
321
|
+
**Pre-release (v3.10 line) — round-2 audit tail, batch 3 (1 MED + 2 LOW); closes the round-2 audit.**
|
|
322
|
+
|
|
323
|
+
### Fixed
|
|
324
|
+
|
|
325
|
+
- **HTTP-CORS-EXPOSE-SESSION-ID [MED] — browser MCP clients couldn't read the `Mcp-Session-Id` response header** (`src/http-transport.ts`). `applyCors` set `Access-Control-Allow-Headers` (which only lets a browser SEND `Mcp-Session-Id` on a request) but never `Access-Control-Expose-Headers`, so cross-origin JS could not READ the `Mcp-Session-Id` the SDK returns on `initialize` — in stateful mode every follow-up request looked like a brand-new session. Added `res.setHeader("Access-Control-Expose-Headers", "Mcp-Session-Id")` to `applyCors` (runs on every request + preflight).
|
|
326
|
+
- **CLI-SERVEHTTP-RECENCY-FAILLATE [LOW] — serve-http accepted bad advanced-retrieval flags at boot, failing only on the first search** (`src/cli.ts` + new `src/retrieval-opts.ts`). `startHttpServer` builds `prepareServerDeps` lazily (per session, on the first request), so a typo'd `--recency-weight 5` / `--stale-days x` / `--reranker-top-n 0` started the server cleanly and only threw when the first `obsidian_search` ran — unlike stdio `serve`, which validates eagerly at startup. The validation was extracted into a new leaf module `src/retrieval-opts.ts` (`parseRecencyConfig` + `validateServeHttpRetrievalOpts`), called at the serve-http boot (fail-fast) AND reused by `prepareServerDeps` (single source of truth — no validation drift between the two paths). The leaf module also satisfies the `no-internal-imports` Class-A invariant (test files may not value-import `src/server.ts`).
|
|
327
|
+
- **PERIODIC-WW-LOCALE-CONFLATION [LOW, documented-deliberate] — lowercase week tokens resolve to ISO-8601, not Moment's locale-aware weeks** (`src/periodic.ts`). Moment's `ww`/`wo`/`gggg` are locale-aware (locale-dependent week start + numbering) while `WW`/`Wo`/`GGGG` are ISO-8601; `formatToken` intentionally resolves BOTH to ISO. enquire ships no locale database and ISO weeks are the Obsidian Periodic-Notes / Daily-Notes default, so this is the correct locale-independent behavior for filename templates — documented as a deliberate contract in the source + pinned in tests (lowercase == uppercase == ISO), not a silent conflation.
|
|
328
|
+
|
|
329
|
+
### Tests (1270)
|
|
330
|
+
|
|
331
|
+
- +2 in `tests/http-transport.test.ts` (Expose-Headers on preflight + on a real request); +1 in `tests/periodic.test.ts` (ww/wo/gggg == ISO contract); +12 in new `tests/serve-http-opts-validation.test.ts` (`parseRecencyConfig` + `validateServeHttpRetrievalOpts`, each with POSITIVE + NEGATIVE controls including the gating that `--reranker-top-n` is only validated when `--enable-reranker` is set). 1255 → 1270.
|
|
332
|
+
|
|
333
|
+
### Notes
|
|
334
|
+
|
|
335
|
+
- **Post-rc.61 re-sweep (FM-PROTO sibling).** The `frontmatter_set` SET loop (`after[k] = v` in `src/tools/write.ts`) has the same prototype-setter shape as the rc.61 FM-PROTO-KEY-DROP fix, but zod's `.record()` strips a literal `__proto__` key from `args.set` before it reaches the loop (verified empirically) → the SET path is unreachable; rc.61's fix covered the reachable file-derived path (js-yaml resolves an existing `__proto__` frontmatter key to an own property). `bases.ts` `out[k] = fm[k]` is a read-only query-result projection (no write-back), not corruption. No code change needed.
|
|
336
|
+
|
|
337
|
+
> **Lesson:** this CLOSES the round-2 fresh 8-lens audit (7 confirmed: 1 HIGH in rc.60 + 2 MED + 4 LOW across rc.61 + rc.62). The CLI-SERVEHTTP fix's right shape was to extract a single shared validator (used by BOTH the eager serve-http boot and the lazy prepareServerDeps) rather than duplicate the checks — duplicated validation is its own drift class. The leaf-module placement was forced by the `no-internal-imports` invariant, which is exactly the kind of structural guard that keeps test/boilerplate coupling from creeping in.
|
|
338
|
+
|
|
339
|
+
---
|
|
340
|
+
|
|
341
|
+
## [3.10.0-rc.61] — 2026-06-19
|
|
342
|
+
|
|
343
|
+
> **TL;DR:** **Round-2 audit tail, batch 2 — three write-path fidelity fixes.** **WRITE-2 [MED]**: `frontmatter_set` on a note whose existing frontmatter is malformed YAML (e.g. a tab used for indentation) blindly prepended a SECOND `---` block, doubling/corrupting it — now refuses fail-closed. **WRITE-3 [LOW]**: a case-only rename (`Foo.md`→`foo.md`) on a case-insensitive FS was blocked — fixed at both the tool guard and the `vault.renameFile` primitive. **FM-PROTO-KEY-DROP [LOW, rc.58 regression]**: rc.58's `normalizeDateOnly` deep-walk silently dropped a literal `__proto__` frontmatter key — now preserved via `Object.defineProperty`. **1251 → 1255 source tests.**
|
|
344
|
+
|
|
345
|
+
**Pre-release (v3.10 line) — round-2 audit tail, batch 2 (2 LOW + 1 MED write-path).**
|
|
346
|
+
|
|
347
|
+
### Fixed
|
|
348
|
+
|
|
349
|
+
- **WRITE-2 [MED] — `frontmatter_set` on malformed-YAML frontmatter wrote a doubled `---` block** (`src/tools/write.ts`). When a note's existing frontmatter is invalid YAML (e.g. a TAB used for indentation, which YAML forbids and js-yaml@4 rejects — `parseNote` then falls back to treating the whole file as body, so `note.frontmatter` is `{}`), `frontmatterSet` treated it as "no frontmatter" and PREPENDED a fresh `---` block, producing a corrupt file with two frontmatter fences. Fix: `frontmatterSet` now re-parses `note.content` with `parseFrontmatter` up front and, if it throws, REFUSES the edit with a clear "its existing frontmatter is not valid YAML (e.g. a tab used for indentation) — fix it by hand first" error — fail-closed instead of silent corruption. A note that genuinely has no frontmatter still parses cleanly (`{}`) and gets one added as before.
|
|
350
|
+
- **WRITE-3 [LOW] — case-only rename (`Foo.md` → `foo.md`) was blocked on a case-INSENSITIVE filesystem** (`src/tools/write.ts` + `src/vault.ts`). Two layers rejected it: (1) `renameNote`'s "Destination already exists" guard saw the source as the destination (same path, different case), and (2) the `vault.renameFile` primitive uses `link()`+`unlink()` for an atomic exclusive create, which cannot self-replace the same inode. Fix at both layers: `renameNote` skips its existence guard for a case-only path difference and defers to `vault.renameFile` (the authority); `vault.renameFile` gains `isSameInodeCaseRename` (paths differ only in case AND resolve to the same inode) and uses a plain `rename` for that case. A case-SENSITIVE filesystem with a distinct existing `foo.md` still throws `EEXIST` → "Destination already exists" (overwrite required).
|
|
351
|
+
- **FM-PROTO-KEY-DROP [LOW, rc.58 regression] — `frontmatter_set` silently dropped a literal `__proto__` frontmatter key** (`src/frontmatter.ts`). rc.58's `normalizeDateOnly` deep-walk (bare-date fidelity) rebuilt every object with `out[k] = normalizeDateOnly(v)`; for `k === "__proto__"` that assignment hits the object's prototype SETTER rather than creating an own property, so the key vanished on re-stringify (data loss vs a direct `dump`). Fix: build each key with `Object.defineProperty(out, k, { value, enumerable, writable, configurable })`, which creates a real own enumerable data property for ANY key name — including `__proto__` — so it survives the deep-walk and is re-emitted by `dump`.
|
|
352
|
+
|
|
353
|
+
### Tests (1255)
|
|
354
|
+
|
|
355
|
+
- +4 source `it()`: WRITE-2 refuse-on-malformed + a NEGATIVE control (clean no-frontmatter note still gets one added) in `tests/frontmatter-ops.test.ts`; WRITE-3 case-only rename (with a case-insensitive-FS skip guard) in `tests/write.test.ts`; FM-PROTO `__proto__`-key-survives-stringify in `tests/frontmatter.test.ts`. 1251 → 1255.
|
|
356
|
+
|
|
357
|
+
> **Lesson:** WRITE-3 needed fixes at BOTH layers — patching only the audit-cited `vault.renameFile` primitive left the user-facing `renameNote` tool still throwing at its own existence guard; a fix is not complete until the whole user-facing path is exercised. And FM-PROTO is the 5th recursion-pair this session: rc.58's own `normalizeDateOnly` (which closed the bare-date mutation class) regressed `__proto__`-key preservation — the post-merge re-sweep / multi-lens audit is exactly what surfaces "the audit-driven fix recursed its own class".
|
|
358
|
+
|
|
359
|
+
---
|
|
360
|
+
|
|
361
|
+
## [3.10.0-rc.60] — 2026-06-19
|
|
362
|
+
|
|
363
|
+
> **TL;DR:** **Round-2 fresh audit → WRITE-1 [HIGH data-loss], shipped first/isolated.** A 2nd from-scratch 8-lens workflow-audit (30 agents, 3-skeptic verify; deeper into the modules round 1 skimmed) found a genuine HIGH the prior 25 audit rounds missed: `renameNote(overwrite:true)` silently LOSES the source note when the destination backlinks the source. Fixed by excluding the rename destination from the backlink-rewrite plan. The 6 remaining round-2 findings (2 MED / 4 LOW) ship in rc.61–rc.62. **1249 → 1251 source tests.**
|
|
364
|
+
|
|
365
|
+
**Pre-release (v3.10 line) — round-2 audit, WRITE-1 (HIGH, isolated).**
|
|
366
|
+
|
|
367
|
+
### Fixed
|
|
368
|
+
|
|
369
|
+
- **WRITE-1 [HIGH, data-loss] — `obsidian_rename_note({overwrite:true})` silently destroys the source when the destination backlinks the source** (`src/tools/write.ts`). The backlink-rewrite plan loop excluded only the SOURCE file (`e.absPath === fromAbs`), never the DESTINATION. With two distinct notes — `A.md` (source) and an existing `B.md` (destination) that contains `[[A]]` — the sequence was: (1) write source self-refs at the OLD path; (2) `renameFile(A→B, overwrite)` moves A's content onto `B.md` (overwriting B's original); (3) the backlink loop then `writeNote("B.md", B's-pre-rename-rewritten-content)` — **clobbering the just-moved source content.** A.md's content was permanently lost while the response still reported `{from:"A", to:"B"}` success. Root cause (class, not instance): the orchestrator builds the backlink plan against PRE-rename content but applies it POST-rename, so any path the rename mutated (the destination) is stale in the plan. Fix: also skip the destination (`e.absPath === toAbsCheck`) from the plan — its post-rename content IS the moved source (whose self-references were already fixed via `sourcePlan`), so there is nothing to rewrite there. `archiveNote` inherits the fix (it delegates to `renameNote`).
|
|
370
|
+
|
|
371
|
+
### Tests (1251)
|
|
372
|
+
|
|
373
|
+
- +2 source `it()` in `tests/write.test.ts`: a data-loss regression (overwrite:true where the destination backlinks the source → assert the destination holds the moved source content) + a NEGATIVE control (overwrite:true to a destination that does NOT backlink the source still works — proves the skip isn't over-broad). 1249 → 1251.
|
|
374
|
+
|
|
375
|
+
> **Lesson:** the rename orchestrator computes a backlink-rewrite plan against PRE-rename disk content but writes it POST-rename — any path the rename itself mutated (the overwrite destination) is stale in that plan, and the long-standing source-only exclusion was the gap. A fresh multi-lens audit's deeper write-path lens found a real HIGH data-loss bug that 25 prior rounds + the standing test suite never surfaced — behavioral multi-lens audits remain the highest-yield gate, and a data-loss path deserves a dedicated isolated RC.
|
|
376
|
+
|
|
377
|
+
---
|
|
378
|
+
|
|
379
|
+
## [3.10.0-rc.59] — 2026-06-19
|
|
380
|
+
|
|
381
|
+
> **TL;DR:** **Post-rc.58 re-sweep — a 6th OPTDEP-leak sibling (`hnsw.ts`) + the detector's own blind-spot.** The mandated post-merge re-sweep found `hnsw.ts`'s `loadHnswlib` leaks the importing file's abs path via a `const msg = err.message; …throw new Error(\`…${msg}\`)` **indirection** the rc.57 leak-detector was structurally blind to (it matched only DIRECT `${err.message}`) — missed by the 8-lens audit + my rc.55 grep too. RCA: the throw is **fail-soft-caught server-side** (brute-force fallback → operator stderr, not the client) → **LOW**, but the fix + detector-strengthening are the durable close. Routed `loadHnswlib` through `optionalDepDetail`, added `hnsw.ts` to the inventory (3→6 files), and **strengthened the detector to be indirection-aware AND throw-scoped** (flags a `const`-captured error message interpolated in a `throw`, ignores server-side `stderr` logs). **1249 source tests unchanged.**
|
|
382
|
+
|
|
383
|
+
**Pre-release (v3.10 line) — post-rc.58 re-sweep.**
|
|
384
|
+
|
|
385
|
+
### Fixed
|
|
386
|
+
|
|
387
|
+
- **OPTDEP leak, 6th sibling [LOW] — `hnsw.ts` `loadHnswlib` abs-path leak via a `const msg` indirection** (`src/hnsw.ts`). `await import("hnswlib-node")`'s catch did `const msg = err instanceof Error ? err.message : String(err)` then `throw new Error(\`…Underlying error: ${msg}\`)` — Node's `ERR_MODULE_NOT_FOUND` message embeds the importing file's absolute path. RCA: server.ts catches the HNSW build/load failure **fail-soft** (→ brute-force semantic search → the abs path lands in the operator's own stderr, NOT in a client response), so this is LOW (operator-side), not the client-facing MED that OPTDEP-SQLITE was. Fixed for defense-in-depth + consistency: the loader now routes through `optionalDepDetail(err)` (code only).
|
|
388
|
+
|
|
389
|
+
### Changed
|
|
390
|
+
|
|
391
|
+
- **`tests/optional-dep-leak-invariant.test.ts` detector — now indirection-aware AND throw-scoped** (the real durable fix; closes the blind-spot that let `hnsw.ts` escape the rc.57 inventory + detector + 8-lens audit). It previously matched only a DIRECT `${err.message}` / `${String(err)}` token *anywhere*; it now (a) collects `const`-captured error-message variables and (b) flags `err.message` / `String(err)` / any captured var interpolated inside a `throw new Error(...)` (the client-facing sink), while NOT flagging a server-side `process.stderr.write(… ${msg})` (operator's own machine). Inventory extended 3 → 6 files (adds `hnsw.ts`). NEGATIVE control extended to prove the indirection IS caught and the stderr case is NOT (so the detector isn't vacuous and doesn't over-flag). `tools/search.ts`'s `signalErrors.* = msg` chokepoint was investigated and left as-is — its upstream import/model sources are all sanitized (rc.45/rc.57), so it is fed path-free (not a confirmed live leak).
|
|
392
|
+
|
|
393
|
+
### Tests (1249)
|
|
394
|
+
|
|
395
|
+
- No new source `it()` — the detector rewrite + the extended direct/indirection/stderr NEGATIVE-control assertions stay within the existing `optional-dep-leak-invariant` tests. 1249 unchanged.
|
|
396
|
+
|
|
397
|
+
> **Lesson:** a leak-class detector must model the SINK (a thrown Error reaching the client), not a surface token. The rc.57 detector matched the `${err.message}` token but was blind to the `const msg = err.message; …${msg}` indirection AND didn't distinguish a `throw` (client-facing) from a `process.stderr.write` (operator-side) — so `hnsw.ts` slipped past it, the inventory, AND the multi-agent audit. The throw-scoped indirection-aware detector ends the OPTDEP sub-class structurally. This is the 4th audit-fix-recurs-its-own-class pair this session — the post-merge re-sweep keeps earning its keep.
|
|
398
|
+
|
|
399
|
+
---
|
|
400
|
+
|
|
401
|
+
## [3.10.0-rc.58] — 2026-06-19
|
|
402
|
+
|
|
403
|
+
> **TL;DR:** **Fresh 8-lens audit — Batch C+D (3 LOW correctness), closes the audit.** **CT-LASTINDEXOF-COLLISION**: `chatThreadAppend`'s rc.55 `lastIndexOf` could match a heading-like line inside the user's content → wrong `line_start`/past EOF; anchored to the appended block (collision-proof). **CONC-1**: documented the read-modify-write lost-update contract (concurrent same-note appends must be serialized). **FM-DATE-SILENT-MUTATION**: a bare `created: 2026-01-15` was silently rewritten to `2026-01-15T00:00:00.000Z` on any unrelated `frontmatter_set` (breaking date-only Dataview queries + falsifying the rc.54 "ISO dates unaffected" claim) → `stringifyFrontmatter` now renders date-only Dates as `YYYY-MM-DD`. **1246 → 1249 source tests.** Closes the 5-finding fresh audit (1 HIGH + 1 MED + 3 LOW across rc.57+rc.58).
|
|
404
|
+
|
|
405
|
+
**Pre-release (v3.10 line) — fresh-audit Batch C+D (correctness).**
|
|
406
|
+
|
|
407
|
+
### Fixed
|
|
408
|
+
|
|
409
|
+
- **CT-LASTINDEXOF-COLLISION [LOW, recursion of rc.55] — `chat_thread_append` `line_start` could point into user content** (`src/tools/read.ts`). rc.55 derived the line range from `newBody.lastIndexOf(headingMarker)`; if `args.content` embedded a line byte-identical to the freshly-generated `### <role> · <timestamp>` marker (same-second timestamp), `lastIndexOf` matched the **copy inside the content** → `line_start` into user content and `line_end` past EOF (violating the rc.50 no-past-EOF property). Reachability is very low (the timestamp is the server's wall-clock second, not client-controllable), hence LOW. Fixed by anchoring to the **appended block**: `trimmed.length + toAppend.indexOf(headingMarker)` — `toAppend` contains exactly one real heading, which always precedes any user-content copy within `messageBlock`, so first-occurrence `indexOf` is collision-proof.
|
|
410
|
+
- **CONC-1 [LOW] — concurrent appends to the same thread note are a lost-update window** (`src/tools/read.ts`). `chatThreadAppend` is read-modify-`writeNote(overwrite)`, not an atomic `O_APPEND` like `Vault.appendNote`; two appends to the same `note_path` that overlap (a client not awaiting, or concurrent serve-http requests) can drop one message (both read body B; the second write B+msg2 overwrites B+msg1). The TSDoc's understated "last-write-wins" note is rewritten to an explicit **CONCURRENCY CONTRACT**: callers MUST serialize appends to a given thread note. The structural fix (atomic-append the message branch) is deferred — the heading-injection / new-note branches genuinely need a full write, and conflating them risks re-opening the rc.50→rc.55 line-arithmetic class.
|
|
411
|
+
- **FM-DATE-SILENT-MUTATION [LOW] — `frontmatter_set` rewrote a bare date to a full ISO timestamp** (`src/frontmatter.ts`). js-yaml resolves a bare/unquoted `created: 2026-01-15` to a midnight-UTC `Date`, and a naive `dump` re-serialized it as `2026-01-15T00:00:00.000Z` — so a `frontmatter_set` touching ANY other key silently appended a midnight time to every bare date (breaks date-only Dataview/Templater queries) AND falsified the rc.54 header's "ISO dates … unaffected" claim (true only for *quoted* dates). `stringifyFrontmatter` now deep-walks `data` and renders any `Date` with no time-of-day component (midnight UTC) as a `YYYY-MM-DD` string (so the date survives as `'2026-01-15'`, the text preserved, no spurious time); a genuine non-midnight timestamp is left as a full ISO string. The `frontmatter.ts` header claim is corrected to describe this.
|
|
412
|
+
|
|
413
|
+
### Tests (1249)
|
|
414
|
+
|
|
415
|
+
- +3 source `it()`: 1 in `tests/chat-thread.test.ts` (CT-LASTINDEXOF collision-proof) + 2 in `tests/frontmatter.test.ts` (FM-DATE round-trip POSITIVE + non-midnight-timestamp NEGATIVE control). 1246 → 1249.
|
|
416
|
+
|
|
417
|
+
> **Lesson:** rc.58 closed a recursion of rc.55 (CT-LASTINDEXOF — the rc.55 `lastIndexOf` fix introduced a new collision edge) and a residual of rc.54's own header overclaim (FM-DATE — "ISO dates unaffected" was only true for quoted dates) — the project's signature "an audit-driven fix recurs its own class" pattern, the 3rd recursion-pair this session (rc.54→FM-SCALAR-DATE, rc.55→OPTDEP-SQLITE, rc.55→CT-LASTINDEXOF). The fresh-audit-then-fix-then-re-sweep loop is exactly what surfaces these.
|
|
418
|
+
|
|
419
|
+
---
|
|
420
|
+
|
|
421
|
+
## [3.10.0-rc.57] — 2026-06-19
|
|
422
|
+
|
|
423
|
+
> **TL;DR:** **Fresh 8-lens ultracode audit — Batch A+B (security): a DQL CPU-DoS [HIGH] + a new abs-path-leak instance [MED].** A from-scratch multi-lens workflow-audit (24 agents, 3-skeptic adversarial verify) of the shipped rc.54→rc.56 line returned 5 confirmed findings (1 HIGH / 1 MED / 3 LOW). This RC ships the two security ones. **DQL-PARSE-QUADRATIC-DOS [HIGH]**: the always-registered read-only `obsidian_dataview_query` had no length cap and an O(n²) clause tokenizer → a long query pins the main event loop (DoS for all serve-http clients). **OPTDEP-SQLITE-PATH-LEAK-EMBEDDB [MED]**: `embed-db.ts` + `fts5.ts` `better-sqlite3` import errors leaked the host abs path to clients (a new instance of the rc.55 OPTDEP class my line-oriented grep-sweep missed). Both fixed with a fail-closed cap / shared sanitizer + an extended structural inventory invariant. **1240 → 1246 source tests.**
|
|
424
|
+
|
|
425
|
+
**Pre-release (v3.10 line) — fresh-audit Batch A+B (security).**
|
|
426
|
+
|
|
427
|
+
### Fixed
|
|
428
|
+
|
|
429
|
+
- **DQL-PARSE-QUADRATIC-DOS [HIGH] — `obsidian_dataview_query` CPU-DoS** (`src/dql.ts`, `src/tool-registry.ts`). The tool is registered unconditionally (read-only, no CLI gate) with `query: z.string().min(1)` and **no `.max()`**; `splitClauses` re-allocated + upper-cased the whole remaining tail (`input.slice(i).toUpperCase()`) at every whitespace boundary → **O(n²)** on the main event loop, so a long query string from any bearer-auth serve-http client pins the server (denial of service for all concurrent clients), well below the HTTP body cap. Fixed: (1) new `MAX_DQL_QUERY_LEN = 4096` enforced **fail-closed in `parseDql`** (the shared sink — every path goes through it), mirroring `MAX_QUESTION_PATTERN_LEN` / `MAX_LIKE_PATTERN_LEN`; (2) a zod `.max(MAX_DQL_QUERY_LEN)` at the tool boundary (defense-in-depth + clean client error); (3) **linearized** the `splitClauses` tokenizer to a fixed-length per-keyword compare (`input.slice(i, i + k.length).toUpperCase()`), no whole-tail allocation/upcasing — identical behavior, now O(n).
|
|
430
|
+
- **OPTDEP-SQLITE-PATH-LEAK-EMBEDDB [MEDIUM] — better-sqlite3 import error leaked the host abs path** (`src/embed-db.ts`, `src/fts5.ts`). Their `await import("better-sqlite3")` loaders interpolated raw `err.message` into the thrown Error; Node's `ERR_MODULE_NOT_FOUND` message embeds the importing file's **absolute path** (`imported from /Users/<you>/.../dist/embed-db.js`), and on a serve-http `obsidian_search` (embed-db file present but the optional dep unresolvable) that error reaches the client via `signal_errors.embeddings`. A new instance of the rc.55 OPTDEP-MODULE-PATH-LEAK class — the rc.55 invariant inventory was `[ocr, pdf, embeddings]`, so the two sqlite loaders were the missed siblings (and my rc.55 follow-up sweep used a LINE-oriented `grep` that structurally can't see the multi-line `${…}` interpolation here). Fixed: both loaders (outer import + inner native-binding probe) route through `optionalDepDetail` (code only, no path).
|
|
431
|
+
|
|
432
|
+
### Added
|
|
433
|
+
|
|
434
|
+
- **`tests/parser-input-cap-invariant.test.ts`** — curated structural guard: every always-registered parser-fed tool input (`obsidian_open_questions` `pattern`, `obsidian_dataview_query` `query`) must carry a `.max(<cap>)` in its registered zod schema, else CI fails (+ NEGATIVE control proving the detector isn't vacuous). Closes the "unbounded client string → superlinear parser" DoS class structurally.
|
|
435
|
+
- **`tests/dql.test.ts`** — DQL DoS-closure tests: an over-length query is rejected in O(1) (a 2 MB pathological input rejected <50ms, never entering the tokenizer); a maximally-pathological query AT the cap parses fast (<250ms, proving the linearized tokenizer); a valid query just under the cap parses (NEGATIVE control — not over-capping).
|
|
436
|
+
- **`tests/optional-dep-leak-invariant.test.ts`** inventory extended 3 → 5 files (`embed-db.ts`, `fts5.ts`) so the next better-sqlite3-loader leak fails CI.
|
|
437
|
+
|
|
438
|
+
### Tests (1246)
|
|
439
|
+
|
|
440
|
+
- +6 source `it()`: 3 in `tests/dql.test.ts` (DQL length-cap describe) + 3 in `tests/parser-input-cap-invariant.test.ts`. The optional-dep-leak inventory extension is a list change (no new `it()`). 1240 → 1246.
|
|
441
|
+
|
|
442
|
+
> **Lesson:** my rc.55 OPTDEP fix was an INSTANCE fix (3 files) whose sibling-sweep used a LINE-oriented `grep` that structurally cannot see a multi-line `${…}` interpolation — so embed-db.ts/fts5.ts escaped until this audit read the actual code paths. The durable close is the inventory invariant *covering the files* (the JS detector was always multi-line-capable), not a one-off grep — the project's signature "instance fix ≠ class fix" recursion, now ended for the sqlite loaders. And: an always-registered tool with an unbounded string input is a DoS until proven capped — the parser-input-cap invariant makes that a permanent gate.
|
|
443
|
+
|
|
444
|
+
---
|
|
445
|
+
|
|
446
|
+
## [3.10.0-rc.56] — 2026-06-19
|
|
447
|
+
|
|
448
|
+
> **TL;DR:** **Closes the rc.53 audit — its final 2 LOW findings** (rc.54 closed 16, rc.55 closed 3, of 21 confirmed). **RS-3**: `docs/QUICKSTART.md` still cited `pdfjs-dist@5.7+` as the Node-floor lowest-common-denominator after the rc.52 5→6 bump → `pdfjs-dist@6+`. **FM-3** (documented-rejection verdict): tab-indented YAML frontmatter throws on js-yaml@4 → `parseNote` whole-body fallback — verified NOT a migration regression (the YAML spec forbids tabs for indentation; js-yaml@3/gray-matter enforced it identically) and not data loss (the frontmatter text stays indexed in the body), so it's documented + pinned with a throw-contract test rather than "fixed". **1239 → 1240 source tests.**
|
|
449
|
+
|
|
450
|
+
**Pre-release (v3.10 line) — rc.53 audit closure.**
|
|
451
|
+
|
|
452
|
+
### Fixed
|
|
453
|
+
|
|
454
|
+
- **RS-3 [LOW] — stale `pdfjs-dist@5.7+` reference in QUICKSTART** (`docs/QUICKSTART.md`, 2 sites). The rc.52 bump moved the optional dep to `^6.0.227`, but the Node-floor rationale ("`pdfjs-dist@5.7+` requires `>=22.13.0`") still named the old major. Updated to `pdfjs-dist@6+` (v6 has the same `>=22.13.0` engines floor, so the Node-version rationale is unchanged).
|
|
455
|
+
|
|
456
|
+
### Documented (rejection verdict)
|
|
457
|
+
|
|
458
|
+
- **FM-3 [LOW] — tab-indented frontmatter is NOT a migration regression.** The audit flagged that js-yaml@4 throws on tab-indented YAML ("tab characters must not be used in indentation"), after which `parseNote` falls back to treating the whole file as body. Verified this is **not** a behavior change from dropping gray-matter: the YAML spec forbids tabs for indentation and js-yaml@3 (gray-matter's engine) rejected it identically — and on the fallback the frontmatter **text stays indexed and searchable in the body** (no data loss), it simply isn't parsed into structured `data`. Documented in the `src/frontmatter.ts` header and pinned with a throw-contract test, per the project's documented-rejection pattern (don't "fix" a non-bug).
|
|
459
|
+
|
|
460
|
+
### Tests (1240)
|
|
461
|
+
|
|
462
|
+
- +1 source `it()` in `tests/frontmatter.test.ts` (tab-indented frontmatter throws — pins the FM-3 contract). 1239 → 1240.
|
|
463
|
+
|
|
464
|
+
> **Lesson:** an audit finding can be REAL-but-not-a-bug — verify the claimed regression against the prior behavior (here: both YAML engines reject tabs per spec, and the text isn't lost) before "fixing" it; the documented-rejection verdict + a contract test is the right close, not a code change that papers over a non-issue. **This concludes the entire rc.53 audit cascade (21 confirmed + 1 recursion, all shipped or reasoned-rejected across rc.54 → rc.55 → rc.56).**
|
|
465
|
+
|
|
466
|
+
---
|
|
467
|
+
|
|
468
|
+
## [3.10.0-rc.55] — 2026-06-19
|
|
469
|
+
|
|
470
|
+
> **TL;DR:** **Independent code-correctness batch — 3 findings from the rc.53 audit + 1 recursion the post-rc.54 audit caught.** **CT-LINE-OFFBY1** (`chat_thread_append` `line_start` pointed one line before the appended `### role` heading; new-note branch hardcoded the blank line) → derived from the heading offset across all 3 branches. **CHUNK-SURROGATE-SPLIT** (FTS5 oversize-line hard-cut split surrogate pairs → lone surrogates in the index) → surrogate-safe cut. **OPTDEP-MODULE-PATH-LEAK-02** (optional-dep `import()` errors echoed Node's "imported from /Users/…/dist/…" abs path to serve-http clients — abs-path-leak sibling outside rc.49's Vault scope) → shared `optionalDepDetail` surfaces only the error code, + inventory invariant. **FM-SCALAR-DATE** (rc.54's coercion let a bare top-level Date scalar slip through — a recursion of the class rc.54 closed) → plain-object check. **1233 → 1239 source tests.**
|
|
471
|
+
|
|
472
|
+
**Pre-release (v3.10 line) — code-correctness batch.**
|
|
473
|
+
|
|
474
|
+
### Fixed
|
|
475
|
+
|
|
476
|
+
- **CT-LINE-OFFBY1 [MEDIUM, recurrence of rc.50 CODE-2] — `chat_thread_append` `line_start` pointed one line before the message heading** (`src/tools/read.ts`). For the existing-thread branch `line_start` counted newlines in the pre-append body + 1 → the prior content line, one line above the new `### role · ts` heading; the new-note branch hardcoded `4`, which is the blank line (the heading is line 5). All three branches now derive `line_start` from the heading marker's offset in the FINAL written content, and `line_end` from the trimmed message-block's newline count — addressing the actual file and pointing AT the heading (rc.50's no-past-EOF property preserved).
|
|
477
|
+
- **CHUNK-SURROGATE-SPLIT [MEDIUM] — FTS5 oversize-line hard-cut split surrogate pairs** (`src/fts5.ts`). `chunkContent`'s single-line hard-cut used `slice(i, i + maxChars)`, which works on UTF-16 code UNITS — a boundary landing mid-emoji emitted a lone surrogate (a corrupt code point) into the indexed chunk. The cut now backs off by one when the boundary unit is a high surrogate, so a pair is never split (a chunk may be `maxChars − 1` units; re-joining the chunks is lossless).
|
|
478
|
+
- **OPTDEP-MODULE-PATH-LEAK-02 [HIGH, abs-path-leak sibling] — optional-dep import errors leaked the host abs path** (`src/ocr.ts`, `src/pdf.ts`, `src/embeddings.ts`). A failed `await import("tesseract.js" | "@napi-rs/canvas" | "pdfjs-dist" | "@huggingface/transformers")` threw an Error that interpolated `err.message` — Node's `ERR_MODULE_NOT_FOUND` message embeds the importing file's ABSOLUTE path (`Cannot find package 'X' imported from /Users/<you>/.../dist/ocr.js`), leaking the host filesystem layout to bearer-auth serve-http clients (the module-resolution sibling of the rc.45/rc.49 vault-fs leak class, outside the Vault's `*Safe` wrappers). New leaf `src/optional-dep.ts` `optionalDepDetail(err)` surfaces only the error CODE; all 6 import-catches route through it.
|
|
479
|
+
- **FM-SCALAR-DATE [post-rc.54-audit recursion] — rc.54's non-mapping coercion let a Date slip through** (`src/frontmatter.ts`). rc.54's `typeof loaded === "object" && !Array.isArray(loaded)` check accepted a bare top-level Date scalar (`---\n2026-01-01\n---`, which js-yaml resolves to a `Date` instance) as `data` — a recursion of the very FM-SCALAR class rc.54 closed. Tightened to a PLAIN-object check (`isPlainObject`: rejects `Date`/`RegExp`/array/`null`, accepts a YAML mapping) + a Date-scalar regression test.
|
|
480
|
+
|
|
481
|
+
### Added
|
|
482
|
+
|
|
483
|
+
- **`src/optional-dep.ts`** — `optionalDepDetail(err)`, a privacy-safe detail string (`error code: <code>`) for failed optional-dependency imports; never echoes `err.message` (which embeds the abs path).
|
|
484
|
+
- **`tests/optional-dep-leak-invariant.test.ts`** — curated-inventory invariant: a raw `${err.message}` / `${String(err)}` interpolation in any optional-dep loader (`ocr.ts`/`pdf.ts`/`embeddings.ts`) fails CI (+ NEGATIVE control proving the detector isn't vacuous + a `optionalDepDetail` unit asserting no path leaks).
|
|
485
|
+
|
|
486
|
+
### Tests (1239)
|
|
487
|
+
|
|
488
|
+
- +6 source `it()`: 3 in `tests/optional-dep-leak-invariant.test.ts` (2 POSITIVE + 1 NEGATIVE control) + 1 in `tests/chat-thread.test.ts` (heading-line across all 3 branches) + 1 in `tests/fts5.test.ts` (surrogate-safe hard-cut) + 1 in `tests/frontmatter.test.ts` (Date-scalar coercion). 1233 → 1239.
|
|
489
|
+
|
|
490
|
+
> **Lesson:** the marathon's audit-after-each-stage caught rc.54's own FM-SCALAR coercion recursing one RC later (a bare Date slipped the `typeof === "object"` check) — and the durable close for the optional-dep leak is the inventory invariant, mirroring rc.49's Vault leak invariant: convert "did every import catch avoid echoing the path?" into a self-checking gate.
|
|
491
|
+
|
|
492
|
+
---
|
|
493
|
+
|
|
494
|
+
## [3.10.0-rc.54] — 2026-06-19
|
|
495
|
+
|
|
496
|
+
> **TL;DR:** **Frontmatter-migration hardening — overclaim #20 + a real corruption bug + the gray-matter doc-drift sweep.** The post-rc.53 state-driven audit confirmed the js-yaml@4 migration is fundamentally SOUND, but rc.53's **"byte-identical port"** claim was an overclaim (**#20**): js-yaml@4 is YAML **1.2** vs gray-matter's js-yaml@3 YAML **1.1**, so SCALAR resolution diverges (`0755`→755 not 493, `12:34:56`→string not 45296, `1_000`→string not 1000). The claim is now scoped to STRUCTURAL parsing and the divergence pinned as a deliberate contract. Also fixes a **real corruption bug** (a non-mapping frontmatter doc was cast to `Record` and would be written back char-indexed by `frontmatter_set`), a **pre-existing SECURITY.md enforcement overclaim** (`.base` YAML "no anchor-expansion / YAML bomb rejected" — js-yaml does neither), the full gray-matter→js-yaml doc-drift sweep, and SC-1 (drop the `@huggingface/transformers` dev/optional dep duplication). New structural guard: `tests/no-graymatter-invariant.test.ts`. **1224 → 1233 source tests.**
|
|
497
|
+
|
|
498
|
+
**Pre-release (v3.10 line) — post-rc.53 audit response.**
|
|
499
|
+
|
|
500
|
+
### Fixed
|
|
501
|
+
|
|
502
|
+
- **Overclaim #20 — rc.53 "byte-identical port" (claimed-guarantee-vs-reality).** `src/frontmatter.ts` ran on js-yaml@4 (YAML 1.2 core) while gray-matter used js-yaml@3 (YAML 1.1); they resolve some scalars DIFFERENTLY — bare octal `0755`→`755` (was `493`), leading-zero `0888`→`888` (was `"0888"`), sexagesimal `12:34:56`→`"12:34:56"` (was int `45296`), underscore `1_000`→`"1_000"` (was `1000`). The rc.53 differential corpus contained none of these shapes, so the diff "passed" while incomplete on the scalar dimension. The header comment now scopes "byte-identical" to STRUCTURAL parsing and documents the divergence; `tests/frontmatter.test.ts` pins the 4 scalar resolutions as a deliberate js-yaml@4 contract. CHANGELOG rc.53 wording corrected inline.
|
|
503
|
+
- **FM-SCALAR (corruption) — a NON-mapping frontmatter document was cast to `Record`.** A frontmatter block that's a bare scalar (`---\nhello\n---`) or a sequence (`---\n- a\n- b\n---`) was returned as `data` and would be spread char-indexed by `frontmatter_set`, writing corrupt YAML back. `parseFrontmatter` now coerces any non-object/array top-level document to `{}` (gray-matter parity). +2 guards.
|
|
504
|
+
- **SECURITY.md `.base` enforcement overclaim (claimed-guarantee-vs-code-guard).** The threat-model claimed YAML is parsed "via `SAFE_SCHEMA` … No anchor-expansion … A YAML bomb … is rejected at parse time" — but js-yaml (v3 AND v4) resolves anchors/aliases and has no billion-laughs guard, so this was a PRE-EXISTING overclaim (true even under gray-matter). Rewritten to the enforced reality: js-yaml@4 default `load` (YAML 1.2 core schema, safe-by-default — no `!!js/function`), the merge-key quadratic DoS fixed in v4, alias bombs NOT specifically rejected and bounded only by the single-user local-vault threat model.
|
|
505
|
+
- **gray-matter doc-drift sweep.** README, SECURITY.md, `docs/api.md`, CONTRIBUTING.md, `typedoc.json`, and the `read.ts` / `write.ts` TSDoc still named gray-matter as the live frontmatter engine ~1 RC after its removal → all updated to js-yaml@4 (CONTRIBUTING mandatory-deps `gray-matter`→`js-yaml`; `typedoc.json` 44→45 tools).
|
|
506
|
+
- **SC-1 — `@huggingface/transformers` was declared in BOTH `devDependencies` and `optionalDependencies`.** Removed the dev duplicate (it's a runtime-optional dep — needed for `build-embeddings` / `--enable-reranker`). This also correctly scopes its `protobufjs` subtree as a prod-optional dependency in `npm audit` (already covered by the `overrides` → 7.6.4 pin).
|
|
507
|
+
|
|
508
|
+
### Added
|
|
509
|
+
|
|
510
|
+
- **`tests/no-graymatter-invariant.test.ts`** — structural guard (auto-tracked by the META-invariant): fails CI if `gray-matter` reappears as a declared dependency in any `package.json` section OR is imported/required in any `src/**/*.ts` module. Prose mentions in comments (documenting the port's provenance) are intentionally allowed; the detector matches only an actual import/require (+ NEGATIVE control proving it isn't vacuous).
|
|
511
|
+
- **`tests/frontmatter.test.ts`** scalar-contract block — 4 `it()`s pinning the YAML-1.2 resolutions (octal/leading-zero/sexagesimal/underscore) as a deliberate, documented contract + 2 non-mapping-coercion guards (bare scalar → `{}`, sequence → `{}`).
|
|
512
|
+
|
|
513
|
+
### Tests (1233)
|
|
514
|
+
|
|
515
|
+
- +9 source `it()`: 6 in `tests/frontmatter.test.ts` (4 scalar-contract + 2 FM-SCALAR coercion) + 3 in the new `tests/no-graymatter-invariant.test.ts` (2 POSITIVE + 1 NEGATIVE control). 1224 → 1233.
|
|
516
|
+
|
|
517
|
+
> **Lesson:** a parser migration's differential corpus must cover the SCALAR-RESOLUTION dimension, not just the structural split — two parsers can be byte-identical on delimiters/stringify and still resolve `0755` differently. "Byte-identical port" is exactly the enforced-guarantee claim that the marathon's audit-after-each-stage gate is meant to catch — and it did, one RC later.
|
|
518
|
+
|
|
519
|
+
---
|
|
520
|
+
|
|
521
|
+
## [3.10.0-rc.53] — 2026-06-19
|
|
522
|
+
|
|
523
|
+
> **TL;DR:** **Dropped gray-matter → js-yaml@4; the js-yaml advisory is now RESOLVED, not allowlisted (tree fully clean).** gray-matter@4 hard-binds js-yaml@3's removed `safeLoad`/`safeDump`, which pinned the vulnerable js-yaml@3 (GHSA-h67p-54hq-rp68) in the tree with no v3 fix. Replaced it with a tiny in-repo `src/frontmatter.ts` (a faithful PORT of gray-matter's split + stringify, differentially validated byte-identical on the STRUCTURAL paths — delimiter split + stringify — over a broad corpus before removal) on **js-yaml@4.2.0** (whose `load`/`dump` are safe-by-default). The scoped audit `ALLOWLIST` is now **empty** — strictest posture. **1213 → 1224 source tests.** _(Correction in rc.54: "byte-identical" holds for STRUCTURAL parsing only — js-yaml@4 is YAML 1.2, so SCALAR resolution diverges from gray-matter's js-yaml@3/YAML-1.1 on octal/sexagesimal/underscore shapes; see the rc.54 entry.)_
|
|
524
|
+
|
|
525
|
+
**Pre-release (v3.10 line) — gray-matter → js-yaml@4 migration (resolves #170).**
|
|
526
|
+
|
|
527
|
+
### Changed
|
|
528
|
+
|
|
529
|
+
- **Replaced `gray-matter` with `src/frontmatter.ts` + `js-yaml@^4.2.0`.** New `parseFrontmatter` / `stringifyFrontmatter` port gray-matter's exact semantics — the `---` delimiter split, the `----` (4-dash) guard, the comment-only-emptiness check, the leading CR/LF strip after the closing fence, the UTF-8 BOM strip, and the `newline()` join — with only the YAML engine swapped to js-yaml@4 (`load`/`dump` = the v3 `safeLoad`/`safeDump` semantics). `content` stays a verbatim suffix of the input, so `parser.ts`'s `bodyStartLine` `lastIndexOf` is unaffected. Swapped at all call sites: `parser.ts` (parseNote), `tools/meta.ts` (validate_note_proposal), `tools/write.ts` (composeNote + frontmatterSet), `bases.ts` (queryBase frontmatter + `parseBase` — `SAFE_SCHEMA` dropped, v4 `load` is safe-by-default). Added `@types/js-yaml@^4` (real types — no more structural casts). gray-matter removed from `dependencies`.
|
|
530
|
+
- **De-allowlisted GHSA-h67p-54hq-rp68.** With gray-matter gone, no vulnerable js-yaml remains in the tree; the `scripts/check-audit.mjs` `ALLOWLIST` is back to empty (the strictest posture). Resolves the rc.50-accepted advisory at the root (#170).
|
|
531
|
+
|
|
532
|
+
### Added
|
|
533
|
+
|
|
534
|
+
- **`tests/frontmatter.test.ts`** — standalone CI guard for the new module (simple/empty/comment-only/`----`-guard/CRLF/BOM-via-parser/verbatim-suffix parse cases + round-trip + date-string fidelity + a malformed-YAML-throws NEGATIVE control). A dev-only differential test (vs gray-matter, byte-identical on the structural-parse corpus; see the rc.54 scalar-resolution caveat) validated the port pre-removal and was then deleted (it imported the removed dep).
|
|
535
|
+
|
|
536
|
+
### Tests (1224)
|
|
537
|
+
|
|
538
|
+
- +11 source `it()` in `tests/frontmatter.test.ts`; the 4 call-site swaps are covered by the existing parser/write/bases/meta/frontmatter-ops suites (all green — the differential test confirmed parity, incl. the BOM case the corpus initially missed). 1213 → 1224.
|
|
539
|
+
|
|
540
|
+
> **Lesson:** replacing a battle-tested parser is safe ONLY with a differential test against the incumbent over a broad corpus *before* removing it — and even then the corpus has gaps (the BOM case slipped the diff corpus but was caught by the pre-existing parser BOM test, then folded into the port). A migration's verification is the differential diff, not the new code's own tests.
|
|
541
|
+
|
|
542
|
+
---
|
|
543
|
+
|
|
544
|
+
## [3.10.0-rc.52] — 2026-06-18
|
|
545
|
+
|
|
546
|
+
> **TL;DR:** **Dependency bumps — `pdfjs-dist` 5→6 (closes the deferred major) + a newly-surfaced `hono` advisory batch.** Bumped `pdfjs-dist` `^5.7.284 → ^6.0.227` (v6 engines `>=22.13.0` match ours exactly; the Node `legacy/build` import + `getDocument` API are unchanged — verified by the full PDF/OCR suite + a real-extraction smoke). The bump's `npm install` surfaced **5 newly-published `hono` advisories** (1 high CORS + 4 moderate, on the prod transitive via MCP SDK → @hono/node-server, all `<=4.12.24`) — fixed in-range by bumping the existing `overrides` `hono ^4.12.21 → ^4.12.26` (no major bump). **Tests unchanged (1213).**
|
|
547
|
+
|
|
548
|
+
**Pre-release (v3.10 line) — dependency bumps: pdfjs 5→6 + hono advisory fix.**
|
|
549
|
+
|
|
550
|
+
### Changed
|
|
551
|
+
|
|
552
|
+
- **`pdfjs-dist` `^5.7.284 → ^6.0.227`** (`package.json` optionalDeps + the three install-hint strings in `pdf.ts`/`ocr.ts`/`doctor.ts`). pdfjs v6 requires Node `>=22.13.0 || >=24` — identical to our `engines.node`, so no floor change. We consume the cross-environment `pdfjs-dist/legacy/build/pdf.mjs` build + `getDocument`, whose API is stable across the major (the in-the-wild v6 migration pain is webpack/ESM bundling, which doesn't apply to our Node dynamic import). Verified: all 55 `tests/pdf.test.ts` + `tests/ocr*.test.ts` pass against v6 (they parse real PDF fixtures), `isPdfjsAvailable()` true, full suite green. Closes the rc.48-deferred major bump (dependabot #177).
|
|
553
|
+
- **`hono` override `^4.12.21 → ^4.12.26`** — the pdfjs `npm install` re-resolved hono and surfaced 5 newly-published advisories on `hono <=4.12.24` (prod transitive via `@modelcontextprotocol/sdk` → `@hono/node-server`): GHSA-88fw-hqm2-52qc (high — CORS wildcard reflects Origin with credentials), + 4 moderate (serve-static path traversal on Windows; AWS-Lambda/Lambda@Edge header/cookie handling; body-limit bypass). Most are Lambda-adapter-specific (not our Node `serve-http`), but all are cleared by the in-range bump to 4.12.26 (no `--force`). Scoped audit gate green (only the documented js-yaml GHSA remains allowlisted).
|
|
554
|
+
|
|
555
|
+
### Tests (1213)
|
|
556
|
+
|
|
557
|
+
- No new tests — dependency bump + hint-string edits + an override version. Coverage of the bump is the existing PDF/OCR suites against the real v6 package (all green). 1213 unchanged.
|
|
558
|
+
|
|
559
|
+
---
|
|
560
|
+
|
|
561
|
+
## [3.10.0-rc.51] — 2026-06-16
|
|
562
|
+
|
|
563
|
+
> **TL;DR:** **Re-audit batch 3 (final) — docs-drift LOWs + class-D structural guards.** Two claim-vs-reality doc drifts the gates didn't catch: `docs/api.md` described a **non-existent `--hnsw-persist`** CLI flag (only `--no-hnsw-persist` exists; persistence is opt-out default-on), and `SECURITY.md` asserted a **fixed 4 MB** HTTP body cap while the code DERIVES `max(4 MB, max-file-bytes × 1.5)` (7.5 MB at the default). Both fixed + pinned with a new invariant so they can't recur. **1210 → 1213 source tests.** This closes the rc45-48 re-audit (1 HIGH + 2 MED + 3 LOW all shipped).
|
|
564
|
+
|
|
565
|
+
**Pre-release (v3.10 line) — re-audit batch 3: docs drift + class-D guards.**
|
|
566
|
+
|
|
567
|
+
### Fixed
|
|
568
|
+
|
|
569
|
+
- **DOC-HNSW-PERSIST-PHANTOM-FLAG [LOW]** (`docs/api.md`). The `--use-hnsw` row's description said the index reloads "from `.hnsw.bin` sidecar if `--hnsw-persist`" — but `--hnsw-persist` never existed; persistence is **on by default**, opt-out via `--no-hnsw-persist` (correctly stated two rows down). Reworded to "reloaded from the `.hnsw.bin` sidecar by default — opt out with `--no-hnsw-persist`".
|
|
570
|
+
- **DOC-SECURITY-HTTP-BODY-CAP-STALE [LOW]** (`SECURITY.md`). The body-bomb mitigation claimed a "Per-request body size cap (4 MB)", but `http-transport.ts` `deriveHttpBodyCap` computes `max(4 MB, maxFileBytes × 1.5)` = **7.5 MB** at the default 5 MB file cap (stale since v3.7.12 M4; `docs/http-transport.md` was already correct). Updated to the derived formula.
|
|
571
|
+
|
|
572
|
+
### Added
|
|
573
|
+
|
|
574
|
+
- **`tests/cli-flag-docs-invariant.test.ts` (class-D structural guard).** (1) Scans `docs/api.md` for every `--flag` token and asserts each resolves to a real commander `.option()` in `src/cli.ts` (+ builtins) — a phantom flag in the canonical flag-doc surface now fails CI; ships a NEGATIVE control (flags `--hnsw-persist`, clears `--no-hnsw-persist`). (2) Pins the `SECURITY.md` body-bomb line to the *derived* wording (`--max-file-bytes` + `1.5×`) so it can't drift back to a bare fixed number. OIA Check 3 validates subcommands, not `--flag` tokens in prose — these close that gap.
|
|
575
|
+
|
|
576
|
+
### Tests (1213)
|
|
577
|
+
|
|
578
|
+
- +3 source `it()` (flag-in-docs scan + NEGATIVE control + body-cap wording pin). 1210 → 1213.
|
|
579
|
+
|
|
580
|
+
> **Re-audit closed:** the 6-lens state-driven re-audit of the rc.45→rc.48 line (1 HIGH + 2 MEDIUM + 3 LOW) is fully shipped — rc.49 (abs-path-leak class), rc.50 (js-yaml phantom dep + protobufjs advisory + scoped audit gate + chatThreadAppend), rc.51 (docs drift). Tracked follow-ups: js-yaml advisory de-allowlist (gray-matter YAML-engine migration), pdfjs-dist 5→6 (PR #177).
|
|
581
|
+
|
|
582
|
+
---
|
|
583
|
+
|
|
584
|
+
## [3.10.0-rc.50] — 2026-06-16
|
|
585
|
+
|
|
586
|
+
> **TL;DR:** **Re-audit batch 2 — supply-chain (phantom dep + newly-published advisories) + the chatThreadAppend line-drift LOW.** Declaring the previously-phantom `js-yaml` dep surfaced that `npm audit` had gone red **project-wide**: two newly-published advisories on transitive deps (`js-yaml` moderate via gray-matter; `protobufjs` high+moderate via the dev/optional `@huggingface/transformers`→onnxruntime). protobufjs is **fixed** (override → 7.6.4, in-range, no break). js-yaml has **no fix that doesn't break gray-matter** (v4 removed `safeLoad`), so it's accepted via a new **scoped audit gate** that fails on every advisory except documented ones — keeping the bar strict for all else. Plus the chatThreadAppend line_start/line_end range fix (CODE-2) and a phantom-import inventory invariant. **1204 → 1210 source tests.**
|
|
587
|
+
|
|
588
|
+
**Pre-release (v3.10 line) — re-audit batch 2: supply-chain + line-drift.**
|
|
589
|
+
|
|
590
|
+
### Fixed
|
|
591
|
+
|
|
592
|
+
- **SC-PHANTOM-JSYAML-01 [MEDIUM] — `js-yaml` was a phantom (undeclared) runtime dep** (`package.json`, `src/bases.ts`). `bases.ts` does `await import("js-yaml")` for a CORE feature (`.base` parsing) but js-yaml was undeclared — it resolved only via gray-matter's transitive pin + npm hoisting (would break under pnpm-no-hoist / Yarn PnP / a gray-matter major). Declared `"js-yaml": "^3.14.0"` (pinned to v3 for the `SAFE_SCHEMA` + 2-arg `load` API the code uses).
|
|
593
|
+
- **protobufjs high+moderate advisories (GHSA-wcpc-wj8m-hjx6 + GHSA-f38q-mgvj-vph7) — fixed via `overrides`.** A newly-published high DoS (+ moderate property-shadow) on protobufjs `<=7.6.2`, reached transitively through the dev/optional `@huggingface/transformers` → onnxruntime-web. In-range fix (no major bump): `overrides: { "protobufjs": "^7.6.4" }`.
|
|
594
|
+
- **CODE-2-chatthread-line-drift [LOW] — `chatThreadAppend` reported line numbers past EOF** (`src/tools/read.ts`). `line_start` counted newlines in the un-stripped `body` while the write strips trailing newlines (`newBody = body.replace(/\n+$/,"") + toAppend`), so the reported range drifted UP by the stripped-newline count (could point past EOF). Now counts in the trimmed string actually written; `line_end` advances by the appended block's newline COUNT (not `split("\n").length`, which over-counts by one). Fixed the sibling imprecision in the new-note branch too.
|
|
595
|
+
|
|
596
|
+
### Added
|
|
597
|
+
|
|
598
|
+
- **Scoped `npm audit` gate (`scripts/check-audit.mjs` + `check:audit`).** Replaces the bare `npm audit --audit-level` calls in `prepublishOnly` + `ci.yml` + `release.yml` with one wrapper that keeps the same thresholds (prod ≥ moderate, dev ≥ high) but fails on every advisory EXCEPT documented ones in an `ALLOWLIST` (each with a written rationale + resolution path). Currently allowlists only **GHSA-h67p-54hq-rp68** (js-yaml merge-key DoS): no v3 fix; js-yaml@4.2.0 removes `safeLoad`, breaking gray-matter at import; the DoS needs attacker-controlled YAML in the user's OWN local vault. Tracked for resolution (migrate frontmatter parsing to js-yaml@4 / drop gray-matter). This is the project's documented-rejection pattern applied to supply-chain — accept *with reasoning, in a reviewable place*, not by lowering the gate.
|
|
599
|
+
- **`tests/phantom-import-invariant.test.ts`** — scans every dynamic `import("x")` in `src/` and fails CI if its package root isn't in `dependencies`/`optionalDependencies` (node: builtins + relative excluded). The next phantom dynamic dep can't ship. + NEGATIVE control.
|
|
600
|
+
- **`tests/check-audit.test.ts`** — unit-tests the scoped gate's pure core (an un-allowlisted advisory fails; the documented one passes; below-threshold ignored) + a drift guard pinning the allowlist to its single documented entry.
|
|
601
|
+
|
|
602
|
+
### Tests (1210)
|
|
603
|
+
|
|
604
|
+
- +6 source `it()`: phantom-import invariant (scan + NEGATIVE), check-audit gate (3), chatThreadAppend range-within-file (1). 1204 → 1210.
|
|
605
|
+
|
|
606
|
+
> **Note:** `ci.yml` + `release.yml` audit steps were switched to the wrapper — this PR touches `.github/workflows/`, so it requires a maintainer web-UI merge (CI token lacks `workflow` scope).
|
|
607
|
+
|
|
608
|
+
---
|
|
609
|
+
|
|
610
|
+
## [3.10.0-rc.49] — 2026-06-15
|
|
611
|
+
|
|
612
|
+
> **TL;DR:** **abs-path-leak class — TRUE root closure + inventory invariant (HIGH).** A multi-lens re-audit of the shipped rc.45→rc.48 line found that rc.45's abs-path-leak fix had **recursed its own class**: it wrapped 3 READ sinks (readFile/readBinaryFile/stat) and claimed the class closed "at the SOURCE every caller funnels through", but the **write path** (`writeNote`/`renameFile`/`appendNote` — HIGH) and `readNote` (the primary read funnel — MEDIUM) still threw RAW fs errors embedding the host's absolute vault/home path to any bearer-token serve-http client. rc.49 routes **every** `fs` sink in `Vault` through sanitizing wrappers, adds a behavioral leak test (which caught a *further* residual — `resolveSafePath`'s `realpath` leaked `ENOTDIR` with the abs path), and ships a P0 inventory invariant so the next sink physically cannot escape. **1200 → 1204 source tests.**
|
|
613
|
+
|
|
614
|
+
**Pre-release (v3.10 line) — re-audit batch 1: abs-path-leak class true closure.**
|
|
615
|
+
|
|
616
|
+
### Fixed
|
|
617
|
+
|
|
618
|
+
- **HIGH (RC45-WRITEPATH-LEAK) + MEDIUM (CODE-1) — abs-path-leak class residual across the write path + readNote** (`src/vault.ts`). rc.45 sanitized only `readFile`/`readBinaryFile`/`stat`. The re-audit confirmed `writeNote` (`fs.mkdir`/`fs.writeFile`/`fs.stat`), `renameFile` (`fs.rename`/`fs.link`/`fs.copyFile`/`fs.unlink`/`fs.stat`/`fs.mkdir`), `appendNote` (`fs.open`/`fs.stat`), and `readNote` (`fs.stat`/`fs.readFile`) all threw raw errors embedding the absolute host path — reaching MCP clients verbatim on `serve-http` (e.g. `replaceInNotes` surfaces `err.message`). Root fix: new private sanitizing wrappers (`statSafe`/`realpathSafe`/`readFileSafe`/`writeFileSafe`/`mkdirSafe`/`openSafe`/`renameSafe`/`linkSafe`/`copyFileSafe`/`unlinkSafe`); **every** raw `fs` sink in the `Vault` class (27 call sites, incl. the cache/startup internals) now routes through them. `err.code` is preserved, so the `EEXIST`/`EXDEV` control flow in writeNote/renameFile is unchanged.
|
|
619
|
+
- **HIGH residual found by the behavioral test — `resolveSafePath`'s `realpath` leaked the abs path** (`src/vault.ts`). The first static sweep excluded `realpath` (assumed always `.catch`-guarded); the new behavioral test proved `appendNote("file.md/child")` threw `ENOTDIR: … realpath '<abs>'` raw. Added `realpathSafe` + converted the two unguarded `realpath` sinks (`resolveSafePath`, the startup root-resolve). This is exactly why a behavioral test (not just a static grep) is required for a leak class.
|
|
620
|
+
|
|
621
|
+
### Added
|
|
622
|
+
|
|
623
|
+
- **`tests/abs-path-leak-invariant.test.ts` (P0 inventory invariant).** A pure detector parses the `Vault` class and fails CI if any method contains a raw (non-`.catch`-guarded) `fs` sink without referencing `sanitizeFsError` (or an explicit `abs-path-safe` exemption). Converts "did we sanitize every fs sink?" (recursion-prone — rc.45 missed 23 of 27) into a self-checking gate; ships a NEGATIVE control (flags a leaky method, clears a sanitized one, ignores a `.catch` probe + an exemption marker). Plus **behavioral leak tests** in `tests/security.test.ts`: `readNote` + the write path (`writeNote`/`renameFile`/`appendNote`) on a forced fs error assert the message never contains the vault root.
|
|
624
|
+
|
|
625
|
+
### Tests (1204)
|
|
626
|
+
|
|
627
|
+
- +4 source `it()`: the inventory invariant (scan + NEGATIVE control) and 2 behavioral leak tests (readNote; write-path trio). 1200 → 1204. The 27-site wrapper conversion is covered by the full vault/security/write/read suites (all green — `err.code` preserved, no behavioral change).
|
|
628
|
+
|
|
629
|
+
> **Re-audit sequencing:** rc.50 = `js-yaml` phantom dep (MED) + `chatThreadAppend` line drift (LOW) + phantom-import OIA; rc.51 = docs drift (api.md `--hnsw-persist` phantom flag, SECURITY.md body-cap) + class-D guards. The remaining meta P0 hardening (golden-value scoring spec, unicode fuzz, enforcement-verb OIA) follows.
|
|
630
|
+
|
|
631
|
+
---
|
|
632
|
+
|
|
633
|
+
## [3.10.0-rc.48] — 2026-06-15
|
|
634
|
+
|
|
635
|
+
> **TL;DR:** **Ultracode audit fix-batch 6 — docs/TSDoc-drift + the prompts-table invariant + a frontmatter-fidelity LOW.** Closes the RCA's `help-parser-tsdoc-drift` ×3 (the overclaim #16 OCR "download on first use" residual in `tool-registry.ts`, and a `ServeOptions` TSDoc naming two tools that don't exist) + `structural-defense-scope` ×1 (the docs/api.md prompts table was stale at **10 of 19** with no invariant pinning it — now backfilled to 19 + guarded). Plus the verified-real `frontmatterSet` trailing-newline fidelity LOW, the STABILITY always-on header (33→34), and a dead `files[]` entry. **1197 → 1200 source tests.** The `pdfjs-dist` 5→6 major bump (PR #177) is split into its own RC for isolated verification.
|
|
636
|
+
|
|
637
|
+
**Pre-release (v3.10 line) — audit fix-batch 6: docs/TSDoc drift + structural guard.**
|
|
638
|
+
|
|
639
|
+
### Fixed
|
|
640
|
+
|
|
641
|
+
- **MEDIUM — `obsidian_ocr_pdf` description claimed runtime language-pack download** (`src/tool-registry.ts`, overclaim #16 residual). The tool description + the inline `lang`-schema comment still said trained-data "download on first use", but rc.10 made OCR offline-enforced (`assertOcrLangsInstalled` fails closed; the worker is `cacheMethod: "readOnly"` — zero outbound calls). Rewritten to the enforced reality: packs must be pre-installed via `enquire-mcp install-ocr-lang <code>`.
|
|
642
|
+
- **LOW — `ServeOptions.enableWrite` TSDoc named two non-existent tools** (`src/server.ts`). It listed `obsidian_append_note` + `obsidian_rename_file`; the real handlers are `obsidian_append_to_note` + `obsidian_rename_note`. Corrected.
|
|
643
|
+
- **LOW — `frontmatterSet` altered the body's trailing-newline state** (`src/tools/write.ts`, roundtrip-serialization-fidelity). `matter.stringify` always appends a `\n`; a frontmatter-only edit on a body saved without a trailing newline silently introduced one. gray-matter's `.content` faithfully preserves the original state, so the fix drops the stringify-added newline iff the original body lacked one. Verified empirically against gray-matter round-trip behavior.
|
|
644
|
+
- **LOW — STABILITY.md "always-on (33)" header off by one** (`STABILITY.md`). The list under it has 34 entries (incl. `obsidian_stale_notes`); the header said 33. → 34.
|
|
645
|
+
- **LOW — dead `files[]` entry** (`package.json`). `docs/api-reference` is GH-Pages-generated (not git-tracked, never produced before `npm pack` — `prepublishOnly` doesn't run `docs:api`), so it never shipped in the tarball. Removed the misleading entry.
|
|
646
|
+
|
|
647
|
+
### Added
|
|
648
|
+
|
|
649
|
+
- **docs/api.md prompts-table invariant** (`tests/docs-consistency.test.ts`). The prompts table was stale at 10 of 19 with nothing guarding it. Backfilled the 9 missing prompts (`search_with_query_expansion`, `vault_synth`, `vault_wiki_compile`, `vault_lint_extended`, `vault_capture`, `vault_persona_search`, `vault_automation_setup`, `vault_research`, `vault_synthesis_page`) + a new invariant asserting every `registerPrompt()` in `src/prompts.ts` appears in the api.md prompts section (with a NEGATIVE control proving the detector flags an absent prompt).
|
|
650
|
+
|
|
651
|
+
### Tests (1200)
|
|
652
|
+
|
|
653
|
+
- +3 source `it()`: the api.md prompts-table invariant + its NEGATIVE control (`tests/docs-consistency.test.ts`), and the `frontmatterSet` trailing-newline fidelity test (`tests/frontmatter-ops.test.ts`). 1197 → 1200.
|
|
654
|
+
|
|
655
|
+
> **Deferred to its own RC:** `pdfjs-dist` `^5.7.284` → `^6.0.227` (PR [#177](https://github.com/oomkapwn/enquire-mcp/pull/177)) — a MAJOR bump that needs isolated verification (install + PDF-render + OCR-canvas paths) rather than coupling an untested upgrade to this clean docs batch.
|
|
656
|
+
|
|
657
|
+
---
|
|
658
|
+
|
|
659
|
+
## [3.10.0-rc.47] — 2026-06-15
|
|
660
|
+
|
|
661
|
+
> **TL;DR:** **Ultracode audit fix-batch 5 — the range-arithmetic (body-relative line-number) class.** The 58-agent RCA workflow confirmed two MEDIUM siblings of this class: `obsidian_open_questions` and `readNote(format:"map")` both reported line numbers indexed on the *frontmatter-stripped* body (`i + 1`), so for any note with YAML frontmatter the reported `line` was short by the frontmatter length — an agent jumping to that line would land too early. Both now use the parser's already-exposed `bodyStartLine` to report **file-absolute** lines. **1196 → 1197 source tests.**
|
|
662
|
+
|
|
663
|
+
**Pre-release (v3.10 line) — audit fix-batch 5: range-arithmetic / file-absolute line numbers.**
|
|
664
|
+
|
|
665
|
+
### Fixed
|
|
666
|
+
|
|
667
|
+
- **MEDIUM — `obsidian_open_questions` reported body-relative line numbers** (`src/tools/meta.ts`). `getOpenQuestions` split `parsed.body` (which excludes frontmatter) and emitted `lineNo: i + 1`, so a `Q:`/`TODO?` marker in a note with YAML frontmatter was reported off by the frontmatter line count. Now `lineNo: parsed.bodyStartLine + i` (file-absolute; `bodyStartLine` is 1 when there's no frontmatter, so frontmatter-less notes are unchanged).
|
|
668
|
+
- **MEDIUM — `readNote(format:"map")` reported body-relative heading line numbers** (`src/tools/read.ts`). `extractHeadings(parsed.body)` emitted `line: i + 1` over the frontmatter-stripped body. The helper now takes `bodyStartLine` and emits `line: bodyStartLine + i` (file-absolute); the caller passes `parsed.bodyStartLine`.
|
|
669
|
+
|
|
670
|
+
### Tests (1197)
|
|
671
|
+
|
|
672
|
+
- `tests/lint.test.ts`: new `it` — a frontmatter'd note whose `Q:` marker is on file line 8 (frontmatter lines 1-4, blank 5, heading 6, blank 7, marker 8); asserts `q.line === 8` (file-absolute) and `!== 4` (the old body-relative value).
|
|
673
|
+
- `tests/tools.test.ts`: extended the existing `readNote format:"map"` test to assert `headings[0]` is `{ text: "Top heading", line: 6 }` (file-absolute) and `!== 2` (old body-relative). No new `it` (assertion added in place).
|
|
674
|
+
- Net source `it()`: 1196 → 1197.
|
|
675
|
+
|
|
676
|
+
> **Deferred to rc.48** (verified-real but distinct class): `frontmatterSet` round-trips the body through `matter.stringify`, which appends a trailing newline to a body saved without one (LOW, roundtrip-serialization-fidelity — needs gray-matter newline-state handling + a fidelity test, batched with the other LOWs). The RCA-flagged `embeddingsSearch` `db.open()` path-leak was **refuted** (it's guarded by the rc.34 fail-soft `peekEmbedDbMeta`); `chatThreadAppend` line_end and `--late-chunk-context 0` were not confirmed by the RCA and are left unchanged.
|
|
677
|
+
|
|
678
|
+
---
|
|
679
|
+
|
|
680
|
+
## [3.10.0-rc.46] — 2026-06-15
|
|
681
|
+
|
|
682
|
+
> **TL;DR:** **Ultracode full-project audit, fix-batch 4/4 — the Unicode NFC name-resolution CLASS.** rc.43's `foldKey()` (G1) fixed ONE instance of the macOS NFC/NFD name-resolution bug; an RCA + an exhaustive source-signature sweep found the SAME bug live in **14 name-comparison sites across 5 files** (the sweep caught 2 the RCA's hand-enumeration had missed). Every site folded a note name with `.toLowerCase()` but no `.normalize("NFC")`, so on macOS (APFS returns filenames decomposed/NFD; wikilinks and titles are usually composed/NFC) an accented name like a `[[café]]` link silently failed to resolve to its `café.md` file in the wikilink graph, title lookups, `.base` `linksTo`/`file.name ==`, fuzzy title 3-grams, and similar-note suggestions. Fixed at the root with a new shared `src/name-fold.ts` (`foldName`) + a **P0 inventory invariant** that fails CI on any future unfolded site. **1191 → 1196 source tests.**
|
|
683
|
+
|
|
684
|
+
**Pre-release (v3.10 line) — audit fix-batch 4/4: NFC name-resolution class.**
|
|
685
|
+
|
|
686
|
+
### Fixed
|
|
687
|
+
|
|
688
|
+
- **NFC name-resolution class (14 sites, 5 files)** — names compared/keyed without Unicode normalization silently failed to match across NFC/NFD forms on macOS. New dependency-free leaf module `src/name-fold.ts` exports `foldName(s) = s.normalize("NFC").toLowerCase()`; every site now routes through it (and `tools/meta.ts`'s `foldKey` is now `foldName(stripMd(s))`):
|
|
689
|
+
- `src/communities.ts` — wikilink-graph basename index build + lookup (`buildWikilinkGraph`); accented `[[links]]` now form graph edges.
|
|
690
|
+
- `src/vault.ts` — `findByTitle` / `findAllByTitle` (both the query norm and the per-entry match); write tools resolve accented titles + still fail-loud on true duplicates.
|
|
691
|
+
- `src/bases.ts` — `linksTo()` outbound-set build + query, and `file.name ==` / `!=` (both `want` and `got`).
|
|
692
|
+
- `src/tools/meta.ts` — `lint_vault_wiki` `titleSet` build + the capitalised-phrase lookup.
|
|
693
|
+
- `src/tools/search.ts` — `find_similar` title 3-gram construction.
|
|
694
|
+
- `src/tools/write.ts` — `suggestSimilar` target/basename/relPath folding.
|
|
695
|
+
- rc.43's `foldKey()` was an INSTANCE fix that never triggered a sibling sweep; even the RCA's hand-enumeration missed `search.ts` + `write.ts` — only an exhaustive signature grep found all 14, which is why the durable fix is the inventory invariant below, not the 14 edits.
|
|
696
|
+
|
|
697
|
+
### Added
|
|
698
|
+
|
|
699
|
+
- **`tests/name-fold-invariant.test.ts` (P0 inventory invariant).** A pure detector greps every `src/**/*.ts` for the class signature — an extension-strip (`/\.md$/i` or `/\.base$/i`) or `stripMd`/`stripMdExt` call immediately followed by `.toLowerCase()` (i.e. a note name folded without NFC) — and fails CI on any match, so a new unfolded comparison site can never ship. Ships with a NEGATIVE control proving the detector flags the real bug shapes (and does not flag the correct `foldName(...)` form), plus a `foldName` unit covering NFC↔NFD equality, ASCII regression, and a NOT-accent-stripping control. Converts "did we NFC-fold every name comparison?" (recursion-prone) into a self-checking gate — same move as the rc.25 ReDoS fuzz and the rc.36 resource-bound manifest.
|
|
700
|
+
|
|
701
|
+
### Tests (1196)
|
|
702
|
+
|
|
703
|
+
- +5 source `it()` in `tests/name-fold-invariant.test.ts` (3 `foldName` units + 2 invariant: the src-scan gate + the detector NEGATIVE control). 1191 → 1196. The 14 code fixes are covered by the existing communities / bases / vault / meta / search / write / wikilink-nfc suites (all green).
|
|
704
|
+
|
|
705
|
+
---
|
|
706
|
+
|
|
707
|
+
## [3.10.0-rc.45] — 2026-06-15
|
|
708
|
+
|
|
709
|
+
> **TL;DR:** **Ultracode full-project audit, fix-batch 3/4 — the abs-path-leak CLASS (HIGH + privacy brand).** An RCA-driven re-sweep proved that rc.43's **G1** (NFC) and the audit's **M3** (path-leak) were per-INSTANCE fixes, not CLASS fixes — sibling leak sinks were still live. This batch closes the error-path information-disclosure class at the source: `Vault.stat`/`readFile`/`readBinaryFile` now sanitize fs errors (strip the absolute vault root, keep `code` + the ENOENT shape) so EVERY caller returns vault-relative errors to MCP clients; the OCR install-hint throw no longer leaks the host tessdata/home dir; and the offline model-load error no longer surfaces the raw transformers.js cause (which embeds the `~/.cache/huggingface` absolute path). Two tests that had PINNED the leaked text were flipped to negative controls. **1189 → 1191 source tests.** rc.46 carries the NFC name-resolution class; rc.49 adds the matching structural OIA guard.
|
|
710
|
+
|
|
711
|
+
**Pre-release (v3.10 line) — audit fix-batch 3/4: abs-path-leak class.**
|
|
712
|
+
|
|
713
|
+
### Fixed
|
|
714
|
+
|
|
715
|
+
- **M3 [HIGH, privacy] — fs errors echoed the absolute vault path to MCP clients (root-class fix)** (`src/vault.ts`). `Vault.stat` (and the `chatThreadRead` sibling path, and every other fs-touching reader) surfaced raw Node `Error.message`/`Error.path` like `ENOENT: no such file or directory, stat '/Users/alex/Documents/Obsidian Vault/Nope.md'` — on `serve-http` that absolute path fingerprints the host (home dir, vault location). rc.43's G1 and the audit's M3 were each patched as a single sink; the RCA re-sweep found the siblings still live. Fixed at the SOURCE: new `private sanitizeFsError(err)` strips `this.root` (+ `path.sep`) from `err.message` and from `err.path`/`err.dest`, while preserving `err.code` and the canonical `ENOENT: … '<relative>'` message shape; it wraps `stat`, `readFile`, and `readBinaryFile`, so the leak is closed for ALL callers, not just the one the auditor named. Verified at runtime: `leaksRoot=false code=ENOENT msg="ENOENT: no such file or directory, stat 'Nope.md'"`.
|
|
716
|
+
- **HIGH — OCR install hint leaked the host tessdata/home directory** (`src/ocr.ts`). `assertOcrLangsInstalled`'s fail-closed throw appended `— downloads <lang>.traineddata into ${dir}`, where `${dir}` is the resolved absolute tessdata path (under the host home dir) — echoed to MCP clients when an OCR query hits a missing language pack. Dropped `${dir}`; the actionable `install-ocr-lang <code>` command stays.
|
|
717
|
+
- **MEDIUM — offline model-load error surfaced the raw transformers.js cause (abs cache path)** (`src/embeddings.ts`). `offlineModelLoadError` appended `Original: ${original}`, but the raw transformers.js error embeds the absolute model-cache path (`~/.cache/huggingface/...`). Dropped the raw cause; the message still names the model alias + hfId and the actionable `build-embeddings` hint, and still restates the "zero outbound network calls" guarantee.
|
|
718
|
+
|
|
719
|
+
### Tests (1191)
|
|
720
|
+
|
|
721
|
+
- `tests/security.test.ts`: new `describe("Vault — error-message privacy (rc.45 abs-path-leak class)")` — `stat`/`readFile`/`readBinaryFile` on a missing note assert the message does NOT contain the vault root, `code === "ENOENT"`, and the message still names the relative path; + a POSITIVE control that reading a present file still works.
|
|
722
|
+
- `tests/embeddings-offline.test.ts`: the two tests that had asserted the leaked cause text (`/ENOENT cache miss/`, `/raw string failure/`) were flipped to `not.toMatch` ("test pinned the bug" pattern) + a NEGATIVE control proving `/Users/secret/.cache/huggingface` is absent from the message while the slash-free model id is still named.
|
|
723
|
+
- Net source `it()`: 1189 → 1191.
|
|
724
|
+
|
|
725
|
+
### Coverage
|
|
726
|
+
|
|
727
|
+
- **Per-file floor — `src/embeddings.ts` branches 28 → 27 (documented reduction).** The privacy fix removed the `original instanceof Error ? original.message : String(original)` ternary from `offlineModelLoadError` (the raw cause is no longer surfaced), which deleted two previously-*covered* branch arms — mechanically lowering this integration-dep-heavy file's branch ratio from 29.82% to 27.27% (15/55). Lifting it back would require a real model load (the remaining uncovered branches are `applyOfflineEnv` + the offline-catch path), and synthesizing a branch purely to hold coverage would be test-theater. Floor lowered by 1pp per the documented-reduction rule; the inline `check-per-file-coverage.mjs` comment is synced.
|
|
728
|
+
|
|
729
|
+
---
|
|
730
|
+
|
|
731
|
+
## [3.10.0-rc.44] — 2026-06-09
|
|
732
|
+
|
|
733
|
+
> **TL;DR:** **Ultracode full-project audit, fix-batch 2/4 — serve / watcher / OCR robustness.** Four code MEDIUMs + one INFO: a canvas-OOM DoS guard bypass (M2), a silently-dead watcher embed/HNSW sync when FTS is off (M5), a shutdown ordering bug that dropped in-flight watcher events (M6), an unbounded directory-walk recursion depth (G2 critic gap), and an OCR language-pack pre-flight that accepted an unreadable `.gz` (INFO). **1189 source tests** (two existing tests were rewritten — they had encoded the buggy behavior, the "test pinned the bug" pattern).
|
|
734
|
+
|
|
735
|
+
**Pre-release (v3.10 line) — audit fix-batch 2/4: serve/watcher/OCR robustness.**
|
|
736
|
+
|
|
737
|
+
### Fixed
|
|
738
|
+
|
|
739
|
+
- **M2 [MEDIUM, security] — `clampOcrScale` floor defeated the canvas-OOM cap** (`src/ocr.ts`). `Math.max(0.1, …)` overrode `MAX_OCR_CANVAS_DIM` once `5000/maxBaseDim < 0.1`, i.e. for any page side > 50,000 pt — pdfjs does NOT enforce the PDF spec's 14,400 pt MediaBox limit, so a crafted `/MediaBox [0 0 1000000 1000000]` rendered at the floored 0.1 → 100,000 px → a ~40 GB RGBA canvas → OOM (the exact DoS the rc.10 guard exists to stop; the function's own TSDoc claimed the rendered side "never exceeds" the cap). Dropped the floor (the cap-derived ratio is the safe ceiling; `requestedScale` is already `[0.5,4]`-clamped upstream so it can't be ≤0) AND hard-cap the final canvas pixels at the call site (defense-in-depth). Normal pages are unaffected. The existing `clampOcrScale` test that asserted `≥ 0.1` (it had encoded the bug) was rewritten to assert the rendered side stays within the cap.
|
|
740
|
+
- **M5 [MEDIUM] — watcher embed-db + HNSW live-sync was silently dead with no FTS index** (`src/watcher.ts`). `handle()`'s `if (!this.ftsIndex) return` early-return preceded ALL unlink/embed-db/HNSW logic, so with a null FTS index (e.g. `--use-hnsw` without `--persistent-index`) every add/change/unlink skipped the embed upsert + `hnsw.applyDiff` — yet `server.ts` had wired `attachEmbed`/`attachHnsw` and printed "watcher embed-db sync enabled" / "HNSW live-update enabled". Now the early-return fires only when BOTH `ftsIndex` and `embedDb` are null (pure cache-invalidation), and every `this.ftsIndex` call is optional-chained so the embed-db + HNSW sync runs regardless of FTS.
|
|
741
|
+
- **M6 [MEDIUM] — `shutdownHttpServer` closed FTS5 before draining the watcher** (`src/http-transport.ts`). The teardown order was cache-flush → `ftsIndex.close()` → `watcher.close()`, but the chokidar watcher stayed live through `closeAll` (≤5s) + `closeServerBounded` (≤3s), so an edit landing in that window enqueued a handler that then ran `ftsIndex.reindexFile(...)` against the just-closed SQLite handle → `handle()`'s catch skipped the rest and the whole event (FTS + embed + HNSW) was dropped. Reordered to drain the watcher FIRST (while FTS + embed-db are open), then close `watcherEmbedDb`, then flush the cache, then close FTS last — now matching `shutdownStdioDeps`.
|
|
742
|
+
- **G2 [MEDIUM, critic gap] — unbounded directory-walk recursion** (`src/vault.ts`). `walk`/`walkAnyExt` skip symlinks (no cycle risk) but had no depth bound, and `capScanEntries(MAX_SCAN_NOTES)` only caps the result array AFTER the whole tree is traversed — so a pathologically deep real tree drove unbounded recursion + readdir I/O before the cap applied. Added `MAX_WALK_DEPTH = 64` (far below any real vault; bounds a hostile/accidental deep tree).
|
|
743
|
+
- **INFO — `ocrLangIsInstalled` accepted an unreadable `.gz`-only pack** (`src/ocr.ts`). It returned true for `<lang>.traineddata.gz`, but the worker is pinned `gzip:false` + `cacheMethod:"readOnly"` and reads only the uncompressed `<lang>.traineddata` — so the pre-flight passed while `createWorker` then failed. Now requires the uncompressed form (which `install-ocr-lang` always writes). The existing test that asserted `.gz` acceptance was rewritten as a NEGATIVE control.
|
|
744
|
+
|
|
745
|
+
### Tests (1189)
|
|
746
|
+
|
|
747
|
+
- `tests/ocr-offline.test.ts`: the `clampOcrScale` "≥0.1 floor" test → rewritten to assert the rendered side ≤ `MAX_OCR_CANVAS_DIM` for a 1,000,000 pt page; the "recognizes a `.gz` pack" test → rewritten to assert a `.gz`-only install is NOT recognized (and the uncompressed form is). Net source `it()` count unchanged (1189). M5/M6/G2 are ordering/structural changes covered by the existing watcher + http-transport + vault integration suites (all green) and verified against the adversarial 3-judge audit.
|
|
748
|
+
|
|
749
|
+
---
|
|
750
|
+
|
|
751
|
+
## [3.10.0-rc.43] — 2026-06-09
|
|
752
|
+
|
|
753
|
+
> **TL;DR:** **First fix-batch of the 14-lens ultracode full-project audit (129 agents, adversarial 3-judge verification) — retrieval & i18n correctness.** The audit confirmed the codebase is exceptionally clean — **0 CRITICAL, 0 HIGH** after verification — with 9 MEDIUM / 24 LOW / 4 INFO + 2 critic gaps across all 14 areas. rc.43 ships the 3 retrieval/i18n-correctness items: **M1** folder-prefix filter returned ZERO results for emoji folder names (UTF-16 vs code-point length mismatch), **M7** community-detection modularity was non-standard (truncated null-model penalty, could invert partition ranking), **G1** wikilink/`find_path` resolution failed across Unicode NFC/NFD forms (`[[café]]` ↔ `café.md` on macOS). **1181 → 1189 source tests.** rc.44–rc.46 carry the remaining MEDIUM/LOW/INFO batches.
|
|
754
|
+
|
|
755
|
+
**Pre-release (v3.10 line) — audit fix-batch 1/4: retrieval & i18n correctness.**
|
|
756
|
+
|
|
757
|
+
### Fixed
|
|
758
|
+
|
|
759
|
+
- **M1 [MEDIUM] — folder filter matched ZERO rows for astral-char (emoji) folder names** (`src/fts5.ts`, `src/embed-db.ts`). The folder-prefix equality bound JS `prefix.length` (UTF-16 code **units**) to SQLite `substr(rel_path, 1, ?)` (which counts code **points**), so any folder whose name contains a non-BMP char (e.g. `📚Books/` — JS length 8, 6 code points) over-read by one and matched nothing. Both sites now use `substr(rel_path, 1, length(?))` and bind the prefix string twice, letting SQLite compute the character length consistently. Empirically reproduced + regression-tested (emoji-folder test in both fts5 + embed-db suites).
|
|
760
|
+
- **M7 [MEDIUM] — `computeModularity` used a non-standard, inflated formula** (`src/communities.ts`). It subtracted the Newman–Girvan null-model penalty `k_i·k_j/2m` ONLY for *adjacent* same-community node pairs (the term lived inside the per-neighbor loop), whereas the standard formula penalizes ALL same-community pairs. The truncated penalty inflated Q and could rank a degenerate single-community partition ABOVE the correct split (auditor reproduced an inverted ranking). Rewritten to the exact decomposition `Q = Σ_c [ in_c/2m − (tot_c/2m)² ]` over per-community degree sums (verified: the natural bridged-triangles split scores 0.3571 > all-in-one 0.0). `computeModularity` is now exported + unit-tested (split > single + a Q≈0 NEGATIVE control).
|
|
761
|
+
- **G1 [MEDIUM, critic gap] — wikilink / `find_path` resolution failed across Unicode normalization forms** (`src/tools/meta.ts`). `findBestMatch`/`indexFor` keyed entries by `stripMd(name).toLowerCase()` with no Unicode normalization, so a `[[café]]` link typed NFC never resolved to a `café.md` file whose name macOS APFS returns in NFD (`"café"` NFC !== `"café"` NFD even after `toLowerCase()`). New `foldKey()` helper NFC-normalizes before case-folding and is applied at every index-build + query site (basename, relPath, joined relative path, endsWith scan). Verified NFC→NFD, NFD→NFC, ASCII regression, and a negative control.
|
|
762
|
+
|
|
763
|
+
### Tests (1189)
|
|
764
|
+
|
|
765
|
+
- +8 source `it()`: emoji-folder filter (fts5 + embed-db), `computeModularity` Newman-decomposition (split > single-community + Q≈0 NEGATIVE control), and new `tests/wikilink-nfc.test.ts` (NFC/NFD/ASCII/negative). 1181 → 1189.
|
|
766
|
+
|
|
767
|
+
### Audit
|
|
768
|
+
|
|
769
|
+
- The full audit (14 lenses · 129 subagents · adversarial 3-judge verify, ≥2/3 to confirm) returned **0 CRITICAL / 0 HIGH** — strong validation of the 20+ prior rounds + structural-invariant apparatus. Remaining confirmed findings (9 MED incl. 3 shipped here, 24 LOW, 4 INFO, 2 critic gaps) are batched into rc.44 (serve/watcher/OCR robustness), rc.45 (privacy/tool-handler correctness), rc.46 (docs/test-infra/script drift + structural guards).
|
|
770
|
+
|
|
771
|
+
---
|
|
772
|
+
|
|
773
|
+
## [3.10.0-rc.42] — 2026-06-09
|
|
774
|
+
|
|
775
|
+
> **TL;DR:** **Full state-driven re-audit (4 independent lenses) of the rc.35→rc.41 line + every doc/test/security surface — found that rc.40 AND rc.41 each shipped a fresh instance of a documented recurring class (the post-merge re-sweep working as designed), and closed both + 2 docs drifts.** 1 HIGH (rc.41 "zero cloud calls during serve" was ASPIRATIONAL — embedder/reranker could CDN-fetch on a cache-miss; now ENFORCED, mirroring OCR), 1 MEDIUM (rc.40 K-3 derive-check was scope-too-narrow — scanned only write.ts, missing read.ts's `chatThreadAppend`), 2 LOW (QUICKSTART `44-tool`; README.zh.md stale test count + missing the rc.41 fleet-memory framing). The code re-sweep otherwise confirmed rc.35→rc.41 sound (no other findings). **1177 → 1181 source tests.**
|
|
776
|
+
|
|
777
|
+
**Pre-release (v3.10 line) — audit response.**
|
|
778
|
+
|
|
779
|
+
### Security
|
|
780
|
+
|
|
781
|
+
- **F1 [HIGH] — "zero cloud calls during serve" is now ENFORCED, not aspirational.** Pre-rc.42, a missing local model cache let transformers.js silently CDN-fetch (~120MB) the embedder/reranker on a serve-time query — the exact "claimed-guarantee vs code-guard" shape of overclaim #16 (OCR), left un-closed for the ML path. Now `serve`/`serve-http` call `setEmbeddingsOffline()` at startup → transformers.js `env.allowRemoteModels = false`, so a model absent from the LOCAL cache **fails closed** (`offlineModelLoadError` with an `install-model`/`build-embeddings` hint) instead of fetching. `build-embeddings`/`install-model` never set the flag, so the one-time online download is unchanged. New **OIA Check 4f** (mirrors 4e for OCR) fails CI if a doc makes the enforced claim while the guard is absent. SECURITY.md updated to document the enforcement. (`src/embeddings.ts`, `src/cli.ts`, `scripts/oia-walk.mjs`, `SECURITY.md`.)
|
|
782
|
+
|
|
783
|
+
### Fixed
|
|
784
|
+
|
|
785
|
+
- **F2 [MED] — K-3 derive-check scope-too-narrow (the rc.40 #11 fix's own recursion).** `fsMutatingExports` scanned only `src/tools/write.ts`, but a real WRITE handler (`chatThreadAppend`) lives in `src/tools/read.ts` — so a new fs-mutating handler added to read.ts would escape BOTH the derive-check AND the layer-1 READ_ONLY-violation scan (which only flags names already in the set), letting it falsely advertise `readOnlyHint`. Now scans `WRITE_HANDLER_SOURCES = [write.ts, read.ts]` (mirrors resource-bound's `SCANNER_SOURCES`), with a sanity assertion that `chatThreadAppend` is detected + the delegating-handler blind spot (`archiveNote`) documented. Same scope-too-narrow class the project has fought (resource-bound rc.18, OIA Check 7 ×2). (`tests/k3-readonly-hint-invariant.test.ts`.)
|
|
786
|
+
- **F3 [LOW] — `docs/QUICKSTART.md` "44-tool" → "45-tool"** (slipped both guards: the scope-completeness `tool-count` pattern was space-only `\b\d{2}\s+tools\b`, and QUICKSTART wasn't in its scope). Closed the class: pattern now also matches the hyphenated `\b\d{2}-tool\b` form, and QUICKSTART joins the scope. (`docs/QUICKSTART.md`, `scripts/scope-completeness-audit.mjs`.)
|
|
787
|
+
- **F4 [LOW] — `README.zh.md` re-synced with the rc.41 English README**: test-count lower-bound `1100+` → `1170+` (drifted 77 below actual), and the rc.41 server-fleet-memory counter-positioning clause added to the "扎根于原文" (Grounded) paragraph.
|
|
788
|
+
|
|
789
|
+
### Tests (1181)
|
|
790
|
+
|
|
791
|
+
- New `tests/embeddings-offline.test.ts` (+4) — the F1 serve-offline surface: `setEmbeddingsOffline`/`isEmbeddingsOffline` toggle (default ONLINE so build-embeddings can download — NEGATIVE control) + `offlineModelLoadError` is a fail-closed error naming the model + an actionable hint + preserving the cause (+ a non-Error-cause NEGATIVE control). CI-safe (pure surface, no optional dep / model download); the cli.ts wiring is structurally guarded by OIA Check 4f.
|
|
792
|
+
- F2 widens the existing rc.40 K-3 derive test (no net `it()` change there). 1177 → 1181.
|
|
793
|
+
|
|
794
|
+
### Method
|
|
795
|
+
|
|
796
|
+
- **The re-audit is the value**: 4 parallel lenses (code · docs · tests/CI · security/privacy) on the *shipped* commit. Code re-sweep confirmed rc.35→rc.41 sound, but the docs + security + test-infra lenses each caught a residual the change-driven sweeps missed — and notably BOTH of the two newest RCs had recursed a known class (rc.40 → scope-too-narrow, rc.41 → claimed-guarantee-vs-code-guard). Caught in-house before an external auditor; both closed with a structural class-defense, not just an instance fix.
|
|
797
|
+
|
|
798
|
+
---
|
|
799
|
+
|
|
800
|
+
## [3.10.0-rc.41] — 2026-06-09
|
|
801
|
+
|
|
802
|
+
> **TL;DR:** **Counter-positioning docs (caura-memclaw study output) — sharpens the "Grounded, not extracted" brand against the *server-fleet*-memory category.** The hero framing (README + COMPARISON) already distinguished enquire from the *chat-memory* cohort (mem0/Zep/Supermemory — which *extract* facts from chat logs). It now ALSO distinguishes from multi-tenant *fleet*-memory platforms (server-side stores that paraphrase agent traffic into a shared database): enquire is **single-user, local-first, zero cloud calls during serve** — one vault you own, read, edit, and delete yourself. Docs-only; no count or claim-surface change. **1177 source tests unchanged.**
|
|
803
|
+
|
|
804
|
+
**Pre-release (v3.10 line) — positioning.**
|
|
805
|
+
|
|
806
|
+
### Changed
|
|
807
|
+
|
|
808
|
+
- **`README.md` hero + `docs/COMPARISON.md` intro** — the "Grounded, not extracted" framing gains a server-fleet-memory contrast clause. Prompted by a competitive study (caura-ai/caura-memclaw, a multi-tenant fleet-memory platform): the existing copy countered only the conversation-memory cohort, leaving the fleet-memory category unaddressed. No new factual/numeric claim — "zero cloud calls during serve" is the existing SECURITY.md/package.json claim, restated in the positioning context.
|
|
809
|
+
|
|
810
|
+
### Notes
|
|
811
|
+
|
|
812
|
+
- Marketing assets (a problem-first content blog, a landing page, Discord) remain maintainer-driven and are **never** committed to this public repo (standing rule). The one remaining borrowable *technical* idea from the study — closed-loop retrieval feedback (a "Karpathy Loop" `mark_useful`-style tool) — is a net-new persistent-state feature with data-at-rest + right-to-erasure implications; it's captured for the v3.10.x feature line and best sequenced after the v3.10 → `@latest` promotion (so the promotion audit isn't complicated by a fresh feature) and with a dedicated privacy-design pass.
|
|
813
|
+
|
|
814
|
+
---
|
|
815
|
+
|
|
816
|
+
## [3.10.0-rc.40] — 2026-06-09
|
|
817
|
+
|
|
818
|
+
> **TL;DR:** **Closes the entire LOW/INFO tail of the wq9ml34gr workflow-audit — the audit is now fully shipped (1 CRIT + 4 MED + 7 LOW/INFO, all closed).** Seven low-stakes items: two watcher concurrency-hardening fixes (close-window event race + HNSW dirty-flag race — both were lost-fast-reload-only, the signature-guard already rebuilt), `--stale-days` doc/help/TSDoc honesty (it tunes recency re-ranking only — the `stale` flag is a hardcoded 365), two test-infra invariant broadenings (K-3 derives write handlers from source; resource-bound detects parallel-fanout scanners), eval `hits_relevant` dedup (INFO), and dropping an unused `id-token: write` grant. **1174 → 1177 source tests.**
|
|
819
|
+
|
|
820
|
+
**Pre-release (v3.10 line) — audit tail (LOW/INFO).**
|
|
821
|
+
|
|
822
|
+
### Fixed
|
|
823
|
+
|
|
824
|
+
- **`src/watcher.ts` #6 — close()-window event race.** `close()` now stops the chokidar watcher FIRST (before draining the queue + flushing), and `onChange`/`handle` early-return when `closed` — so an edit landing mid-shutdown can't apply a live HNSW diff the just-persisted sidecar wouldn't reflect (pre-rc.40: a lost fast-reload; the signature-guard rebuilt on next serve, no corruption).
|
|
825
|
+
- **`src/watcher.ts` #7 — flushHnswToDisk dirty-flag race.** Clears `hnswDirty` BEFORE the `saveTo` await (re-set to true on failure) so a concurrent `applyDiff` that re-marks dirty during the write isn't clobbered by a late `= false` — the index stays correctly dirty → next serve rebuilds rather than trusting a sidecar that predates the diff.
|
|
826
|
+
- **`--stale-days` honesty (#9)** — `src/cli.ts` help, `src/server.ts` `ServeOptions` TSDoc, and `docs/api.md` no longer claim `--stale-days` is "the threshold behind the `stale` flag." It only tunes the recency RE-ranking half-life (active when `--recency-weight > 0`); the `stale` flag on hits always uses the fixed 365-day default. (Behavior was always correct + test-pinned; the docs over-claimed.)
|
|
827
|
+
- **`src/eval.ts` #13 (INFO)** — `hits_relevant` now counts DISTINCT relevant paths (Set), mirroring the rc.33 dedup in `recallAtK`/`ndcgAtK`, so a duplicate path can't print `N/M` with N>M. Unreachable at the default note granularity (paths unique); pins the contract for block-granularity callers.
|
|
828
|
+
- **`.github/workflows/dist-tag-cleanup.yml` #14 (INFO)** — dropped the unused `id-token: write` permission (dist-tag rm auths via `NPM_TOKEN`; no OIDC step). Least-privilege.
|
|
829
|
+
|
|
830
|
+
### Tests (1177)
|
|
831
|
+
|
|
832
|
+
- **K-3 invariant #11** — new `fsMutatingExports(src)` derives fs/vault-mutating exported handlers from `write.ts` source; asserts ⊆ `KNOWN_WRITE_HANDLERS` so a NEW mutating handler wired under READ_ONLY (which would falsely advertise `readOnlyHint`) fails CI — closing the "did-we-remember-to-add-it" gap the erasure/resource-bound inventory invariants close. + NEGATIVE control (flags an untracked `fs.unlink` export).
|
|
833
|
+
- **resource-bound invariant #12** — `discoverScanners` now also matches parallel-fanout iteration (`Promise.all` / `.map(async` / `for await`), not only a literal `for (`, so a whole-vault reader written as a pure `Promise.all(entries.map(async …readNote))` can't escape the cap-or-exempt gate. + NEGATIVE fixture. (No new scanner discovered in current src — confirmed no cascade.)
|
|
834
|
+
- +3 source `it()` (K-3 ×2, resource-bound ×1); docs count 1174 → 1177. The watcher #6/#7 fixes are covered by the existing watcher invariant tests (which still pass — the close still drains + flushes; behavior hardened, not changed).
|
|
835
|
+
|
|
836
|
+
### Method
|
|
837
|
+
|
|
838
|
+
- The full LOW/INFO tail from the rc.36 multi-agent behavioral/threat audit, batched into one RC (not N reactive patches). With this, every confirmed finding (1 CRIT rc.36 · 4 MED rc.37/38 · 7 LOW/INFO rc.40 · the rc.39 sink-bound from the mandated re-sweep) is shipped — the audit is closed.
|
|
839
|
+
|
|
840
|
+
---
|
|
841
|
+
|
|
842
|
+
## [3.10.0-rc.39] — 2026-06-09
|
|
843
|
+
|
|
844
|
+
> **TL;DR:** **The mandated post-rc.36 ReDoS re-sweep + its architectural fix (HIGH) — permanently ends the 4×-recurring ReDoS class.** A broader-generator 20,000-pattern re-sweep confirmed rc.36's specific fix is sound (all known shapes flagged, default accepted, no regression) **but surfaced the inherent undecidable residual: 80 SAFE-classified nested patterns genuinely hang V8** — e.g. the 37-char `\W?(([ca]*?){0,3}|c{2,5}b{2,5}){0,3}$` (confirmed: hangs indefinitely on a ~50-char line). This is NOT an rc.36 regression — it's the long-standing limit of *any* static ReDoS denylist (the detection problem is undecidable). Rather than chase 80 more hand-rules (the EDA rabbit hole CLAUDE.md forbids), the fix **bounds the SINK**: `obsidian_open_questions` now matches a caller-supplied pattern on a **worker thread with a hard wall-clock budget** (`MAX_QUESTION_SCAN_MS`=5000), so the main event loop can **never** hang for any pattern, and a pattern that blows the budget is rejected fail-closed. `isCatastrophicRegex` stays as the cheap pre-filter; the safe default pattern stays inline (zero overhead). **1170 → 1174 source tests.**
|
|
845
|
+
|
|
846
|
+
**Pre-release (v3.10 line) — security: ReDoS sink-bound (the architectural class-ender).**
|
|
847
|
+
|
|
848
|
+
### Security
|
|
849
|
+
|
|
850
|
+
- **`src/tools/meta.ts` — hard ReDoS sink-bound for `obsidian_open_questions` (HIGH; closes the static-detector residual the rc.36 re-sweep confirmed).** New `matchLinesBounded(pattern, lines, budgetMs)` runs the caller pattern's per-line matching on a **worker thread** and races a `budgetMs` timeout: the worker isolates V8's backtracking off the main event loop, and on timeout it's terminated and the request is rejected fail-closed (`"matching exceeded the Nms safe budget"`). `getOpenQuestions` is refactored to collect candidate lines + metadata, then match via the worker (caller pattern) or inline (safe default). An invalid pattern rejects with a clear error. **This makes detector completeness moot — no pattern, however crafted, can hang the server.**
|
|
851
|
+
- Kept `isCatastrophicRegex` + `MAX_QUESTION_PATTERN_LEN` as the cheap pre-filter (rejects obvious shapes without spawning a worker). The worker-budget is the hard backstop for everything the best-effort denylist misses (ReDoS detection is undecidable). New `MAX_QUESTION_SCAN_MS` constant; an optional `scanBudgetMs` arg (NOT in the MCP tool schema, so not caller-settable) lets tests use a short budget.
|
|
852
|
+
- **Why the architectural fix over more rules:** the re-sweep proved a static analyzer for an undecidable property is forever incomplete (rc.21/24/25/36 each closed a shape; the next is always findable). Bounding the sink ends the class permanently — the project's own "fix the class architecturally, don't chase EDA-precise detection" rule.
|
|
853
|
+
|
|
854
|
+
### Tests (1174)
|
|
855
|
+
|
|
856
|
+
- `tests/redos-guard.test.ts`: +4 — `matchLinesBounded` describe (safe pattern → first-capture matches; a pattern `isCatastrophicRegex` MISSES → rejected *within the budget*, asserting it did NOT hang; invalid pattern → clear error) + a `getOpenQuestions` end-to-end test that a detector-missed catastrophic pattern (base64-decoded at runtime for CodeQL js/redos hygiene) is rejected via the worker bound on a vault seeded with a long line. Docs test-count bumped 1170 → 1174.
|
|
857
|
+
|
|
858
|
+
### Method
|
|
859
|
+
|
|
860
|
+
- The re-sweep is the MANDATED post-merge step for a recurring-class fix (it's how rc.24's fix surfaced rc.25's CRITICAL). It correctly returned a mixed verdict: rc.36 sound, but the residual real — leading to the architectural fix the maintainer approved over reactive detector-patching.
|
|
861
|
+
|
|
862
|
+
---
|
|
863
|
+
|
|
864
|
+
## [3.10.0-rc.38] — 2026-06-09
|
|
865
|
+
|
|
866
|
+
> **TL;DR:** **Correctness + resource batch (audit #5 + #2, both MEDIUM).** **#5:** a `.base` filter `not: '<unevaluated predicate>'` (a typo, or `inDate(...)` / any formula predicate — none of which `obsidian_query_base` evaluates) **returned EVERY row instead of none.** v3.6.2 HN-2 fail-closes an unevaluable predicate to `false` = "exclude the row"; `not` blindly negated that to `true` = "include" — silently over-including the whole scan, the exact over-inclusion HN-2 (and SECURITY.md) say is prevented, reachable through negation. Fixed by evaluating the `not` child against a **fresh `unevaluated` probe** (the real set is shared across rows, so a naive size-delta only catches the first row) and excluding if the child touched any unevaluable predicate. **#2:** the embedder / BGE-reranker **ONNX InferenceSession was rebuilt on EVERY query** — `loadEmbedder`/`loadReranker` re-ran `pipeline()` / `from_pretrained()` per call (~110–120MB session init each), so a long-lived `serve --enable-reranker` paid full model-init latency per `obsidian_search` and N concurrent authenticated queries spun up N simultaneous sessions (a memory-spike vector + a direct hit to the headline sub-10ms claim). Now **cached per-alias** at the module level. **1168 → 1170 source tests** (+2 for #5; #2's behavioral path is gated-smoke + build-verified, as the whole model path is).
|
|
867
|
+
|
|
868
|
+
**Pre-release (v3.10 line) — correctness + resource.**
|
|
869
|
+
|
|
870
|
+
### Fixed
|
|
871
|
+
|
|
872
|
+
- **`src/bases.ts` — `not:` inverted the fail-closed semantics for unevaluated predicates (#5, MEDIUM correctness).** `evalFilter`'s `if ("not" in f) return !evalFilter(f.not, ctx)` negated the `false` that an unknown/typo/unparseable predicate (incl. `inDate(...)`) fail-closes to — turning "exclude" into "include every row." Now: `const probe = new Set(); const inner = evalFilter(f.not, { ...ctx, unevaluated: probe }); merge probe → ctx.unevaluated; if (probe.size > 0) return false; return !inner;`. A fresh probe is required because `ctx.unevaluated` is shared across all rows, so a same-set size delta only fires on the first row that hits the predicate (the bug the first fix attempt hit, caught by the new test). Contradicted SECURITY.md ("Predicate strings that don't match any pattern … treated as false (fail-closed … exclude the row rather than over-include it)").
|
|
873
|
+
- **`src/embeddings.ts` — embedder/reranker ONNX session rebuilt per query (#2, MEDIUM resource/latency).** `loadEmbedder` / `loadReranker` now check a module-level `Map<alias, Promise<Embedder|Reranker>>` and reuse the handle; the build is extracted to private `buildEmbedder` / `buildReranker`. The promise-cache also collapses a concurrent first-load thundering-herd, and a rejected load is evicted so a later call can retry. The `rerankerOverride` test seam (search.ts) is unaffected (it bypasses `loadReranker`).
|
|
874
|
+
|
|
875
|
+
### Tests (1170)
|
|
876
|
+
|
|
877
|
+
- `tests/bases.test.ts`: +2 — `not:` over an unevaluated `inDate(...)` and over a typo'd predicate both fail-closed to 0 matches (the existing "filters via not" over a KNOWN predicate is the positive control). #2's cache is build-verified (types) + exercised by the gated `reranker-smoke` (which runs `buildReranker`); the model-load path is gated behind real weights codebase-wide, so no flaky real-load unit test was added. Docs count bumped 1168 → 1170.
|
|
878
|
+
|
|
879
|
+
### Method
|
|
880
|
+
|
|
881
|
+
- Continuation of the rc.36 multi-agent behavioral/threat audit (correctness/resource dimensions). The two MEDIUMs are orthogonal to the rc.36 ReDoS detector and the rc.37 erasure paths. Shipped after rc.37 confirmed published on `@rc`.
|
|
882
|
+
|
|
883
|
+
---
|
|
884
|
+
|
|
885
|
+
## [3.10.0-rc.37] — 2026-06-09
|
|
886
|
+
|
|
887
|
+
> **TL;DR:** **Privacy / right-to-erasure batch (audit #3/#4 MEDIUM + #8 LOW).** The cross-vault `prune` GC silently **left full note bodies on disk forever**: its whitelist regex (`ENQUIRE_CACHE_ARTIFACT`) omitted the `<hash>.json` parse cache (written by `saveDiskCache`, holds every note's raw body), so decommissioning a vault and running `prune` deleted its `.fts5.db`/`.embed.db`/HNSW sidecars but kept its full-text `.json` (and any `.json.tmp`) — a GDPR-shaped right-to-erasure gap (same class as rc.34 P-2 / rc.36 F-2). Fixed the regex to cover `json` + the `.tmp` leftover, and — the structural half — the **erasure-completeness invariant now patrols the `prune` eraser too** (it previously only checked the 3 per-vault `clear-*` erasers; the `prune` surface was unguarded, which is exactly why #3 shipped). Plus #8: an emptied `--use-hnsw` embed-db now erases its stale `.hnsw.bin` + `.hnsw.meta.json` sidecars (the `.meta.json` carries deleted notes' `text_preview`). All local-only (mode-0600, no remote disclosure), behind opt-in preconditions. **1164 → 1168 source tests** (+4).
|
|
888
|
+
|
|
889
|
+
**Pre-release (v3.10 line) — privacy / right-to-erasure.**
|
|
890
|
+
|
|
891
|
+
### Security / privacy
|
|
892
|
+
|
|
893
|
+
- **`src/fts5.ts` — `prune` left the `.json` parse cache (full note bodies) on disk (#3, MEDIUM right-to-erasure).** `ENQUIRE_CACHE_ARTIFACT` now matches `<hash>.{json,fts5.db,embed.db,hnsw.bin,hnsw.meta.json}` + the `-wal`/`-shm`/`.tmp` sidecars (was missing `json` and `.tmp`). A decommissioned vault's `<hash>.json` (and any crash-left `<hash>.json.tmp`), both holding raw note bodies, are now GC'd by `prune` like every other family. Help text (`cli.ts`) + `docs/api.md` updated to list the `.json` family.
|
|
894
|
+
- **`tests/erasure-invariant.test.ts` — the erasure invariant now patrols the cross-vault `prune` eraser (#4, MEDIUM structural).** It previously asserted only that the 3 per-vault `clear-*` erasers reference every artifact suffix; the `prune` whitelist was a 4th deletion authority with NO coverage — which is why #3 shipped undetected. New "prune covers every per-vault writer family" block asserts `planCachePrune` selects a representative filename of each writer family (`.json`, `.json.tmp`, `.fts5.db`, WAL, `.embed.db`, `.hnsw.bin`, `.hnsw.meta.json`) for an OTHER vault, with a NEGATIVE control that replays the literal pre-rc.37 regex (missing `.json`) and proves it leaves the parse cache. `writers ⊆ prune-eraser`, structurally.
|
|
895
|
+
- **`src/server.ts` — emptied `--use-hnsw` embed-db left stale HNSW sidecars (#8, LOW right-to-erasure residual).** When `getAllVectors()` is empty no index is built, so there was no `saveTo` to overwrite a prior `<base>.hnsw.bin` + `.hnsw.meta.json` — and the `.meta.json` carries deleted notes' `text_preview`. The empty branch now unlinks both sidecars (best-effort, persist-gated), mirroring `EmbedDb.clearOnDisk`'s sidecar-erase minus deleting the (valid, empty) db.
|
|
896
|
+
|
|
897
|
+
### Tests (1168)
|
|
898
|
+
|
|
899
|
+
- +4 source `it()`: `tests/cache-prune.test.ts` (the `.json` parse-cache coverage test), `tests/erasure-invariant.test.ts` (the prune-coverage family loop + its NEGATIVE control + the #8 server.ts structural assertion). Existing prune test's "all 4 types" case extended to "all 5 families + tmp". Docs test-count bumped 1164 → 1168 across README ×4, package.json, llms.txt, AGENTS, COMPARISON, ROADMAP.
|
|
900
|
+
|
|
901
|
+
### Method
|
|
902
|
+
|
|
903
|
+
- Continuation of the rc.36 multi-agent behavioral/threat audit (the privacy/erasure dimension). Each finding adversarially re-verified against current code before fixing. The orthogonal-module batch (no overlap with the rc.36 ReDoS detector) ships after rc.36 confirmed published on `@rc`.
|
|
904
|
+
|
|
905
|
+
---
|
|
906
|
+
|
|
907
|
+
## [3.10.0-rc.36] — 2026-06-09
|
|
908
|
+
|
|
909
|
+
> **TL;DR:** **CRITICAL ReDoS fix — the 4th recurrence of the class, found by a fresh behavioral/threat audit.** `isCatastrophicRegex` (the `obsidian_open_questions` guard) computed its catastrophe verdict ONLY when a quantified GROUP closed (`)` pop), so a **bare top-level run of adjacent overlapping unbounded quantifiers** — `\w*\w*\w*\w*\w*\w*\w*\w*$` (25 chars, under the 200-char cap) — was classified SAFE and compiled. On a single ~45-char word-run line it hangs V8 **~16s** (and `a*a*$` is ~1s at 2000 chars / ~68s at 8000), and `obsidian_open_questions` is **always-registered**, so any token-bearing client could freeze a `serve-http` instance for all clients. Fixed by evaluating a new `frameAdjacentOverlap` check on the TOP frame (never popped) **and** every group body. Overlap is decided by **probing actual single-char regex membership** (delegating the char-class truth-table to V8 — so disjoint broad pairs like `\d*\s*` / `[#.]+\s+` stay accepted while cross-class `\w*\d*` is caught), with a `.`-greedy **absorber tail-exemption** that keeps the shipped default `…\s*[:\-]?\s*(.+)$` safe (it is benign ONLY because `(.+)` absorbs the tail). **Overclaim #19**: the detector TSDoc claimed it "never UNDER-flags," which was false. **The durable fix is the fuzz**: the rc.25 generative ReDoS fuzz never caught this because its GENERATOR only emitted quantified groups — extended `genPattern` to emit bare top-level concatenations (SAFE corpus 43 → 390), so the next top-level under-flag fails CI empirically. Canonical source-`it()` count unchanged (1164); rc.36 adds 20 data-driven guard cases + a corpus-floor assertion.
|
|
910
|
+
|
|
911
|
+
**Pre-release (v3.10 line) — security: ReDoS recurrence #4 (CRITICAL).**
|
|
912
|
+
|
|
913
|
+
### Security
|
|
914
|
+
|
|
915
|
+
- **`src/tools/meta.ts` `isCatastrophicRegex` — top-level adjacent-quantifier bypass (CRITICAL, remote DoS on bearer-auth `serve-http`).** The catastrophe `return true` lived only inside the `if (ch === ")")` pop branch; frame 0 is never popped, so a bare `\w*\w*…$` / `a*a*$` / `(a)*(a)*$` sequence reached `return false` unflagged. New `frameAdjacentOverlap(body, exemptByAbsorber)` walks atoms left-to-right and flags ≥2 adjacent (zero-width anchors + min-zero/nullable atoms transparent; a mandatory atom breaks the run) unbounded-quantified atoms with **overlapping** match sets. Called on the top frame (`exemptByAbsorber=true`, full tail visible → `.`-greedy absorber exemption applies) and on each group body at pop (`false` — external continuation unknown → over-flag; catches `(\w*\w*)x`).
|
|
916
|
+
- Overlap via **`atomsOverlap`**: probe `OVERLAP_PROBES` (a representative ASCII alphabet) against each atom's `^(?:atom)$` single-char matcher — no hand-maintained class truth table (which would itself be an under-flag bug surface). Correct for literals, `.`, char-classes, and shorthand overlaps (`\w`⊃`\d` → caught; `\w`∩`\s`=∅ → accepted).
|
|
917
|
+
- **Absorber exemption** (`isUniversalAbsorber` + `tailIsBenign`): a run followed by a `.`-greedy `.+`/`.*` (optionally grouped, e.g. `(.+)`) is benign — it consumes any tail and reaches the end-anchor without forced redistribution. This is precisely why the default `open_questions` pattern (`…\s*[:\-]?\s*(.+)$`) is SAFE (~0.1ms) while `…\s*[:\-]?\s*$` is CATASTROPHIC (~12s).
|
|
918
|
+
- **`tests/redos-fuzz.test.ts` — generator blind spot closed (the root reason rc.21/24/25 didn't end the class).** `genPattern` always wrapped output in a quantified group `(…)${q}$`, so the bare top-level shape that hung in rc.36 could never be generated → the empirical net had a hole exactly where the bug lived. Now ~40% of patterns are a bare `seq seq $` concatenation. The SAFE-classified corpus that reaches the timed worker grew **43 → 390**; added `expect(safeChecked).toBeGreaterThan(80)` + `expect(bareTopLevelSafe).toBeGreaterThan(20)` so corpus starvation (finding #10 from the audit) can't silently return.
|
|
919
|
+
|
|
920
|
+
### Tests (1164)
|
|
921
|
+
|
|
922
|
+
- `tests/redos-guard.test.ts`: +12 catastrophic cases (`a*a*$`, `a*a*a*$`, `\w*…\w*$`, `\w*\w*$`, `.*.*$`, `\s*\s*$`, `\s*[:\-]?\s*$`, `a*x?a*$`, `\w*\d*$`, `(a)*(a)*$`, `(\w*\w*)x`, `a*a*b$`) + 8 safe-precision cases (`a*b*$`, `a*b*c*$`, `\d*\s*x`, `\w+\s+`, `[#.]+\s+`, `a*xa*$`, `\s*\s*(.+)$`, `\s*[:\-]?\s*(.+)$`). All are data-driven entries on the existing array-loop `it()`s, so the **canonical source-`it()` count is unchanged at 1164** (runtime test count rises; the two arrays are each other's positive/NEGATIVE controls).
|
|
923
|
+
|
|
924
|
+
### Method
|
|
925
|
+
|
|
926
|
+
- Found by a fresh **multi-agent behavioral + threat-model audit** (7 dimensions, each finding adversarially re-verified against current code) — the external-lens, runtime-behavior sweep the drift/claim-driven home-grown gates are structurally blind to. The same audit surfaced a MEDIUM/LOW batch (embedder/reranker session cache, `prune` parse-cache right-to-erasure, `.base` `not:` fail-closed inversion, watcher close-window race, …) shipping in follow-up RCs.
|
|
927
|
+
|
|
928
|
+
---
|
|
929
|
+
|
|
930
|
+
## [3.10.0-rc.35] — 2026-06-08
|
|
931
|
+
|
|
932
|
+
> **TL;DR:** **README reorder (maintainer request) — lead with the competitive case.** Moved the **"🏆 Why it's the best"** section (the side-by-side comparison table + the "six features no other Obsidian-MCP has" framing + the Karpathy strategic claim) up to sit **immediately before "⚡ Quick start"** (right after "The solution"), so an evaluator sees the differentiation before the install steps. The hero's one-line `claude mcp add …` + the "30-second install" nav link keep "try it now" reachable from the very top, so the install isn't buried. Done as a **deterministic boundary-based move** (section heading → next heading), not a hand cut-paste, so the 20-row table relocated intact. **README-only — no code, no new/changed claims; the hero stat line stays the first `**N tools**` so the docs-consistency first-match regexes are unaffected; 1164 tests unchanged.**
|
|
933
|
+
|
|
934
|
+
**Pre-release (v3.10 line) — README section reorder.**
|
|
935
|
+
|
|
936
|
+
### Changed
|
|
937
|
+
|
|
938
|
+
- **`README.md` — "🏆 Why it's the best" moved above "⚡ Quick start"** (was buried after "Use cases" / "When NOT to use it" / "API reference"). New top-level flow: hero → The problem → The solution → **Why it's the best** → Quick start → Set up in your AI agent → Use cases → When NOT to use it → API reference → How retrieval works → … No content changed — only the section's position. Separators verified (no double `---`); the removed-from spot (API reference → How retrieval works) closes cleanly.
|
|
939
|
+
|
|
940
|
+
### Tests (1164)
|
|
941
|
+
|
|
942
|
+
- None — README-only reorder; no claims added/changed (docs-consistency green; the hero `**45 tools · …**` stat line remains the first tool-count match, so the relocated `**45 production tools**` comparison row doesn't shift any first-match regex). 1164 unchanged.
|
|
943
|
+
|
|
944
|
+
### Files changed
|
|
945
|
+
|
|
946
|
+
- `README.md` (section reorder), `CLAUDE.md` (roll-up rc.34 → rc.35 + the reorder note).
|
|
947
|
+
- version bump 3.10.0-rc.34 → 3.10.0-rc.35.
|
|
948
|
+
|
|
949
|
+
---
|
|
950
|
+
|
|
951
|
+
## [3.10.0-rc.34] — 2026-06-08
|
|
952
|
+
|
|
953
|
+
> **TL;DR:** **RCA re-sweep of the rc.33 fix — the same bug had a SIBLING, and it was worse.** The post-rc.33 re-sweep (mandated by the project's "fix the class, not the instance" rule) found that `peekEmbedDbMeta` — the embed-db twin of the `peekFtsMetaSafe` function hardened in rc.33 — has the **identical `new Database()`-outside-the-try shape**, and it's called **UNGUARDED** in two hot spots the FTS one wasn't: `embeddingsSearch` (`tools/search.ts`, the peek runs *before* that function's own try/catch) and two CLI subcommands (`cli.ts`). So a **corrupt / unreadable / directory `.embed.db` would error the `embeddings_search` tool and crash those CLI subcommands** (vs the rc.33 FTS case, which only bit startup). Hardened it the same way: `new Database()` + the meta queries now sit inside one try → any failure returns `null` (treated as "no embed-db" — the existing graceful-degrade path). **`src/embed-db.ts` + tests only; +3 tests → 1164.** This is the re-sweep discipline paying off: the rc.33 instance fix's mandatory sibling-scan caught a higher-impact instance of the same class.
|
|
954
|
+
|
|
955
|
+
**Pre-release (v3.10 line) — post-rc.33 RCA re-sweep (peekEmbedDbMeta sibling).**
|
|
956
|
+
|
|
957
|
+
### Fixed
|
|
958
|
+
|
|
959
|
+
- **`src/embed-db.ts` — `peekEmbedDbMeta` now truly never throws** (RCA sibling of rc.33's `peekFtsMetaSafe` fix). `new Database()` was outside the function's only try/catch (which guarded just the dep *load*), so a corrupt / unreadable / not-a-DB / directory `.embed.db` made the peek throw. Unlike the FTS peek (startup-only), this one is called UNGUARDED on the `embeddings_search` hot path (`tools/search.ts`, before that function's `try`) and in `cli.ts` subcommands — so the throw errored the search tool / crashed the CLI instead of degrading. Now the DB-open + read are wrapped → any failure → `null`. (`server.ts` call sites were already inside try/catch; this also makes them cleaner.)
|
|
960
|
+
|
|
961
|
+
### Tests (1164)
|
|
962
|
+
|
|
963
|
+
- +3 (`tests/embed-db.test.ts`): `peekEmbedDbMeta` returns `null` (not throws) for a non-existent file, a **directory** path, and a **corrupt non-SQLite** file. Non-vacuous when `better-sqlite3` is present (CI): pre-fix `new Database()` threw → the directory/corrupt cases failed. 1161 → 1164.
|
|
964
|
+
|
|
965
|
+
### Files changed
|
|
966
|
+
|
|
967
|
+
- `src/embed-db.ts` (peekEmbedDbMeta wrap), `tests/embed-db.test.ts` (+3), count-bump (`1161 → 1164`) in `README.md` / `package.json` / `llms.txt` / `AGENTS.md` / `ROADMAP.md` / `docs/COMPARISON.md` / `CLAUDE.md`.
|
|
968
|
+
- version bump 3.10.0-rc.33 → 3.10.0-rc.34.
|
|
969
|
+
|
|
970
|
+
---
|
|
971
|
+
|
|
972
|
+
## [3.10.0-rc.33] — 2026-06-08
|
|
973
|
+
|
|
974
|
+
> **TL;DR:** **Post-rc.31 audit response — code correctness (batch 2/2).** Ships the code findings from the 3-lens audit (rc.32 was docs/test-infra). The headline: **FTS5 (`--persistent-index`) now fails soft to TF-IDF instead of hard-crashing serve** when `better-sqlite3` is missing/unbuilt — closing the **"auto-degrades gracefully: works with any subset of signals" claimed-guarantee gap** (the embed-db / PDF / HNSW paths already fail-soft; FTS5 was the lone hard-crash). Writing the E2E test for it **surfaced a DEEPER latent bug the audit didn't name**: `peekFtsMetaSafe` — the pre-open metadata peek, which runs BEFORE the fail-soft try/catch — wrapped the better-sqlite3 *load* but not `new Database()`, so a **corrupt / unreadable / directory persistent-index file crashed serve at startup** (a function literally named "Safe" could throw). Both fixed: `peekFtsMetaSafe` now truly never throws (any failure → `null`), and the open/sync path degrades to TF-IDF with a loud stderr warning. Plus two eval-correctness polishes the audit flagged: `recallAtK`/`ndcgAtK` **dedupe duplicate relevant paths** (a path repeated in the result list no longer inflates recall past 1.0 / DCG past the ideal — pre-existing, unreachable via the eval path, now correct for any caller) and `formatEvalResult` uses a **dynamic id-column width** (ids > 15 chars no longer shift every following column). **+5 tests → 1161.** The FTS5 fail-soft is verified by a new E2E test that forces `ftsIndex.open()` to fail (points `--index-file` at a directory) and asserts serve still completes the MCP handshake + answers `tools/list`.
|
|
975
|
+
|
|
976
|
+
**Pre-release (v3.10 line) — post-rc.31 audit, batch 2/2 (code correctness).**
|
|
977
|
+
|
|
978
|
+
### Fixed
|
|
979
|
+
|
|
980
|
+
- **`src/server.ts` — FTS5 `--persistent-index` fails soft to TF-IDF** (was: re-throw → serve crash). On any `ftsIndex.open()`/sync failure (most commonly `better-sqlite3` missing/unbuilt + `--persistent-index`, e.g. the Docker introspection image or a failed native build), serve now sets `ftsIndex = null` (exactly the heavily-tested no-`--persistent-index` state) + emits a stderr warning, instead of a hard crash with an unactionable "npm rebuild" stack trace. Parity with the already-fail-soft PDF / embed-db / HNSW paths.
|
|
981
|
+
- **`src/fts5.ts` — `peekFtsMetaSafe` now truly never throws** (latent bug surfaced while testing the above). `new Database()` + the meta queries were OUTSIDE the function's only try/catch (which guarded just the dep *load*), so a corrupt / unreadable / not-a-DB / directory index file made the pre-open peek throw and crash serve before the fail-soft could engage. Now the whole DB-open + read is wrapped → any failure returns `null`.
|
|
982
|
+
- **`src/eval.ts` — `recallAtK` + `ndcgAtK` dedupe duplicate relevant paths.** A relevant path repeated in the result list counted multiple times (recall could exceed 1.0; DCG could exceed the ideal). Now each relevant path is credited once (recall counts the distinct set; ndcg credits the first rank). Pre-existing + unreachable via the eval path (default `note` granularity yields one hit per path) — defensive correctness for any caller.
|
|
983
|
+
- **`src/eval.ts` — `formatEvalResult` dynamic id-column width.** Per-query ids longer than 15 chars previously overflowed the fixed pad and shifted every following column; now the id column sizes to the widest id (mirrors `formatEvalMatrix`).
|
|
984
|
+
|
|
985
|
+
### Tests (1161)
|
|
986
|
+
|
|
987
|
+
- +5: `tests/e2e-handlers.test.ts` FTS5 fail-soft E2E (CI-GUARD that serve came up degraded + `tools/list` still answers — forces the failure via `--index-file <dir>`; revert-verified: restoring the re-throw crashes startup → the handshake times out → the guard fails); `tests/eval.test.ts` recallAtK-dedupe, ndcgAtK-dedupe, formatEvalResult long-id alignment (each fails pre-fix). 1156 → 1161.
|
|
988
|
+
|
|
989
|
+
### Files changed
|
|
990
|
+
|
|
991
|
+
- `src/server.ts` (FTS5 open fail-soft), `src/fts5.ts` (peekFtsMetaSafe wrap), `src/eval.ts` (recall/ndcg dedupe + dynamic id width), `tests/e2e-handlers.test.ts` (+2), `tests/eval.test.ts` (+3), count-bump (`1156 → 1161`) in `README.md` / `package.json` / `llms.txt` / `AGENTS.md` / `ROADMAP.md` / `docs/COMPARISON.md` / `CLAUDE.md`.
|
|
992
|
+
- version bump 3.10.0-rc.32 → 3.10.0-rc.33.
|
|
993
|
+
|
|
994
|
+
---
|
|
995
|
+
|
|
996
|
+
## [3.10.0-rc.32] — 2026-06-08
|
|
997
|
+
|
|
998
|
+
> **TL;DR:** **Post-rc.31 audit response — docs + test-infra (batch 1/2).** Ran a from-scratch 3-lens audit (code · docs · test/process, via the Agent tool — NOT Workflow) of the rc.27→rc.31 seeklink batch; every finding **per-item re-verified against the actual code** (anti-overclaim). Verdict: the batch is **exceptionally clean — 0 CRITICAL, 0 HIGH** (the same-PR-invariant discipline held). This RC ships the docs/test-infra findings (the one MEDIUM + LOWs; the code findings follow as rc.33): **(1)** the **CLAUDE.md status roll-up** was frozen at `@rc`=rc.26 while the real @rc was rc.31 — the recurring **α-class "status section stale"** (rc.12 / rc.4 / v3.7.4 / v3.7.9 / v3.8.4 …). Updated it to rc.32 + the seeklink/audit summary, **and finally made it STRUCTURAL**: `check-version-consistency.mjs` now enforces the roll-up's `(current roll-up; \`@rc\`=<version>…)` marker == `package.json` on every `-rc.N` build, so a frozen roll-up fails CI (detection-power verified: it flagged the rc.32-vs-rc.31 mismatch before the bump). **(2)** three rc.28 tool-count guards were **vacuous-on-deletion** (caught a stale number but passed if the phrasing was removed) → presence-asserted. **(3)** the eval **`error` failure-bucket** is now asserted end-to-end (thrown query → `failure_bucket:"error"` + aggregate). **(4)** the rc.27 CHANGELOG "AGENTS.md ×2" advisory-sync count was an overcount → corrected. **No `src/` behavior change; 1156 tests unchanged.**
|
|
999
|
+
|
|
1000
|
+
**Pre-release (v3.10 line) — post-rc.31 audit, batch 1/2 (docs + test-infra).**
|
|
1001
|
+
|
|
1002
|
+
### Tooling (structural enforcement)
|
|
1003
|
+
|
|
1004
|
+
- **`check-version-consistency.mjs` gains a CLAUDE.md roll-up `@rc`-currency guard** (the α-class structural defense). On any `-rc.N` build it asserts the roll-up's `(current roll-up; \`@rc\`=X.Y.Z-rc.N …)` marker equals `package.json`'s version. NOT counted among the 7 published-version surfaces (it's a status-summary claim, not a published-version file) — the "7 surfaces" wording stays accurate. This converts the 6×-recurring "CLAUDE.md status stale" α-class from a discipline into a CI gate.
|
|
1005
|
+
- **`tests/docs-consistency.test.ts`** — the three rc.28 tool-count guards (`**N production tools**`, `| Tool count | N |`, `N tool implementations`) now **presence-assert** before checking the value, so they catch both a stale number AND the phrasing being deleted (the rc.30 zh-numeric `it()` already did this; this brings the rc.28 guards to the same bar).
|
|
1006
|
+
|
|
1007
|
+
### Tests / docs
|
|
1008
|
+
|
|
1009
|
+
- **`tests/eval.test.ts`** — the "survives a query that throws" test now asserts `per_query[1].failure_bucket === "error"` + `diagnostics.failure_buckets.error === 1` (end-to-end wiring of rc.31's classifier, previously only unit-tested).
|
|
1010
|
+
- **`CLAUDE.md`** — status roll-up advanced rc.26 → rc.32 + the seeklink batch (rc.27→rc.31) and this audit summarized.
|
|
1011
|
+
- **`CHANGELOG.md`** — rc.27's "advisory-gate-count … `AGENTS.md` ×2" corrected to "`AGENTS.md` (count header + advisory list)" (the "5 advisory" count appears once in AGENTS; the 2nd edit was the advisory list).
|
|
1012
|
+
|
|
1013
|
+
### Rejected (with reasoning)
|
|
1014
|
+
|
|
1015
|
+
- **"advisory CI-gate count (5) is unpinned by an invariant"** (audit LOW) — **rejected.** 3 of the 5 advisory checks (CodeQL ×2 + Analyze) come from GitHub default-setup, not repo files, so the count can't be structurally derived; the **required** count (9) IS pinned (it's what gates releases). Documenting it as deliberately-unpinned.
|
|
1016
|
+
|
|
1017
|
+
### Tests (1156)
|
|
1018
|
+
|
|
1019
|
+
- None — all additions are inline assertions in existing tests + a script guard (no new `it()`). 1156 unchanged.
|
|
1020
|
+
|
|
1021
|
+
### Files changed
|
|
1022
|
+
|
|
1023
|
+
- `scripts/check-version-consistency.mjs` (α-class roll-up guard), `tests/docs-consistency.test.ts` (3 presence asserts), `tests/eval.test.ts` (error-bucket assertion), `CLAUDE.md` (roll-up rc.26→rc.32 + batch summary), `CHANGELOG.md` (rc.27 ×2 correction).
|
|
1024
|
+
- version bump 3.10.0-rc.31 → 3.10.0-rc.32.
|
|
1025
|
+
|
|
1026
|
+
---
|
|
1027
|
+
|
|
1028
|
+
## [3.10.0-rc.31] — 2026-06-07
|
|
1029
|
+
|
|
1030
|
+
> **TL;DR:** **Eval failure-bucket diagnostics (seeklink-inspired) — turn "the score is low" into "*why* it's low".** The eval harness now classifies every query into a retrieval-failure bucket — `hit_rank_1` / `hit_top_k` / `miss` / `no_labels` / `error` — surfaced per-query and as an aggregate `diagnostics.failure_buckets` counter, plus a breakdown line in the CLI report. This is the half of seeklink's `failure_bucket` idea that fits enquire **safely**: the buckets are derived **only from the already-scored top-K** results, so the metric numbers (NDCG/Recall/MRR) are **byte-identical** — zero behavior change, zero extra retrieval cost. **`answer_contains` answerability is deliberately DEFERRED** (honest scoping): a faithful version needs the full matched-chunk text, but `SearchHybridHit` only carries a ~120-char `snippet`, so a snippet-based check would systematically *under-report* (phrase in the chunk but outside the preview) — a misleading metric this project won't ship. The deeper seeklink "candidate-gen miss vs ranking-budget miss" split is likewise deferred (it needs a retrieval wider than K, which would change the reranker budget and break historical comparability) — both deferrals are documented inline in `eval.ts`. **`src/eval.ts` + tests only; +12 tests → 1156.**
|
|
1031
|
+
|
|
1032
|
+
**Pre-release (v3.10 line) — eval failure-bucket diagnostics (seeklink-inspired).**
|
|
1033
|
+
|
|
1034
|
+
### Added
|
|
1035
|
+
|
|
1036
|
+
- **`FailureBucket` type + `classifyFailureBucket()` + `tallyFailureBuckets()`** (`src/eval.ts`, all exported + pure) — classify a query's outcome from its scored top-K paths; `error` (threw) takes precedence, then `no_labels`, `hit_rank_1`, `hit_top_k`, `miss`.
|
|
1037
|
+
- **`EvalQueryScore.failure_bucket`** (per query) + **`EvalResult.diagnostics.failure_buckets`** (aggregate counter; optional so hand-built results like `run-benchmarks.mjs` stay valid — `runEval` always populates it).
|
|
1038
|
+
- **`formatEvalResult` failure-bucket breakdown** — a `failure buckets: hit@1=… hit@k=… miss=… …` line + a `bucket` column in `--per-query` mode.
|
|
1039
|
+
|
|
1040
|
+
### Deferred (documented inline in `eval.ts`, with reasons)
|
|
1041
|
+
|
|
1042
|
+
- **`answer_contains` answerability** — needs full chunk text; `SearchHybridHit` exposes only a ~120-char snippet, so a snippet-based check would under-report and mislead. Revisit if `searchHybrid` ever returns the full matched chunk.
|
|
1043
|
+
- **`miss` → candidate-gen-miss vs ranking-budget/reranker-ordering-miss split** — needs a retrieval wider than K, which would change the reranker's candidate budget and thus the scored numbers (breaking historical comparability). Needs first-stage-diagnostics plumbing from `searchHybrid` first.
|
|
1044
|
+
|
|
1045
|
+
### Tests (1156)
|
|
1046
|
+
|
|
1047
|
+
- +12 (`tests/eval.test.ts`): `classifyFailureBucket` (5 positive + 3 NEGATIVE controls), `tallyFailureBuckets` (complete-counter + empty-list NEGATIVE), `runEval` populates `failure_bucket` + `diagnostics`, and `formatEvalResult` renders/omits the breakdown (positive + NEGATIVE). 1144 → 1156.
|
|
1048
|
+
|
|
1049
|
+
### Files changed
|
|
1050
|
+
|
|
1051
|
+
- `src/eval.ts` (FailureBucket type + classifier + tally + interface fields + renderer), `tests/eval.test.ts` (+12), count-bump in `README.md` / `package.json` / `llms.txt` / `AGENTS.md` / `ROADMAP.md` / `docs/COMPARISON.md` / `CLAUDE.md`.
|
|
1052
|
+
- version bump 3.10.0-rc.30 → 3.10.0-rc.31.
|
|
1053
|
+
|
|
1054
|
+
---
|
|
1055
|
+
|
|
1056
|
+
## [3.10.0-rc.30] — 2026-06-07
|
|
1057
|
+
|
|
1058
|
+
> **TL;DR:** **Bilingual `README.zh.md` (中文) — reach into the Chinese PKM / Obsidian / dev community (seeklink-inspired).** Added a complete, faithful **Chinese README** mirroring every section (problem/solution, grounded-not-extracted + freshness, quick start, use cases, "when NOT to use it", the full capability table, the 7-tier retrieval ladder, Trust, FAQ) — capitalizing on enquire's *already-shipped* multilingual + CJK (`Intl.Segmenter`) support that was previously under-marketed. A `[English] · [中文]` switcher sits at the top of **both** READMEs, and `README.zh.md` ships in the npm tarball (`package.json#files`). Honest disclaimer up top: the **English README is authoritative** (it updates every release). Per the rc.14 "new docs surface with numeric claims needs an invariant in the same PR" rule, `docs-consistency.test.ts` now **pins the zh numeric claims**: tool count (`45 个工具`) and prompt count (`19 个 MCP 提示词`) exact against `TOOL_MANIFEST`, and the test count as a **drift-proof lower bound** (`1100+ 单元测试`, mirroring AGENTS.md's `X+ tests`) so it stays valid as the suite grows. **Docs/tests only — zero `src/` runtime change. +1 test (the zh invariant) → 1144.**
|
|
1059
|
+
|
|
1060
|
+
**Pre-release (v3.10 line) — bilingual README.zh.md (seeklink-inspired).**
|
|
1061
|
+
|
|
1062
|
+
### Added
|
|
1063
|
+
|
|
1064
|
+
- **`README.zh.md`** — complete Chinese translation of the README, all sections present (tables kept; code blocks verbatim). Markets the existing 50+-language / CJK retrieval to a Chinese-speaking audience.
|
|
1065
|
+
- **`[English] · [中文]` language switcher** at the top of both `README.md` and `README.zh.md`.
|
|
1066
|
+
- **`README.zh.md` added to `package.json#files`** so it ships to npm alongside the English README.
|
|
1067
|
+
|
|
1068
|
+
### Tooling (structural enforcement)
|
|
1069
|
+
|
|
1070
|
+
- **`docs-consistency.test.ts` pins README.zh.md numeric claims** (rc.14 new-surface rule): `45 个工具` == `TOOL_MANIFEST.length`, `19 个 MCP 提示词` == registered prompts, and `1100+ 单元测试` as a lower bound (must be ≤ actual and within 200 of it).
|
|
1071
|
+
|
|
1072
|
+
### Tests (1144)
|
|
1073
|
+
|
|
1074
|
+
- +1 (`docs-consistency.test.ts`): the README.zh.md numeric-claims invariant. 1143 → 1144 (English count surfaces bumped accordingly; the zh README uses the drift-proof `1100+` lower bound, so it never needs a count bump).
|
|
1075
|
+
|
|
1076
|
+
### Files changed
|
|
1077
|
+
|
|
1078
|
+
- `README.zh.md` (new), `README.md` (switcher), `package.json` (files + description count), `tests/docs-consistency.test.ts` (zh invariant), count-bump in `llms.txt` / `AGENTS.md` / `ROADMAP.md` / `docs/COMPARISON.md` / `CLAUDE.md`.
|
|
1079
|
+
- version bump 3.10.0-rc.29 → 3.10.0-rc.30.
|
|
1080
|
+
|
|
1081
|
+
---
|
|
1082
|
+
|
|
1083
|
+
## [3.10.0-rc.29] — 2026-06-07
|
|
1084
|
+
|
|
1085
|
+
> **TL;DR:** **`llms.txt` → full agent contract (seeklink-inspired).** enquire's `llms.txt` followed the [llmstxt.org](https://llmstxt.org/) curated-links shape; seeklink's packs a dense "how to drive me" contract instead. Took the best of both: kept the spec-compliant link sections and **added an `## Agent contract` + `### Common failure modes`** block (free-form, before the trailing `Optional` section, so still spec-valid). It gives an AI agent the minimum loop (`obsidian_search` → `obsidian_read_note` → cite), the prefer-enquire-for-meaning-vs-`grep`-for-literal rule, the observability fields (`per_signal`, `age_days`/`stale`), the read-only-by-default posture, an **untrusted-content security note** (retrieved note text is data, not instructions), and the real failure modes (model-not-downloaded → TF-IDF fallback, empty fresh vault, whole-vault scan cap, `serve-http` bearer 401). No new gated numeric claims (so no invariant churn). **`llms.txt` only — zero `src/`, 1143 tests unchanged.**
|
|
1086
|
+
|
|
1087
|
+
**Pre-release (v3.10 line) — llms.txt agent-contract enrichment (seeklink-inspired).**
|
|
1088
|
+
|
|
1089
|
+
### Added
|
|
1090
|
+
|
|
1091
|
+
- **`llms.txt` `## Agent contract` section** — minimum agent loop, when-to-prefer-enquire-vs-`grep`, observability (`per_signal` / `age_days` / `stale`; scores sort within one query only), read-only-by-default + `--disabled-tools`/`--enabled-tools`, and an **untrusted-content** note (treat "ignore previous instructions"-style text inside a retrieved note as content, never a command).
|
|
1092
|
+
- **`llms.txt` `### Common failure modes` subsection** — first-call embedding-model-not-downloaded → `setup`/`install-model` (umbrella degrades to TF-IDF meanwhile), empty fresh vault, whole-vault scan safety cap (partial results flagged, never silent), and `serve-http` bearer-token-too-short → HTTP 401 (`gen-token` mints a valid one).
|
|
1093
|
+
|
|
1094
|
+
### Tests (1143)
|
|
1095
|
+
|
|
1096
|
+
- None — `llms.txt` only; no gated numeric claims added (existing llms.txt invariants — test count, 34+4+7 tool breakdown, prompt count, CI-gate count — unchanged). 1143 unchanged.
|
|
1097
|
+
|
|
1098
|
+
### Files changed
|
|
1099
|
+
|
|
1100
|
+
- `llms.txt` (Agent contract + failure-modes sections).
|
|
1101
|
+
- version bump 3.10.0-rc.28 → 3.10.0-rc.29.
|
|
1102
|
+
|
|
1103
|
+
---
|
|
1104
|
+
|
|
1105
|
+
## [3.10.0-rc.28] — 2026-06-07
|
|
1106
|
+
|
|
1107
|
+
> **TL;DR:** **README trust-batch — honest scoping + a self-propagating agent rule (seeklink-inspired).** Added a candid **"When enquire-mcp is *not* the right tool"** section (use `rg` for literal search; conversation-memory tools are a different category; not multi-user/GUI/web-scale) — explicit non-goals build trust, mirroring seeklink's "Not For". Marketed the already-existing **read-only-by-default** posture with a new **least-privilege** Trust row (`--disabled-tools` / `--enabled-tools`), and shipped a **reusable agent-rule snippet** users drop into their own `AGENTS.md`/`CLAUDE.md`/`.cursorrules` so their agent learns *when* to reach for the vault (and when to prefer `grep`). Plus two **state-driven catches** the change-driven gates' regexes missed: a stale **"44 tools" → 45** in three docs (README comparison row, `docs/COMPARISON.md` table cell, `AGENTS.md` file-tree — the 45th tool `obsidian_stale_notes` shipped in v3.10 but these phrasings never updated; one even contradicted its own 34+4+7=45 breakdown), and a **broken Karpathy gist link** in the README (404 — every other reference used the correct id). Per the "drift demands a structural defense" rule, the same PR **extends the tool-count invariants** to pin the missed phrasings (`**N production tools**`, `| Tool count | N |`, `N tool implementations`). **Docs/tests only — zero `src/` runtime change, 1143 tests unchanged.**
|
|
1108
|
+
|
|
1109
|
+
**Pre-release (v3.10 line) — README trust-batch (seeklink-inspired) + tool-count drift class.**
|
|
1110
|
+
|
|
1111
|
+
### Added
|
|
1112
|
+
|
|
1113
|
+
- **README "🚫 When enquire-mcp is *not* the right tool"** — honest non-goals: literal search (`rg`), conversation-memory category (mem0/Zep/Supermemory), multi-user/hosted, non-Markdown sources, GUI/plugin, web-scale corpora. Trust through candor.
|
|
1114
|
+
- **README "Reusable agent rule" snippet** — a copy-paste block for any `AGENTS.md`/`CLAUDE.md`/`.cursorrules` telling the agent to search the vault first for conceptual recall and use `grep`/`rg` for literal strings (the self-propagating-adoption pattern borrowed from seeklink's Agent Notes).
|
|
1115
|
+
- **README Trust "Least privilege" row** — markets the existing `--disabled-tools`/`--enabled-tools` surface-subsetting (e.g. a read-only research agent gets only `obsidian_search` + `obsidian_read_note`).
|
|
1116
|
+
|
|
1117
|
+
### Fixed
|
|
1118
|
+
|
|
1119
|
+
- **Stale "44 tools" → 45 in three docs** (README comparison row, `docs/COMPARISON.md` "Tool count" cell, `AGENTS.md` file-tree). The 45th tool (`obsidian_stale_notes`) shipped in the v3.10 line but these phrasings drifted; the README row even contradicted its own "34 + 4 + 7 = 45" breakdown.
|
|
1120
|
+
- **Broken Karpathy LLM-Wiki gist link in the README** (`…914927…` → 404). Corrected to the canonical id (`…914893…`, HTTP 200) used everywhere else in the codebase.
|
|
1121
|
+
|
|
1122
|
+
### Tooling (structural enforcement)
|
|
1123
|
+
|
|
1124
|
+
- **Extended the tool-count invariants** (`tests/docs-consistency.test.ts`) to close the phrasings the existing `**N tools**` regex couldn't see — all pinned to `TOOL_MANIFEST.length`: `**N production tools**` (README), `| Tool count | N |` (COMPARISON table cell), `N tool implementations` (AGENTS file-tree). This is the "a drift finding demands a full-surface sweep + structural defense" rule — the instance fix alone would let the class recur.
|
|
1125
|
+
|
|
1126
|
+
### Tests (1143)
|
|
1127
|
+
|
|
1128
|
+
- No new `it()` — the new assertions extend three existing tool-count tests (no canonical-count change). 1143 unchanged.
|
|
1129
|
+
|
|
1130
|
+
### Files changed
|
|
1131
|
+
|
|
1132
|
+
- `README.md` (Not-For section, agent-rule snippet, least-privilege Trust row, 44→45 comparison row, Karpathy link fix), `docs/COMPARISON.md` (Tool count 44→45), `AGENTS.md` (44→45 file-tree), `tests/docs-consistency.test.ts` (3 invariant extensions).
|
|
1133
|
+
- version bump 3.10.0-rc.27 → 3.10.0-rc.28.
|
|
1134
|
+
|
|
1135
|
+
---
|
|
1136
|
+
|
|
1137
|
+
## [3.10.0-rc.27] — 2026-06-07
|
|
1138
|
+
|
|
1139
|
+
> **TL;DR:** **Docker / Glama discoverability — a borrowed lesson from `seeklink`.** MCP directories (Glama, and through Glama the `awesome-mcp-servers` listing) introspect a server by **building its Dockerfile** and completing an MCP handshake + `tools/list` over stdio. enquire shipped `glama.json` long ago but had **no Dockerfile**, so that check couldn't build it. Added a minimal, reproducible **multi-stage `Dockerfile`** that builds from source and serves the **read-only-by-default** MCP over stdio against a baked sample vault — it installs deps with `--ignore-scripts` so `tsc` resolves the optional-dep types with **no native toolchain**, then **prunes optional from the slim runtime**: each native dep loads via lazy `await import()` only when a heavy tool is *called*, so `tools/list` introspection works without them (umbrella search degrades to pure-JS TF-IDF; full FTS5/embeddings/PDF retrieval uses the npm install path). _(The first CI pass caught that `--omit=optional` broke `tsc` — the optional packages are referenced in typed dynamic imports — exactly why the `docker` job exists; corrected to `--ignore-scripts` + prune-optional.)_ Plus a `.dockerignore` for a lean context. Made it **structural** with `tests/docker-glama-invariant.test.ts`: the Dockerfile must invoke the real bin (`dist/index.js`), run `serve`, and use a Node base image whose major ≥ `engines.node` floor; `glama.json` must be valid + list the owner — each with a real NEGATIVE control. **Infra/docs/tests only — zero `src/` runtime change.** The canonical install path stays `npm install -g @oomkapwn/enquire-mcp`; the image is for directory introspection + quick container trials.
|
|
1140
|
+
|
|
1141
|
+
**Pre-release (v3.10 line) — Docker/Glama discoverability (seeklink-inspired).**
|
|
1142
|
+
|
|
1143
|
+
### Added
|
|
1144
|
+
|
|
1145
|
+
- **`Dockerfile`** — multi-stage (build → slim runtime). Build stage: `npm ci --ignore-scripts` (optional deps present so `tsc` resolves their typed dynamic imports — `hnswlib-node` / `pdfjs-dist` / `tesseract.js` / `@napi-rs/canvas` — but never natively compiled → no python/make/g++) → `npm run build` → `npm prune --omit=dev --omit=optional` (slim runtime). Runtime stage: `node:22-slim` (matches `engines.node` ≥ 22) with `dist/` + prod deps + a baked `/vault/welcome.md`. `ENTRYPOINT ["node","dist/index.js"]` + `CMD ["serve","--vault","/vault"]` — read-only by default, so an introspection harness can never mutate. Header documents the real-use path (`docker run -i -v /abs/vault:/vault …`).
|
|
1146
|
+
- **`.dockerignore`** — keeps the build context lean + deterministic (excludes `node_modules`, `dist`, `.git`, `tests`, `docs/audits`, `assets`, etc.).
|
|
1147
|
+
|
|
1148
|
+
### Tooling (structural enforcement)
|
|
1149
|
+
|
|
1150
|
+
- **`tests/docker-glama-invariant.test.ts`** — pins the two files the directory check depends on. Asserts (1) the Dockerfile invokes `dist/index.js` + runs `serve`, (2) every `FROM node:<major>` base image major ≥ the `engines.node` floor (catches a future engines bump outrunning the base image → unsupported runtime), (3) `glama.json` is valid JSON with a `glama.ai` `$schema` + the owner in `maintainers`. Pure analyzers (`analyzeDockerfile` / `engineNodeMajorFloor` / `validateGlamaConfig`) are driven by 5 NEGATIVE controls (no-bin/no-serve Dockerfile, sub-floor base image, missing engines, invalid JSON, missing owner+schema) so the guard is provably non-vacuous. Auto-scanned by the META-invariant (`*-invariant.test.ts`).
|
|
1151
|
+
- **CI `docker` job (`.github/workflows/ci.yml`, advisory).** Anti-overclaim: the image couldn't be built in this dev environment, so a new job actually `docker build`s it, smoke-runs `--help`, and performs a **`tools/list` stdio introspection** (the exact MCP handshake Glama does) asserting `obsidian_search` comes back — turning "Glama-introspectable" into an *enforced* claim and guarding the Dockerfile against rot. Advisory (not in the branch-protection required set → never blocks a merge); uses only the already-SHA-pinned `checkout` + preinstalled `docker` (no new action to pin, no `npm ci` → OIA Checks 9/10 N/A). Advisory gate count `4 → 5` synced across README ×2, `llms.txt`, `AGENTS.md` (count header + advisory list).
|
|
1152
|
+
|
|
1153
|
+
### Docs
|
|
1154
|
+
|
|
1155
|
+
- Test-count surfaces bumped `1135 → 1143` (README badge/hero/trust-row/test-cmd, `package.json` description, `llms.txt`, `AGENTS.md`, `ROADMAP.md`, `docs/COMPARISON.md`, `CLAUDE.md` roll-up) — the docs-consistency invariant pins these to the live `it()` count.
|
|
1156
|
+
|
|
1157
|
+
### Tests (1143)
|
|
1158
|
+
|
|
1159
|
+
- +8 (`tests/docker-glama-invariant.test.ts`): 3 positive (real Dockerfile + glama.json assertions) + 5 NEGATIVE controls. 1135 → 1143.
|
|
1160
|
+
|
|
1161
|
+
### Files changed
|
|
1162
|
+
|
|
1163
|
+
- `Dockerfile` (new), `.dockerignore` (new), `tests/docker-glama-invariant.test.ts` (new), `.github/workflows/ci.yml` (advisory `docker` job).
|
|
1164
|
+
- count-bump (`1135 → 1143`) in `README.md`, `package.json`, `llms.txt`, `AGENTS.md`, `ROADMAP.md`, `docs/COMPARISON.md`, `CLAUDE.md`; advisory-gate-count bump (`4 → 5`) in `README.md` ×2, `llms.txt`, `AGENTS.md` (count header + advisory list).
|
|
1165
|
+
- version bump 3.10.0-rc.26 → 3.10.0-rc.27.
|
|
1166
|
+
|
|
1167
|
+
---
|
|
1168
|
+
|
|
1169
|
+
## [3.10.0-rc.26] — 2026-06-06
|
|
1170
|
+
|
|
1171
|
+
> **TL;DR:** **SYS-1 — supply-chain content-pin (M-9 completion).** The release workflow's one external `run:` download — the `mcp-publisher` CLI that runs with our **OIDC publish identity** on a stable release — was *tag*-pinned (`v1.7.9`, rc.33 M-9) but tag-pins are **mutable** (a tag can be force-moved, a release asset re-uploaded). Now it's **content-pinned**: the tarball's SHA256 is verified (`sha256sum -c`, fail-closed) before it's extracted/executed. Made it **structural** by extending **OIA Check 9b**: a tag-pinned release-archive (`releases/download/<tag>/…\.tar.gz`) `run:` download must ALSO carry a SHA256 verification in the same workflow, else CI fails (`RUN-DOWNLOAD-UNVERIFIED`; detection-power inject/revert-verified). Verified the deferred SYS-1 items against current code first (anti-overclaim): **H-3** paired-sink PDF/OCR parity was **already closed in rc.33**, and Check 9b's `releases/latest` guard already existed — so the genuine residual was just the tag→content upgrade. **Workflow/script/docs only — zero `src/`, 1135 tests unchanged.** Closes the rc.36 meta-audit's two named "deferred behavioral dimensions".
|
|
1172
|
+
|
|
1173
|
+
**Pre-release (v3.10 line) — SYS-1: deferred behavioral-defense dimensions.**
|
|
1174
|
+
|
|
1175
|
+
### Security (supply-chain)
|
|
1176
|
+
|
|
1177
|
+
- **M-9 completion — `mcp-publisher` download is now SHA256 content-pinned (was tag-pinned).** `release.yml`'s registry-publish step downloads the official `mcp-publisher` CLI from a GitHub release. rc.33 pinned it to the `v1.7.9` tag (closed `releases/latest`), but a tag is not immutable. Now: download to a file → `echo "<sha256> mcp-publisher.tar.gz" | sha256sum -c -` (fail-closed) → extract. The pinned hash (`ab12…81ac`, linux/amd64 — the `ubuntu-latest` runner arch) is bumped *deliberately together with* the tag. This binary runs with the workflow's OIDC identity, so content-pinning it is the highest-value spot for the strongest supply-chain defense. (The download was also simplified from `uname`-portable to explicit `linux_amd64`, matching the fixed runner.)
|
|
1178
|
+
|
|
1179
|
+
### Tooling (structural enforcement)
|
|
1180
|
+
|
|
1181
|
+
- **OIA Check 9b extended — release-archive `run:` downloads must be SHA256-verified.** Check 9b already flagged `releases/latest` (moving URL). It now ALSO flags a tag-pinned release **archive** (`releases/download/<tag>/…\.tar.gz|.tgz|.zip`) `curl`/`wget` that lacks a `sha256sum -c` / `shasum -a 256 -c` anywhere in the same workflow file (`RUN-DOWNLOAD-UNVERIFIED`). This converts the content-pin from a one-time fix into a permanent gate — the rc.36 "internalize the lens as an inventory invariant" move. Detection-power verified: stripping the `sha256sum -c` line flags `release.yml:240`; restored → clean. (Check 9b is a sub-check of Check 9 — the canonical OIA top-level count stays 12.)
|
|
1182
|
+
|
|
1183
|
+
### Docs
|
|
1184
|
+
|
|
1185
|
+
- **`CLAUDE.md`** — the rc.36 "remaining uncovered behavioral dimensions" note marked M-9 (→ rc.26) and H-3 (→ rc.33) **closed**, with the still-uncovered set named honestly (generalized enforcement-verb→code-guard taxonomy; the accepted `block`-granularity FTS5↔embed chunk-index divergence). Status roll-up extended rc.25 → rc.26.
|
|
1186
|
+
|
|
1187
|
+
### Tests (1135)
|
|
1188
|
+
|
|
1189
|
+
None — workflow/script/docs only; no `src/` or test change. Check 9b's new branch is verified by the inject/revert detection-power run (OIA checks run via `npm run check:oia`, not vitest). 1135 unchanged.
|
|
1190
|
+
|
|
1191
|
+
### Files changed
|
|
1192
|
+
|
|
1193
|
+
- `.github/workflows/release.yml` (mcp-publisher download → file + `sha256sum -c` + extract), `scripts/oia-walk.mjs` (Check 9b archive-checksum requirement), `CLAUDE.md` (dimension-status note + roll-up).
|
|
1194
|
+
- version bump 3.10.0-rc.25 → 3.10.0-rc.26.
|
|
1195
|
+
|
|
1196
|
+
---
|
|
1197
|
+
|
|
1198
|
+
## [3.10.0-rc.25] — 2026-06-06
|
|
1199
|
+
|
|
1200
|
+
> **TL;DR:** **Round-2 audit — LOW docs-currency batch (final; docs-only).** Five un-gated currency-drift surfaces the change-driven gates don't watch, all flagged by the round-2 state-driven docs sweep: (1) **`CLAUDE.md`** status section was frozen at `v3.9.0-rc.35` (header still said "v3.8.x stable + v3.9.0 architectural") — added a condensed v3.9.0-stable→v3.9.1→v3.10.0-rc.1→rc.25 roll-up + moved the "(current)" marker + updated the title; (2) **`llms.txt`** "what's new" list stopped at rc.3, omitting the v3.10 freshness flagship (contradicting llms.txt's own header) — added it; (3) **`docs/benchmarks.md`** metric-validity said "through the v3.9.0-rc cascade" — extended to the v3.10 line (recency re-rank is off-by-default, a provable no-op, so the numbers are unchanged); (4) **`README.md`** highlight reel stopped at "v3.9.0 stable" — appended a v3.10 (`@rc`) freshness entry; (5) **`docs/api.md`** described the freshness boolean as an "over-one-year flag (≥ 365)" without naming it — now `stale` flag (≥ `--stale-days`, default 365). **Docs-only — zero `src/`, 1135 tests unchanged. This closes the round-2 (post-MED) audit** (rc.23 HIGH shutdown + rc.24 LOW code + rc.25 LOW docs).
|
|
1201
|
+
|
|
1202
|
+
**Pre-release (v3.10 line) — round-2 audit; LOW docs-currency batch (final).**
|
|
1203
|
+
|
|
1204
|
+
### Docs
|
|
1205
|
+
|
|
1206
|
+
- **`CLAUDE.md`** — title `v3.8.x stable + v3.9.0 architectural` → `v3.9.x stable maintenance + v3.10 forgetting-aware line`; added a single condensed status roll-up entry (v3.9.0 STABLE promotion → v3.9.1 → the full v3.10.0-rc.1→rc.25 line: staleness, bug-report batch, MED audit M1–M10, round-2 re-sweep incl. the rc.23 shutdown HIGH) marked "(current)"; removed "(current)" from the rc.35 entry. (Internal process doc — not packaged; the recurring "CLAUDE.md status frozen" α-drift the project's own anti-pattern list names.)
|
|
1207
|
+
- **`llms.txt`** — added a `v3.10+ (@rc)` line to the recent-features list (forgetting-aware freshness + frontmatter-aware search), resolving the list-vs-header self-contradiction on an AI-discoverability surface.
|
|
1208
|
+
- **`docs/benchmarks.md`** — metric-validity currency extended from "the v3.9.0-rc cascade" to "the v3.10 line", with the explicit note that the rc.5 recency re-rank is off by default (`--recency-weight 0` = provable no-op) so default-config numbers are unchanged.
|
|
1209
|
+
- **`README.md`** — highlight reel gained a `v3.10` (`@rc`) entry (freshness + frontmatter-aware search), so it no longer lags the README's own hero differentiator.
|
|
1210
|
+
- **`docs/api.md`** — the v3.10 freshness boolean is now named (`stale`) and `365` is shown as the `--stale-days` default rather than an absolute.
|
|
1211
|
+
|
|
1212
|
+
### Tests (1135)
|
|
1213
|
+
|
|
1214
|
+
None — docs-only RC; no `src/` or test change. 1135 unchanged.
|
|
1215
|
+
|
|
1216
|
+
### Files changed
|
|
1217
|
+
|
|
1218
|
+
- `CLAUDE.md` (title + status roll-up), `llms.txt`, `docs/benchmarks.md`, `README.md`, `docs/api.md`.
|
|
1219
|
+
- `scripts/check-per-file-coverage.mjs` — refreshed the stale `watcher.ts` inline coverage comment (60.69% → 61.83%; rc.24's unlink-gate change + test raised it; caught by OIA Check 6 against the fresh `coverage-summary.json`).
|
|
1220
|
+
- version bump 3.10.0-rc.24 → 3.10.0-rc.25.
|
|
1221
|
+
|
|
1222
|
+
### Method note
|
|
1223
|
+
|
|
1224
|
+
This concludes the **round-2 (post-MED) audit** — a 3-agent pass on the shipped rc.22 commit (per the CLAUDE.md "re-run a focused audit after a class-closing release" rule). It returned 1 HIGH (rc.23 — a regression in our own rc.19, empirically reproduced + fix-verified before shipping), 3 LOW code (rc.24), and 5 LOW docs-currency (rc.25). No CRITICAL; `src/` remains exceptionally clean. The HIGH validated the meta-lesson: the home-grown gates are drift/claim-driven and structurally blind to runtime behavior, so the external-lens re-sweep after each batch is not optional.
|
|
1225
|
+
|
|
1226
|
+
---
|
|
1227
|
+
|
|
1228
|
+
## [3.10.0-rc.24] — 2026-06-06
|
|
1229
|
+
|
|
1230
|
+
> **TL;DR:** **Round-2 audit — LOW code batch (3 fixes).** The same post-MED re-sweep that found the rc.23 HIGH surfaced three LOWs, all verified against current code: (1) **`obsidian_query_base` (`bases.ts queryBase`)** was the lone uncapped member of the `capScanEntries` whole-vault-scanner class — always-registered + bearer-reachable, it reads every matched note's full body with no cap; the `resource-bound-invariant` missed it because `bases.ts` is outside `SCANNER_SOURCES` AND it uses `listFilesByExtension`+`readFile` (not the `listMarkdown`+`readNote` shape the heuristic detects). Capped via `capScanEntries` + a standalone invariant assertion (mirroring the `buildWikilinkGraph` one). (2) **`parser.ts bodyStartLine`** used `source.indexOf(body)`, which false-matches a degenerate note whose entire body text also appears verbatim in a frontmatter line (`---\nx: hi\n---\nhi` → wrong line) → `lastIndexOf` (body is the suffix) + empty-body guard. (3) **`watcher.ts handle()`** rc.20's exclude re-check skipped ALL kinds, so an excluded path's `unlink` never purged its rows (orphan index entries for a deleted-but-excluded note) → gate only add/change; `unlink` always cleans up. **1132 → 1135 tests.** Docs-currency LOWs → rc.25.
|
|
1231
|
+
|
|
1232
|
+
**Pre-release (v3.10 line) — round-2 audit; LOW code batch.**
|
|
1233
|
+
|
|
1234
|
+
### Fixed
|
|
1235
|
+
|
|
1236
|
+
- **LOW (DoS-cap completeness) — `obsidian_query_base` uncapped whole-vault content scan.** `queryBase` (`src/bases.ts`) walks every `.md` note and reads its full body (`limit` applied only AFTER the walk, by design, so it can't bound the scan). It's always-registered + bearer-reachable on `serve-http` — the same shape as `runDql`, which got a `capScanEntries` cap in rc.18 (M4). Now `queryBase`'s scan is wrapped in `capScanEntries(..., "obsidian_query_base")`. The `resource-bound-invariant` couldn't see it (its `discoverScanners` heuristic requires `listMarkdown`+`readNote`; `queryBase` uses `listFilesByExtension`+`readFile`, and `bases.ts` wasn't in `SCANNER_SOURCES`), so a standalone assertion (`bases.queryBase caps its scan via capScanEntries`) now guards it explicitly — mirroring the existing `communities.buildWikilinkGraph` separate assertion. (Broadening the heuristic was rejected: it would then sweep doctor/vault/communities/server/media for the `listFilesByExtension` shape and risk false-positive cascades.)
|
|
1237
|
+
- **LOW (correctness) — `parser.ts bodyStartLine` false-early match.** rc.17 computed the body's file line via `source.indexOf(body)` to make embedding line-citations file-absolute. For a degenerate note whose whole body text also appears verbatim inside a frontmatter line, `indexOf` matches the frontmatter occurrence → too-early a line → an embedding chunk's `line_start` could point inside the frontmatter (the exact deep-link mis-pointing rc.17 fixed, for this shape). Now `source.lastIndexOf(body)` (the body is always the SUFFIX of source) + an empty-body guard (`body.length > 0 ? lastIndexOf : -1`).
|
|
1238
|
+
- **LOW (correctness) — watcher skipped `unlink` cleanup for excluded paths.** rc.20's M7 defense-in-depth re-check (`if (isExcluded) return`) gated ALL event kinds. A note indexed before an exclusion was added, then deleted, never had its rows dropped (orphan FTS5/embed entries — hidden from results by the terminal `isExcluded` filter, but stale on disk). Now the gate is `if (kind !== "unlink" && isExcluded) return` — only the INDEXING ops (add/change) skip; a delete always purges, since removing content is never a privacy risk.
|
|
1239
|
+
|
|
1240
|
+
### Tests (1135)
|
|
1241
|
+
|
|
1242
|
+
`tests/resource-bound-invariant.test.ts` +1 (`bases.queryBase caps its scan via capScanEntries`). `tests/parser.test.ts` +1 (degenerate: body text also in frontmatter → `bodyStartLine` anchors to the real body line, not the frontmatter occurrence — fails with the old `indexOf`). `tests/watcher.test.ts` +1 (excluded `unlink` proceeds to cleanup — the discriminator vs the excluded `change` which stays gated). 1132 → 1135.
|
|
1243
|
+
|
|
1244
|
+
### Files changed
|
|
1245
|
+
|
|
1246
|
+
- `src/bases.ts` (`capScanEntries` import + `queryBase` cap), `src/parser.ts` (`indexOf` → `lastIndexOf` + guard), `src/watcher.ts` (`handle()` gates only add/change on `isExcluded`), `tests/resource-bound-invariant.test.ts` (+1), `tests/parser.test.ts` (+1), `tests/watcher.test.ts` (+1), test-count claims → 1135.
|
|
1247
|
+
- version bump 3.10.0-rc.23 → 3.10.0-rc.24.
|
|
1248
|
+
|
|
1249
|
+
---
|
|
1250
|
+
|
|
1251
|
+
## [3.10.0-rc.23] — 2026-06-06
|
|
1252
|
+
|
|
1253
|
+
> **TL;DR:** **HIGH — `serve-http` hung forever on SIGINT/SIGTERM (a regression of rc.19's own M3 fix).** A post-MED-batch re-sweep (3-agent audit on the rc.22 commit) caught it: rc.19 correctly made shutdown **await** the full teardown before `process.exit(0)` — but `shutdownHttpServer` awaits `server.close()`, and Node's `http.Server.close()` callback fires only once EVERY connection has ended and **does not terminate idle keep-alive sockets**. So any lingering connection (a reverse proxy's keep-alive, a half-open socket, an LB health probe, an SSE stream) made graceful shutdown block **indefinitely**. Pre-rc.19 this latent `close()` hang was masked because the cache-flush handler called `process.exit(0)` on its own; rc.19 removed that hatch and added no bound → "await the drain" became "await forever." **Reproduced** (lingering socket + SIGTERM → process alive >8s, would hang forever; pre-rc.19 exited in 8ms). Fix: a bounded `closeServerBounded()` — close idle keep-alives immediately, then force-close stragglers via `server.closeAllConnections()` after a 3s grace — so shutdown resolves on `close()` completion OR the grace, never never. **Verified: post-fix the same repro exits in ~3.0s, code 0.** This is the textbook recursion-pair (a fix for one shutdown bug shipping another); documented as such. **1130 → 1132 tests.**
|
|
1254
|
+
|
|
1255
|
+
**Pre-release (v3.10 line) — post-MED-batch audit; HIGH regression fix.**
|
|
1256
|
+
|
|
1257
|
+
### Fixed
|
|
1258
|
+
|
|
1259
|
+
- **HIGH — `serve-http` graceful shutdown could hang forever on a lingering connection (regression introduced by rc.19).** `shutdownHttpServer` did `await new Promise(resolve => server.close(() => resolve()))`. `http.Server.close()` waits for ALL open connections to end and never force-closes idle keep-alives, so a single held-open socket blocked the await — and rc.19's `makeHttpShutdownHandler` gates `process.exit(0)` behind that await, so the process never exited. Under an orchestrator (systemd/docker) this escalates to a SIGKILL after the stop-timeout, defeating the very graceful-drain guarantee rc.19 was built to provide. New `closeServerBounded(server, graceMs = HTTP_CLOSE_GRACE_MS=3000)`: registers `server.close()`, immediately calls `server.closeIdleConnections()` (so the common no-in-flight case resolves at once), and arms an unref'd `setTimeout` that force-closes stragglers with `server.closeAllConnections()` after the grace — resolving whichever happens first. Both `shutdownHttpServer` close sites (the `!extras` fast path + the main path, after `registry.closeAll()` drains in-flight MCP requests) route through it. Reproduced + fix-verified empirically (lingering-socket SIGTERM: hang → 3.0s bounded exit).
|
|
1260
|
+
|
|
1261
|
+
### Tests (1132)
|
|
1262
|
+
|
|
1263
|
+
`tests/http-transport.test.ts` +2: `closeServerBounded` resolves within the grace despite a lingering keep-alive socket (the rc.19 hang — pre-fix this never returns) + CONTROL (with nothing lingering it resolves promptly, well under a large grace — proving it resolves on `close()` completion, NOT by always waiting the grace; a naive `setTimeout(resolve, grace)` impl fails this). 1130 → 1132.
|
|
1264
|
+
|
|
1265
|
+
### Files changed
|
|
1266
|
+
|
|
1267
|
+
- `src/http-transport.ts` (new `HTTP_CLOSE_GRACE_MS` + exported `closeServerBounded`; both `shutdownHttpServer` close sites bounded), `tests/http-transport.test.ts` (+2 + `node:http`/`node:net` imports), test-count claims → 1132.
|
|
1268
|
+
- version bump 3.10.0-rc.22 → 3.10.0-rc.23.
|
|
1269
|
+
|
|
1270
|
+
### Method note
|
|
1271
|
+
|
|
1272
|
+
The fix was found by re-running a focused audit on the just-shipped commit (the CLAUDE.md "post-merge re-sweep after a class-closing release" rule) — a 3-agent pass (adversarial diff re-review · docs/process · behavioral/STRIDE) that I commissioned after the MED batch, with the HIGH **empirically reproduced and fix-verified by me** before shipping (not taken on the agent's word). It is a clean recursion-pair instance: rc.19 fixed a shutdown race (flush `exit` beat the drain) and, in removing the `exit` hatch, exposed a latent `server.close()` hang. The remaining audit findings (1 LOW behavioral — `obsidian_query_base` uncapped scan; 2 LOW correctness — parser `indexOf`, watcher unlink-skip; 5 LOW docs currency) ship as rc.24/rc.25.
|
|
1273
|
+
|
|
1274
|
+
---
|
|
1275
|
+
|
|
1276
|
+
## [3.10.0-rc.22] — 2026-06-06
|
|
1277
|
+
|
|
1278
|
+
> **TL;DR:** **Audit MED-batch 7 (final) — M8/M9 test & process integrity.** **M8a (vacuous test):** `security.test.ts`'s "embeddingsSearch filters excluded paths" test was THEATER — it said "we can't test without a model" and **reimplemented** the privacy filter inline (`rawHits.filter(h => !vault.isExcluded(...))`), never running the real code, so `embeddingsSearch`'s two inline filter sites (search.ts ~1100/1106) were uncovered and would stay green even if the guard were deleted. Extracted the filter into a pure, exported `filterExcludedEmbedHits` (the `embeddingsSearch` sibling of rc.8's `pruneExcludedHits`), routed both sites through it, and made the test + a new unit test exercise the REAL helper. **M8b (silent-skip):** the E2E CI-GUARD only asserted `distExists()` — T-3/T-4 had **no** guard at all, and none checked that the server actually spawned, so a spawn failure in CI would silently skip whole suites (incl. T-4's 401-no-bearer auth check). Added/strengthened CI-GUARDs across T-2/T-3/T-4 to assert dist built **and** the process spawned in CI. **M9 (config drift, ι-class):** `package.json` `prepublishOnly` used a single `npm audit --audit-level=high` (all deps) while CI + release both use the stricter two-step (`--omit=dev --audit-level=moderate` then `--include=dev --audit-level=high`) — so prepublish would miss a *moderate prod* vuln CI/release catch. Aligned. **1126 → 1130 tests. This closes the comprehensive-audit MEDIUM batch (M1–M10).**
|
|
1279
|
+
|
|
1280
|
+
**Pre-release (v3.10 line) — audit fix batch 7/7 (M8/M9).**
|
|
1281
|
+
|
|
1282
|
+
### Fixed
|
|
1283
|
+
|
|
1284
|
+
- **M8a — embeddingsSearch privacy filter was untested (vacuous "theater" test).** `tests/security.test.ts`'s embeddingsSearch privacy test reimplemented the `!vault.isExcluded(rel_path)` filter inline and asserted on the reimplementation — it never invoked anything in `src/`, so `embeddingsSearch`'s two inline filter sites had zero behavioral coverage. New pure `filterExcludedEmbedHits<T extends {rel_path}>(hits, isExcluded)` in `src/tools/search.ts` (sibling of `pruneExcludedHits`); both `embeddingsSearch` sites (HNSW refill path + brute-force path) now call it; the security test + a new `search-hybrid.test.ts` unit test (positive + NEGATIVE control) exercise the REAL helper. A regression that drops the guard now fails CI.
|
|
1285
|
+
- **M8b — E2E CI-GUARD silent-skip gaps.** The `tests/e2e-handlers.test.ts` CI-GUARD asserted only `distExists()`, and only T-2 had one — T-3 (HyDE) and T-4 (serve-http) had none, so a failed `spawnServer` / `serve-http` spawn in CI would leave `client`/`proc` null and every test body's `if (!client) return` / `if (!proc) return` would silently skip the suite (including T-4's 401-without-bearer auth assertion). Now each of T-2/T-3/T-4 has a CI-GUARD asserting dist built **and** the server spawned (client non-null / proc non-null + port bound) in CI. Propagates the rc.23 silent-skip→CI-GUARD pattern to the two describes it missed.
|
|
1286
|
+
- **M9 — `prepublishOnly` audit weaker than CI/release (ι-class config drift).** `package.json:prepublishOnly` ran `npm audit --audit-level=high` (high across ALL deps), while `ci.yml` + `release.yml` both run the two-step `npm audit --omit=dev --audit-level=moderate` + `npm audit --include=dev --audit-level=high` — so a **moderate severity prod** vuln would pass prepublish but fail CI/release. Aligned `prepublishOnly` to the identical two-step. (The v3.7.19 ι-class alignment synced release↔CI but missed `prepublishOnly`.)
|
|
1287
|
+
|
|
1288
|
+
### Tests (1130)
|
|
1289
|
+
|
|
1290
|
+
`tests/search-hybrid.test.ts` +2 (`filterExcludedEmbedHits`: removes excluded `rel_path`s preserving order + NEGATIVE control proving it's predicate-driven). `tests/e2e-handlers.test.ts` +2 (T-3 + T-4 CI-GUARDs; T-2's existing guard strengthened to also assert the spawn, net 0). `tests/security.test.ts` rewired to call the real helper (net 0). 1126 → 1130.
|
|
1291
|
+
|
|
1292
|
+
### Files changed
|
|
1293
|
+
|
|
1294
|
+
- `src/tools/search.ts` (new exported `filterExcludedEmbedHits`; `embeddingsSearch` routes both filter sites through it), `tests/security.test.ts` (theater test → real-helper call), `tests/search-hybrid.test.ts` (+2 + import), `tests/e2e-handlers.test.ts` (T-2 guard strengthened + T-3/T-4 guards added), `package.json` (`prepublishOnly` two-step audit + test-count 1130), test-count claims → 1130.
|
|
1295
|
+
- version bump 3.10.0-rc.21 → 3.10.0-rc.22.
|
|
1296
|
+
|
|
1297
|
+
---
|
|
1298
|
+
|
|
1299
|
+
## [3.10.0-rc.21] — 2026-06-06
|
|
1300
|
+
|
|
1301
|
+
> **TL;DR:** **Audit MED-batch 6 — M2/M10 docs integrity.** State-driven docs sweep. **M2 (verified, anti-overclaim):** the total tool-count is ALREADY pinned to `TOOL_MANIFEST.length` (45) across README / STABILITY / COMPARISON / api.md / llms.txt — the audit's "extend docs-consistency to pin them" was already done; the ONE unguarded surface was `ROADMAP.md`, whose "44 tool descriptions" had silently drifted while every guarded surface stayed at 45. Fixed the `44`→`45` + added a `docs-consistency` pin (+ NEGATIVE control) so ROADMAP can't drift again. **M10:** `CITATION.cff` was 2 stables behind (`3.8.8` → `3.9.1`, the current `@latest`); `ROADMAP.md` contradicted itself (the v3.10 forgetting-aware freshness feature listed both as *shipped* under "Already shipped" AND as *open* `[ ]` in Tier-3 — reconciled to `[x]` citing rc.5; the TDQS-pass item was `[ ]` but shipped in rc.7 — marked `[x]`); `server.json`'s subcommand hint got a "run with no subcommand for the full list" suffix (kept representative, NOT an enumerated 15-item list — that would add a drift surface). **api.md "stable v3.9.x" verified accurate (no change).** **1124 → 1126 tests.** M8/M9 test/process → rc.22.
|
|
1302
|
+
|
|
1303
|
+
**Pre-release (v3.10 line) — audit fix batch 6 (M2/M10 docs).**
|
|
1304
|
+
|
|
1305
|
+
### Fixed
|
|
1306
|
+
|
|
1307
|
+
- **M2 — ROADMAP.md tool-count drift + the unguarded surface.** Every *canonical* tool-count surface (README badge+hero+heading, STABILITY header, COMPARISON, api.md first paragraph, llms.txt breakdown) was already pinned to `TOOL_MANIFEST.length` by `docs-consistency.test.ts` — so they all correctly read **45**. `ROADMAP.md`'s "TDQS pass on all **44** tool descriptions" was the lone surface NOT in that invariant set, so it drifted. Fixed to 45 + added `checkRoadmapToolCount` (pure check + positive + NEGATIVE control) pinning ROADMAP's "N tool descriptions" to the manifest. (The audit's "extend docs-consistency to pin them" was already satisfied for the canonical surfaces — only ROADMAP needed closing; documented to avoid claiming a fix that already existed.)
|
|
1308
|
+
- **M10 — CITATION.cff stale.** `version: "3.8.8"` → `"3.9.1"` + `date-released` → `2026-06-01` (the current `@latest`; CITATION updates only on a stable promotion, per its own comment — it had missed the 3.9.0/3.9.1 promotions).
|
|
1309
|
+
- **M10 — ROADMAP self-contradiction.** (a) "Forgetting-aware freshness (v3.10)" is listed under **Already shipped** (rc.4 plumbing + rc.5 opt-in recency re-ranking) but Tier-3 still carried `[ ] "Forgetting-aware" note-staleness scoring` for the same user-facing capability → marked `[x]` (shipped v3.10-rc.5; post-fusion re-rank achieving the Memora-frontier goal) + a cross-reference so the intentional duplication is clear. (b) `[ ] TDQS pass on all 44 tool descriptions` shipped in rc.7 → `[x]` + 45. (Tier-2 items that are only *partially* shipped — rc.14 AI-search bundle, the Obsidian-MCP COMPARISON table — left `[ ]` to avoid overclaiming.)
|
|
1310
|
+
|
|
1311
|
+
### Docs
|
|
1312
|
+
|
|
1313
|
+
- **M10 — server.json subcommand hint.** Appended "— run with no subcommand for the full list" to the positional-arg description so the MCP-Registry hint doesn't undersell the CLI, WITHOUT enumerating all 15 subcommands (an enumerated list would be a fresh drift surface + risks the registry schema's description length; `setup` already subsumes the model/index subcommands).
|
|
1314
|
+
|
|
1315
|
+
### Tests (1126)
|
|
1316
|
+
|
|
1317
|
+
`tests/docs-consistency.test.ts` +2: `ROADMAP.md tool-count claim matches TOOL_MANIFEST` (the M2 pin) + `NEGATIVE: checkRoadmapToolCount flags drift / missing claim`. 1124 → 1126.
|
|
1318
|
+
|
|
1319
|
+
### Files changed
|
|
1320
|
+
|
|
1321
|
+
- `CITATION.cff` (3.8.8 → 3.9.1 + date), `ROADMAP.md` (TDQS + forgetting-aware items → `[x]`; 44 → 45; test total → 1126), `server.json` (subcommand hint suffix), `docs/COMPARISON.md` / `README.md` / `llms.txt` / `AGENTS.md` / `package.json` (test-count 1124 → 1126), `tests/docs-consistency.test.ts` (+2 + `checkRoadmapToolCount`).
|
|
1322
|
+
- version bump 3.10.0-rc.20 → 3.10.0-rc.21.
|
|
1323
|
+
|
|
1324
|
+
---
|
|
1325
|
+
|
|
1326
|
+
## [3.10.0-rc.20] — 2026-06-06
|
|
1327
|
+
|
|
1328
|
+
> **TL;DR:** **Audit MED-batch 5 — M7 privacy / right-to-erasure.** Three privacy-hardening fixes: (1) the HNSW persist **base** was computed independently by the WRITER (`server.ts`) and the ERASER (`EmbedDb.clearOnDisk`) — a duplication that, on drift, would leave the `.hnsw.bin` / `.hnsw.meta.json` sidecars (the meta sidecar carries raw `text_preview`) on disk after `clear-embeddings`, a right-to-erasure gap (the rc.34 P-2 class via a different seam); now both route through ONE shared `hnswPersistBase()` helper, and the erasure-completeness invariant asserts it. (2) The `--watch` handler `handle()` now re-checks `vault.isExcluded()` per file as **defense-in-depth** (chokidar's `ignored` predicate already drops excluded paths; this guards the case where `handle()` is reached another way — mirrors the existing PDF re-check). (3) `SECURITY.md` now documents that privacy filters are **not retroactive** for content already at rest — adding `--exclude-glob` hides matching notes from results immediately (terminal `isExcluded()` filter) but does NOT erase the chunk already written to `.fts5.db` / `.embed.db`; that needs `clear-index` / `clear-embeddings` + rebuild. **1119 → 1124 tests.** M2/M10 docs → rc.21.
|
|
1329
|
+
|
|
1330
|
+
**Pre-release (v3.10 line) — audit fix batch 5 (M7).**
|
|
1331
|
+
|
|
1332
|
+
### Fixed
|
|
1333
|
+
|
|
1334
|
+
- **M7.1 — shared HNSW persist-base helper (right-to-erasure anti-drift).** `server.ts` (writer) derived `persistFile` as `` `${embedFile.replace(/\.embed\.db$/, "")}.hnsw` `` while `EmbedDb.clearOnDisk` (eraser) recomputed the identical expression independently. If either changed, `clear-embeddings` would erase the wrong sidecar path and leave HNSW files (incl. `.hnsw.meta.json`'s raw `text_preview`) on disk. New exported `hnswPersistBase(embedDbFile)` is the single source of truth; both call sites route through it. The `erasure-completeness invariant` now (a) scans the helper for the `.hnsw` suffix (moved out of the eraser method body) and (b) asserts BOTH the eraser AND the writer call `hnswPersistBase` and that the writer no longer recomputes the base inline.
|
|
1335
|
+
- **M7.2 — watcher `handle()` privacy defense-in-depth.** The chokidar `ignored` predicate already drops `--exclude-glob` / `--read-paths` paths before they reach the handler, but `handle()` now ALSO re-checks `vault.isExcluded(relPath)` and returns before any cache-invalidation / index / embed work — so a filtered note can't be indexed even if `handle()` is reached by a direct call or a chokidar edge case. Mirrors the existing defensive PDF re-check.
|
|
1336
|
+
|
|
1337
|
+
### Docs
|
|
1338
|
+
|
|
1339
|
+
- **M7.3 — SECURITY.md: privacy filters are not retroactive for at-rest content.** Added a bullet to the `--read-paths` / `--exclude-glob` posture: a filter added *after* a note was indexed hides it from all tool results immediately (same `isExcluded()` predicate gates search/read/walker) but does NOT erase the copy already on disk (`.fts5.db`, `.embed.db` `text_preview`, `.hnsw.meta.json`); purge via `clear-index` / `clear-embeddings` / `clear-cache` then rebuild. Also updated the "Watcher-aware" bullet to note the new `handle()` re-check.
|
|
1340
|
+
|
|
1341
|
+
### Refactor
|
|
1342
|
+
|
|
1343
|
+
- `hnswPersistBase` lives in `src/embed-db.ts` (alongside `defaultEmbedDbFile`; it owns the `.embed.db` → `.hnsw` relation), imported by `server.ts`. No new import edge beyond server.ts's existing embed-db import; no cycle.
|
|
1344
|
+
|
|
1345
|
+
### Tests (1124)
|
|
1346
|
+
|
|
1347
|
+
`tests/erasure-invariant.test.ts` +3: `hnswPersistBase` behavioral derivation (3 cases) + structural "eraser & writer both route through the helper, writer doesn't inline the base" + NEGATIVE control (the inline-base detector flags the pre-rc.20 shape). The manifest loop now scans `helperFns` so the `.hnsw` suffix is still verified after moving into the helper. `tests/watcher.test.ts` +2: `handle()` skips an excluded path before `invalidateOne` (the M7.2 fix) + POSITIVE control (a non-excluded path DOES reach `invalidateOne`; both build abs paths from the realpath-canonical `v.root` so handle()'s `path.relative` guard doesn't mask the result). 1119 → 1124.
|
|
1348
|
+
|
|
1349
|
+
### Files changed
|
|
1350
|
+
|
|
1351
|
+
- `src/embed-db.ts` (new `hnswPersistBase`; `clearOnDisk` routes through it), `src/server.ts` (writer routes through it), `src/watcher.ts` (`handle()` isExcluded re-check), `SECURITY.md` (non-retroactive note + Watcher-aware update), `tests/erasure-invariant.test.ts` (+3 + `extractFn` + `helperFns` scan), `tests/watcher.test.ts` (+2), test-count claims → 1124.
|
|
1352
|
+
- `scripts/check-per-file-coverage.mjs` — refreshed the stale `http-transport.ts` inline coverage comment (72.85% → 77.61%; rc.19's M3 handler removal raised it; caught by OIA Check 6 against the fresh `coverage-summary.json`).
|
|
1353
|
+
- version bump 3.10.0-rc.19 → 3.10.0-rc.20.
|
|
1354
|
+
|
|
1355
|
+
---
|
|
1356
|
+
|
|
1357
|
+
## [3.10.0-rc.19] — 2026-06-06
|
|
1358
|
+
|
|
1359
|
+
> **TL;DR:** **Audit MED-batch 4 — M3 signal-shutdown race (both transports).** On `SIGINT`/`SIGTERM`, `serve-http` registered **four separate** listeners on the same signal — a cache-`flush` handler, `closeWatcher`, `closeFts`, and `shutdownHttpServer` — and the flush handler called `process.exit(0)` the **moment its fast cache flush resolved**, racing ahead of `shutdownHttpServer`'s up-to-5s in-flight-session drain and **cutting off in-flight requests**. The other three were pure duplication: `shutdownHttpServer` already flushes the cache and closes fts/watcher/embed-db. Fix: ONE `makeHttpShutdownHandler` orchestrator that **awaits** the full graceful teardown, then exits (re-entrancy-guarded). The stdio path (`startServer`) had the same shape (three handlers, the flush one calling `process.exit(0)` on its own — racing the async `watcher.close()`); consolidated into one `shutdownStdioDeps(deps)` that awaits watcher → embed-db → cache → fts before exit. `shutdownStdioDeps` was extracted to `src/shutdown.ts` so it's unit-testable (server.ts is in `no-internal-imports`' RESTRICTED_MODULES — same reason embed-pipeline.ts was split in rc.4). **1113 → 1119 tests.** M7 (privacy/erasure) → rc.20.
|
|
1360
|
+
|
|
1361
|
+
**Pre-release (v3.10 line) — audit fix batch 4 (M3).**
|
|
1362
|
+
|
|
1363
|
+
### Fixed
|
|
1364
|
+
|
|
1365
|
+
- **M3 (HTTP) — the cache-flush SIGINT/SIGTERM handler raced the session drain.** `startHttpServer` registered FOUR listeners on each of `SIGINT`/`SIGTERM`: a persistent-cache `flush` (calling `process.exit(0)` in its `.finally`), `closeWatcher`, `closeFts`, and `shutdown` (= `shutdownHttpServer`). Because the flush's `saveDiskCache` is fast and the registry drain (`closeAll`, up to 5s) is slow, `process.exit(0)` fired **before** in-flight stateful requests finished — exactly the leak `shutdownHttpServer` (v3.8.7 P2-11) was built to prevent. New `makeHttpShutdownHandler(server, exit?)` returns a single, re-entrancy-guarded handler that `await`s `shutdownHttpServer` (drain → close TCP listener → flush cache → close fts/watcher/embed-db) and only THEN exits. The three redundant handlers are removed (their work is wholly subsumed by `shutdownHttpServer`); `beforeExit` keeps a guarded best-effort teardown for the natural-drain path.
|
|
1366
|
+
- **M3 (stdio) — same shape.** `startServer` had three separate signal handlers and the cache-flush one called `process.exit(0)` on its own completion, racing the (async) `watcher.close()`. Consolidated into one orchestrator awaiting `shutdownStdioDeps(deps)` — which closes watcher + embed-db, flushes the persistent cache, then closes fts5, **awaiting each async step** (best-effort: a throw in one step never blocks the rest). The ordering is now deterministic and nothing exits mid-teardown.
|
|
1367
|
+
|
|
1368
|
+
### Refactor
|
|
1369
|
+
|
|
1370
|
+
- **`shutdownStdioDeps` extracted to `src/shutdown.ts`.** `src/server.ts` is in the `no-internal-imports` RESTRICTED_MODULES list ("registration boilerplate"), so a helper there can't be imported by a test — the SAME constraint that drove the rc.4 embed-pipeline extraction. The new module declares a minimal structural `StdioShutdownDeps` interface locally (no import of `ServerDeps`) so there's zero import cycle with the server module; `ServerDeps` structurally satisfies it, so `startServer` passes `deps` directly.
|
|
1371
|
+
|
|
1372
|
+
### Tests (1119)
|
|
1373
|
+
|
|
1374
|
+
`tests/http-transport.test.ts` +2: `makeHttpShutdownHandler` awaits full teardown before exit (asserts exit is NOT synchronous, the TCP listener is closed BEFORE exit fires, and a second signal is a re-entrancy no-op) + NEGATIVE control (a handler that skips the await "exits" while the listener is still up — proving the positive assertion depends on the await). `tests/shutdown.test.ts` (new) +4: ordering watcher→embed-db→cache→fts with awaited async steps; cache flush skipped when persistent cache disabled; best-effort (a throwing step doesn't block the rest); NEGATIVE control (a non-awaiting teardown records the sync "exit" step before the async one finishes). 1113 → 1119.
|
|
1375
|
+
|
|
1376
|
+
### Files changed
|
|
1377
|
+
|
|
1378
|
+
- `src/http-transport.ts` (new `makeHttpShutdownHandler`; four signal handlers → one orchestrator + guarded `beforeExit`), `src/server.ts` (import `shutdownStdioDeps`; three handlers → one orchestrator; drop now-unused `vault`/`ftsIndex`/`watcher` destructuring), `src/shutdown.ts` (new — `StdioShutdownDeps` + `shutdownStdioDeps`), `tests/http-transport.test.ts` (+2), `tests/shutdown.test.ts` (new, +4), test-count claims → 1119.
|
|
1379
|
+
- version bump 3.10.0-rc.18 → 3.10.0-rc.19.
|
|
1380
|
+
|
|
1381
|
+
---
|
|
1382
|
+
|
|
1383
|
+
## [3.10.0-rc.18] — 2026-06-06
|
|
1384
|
+
|
|
1385
|
+
> **TL;DR:** **Audit MED-batch 3 — M4 DoS-cap completeness (`obsidian_dataview_query` + invariant scope).** The audit flagged `obsidian_dataview_query` (`runDql`) as an uncapped whole-vault `readNote`+parse scan reachable over bearer `serve-http`. Root cause: the rc.36 `resource-bound-invariant`'s `SCANNER_SOURCES` covered `read.ts`/`search.ts`/`meta.ts` but NOT `dql.ts`, so `runDql` was never required to be CAP-or-EXEMPT (scope-too-narrow — the recurring class). Fix: cap `runDql` with `capScanEntries` (defense-in-depth — DQL is a *linear* query so a > MAX_SCAN_NOTES vault yields a partial, logged result, never a hang) + add `src/dql.ts` to `SCANNER_SOURCES` so the invariant patrols it. Also fixed a manifest drift my OWN rc.16 introduced: `getOpenQuestions` began calling `capScanEntries` in rc.16 but the manifest still listed it EXEMPT — reclassified CAPPED. **1113 tests unchanged.** M3 (signal-shutdown) → rc.19.
|
|
1386
|
+
|
|
1387
|
+
**Pre-release (v3.10 line) — audit fix batch 3 (M4).**
|
|
1388
|
+
|
|
1389
|
+
### Fixed
|
|
1390
|
+
|
|
1391
|
+
- **M4 — `obsidian_dataview_query` whole-vault scan was uncapped AND invisible to the resource-bound invariant.** `runDql` (`src/dql.ts`) does `vault.listMarkdown()` → per-note `readNote` + frontmatter-eval; it's always-registered + bearer-reachable, but lived OUTSIDE the invariant's `SCANNER_SOURCES`, so the rc.36 "every whole-vault scanner is CAP-or-EXEMPT" completeness check never saw it. Now: the scan is wrapped in `capScanEntries` (defense-in-depth — DQL is O(N) linear, so a > MAX_SCAN_NOTES vault yields a partial result with a logged warning, not a hang), and `src/dql.ts` is added to `SCANNER_SOURCES`. The audit's "like the uncapped graph tools" framing is imprecise (graph tools are O(N²)/graph and MUST cap; DQL is linear) — but a cap is the right defense-in-depth for a bearer-reachable whole-vault tool, and closing the invariant scope is the structural fix.
|
|
1392
|
+
- **rc.16 manifest drift — `getOpenQuestions` was CAPPED in code but EXEMPT in the manifest.** rc.16 (M5) added `capScanEntries` to `getOpenQuestions` but left its `resource-bound-invariant` classification EXEMPT. Reclassified CAPPED (with its `capScanEntries` token) so the manifest matches reality — a post-rc.16 recursion-sweep catch.
|
|
1393
|
+
|
|
1394
|
+
### Tests (1113)
|
|
1395
|
+
|
|
1396
|
+
No new `it()` — the `resource-bound-invariant` now structurally covers `runDql` (CAPPED → must reference `capScanEntries`) and the corrected `getOpenQuestions` classification; the existing `dql.test.ts` exercises the > MAX_SCAN_NOTES cap path (logs the truncation). 1113 unchanged.
|
|
1397
|
+
|
|
1398
|
+
### Files changed
|
|
1399
|
+
|
|
1400
|
+
- `src/dql.ts` (`capScanEntries` import + scan cap), `tests/resource-bound-invariant.test.ts` (`SCANNER_SOURCES` += `src/dql.ts`; `runDql` + `getOpenQuestions` → CAPPED; drop `getOpenQuestions` from EXEMPT).
|
|
1401
|
+
- version bump 3.10.0-rc.17 → 3.10.0-rc.18.
|
|
1402
|
+
|
|
1403
|
+
---
|
|
1404
|
+
|
|
1405
|
+
## [3.10.0-rc.17] — 2026-06-06
|
|
1406
|
+
|
|
1407
|
+
> **TL;DR:** **Audit MED-batch 2/6 — M1 chunking parity (embeddings line citations).** The embedding pipeline chunks the frontmatter-STRIPPED body (to keep YAML out of the vectors) while the FTS5 index chunks the FULL note content — so for any note WITH frontmatter, embeddings / `find_similar` / `semantic_search` stored `line_start`/`line_end` that were BODY-relative (too low by the frontmatter line count), pointing deep-links at the wrong line; and the code comments falsely claimed "identical chunking across BM25 and embeddings." Fix: `parseNote` now exposes `bodyStartLine`, and `embedSingleNote` shifts each chunk's line numbers to FILE-absolute (matching FTS5) — keeping the clean body-only embeddings (no quality regression). Comments corrected; the residual `block`-granularity chunk-INDEX divergence for frontmatter'd notes is documented (the default `note` granularity fuses by path, unaffected). **1108 → 1113 tests.**
|
|
1408
|
+
|
|
1409
|
+
**Pre-release (v3.10 line) — audit fix batch 2/6.**
|
|
1410
|
+
|
|
1411
|
+
### Fixed
|
|
1412
|
+
|
|
1413
|
+
- **M1 — embeddings line citations were body-relative (off by the frontmatter line count) for frontmatter'd notes.** `embedSingleNote` chunks `note.parsed.body` (frontmatter stripped, so YAML never pollutes the vectors), but `chunkContent`'s `lineStart`/`lineEnd` are then relative to the body, not the file — so a hit's deep-link pointed N lines too early (N = the frontmatter line count). `parseNote` now returns `bodyStartLine` (the 1-based file line where the body begins, via `source.indexOf(body)`; 1 with no frontmatter), and `embedSingleNote` adds `(bodyStartLine − 1)` to each chunk's line numbers → FILE-absolute, matching the FTS5 index (which chunks full content). Embedding quality is unchanged (still body-only). Existing embed-dbs apply the corrected lines as notes are re-embedded (on edit, or `enquire-mcp build-embeddings`) — no forced rebuild (proportionate: a full re-embed on every serve for a line-number fix would be disruptive).
|
|
1414
|
+
- **M1 (claim-vs-reality) — `embed-db.ts` header claimed "Same chunking as FTS5 … so chunk identity matches across BM25 and embeddings."** False for markdown (FTS5 chunks full content; embeddings chunk body). Corrected to describe the actual design + the file-absolute line alignment + the `block`-granularity caveat. The `reindexPdfFile` "chunk IDs match" claim is accurate (PDFs have no frontmatter) and left as-is.
|
|
1415
|
+
|
|
1416
|
+
### Docs
|
|
1417
|
+
|
|
1418
|
+
- `searchHybrid` granularity `@param` now notes that in `block` granularity a per-note chunk INDEX may not denote the same span across BM25 (content) and embeddings (body) for frontmatter'd notes — prefer the default `note` granularity (fused by path) for frontmatter-heavy vaults.
|
|
1419
|
+
|
|
1420
|
+
### Tests (1113)
|
|
1421
|
+
|
|
1422
|
+
`tests/embed-pipeline.test.ts` +2 (frontmatter'd note → chunk `lineStart` lands on the file line that actually contains the chunk text, not the `---`; NEGATIVE control: no-frontmatter ⇒ offset 0, line 1). `tests/parser.test.ts` +3 (`bodyStartLine` > 1 with frontmatter; === 1 without — NEGATIVE control; points at the first body line). 1108 → 1113.
|
|
1423
|
+
|
|
1424
|
+
### Files changed
|
|
1425
|
+
|
|
1426
|
+
- `src/parser.ts` (`bodyStartLine` field + computation), `src/embed-pipeline.ts` (file-absolute line offset in `embedSingleNote`), `src/embed-db.ts` (header comment), `src/tools/search.ts` (granularity `@param` caveat), `tests/embed-pipeline.test.ts` (+2), `tests/parser.test.ts` (+3), test-count claims → 1113.
|
|
1427
|
+
- version bump 3.10.0-rc.16 → 3.10.0-rc.17.
|
|
1428
|
+
|
|
1429
|
+
---
|
|
1430
|
+
|
|
1431
|
+
## [3.10.0-rc.16] — 2026-06-05
|
|
1432
|
+
|
|
1433
|
+
> **TL;DR:** **Audit MED-batch 1/6 — retrieval correctness.** A from-scratch 7-agent system audit (core code · transport/CLI · security/STRIDE · privacy · agent-facing surfaces · docs/process · tests), every headline adversarially re-verified against the code, returned **0 CRITICAL / 0 HIGH** for this 30+-round-audited codebase — real findings sat in the apparatus's known behavioral/docs blind spots. This RC fixes the first two verified MEDIUMs. **M5:** `obsidian_open_questions` is documented "oldest-first" but broke at `limit` in vault-WALK order and only THEN sorted — so on a vault with > `limit` questions it returned an arbitrary subset, NOT the oldest. Now collects all (scan capped via `capScanEntries`), sorts oldest-first, slices. **M6:** `HnswIndex.applyDiff` validated vector dim INSIDE the addPoint loop, so a wrong-dim vector threw AFTER some labels were `markDelete`'d and some points added → a half-applied index (silent embed-db↔HNSW divergence in the watcher, which logs + continues rather than rebuilding). Dim is now pre-validated before ANY mutation → atomic for the only caller-data-driven throw. **1104 → 1108 tests.**
|
|
1434
|
+
|
|
1435
|
+
**Pre-release (v3.10 line) — audit fix batch 1/6.**
|
|
1436
|
+
|
|
1437
|
+
### Fixed
|
|
1438
|
+
|
|
1439
|
+
- **M5 — `obsidian_open_questions` returned an arbitrary `limit`-subset, not the oldest** (`src/tools/meta.ts`). The outer loop `break`'d once `out.length >= limit` in `vault.listMarkdown` (readdir) order, then `out.sort(age desc)` ran on that already-truncated set. On a vault with more than `limit` questions, callers asking for "the most-aged open questions" got whichever notes came first in the walk. Now: cap the scan (`capScanEntries`, defense-in-depth like the graph tools), collect ALL matches, sort oldest-first, then `slice(0, limit)` — the documented contract.
|
|
1440
|
+
- **M6 — `HnswIndex.applyDiff` could leave a half-applied index on a dim mismatch** (`src/hnsw.ts`). The `pt.vector.length !== dim` check lived inside the addPoint loop, so a bad vector threw after the `markDelete` loop + earlier `addPoint`s had already mutated the index; the watcher's `syncHnswForFile` catch logs + continues (doesn't rebuild), leaving HNSW out of sync with the freshly-upserted embed-db until the next serve restart (ghost labels / stale `text_preview`). Now ALL dims are pre-validated before any `markDelete`/`resizeIndex`/`addPoint` → applyDiff is atomic for the only caller-data-driven throw (a native addPoint failure after the pre-grow remains the sole, rare, eventually-consistent residual, documented).
|
|
1441
|
+
|
|
1442
|
+
### Security (dependency)
|
|
1443
|
+
|
|
1444
|
+
- **`hono` moderate advisory (transitive, not reachable) — pinned to the patched 4.12.23 via `overrides`.** A fresh `npm audit` moderate advisory hit `hono ≤4.12.20` (GHSA-xrhx-7g5j-rcj5 IPv6-deny bypass + GHSA-3hrh-pfw6-9m5x cookie injection + GHSA-f577-qrjj-4474 JWT scheme + GHSA-2gcr-mfcq-wcc3 mount-prefix), pulled in TRANSITIVELY by `@modelcontextprotocol/sdk@1.29.0` (via `@hono/node-server`). enquire runs its OWN node-`http` transport, not hono, so none of the flagged paths are reachable — but the `audit` CI gate flags the dep tree regardless. Added `"overrides": { "hono": "^4.12.21" }` → resolves to 4.12.23 (in-range for the SDK's `^4.11.4`, non-breaking; build + SDK transport tests green). `npm audit` back to **0 vulnerabilities** (prod-moderate + all-high). Caught by CI's `audit` gate, not my local battery — lesson: run `npm audit` locally too (the battery omitted it).
|
|
1445
|
+
|
|
1446
|
+
### Tests (1108)
|
|
1447
|
+
|
|
1448
|
+
`tests/redos-guard.test.ts` +3 (open-questions oldest-first: limit=1 returns the oldest not the walk-first/newest; limit=2 returns the 2 oldest; full-order regression — names+mtimes chosen so the oldest is never readdir-first, making the revert discriminating). `tests/hnsw.test.ts` +1 (applyDiff wrong-dim throws atomically — the removeLabel survives the failed diff; NEGATIVE control: a valid diff DOES remove it). 1104 → 1108 (the hono override adds no tests).
|
|
1449
|
+
|
|
1450
|
+
### Files changed
|
|
1451
|
+
|
|
1452
|
+
- `src/tools/meta.ts` (M5 + `capScanEntries` import), `src/hnsw.ts` (M6 dim pre-validation), `tests/redos-guard.test.ts` (+3), `tests/hnsw.test.ts` (+1), test-count claims → 1108.
|
|
1453
|
+
- `package.json` + `package-lock.json` (hono `overrides` → 4.12.23, security).
|
|
1454
|
+
- version bump 3.10.0-rc.15 → 3.10.0-rc.16.
|
|
1455
|
+
|
|
1456
|
+
---
|
|
1457
|
+
|
|
1458
|
+
## [3.10.0-rc.15] — 2026-06-03
|
|
1459
|
+
|
|
1460
|
+
> **TL;DR:** **Watcher-test flake stabilized at the root — it was blocking RELEASES, not just PRs.** The rc.13 release run failed at `tests/watcher.test.ts:505` (chokidar FSEvents timing); a re-run published it — but a transient blip must never fail a publish (rc.20 rule). Two root races: **(1)** a brand-new file's FIRST inotify/FSEvents event can be dropped on a loaded runner even after `start()` resolves on `ready`, so a one-shot `writeFile` + `waitFor` can wait forever — the fixed-`setTimeout` warm-ups (rc.7 #36, rc.9 W-FLAKE-2) only approximated a fix; **(2)** the `:505` assertion read the embed-error stderr line IMMEDIATELY after the `fts.totalFiles()` check, but that line is logged a tick LATER in the same handler. Fixed: new `writeAndWaitFor` re-touch-on-miss helper (re-writes the file if the first event is dropped — idempotent reindex) on all 5 new-file-add sites; the lagging embed-error signal is now polled with `waitFor`; `waitFor` default 4000 → 8000 ms. **Verified green 3× back-to-back.** **1104 tests unchanged** (bodies refactored, no `it()` added). **Test-only — zero `src/`.**
|
|
1461
|
+
|
|
1462
|
+
**Pre-release (v3.10 line) — release-reliability fix; no `src/` / behavior change.**
|
|
1463
|
+
|
|
1464
|
+
### Fixed
|
|
1465
|
+
|
|
1466
|
+
- **Watcher test flake blocked the rc.13 release** (`tests/watcher.test.ts:505`, "expected false to be true"). Two root races, both fixed:
|
|
1467
|
+
- **Missed first event.** chokidar can drop the FIRST add event for a brand-new path on a loaded runner (the watch is still arming when the write lands, even though `start()` already resolved on `ready`). New `writeAndWaitFor(filePath, content, cond)` re-writes the file every ~1.2 s while waiting, regenerating a fresh event the watcher reliably catches. Idempotent reindex ⇒ extra writes never change the asserted outcome. Applied to all 5 new-file-add sites (`logged.md`, `added.md`, `added.pdf`, `note-embed.md`, `embed-error.md`).
|
|
1468
|
+
- **Asserting a lagging signal too early.** The `:505` check read the "embed-db sync failed" stderr line right after `fts.totalFiles()>=1`, but that line is logged a tick later in the same handler. Now polled with `waitFor`.
|
|
1469
|
+
- **`waitFor` default 4000 → 8000 ms** — headroom for the full chain (event → `awaitWriteFinish` 250 ms → per-file queue → reindex → embed-sync) under coverage instrumentation + parallel workers, without masking a genuine hang.
|
|
1470
|
+
|
|
1471
|
+
### Method note
|
|
1472
|
+
|
|
1473
|
+
The durable fix the project's fixed-`setTimeout` warm-ups (rc.7 #36, rc.9 W-FLAKE-2) chased for three RCs: the root isn't "wait longer before writing" — it's "regenerate the event if it's dropped" + "poll lagging signals, don't assert them immediately". Per "a transient blip must never fail a release" (rc.20), but fixed at the ROOT (a fixable test race) rather than masked with a retry (the rc.20 npm-ci retry was for an *unfixable* network flake). Verified empirically — watcher suite green 3× back-to-back — per the rc.25 "run it, don't assume" lesson. Deferred (noted): a structural lint flagging `writeFile`-then-`waitFor` in watcher tests was considered and rejected as noisy; the helper is the durable mechanism and the remaining sites are change/unlink (reliable on already-watched files).
|
|
1474
|
+
|
|
1475
|
+
### Tests (1104)
|
|
1476
|
+
|
|
1477
|
+
`tests/watcher.test.ts` — 5 add-sites → `writeAndWaitFor`, embed-error downstream signal → `waitFor`, default timeout 4000→8000. No `it()` added/removed; 1104 unchanged.
|
|
1478
|
+
|
|
1479
|
+
### Files changed
|
|
1480
|
+
|
|
1481
|
+
- `tests/watcher.test.ts` only (new `writeAndWaitFor` helper + 6 site refactors + `waitFor` timeout bump).
|
|
1482
|
+
- version bump 3.10.0-rc.14 → 3.10.0-rc.15.
|
|
1483
|
+
|
|
1484
|
+
---
|
|
1485
|
+
|
|
1486
|
+
## [3.10.0-rc.14] — 2026-06-03
|
|
1487
|
+
|
|
1488
|
+
> **TL;DR:** **`query` + `prune` CLI (bug-report Issues 4, 8) — concludes the actionable bug-report batch.** **(Issue 4)** New `enquire-mcp query "<text>" --vault <path>` runs the SAME hybrid `searchHybrid` the MCP `obsidian_search` tool uses and prints the results — a one-shot CLI search for smoke-tests / CI / debugging without an MCP client (the report had to hand-craft JSON-RPC over stdio to verify retrieval). **(Issue 8)** New `enquire-mcp prune --vault <path>` GCs the per-vault index clutter that accumulates in the cache dir (`clear-cache`/`clear-index` only target the current vault); it removes all OTHER vaults' enquire artifacts, **dry-run by default** (opt in with `--yes`), and — via the pure `planCachePrune` with a strict `<12-hex>.{fts5.db,embed.db,hnsw.bin,hnsw.meta.json}` filter — can NEVER touch a file enquire didn't create. **1098 → 1104 tests.**
|
|
1489
|
+
|
|
1490
|
+
**Minor (pre-release) — v3.10 line; bug-report response batch 3/3 (DX CLI). Concludes the actionable bug-report batch (rc.12 model-path · rc.13 reranker · rc.14 query+prune).**
|
|
1491
|
+
|
|
1492
|
+
### Added
|
|
1493
|
+
|
|
1494
|
+
- **`query` subcommand (Issue 4).** `enquire-mcp query "<text>" --vault <path> [--limit N] [--index-file …] [--json]` — builds/reuses the persistent FTS5 index (peek-safe tokenize, K-1 invariant), runs `searchHybrid`, prints `path:line [kind]` + snippet per hit (or the full JSON with `--json`). Unblocks CLI/CI smoke-testing of retrieval without an MCP client.
|
|
1495
|
+
- **`prune` subcommand (Issue 8).** `enquire-mcp prune --vault <path> [--yes]` — removes cached index artifacts for OTHER vaults, keeping the named one. Dry-run preview by default; `--yes` deletes. Backed by the pure, exported `planCachePrune(entries, keepHash)` whose strict enquire-artifact regex is the safety property (verified: ignores user notes / wrong-shaped hashes / wrong extensions).
|
|
1496
|
+
|
|
1497
|
+
### Fixed
|
|
1498
|
+
|
|
1499
|
+
- **Issue 4 — no CLI search for smoke/CI.** Previously retrieval could only be exercised through the MCP protocol (stdio JSON-RPC). `query` gives a direct CLI path.
|
|
1500
|
+
- **Issue 8 — no GC for accumulated per-vault indexes.** The cache dir grew one index set per vault path/config hash with no cleanup command for OTHER vaults. `prune` is that command. (The root cause for the maintainer's own clutter was the test suite — fixed in rc.11; `prune` is the user-facing GC.)
|
|
1501
|
+
|
|
1502
|
+
### Tests (1104)
|
|
1503
|
+
|
|
1504
|
+
`tests/cache-prune.test.ts` +4 (planCachePrune: selects other vaults, never the kept one, **NEGATIVE control** ignores non-enquire files, empty cases); `tests/cli.test.ts` +2 (`query` prints results, `prune` previews + deletes nothing without `--yes`). 1098 → 1104; claims synced (README ×4, package.json, llms.txt, AGENTS, COMPARISON, ROADMAP).
|
|
1505
|
+
|
|
1506
|
+
### Files changed
|
|
1507
|
+
|
|
1508
|
+
- `src/fts5.ts` (`planCachePrune` + `ENQUIRE_CACHE_ARTIFACT` regex), `src/cli.ts` (`query` + `prune` commands + `searchHybrid`/`planCachePrune` imports), `docs/api.md` (2 subcommand rows), `tests/cache-prune.test.ts` (new, +4), `tests/cli.test.ts` (+2), test-count claims → 1104.
|
|
1509
|
+
- version bump 3.10.0-rc.13 → 3.10.0-rc.14.
|
|
1510
|
+
|
|
1511
|
+
### Known / next
|
|
1512
|
+
|
|
1513
|
+
- **A flaky watcher test (`tests/watcher.test.ts:505`, chokidar FSEvents timing) failed the rc.13 release run** (a re-run published it). Same class as rc.7 #36 / rc.9 W-FLAKE-2, but now blocking *releases*, not just PRs — more severe. **rc.15 will stabilize it** (wait on the watcher's `ready` signal instead of a fixed `setTimeout` warmup) per the "a transient blip must never fail a release" rule (rc.20).
|
|
1514
|
+
|
|
1515
|
+
---
|
|
1516
|
+
|
|
1517
|
+
## [3.10.0-rc.13] — 2026-06-03
|
|
1518
|
+
|
|
1519
|
+
> **TL;DR:** **Reranker observability + pre-cache (bug-report Issues 9, 3, 5).** The cross-encoder reranker was a black box: enabling it triggered a SILENT ~110 MB download on the first query (which could exceed a client's tool-call timeout → unexplained RRF fallback), there was no way to pre-cache it, and the response gave no positive signal it ran. Now: **(Issue 9)** `obsidian_search` emits stderr lifecycle logs (`reranker '<alias>' loading (~110 MB…)` BEFORE the blocking load, `loaded; reranked N pairs` after — failures were already logged) and returns a `reranked: { applied, pairs?, reason? }` field; **(Issue 3)** `install-model` resolves the reranker catalog too, so `enquire-mcp install-model rerank-bge` pre-downloads the cross-encoder (the ~110 MB no longer blocks the first query); **(Issue 5, docs)** `docs/api.md` now states the default reranker is English-tuned and RU/multilingual vaults can leave it off (RRF hybrid already handles them), plus which aliases are verified-working. **1094 → 1098 tests.**
|
|
1520
|
+
|
|
1521
|
+
**Minor (pre-release) — v3.10 line; bug-report response batch 2/3 (reranker cluster).**
|
|
1522
|
+
|
|
1523
|
+
### Added
|
|
1524
|
+
|
|
1525
|
+
- **`SearchHybridResponse.reranked`** — `{ applied: boolean; pairs?: number; reason?: string }`, present ONLY when a reranker was requested. `{applied:true, pairs:N}` after a successful re-score; `{applied:false, reason}` when requested but it didn't run (reason mirrors `signal_errors.reranker` on a load failure, or notes "no candidates"). Closes Issue 9's "silent no-op" — a caller can now tell applied-vs-fell-back without guessing.
|
|
1526
|
+
- **`install-model` accepts reranker aliases** (Issue 3). `alias in RERANKER_MODELS` routes to `loadReranker` + a one-pair smoke; `enquire-mcp install-model rerank-bge` pre-caches the ~110 MB cross-encoder so `serve --enable-reranker` doesn't block on the download at first query. Unknown aliases now fail with BOTH catalogs listed (resolves the `bge` embedding vs `rerank-bge` cross-encoder naming confusion).
|
|
1527
|
+
|
|
1528
|
+
### Fixed
|
|
1529
|
+
|
|
1530
|
+
- **Issue 9 — reranker silently no-op'd with no diagnostics (Medium).** `loadReranker`'s download was silent and `search.ts` logged ONLY failures, so a first-run download that exceeded the client timeout looked identical to a silent failure. Added stderr lifecycle logging (loading… with size / loaded; reranked N pairs) — three distinguishable states — plus the `reranked` response field above. (`signal_errors.reranker` already carried the failure reason; this adds the in-progress + success signals.)
|
|
1531
|
+
|
|
1532
|
+
### Docs
|
|
1533
|
+
|
|
1534
|
+
- **Issue 5 — reranker language guidance.** `docs/api.md` `--enable-reranker` / `--reranker-model` rows now state the default cross-encoder is English-tuned (RRF hybrid handles RU/multilingual well → leave it off with no quality loss), name `rerank-bge` as the only verified-working reranker (the multilingual aliases still fail at `AutoTokenizer`), and point at `install-model` for pre-caching. The empirical RU conclusion from the bug report (RRF-hybrid already correct without reranker) is now documented guidance.
|
|
1535
|
+
|
|
1536
|
+
### Tests (1098)
|
|
1537
|
+
|
|
1538
|
+
`tests/reranker.test.ts` +3 (reranked applied / failed-with-reason / NEGATIVE-control absent-when-not-requested); `tests/cli.test.ts` +1 (install-model unknown-alias lists both catalogs). 1094 → 1098; claims synced (README ×4, package.json, llms.txt, AGENTS, COMPARISON, ROADMAP).
|
|
1539
|
+
|
|
1540
|
+
### Files changed
|
|
1541
|
+
|
|
1542
|
+
- `src/tools/search.ts` (`reranked` field + stderr lifecycle logs), `src/cli.ts` (install-model reranker routing + combined-catalog error + imports), `docs/api.md` (reranker guidance), `tests/reranker.test.ts` (+3), `tests/cli.test.ts` (+1), test-count claims → 1098.
|
|
1543
|
+
- version bump 3.10.0-rc.12 → 3.10.0-rc.13.
|
|
1544
|
+
|
|
1545
|
+
---
|
|
1546
|
+
|
|
1547
|
+
## [3.10.0-rc.12] — 2026-06-03
|
|
1548
|
+
|
|
1549
|
+
> **TL;DR:** **Model-cache path: one resolver, no more lying paths (bug-report Issues 1 + 2).** A fresh-install bug report on a real 236-note vault found `enquire-mcp doctor` printing **"NOT READY — no Xenova model weights found"** on a fully-working **global** install: the cache probe only looked under `process.cwd()/node_modules/…`, but a global `npm i -g` loads the model from the package's OWN nested `node_modules`, resolved relative to the module — never relative to cwd (Issue 1). Relatedly, `install-model` / `setup` printed `cached under ~/.cache/huggingface/` — a path that stays empty — while the help text and two TSDocs named other (also wrong) paths (Issue 2). Both now flow through ONE source of truth, `resolveTransformersCacheDir()` (resolves `@huggingface/transformers` via `createRequire(import.meta.url)` → its package `.cache`, correct for hoisted AND nested layouts), so the diagnostic and the success message can never again disagree with reality. **Verified:** on this machine `doctor` now reports `✓ Embedding model cache — 2 model(s) cached under …/node_modules/@huggingface/transformers/.cache/Xenova/`. **1088 → 1094 tests.**
|
|
1550
|
+
|
|
1551
|
+
**Minor (pre-release) — v3.10 line; bug-report response batch 1/3 (model-path correctness).**
|
|
1552
|
+
|
|
1553
|
+
### Fixed
|
|
1554
|
+
|
|
1555
|
+
- **Issue 1 — `doctor` false-negative on a global install (Medium).** `candidateModelCacheRoots()` (`src/doctor.ts`) probed only a cwd-based path; the model on a global install lives in the package's nested `node_modules/@huggingface/transformers/.cache`, which cwd never reaches → false `NOT READY` on a fully-working setup (panic / needless reinstalls). Now the FIRST candidate is `resolveTransformersCacheDir()` — the module-relative path transformers.js actually loads from — with the cwd probe kept as a local-dev/npx fallback and the HF-Hub conventions kept after it.
|
|
1556
|
+
- **Issue 2 — `install-model` / `setup` printed a wrong, empty cache path (Low).** The success message (`cached under ~/.cache/huggingface/`), the `install-model` description (`~/.cache/huggingface/transformers.js/`), and two `src/embeddings.ts` TSDocs each named a different wrong path. All now print / point to the resolved truth (`resolveTransformersCacheDir()`); the `docs/api.md` `install-model` row was corrected. Root-cause sweep fixed **5 instances** of the wrong-path claim (install-model msg, setup msg, 2 TSDocs, api.md), not just the reported one.
|
|
1557
|
+
|
|
1558
|
+
### Added
|
|
1559
|
+
|
|
1560
|
+
- **`resolveTransformersCacheDir()` + `deriveTransformersCacheDir()`** (`src/embeddings.ts`, exported): single resolver for the transformers.js model-cache dir. Pure derivation slices at the INNERMOST `node_modules/@huggingface/transformers` segment so it's correct for hoisted (`<root>/node_modules/…`) AND nested global-install (`…/enquire-mcp/node_modules/@huggingface/transformers/.cache`) layouts. Resolution-only (no ONNX load) → keeps `doctor`'s fast-read-only promise.
|
|
1561
|
+
- **`tests/transformers-cache-path.test.ts`** (+6): pins the pure derivation incl. the nested global-install layout (the exact Issue-1 shape), with a discriminating NEGATIVE control (no marker → `null`); asserts the live resolver returns the package `.cache` and that `doctor` ranks it first.
|
|
1562
|
+
|
|
1563
|
+
### Method note
|
|
1564
|
+
|
|
1565
|
+
Classic change-driven-vs-state-driven gap: my home gates never run a global install, so the cwd-only probe survived every internal sweep. A real fresh install found it immediately. Fix is the standard transform — collapse N drifting path strings into ONE resolver + a structural test on the derivation, so the diagnostic and the messages are provably consistent. The α-class (TSDoc-drift) rule applied: the wrong-path claims in the `embeddings.ts` TSDocs + the module header comment were corrected in the same commit as the code.
|
|
1566
|
+
|
|
1567
|
+
### Tests (1094)
|
|
1568
|
+
|
|
1569
|
+
`tests/transformers-cache-path.test.ts` +6 source `it()`. 1088 → 1094; claims synced (README ×4, package.json, llms.txt, AGENTS, COMPARISON, ROADMAP).
|
|
1570
|
+
|
|
1571
|
+
### Files changed
|
|
1572
|
+
|
|
1573
|
+
- `src/embeddings.ts` (resolver + helpers + 2 TSDoc path fixes + header comment), `src/doctor.ts` (candidate #0 = resolved cache + stale-comment fix), `src/cli.ts` (install-model + setup messages → resolved path), `docs/api.md` (install-model row), `tests/transformers-cache-path.test.ts` (new), test-count claims → 1094.
|
|
1574
|
+
- version bump 3.10.0-rc.11 → 3.10.0-rc.12.
|
|
1575
|
+
|
|
1576
|
+
---
|
|
1577
|
+
|
|
1578
|
+
## [3.10.0-rc.11] — 2026-06-03
|
|
1579
|
+
|
|
1580
|
+
> **TL;DR:** **Hermetic test cache — found by live-testing the installed server on a real machine.** Driving the installed `enquire-mcp@3.9.1` over JSON-RPC against a real 237-note vault confirmed it works end-to-end (boot, 33 tools, `obsidian_stats`, semantic `obsidian_search` with a built embed-db, path-escape guard) — but also surfaced **~27,000 orphaned files / ~699 MB** sitting in the real user cache (`~/Library/Caches/enquire/`). Root cause: `defaultIndexFile()` (and the embed-db / HNSW sidecars) resolve their dir from `XDG_CACHE_HOME`, and any test that spawns `serve`/`setup`/`build-embeddings`/`index` **without** an explicit `--index-file` fell back to that REAL cache and never cleaned up — weeks of `npm test` accumulated there (mtimes May 8 → Jun 3). Fixed at the root: `tests/setup.ts` now redirects `XDG_CACHE_HOME` to a throwaway temp dir before any test (and every inheriting child spawn) touches the cache. **Verified: real-cache file delta from the suite is now 0.** Structural guard added so it can't regress. **1085 → 1088 tests.**
|
|
1581
|
+
|
|
1582
|
+
**Minor (pre-release) — v3.10 line; test-hygiene fix from live-testing (no `src/` runtime change).**
|
|
1583
|
+
|
|
1584
|
+
### Fixed
|
|
1585
|
+
|
|
1586
|
+
- **Test suite no longer pollutes the real user cache.** `tests/setup.ts` sets `process.env.XDG_CACHE_HOME = mkdtempSync(tmpdir()/enquire-test-cache-…)` (guarded by `if (!XDG_CACHE_HOME)` so CI/devs can override). Because `defaultIndexFile()` keys on `XDG_CACHE_HOME` on every platform and every test child-spawn inherits `process.env` (verified: no test overrides `env` without spreading `process.env`), this redirects ALL fts5 / embed-db / HNSW cache writes — in-process and spawned — to a throwaway dir the OS reclaims. Previously these landed in `~/Library/Caches/enquire/` (macOS) / `~/.cache/enquire/` (Linux) and were never cleaned, so a dev machine that ran the suite over weeks accumulated tens of thousands of orphaned index files. **Verified end-to-end:** real-cache file count is unchanged (Δ0) after a full `npm test`; the only real-cache writes observed during testing came from a separate *real* `serve` run on the actual vault (correct behavior), confirmed by matching the file hash to `sha1("/…/Obsidian Vault")`.
|
|
1587
|
+
|
|
1588
|
+
### Added
|
|
1589
|
+
|
|
1590
|
+
- **`tests/cache-isolation-invariant.test.ts`** (+3): asserts `XDG_CACHE_HOME` is a temp dir during tests and that `defaultIndexFile()` resolves UNDER it, NOT under the real `~/Library/Caches/enquire` (or `~/.cache/enquire`) — so removing the `setup.ts` redirect fails CI. Includes a NEGATIVE control proving the real-cache classifier discriminates (a constant `() => false` would make it vacuous). Meta-invariant-enrolled.
|
|
1591
|
+
|
|
1592
|
+
### Method note
|
|
1593
|
+
|
|
1594
|
+
This is the textbook value of **testing on a real machine** (the project's home-grown gates are drift/claim-driven and structurally blind to filesystem-side-effects like this): the unit suite *caused* the accumulation but never *observed* it, because no gate inspects the real cache dir. The fix is the project's standard transform — root-cause + a permanent inventory/structural invariant (`cache-isolation-invariant`) so the class can't silently return. Severity is **dev-hygiene** (not a product/CI/end-user correctness bug — a real user with one vault gets one index file; CI runners are ephemeral), but it's a real papercut (699 MB on the maintainer's machine) with a clean, verified fix. **The pre-existing local cruft is maintainer-gated to clear** (deleting files is out of scope for the agent) — see the session hand-off for the exact one-liner that keeps the live vault's index and removes the orphans.
|
|
1595
|
+
|
|
1596
|
+
### Tests (1088)
|
|
1597
|
+
|
|
1598
|
+
`tests/cache-isolation-invariant.test.ts` +3 source `it()`. 1085 → 1088; claims synced (README ×4, package.json, llms.txt, AGENTS, COMPARISON, ROADMAP).
|
|
1599
|
+
|
|
1600
|
+
### Files changed
|
|
1601
|
+
|
|
1602
|
+
- `tests/setup.ts` (XDG_CACHE_HOME redirect), `tests/cache-isolation-invariant.test.ts` (new), test-count claims → 1088.
|
|
1603
|
+
- version bump 3.10.0-rc.10 → 3.10.0-rc.11.
|
|
1604
|
+
|
|
1605
|
+
---
|
|
1606
|
+
|
|
1607
|
+
## [3.10.0-rc.10] — 2026-06-02
|
|
1608
|
+
|
|
1609
|
+
> **TL;DR:** **New capability — frontmatter-aware retrieval.** `obsidian_search` gains an optional `filter_frontmatter` map so an agent can scope hybrid search by YAML frontmatter: `{ status: "active", type: ["meeting","decision"] }` → only notes whose frontmatter matches **every** key (strings case-insensitive; an array frontmatter value matches by membership; an array filter value is OR). It's the first genuinely-new *feature* (not polish) since the v3.10 staleness line — Obsidian users live in frontmatter (`status`/`type`/`project`), and **no other Obsidian-MCP can scope semantic search by it**. Opt-in + additive: **absent ⇒ byte-identical** to before (same safe pattern as recency re-ranking). Matching is filter-on-the-fused-candidate-pool, which is already excluded-pruned (rc.8), so no excluded note's frontmatter is read; PDFs (no frontmatter) are excluded without a binary read. **1076 → 1085 tests.**
|
|
1610
|
+
|
|
1611
|
+
**Minor (pre-release) — v3.10 line; new feature: frontmatter-aware retrieval (increment 1/N).**
|
|
1612
|
+
|
|
1613
|
+
### Added
|
|
1614
|
+
|
|
1615
|
+
- **`obsidian_search` `filter_frontmatter?: Record<string, scalar | scalar[]>`** — post-filters fused hits by the note's parsed YAML frontmatter. AND across keys; per key, scalar-equality (strings case-insensitive, numbers/booleans strict, no cross-type coercion) or array-membership; a filter value may be an array for OR. Notes with no frontmatter, or missing a filtered key, are excluded (a filter is a positive assertion). Runs only when the param is passed; filters the candidate pool (so a strict filter can legitimately return < `limit`). Reads candidate frontmatter via the cached `vault.readNote` (graph-boost usually warms it); fail-soft (an unreadable candidate is excluded, honoring the filter).
|
|
1616
|
+
- **Pure exported `frontmatterMatches(frontmatter, filter)`** (+ `FrontmatterFilterValue` / `FrontmatterFilterScalar` types) — the matching semantics, unit-testable in isolation. zod schema added to the `obsidian_search` registration (`z.record` of string→scalar|scalar[]).
|
|
1617
|
+
- **`tests/search-hybrid.test.ts`** (+9): 6 `frontmatterMatches` unit tests (scalar/case-insensitive, array-membership, array-OR + multi-key-AND, number/boolean strictness, missing-key/empty/absent → no match, a NEGATIVE control that discriminates) + 3 integration tests through `searchHybrid` (filter narrows to the matching note; **NEGATIVE control** — no filter returns all three, proving the filter is what narrowed it; array-value OR + multi-key AND).
|
|
1618
|
+
|
|
1619
|
+
### Method note
|
|
1620
|
+
|
|
1621
|
+
This is the deliberate answer to "is the project at a dead-end?" — the *refinement* track had hit diminishing returns, so the next real value is a **new capability**, not another micro-RC. Frontmatter-aware retrieval was chosen because it (1) expands what the product can *do* (not polish), (2) plays to the retrieval core — the project's strength, (3) is deeply Obsidian-native (frontmatter is a first-class Obsidian primitive) with **no competitor parity**, and (4) ships **additively/opt-in** so the critical search path is byte-identical when unused. Rejected alternatives, with reasons: conversation write-back (it's `basic-memory`'s grain and muddies the "grounded, not extracted" differentiator), multi-vault (explicit non-goal in CLAUDE.md), and answer-synthesis (we're a retriever, not a QA generator — would be the kind of overclaim `docs/benchmarks.md` explicitly avoids). The integration test is non-vacuous by construction (the NEGATIVE control returns all three notes when the filter is absent — so a no-op filter implementation fails it), unlike the rc.8 case the revert-verify caught. **Next increments:** rc.11 — `tag` filter parity (FTS has it; hybrid doesn't) + optional boost-by-frontmatter; rc.12 — positioning for the capability.
|
|
1622
|
+
|
|
1623
|
+
### Tests (1085)
|
|
1624
|
+
|
|
1625
|
+
`tests/search-hybrid.test.ts` +9 source `it()`. 1076 → 1085; claims synced (README ×4, package.json, llms.txt, AGENTS, COMPARISON, ROADMAP). Tool count unchanged (45 — this adds a *parameter*, not a tool).
|
|
1626
|
+
|
|
1627
|
+
### Files changed
|
|
1628
|
+
|
|
1629
|
+
- `src/tools/search.ts` (`filter_frontmatter` arg + `frontmatterMatches`/`frontmatterValueMatches`/`frontmatterScalarEq` helpers + matches-loop integration + `fmFilter` hoist), `src/tool-registry.ts` (zod schema), `tests/search-hybrid.test.ts` (+9), `docs/api.md` (args-table row), test-count claims → 1085.
|
|
1630
|
+
- version bump 3.10.0-rc.9 → 3.10.0-rc.10.
|
|
1631
|
+
|
|
1632
|
+
---
|
|
1633
|
+
|
|
1634
|
+
## [3.10.0-rc.9] — 2026-06-02
|
|
1635
|
+
|
|
1636
|
+
> **TL;DR:** **Positioning — a verified, fair head-to-head vs `basic-memory`** (the closest local-markdown-MCP rival), added to the COMPARISON "when to pick something other than enquire-mcp" section. Grounded in a fresh web-research pass (Track B of the promotion plan): `basic-memory` solves the **inverse** problem — it *writes* a knowledge-base **from your AI conversations** (readable markdown, viewable in Obsidian as a GUI), whereas enquire-mcp *recalls the notes you authored*. The entry is intentionally fair-not-sales (calls out exactly when basic-memory is the better pick, and that the two **compose**) and makes the "grounded, not extracted" line concrete with a real, citable example. Docs-only; no overclaim about the competitor (every claim verified against its public repo). **1076 tests unchanged.**
|
|
1637
|
+
|
|
1638
|
+
**Minor (pre-release) — v3.10 line; promotion/positioning increment (no code change).**
|
|
1639
|
+
|
|
1640
|
+
### Changed
|
|
1641
|
+
|
|
1642
|
+
- **`docs/COMPARISON.md`** — "when to pick something other than enquire-mcp" expanded from four cases to **five**: added **`basic-memory` (basicmachines-co)**. It's the closest project in spirit (local-first, markdown, MCP-native, semantic search over a wikilinked knowledge graph, Obsidian as a GUI) but solves the inverse problem — write-memory-from-chat vs recall-what-you-authored — which makes the choice clean and sharpens enquire's "grounded, not extracted" differentiator with a concrete example. Notes that the two compose (basic-memory writes conversation-derived notes; enquire retrieves across the whole authored vault). Kept OUT of the Obsidian-MCP feature matrix (different category) to avoid a misleading row.
|
|
1643
|
+
|
|
1644
|
+
### Method note
|
|
1645
|
+
|
|
1646
|
+
This is the first increment of the **promotion track**. It's grounded in a Firecrawl research pass (PROMO-1), not authored from memory, specifically to avoid competitor-claim overclaim: every `basic-memory` capability stated here was verified against its public GitHub/docs (knowledge-graph, semantic search, wikilinks, MCP-native, Obsidian GUI, conversation-capture). The research also surfaced **discoverability gaps that are maintainer-gated** (not shippable as repo docs) — handed off separately: enquire is absent from the high-intent "best Obsidian MCP server" results (needs stars + listicle presence), the brand search surfaces a stale OpenClaw-directory listing rather than the canonical repo, and the highest-leverage lever remains the published LongMemEval/retrieval score (reference hardware). Glama listing confirmed live (auto-synced from the MCP registry); the "claim" is OAuth-gated. **Deliberately did NOT** churn the README use-cases for marginal on-page SEO — the real high-intent-query gap is off-page (stars/listicles), and quality > keyword-stuffing.
|
|
1647
|
+
|
|
1648
|
+
### Tests (1076)
|
|
1649
|
+
|
|
1650
|
+
No `it()` change (docs-only). 1076 unchanged.
|
|
1651
|
+
|
|
1652
|
+
### Files changed
|
|
1653
|
+
|
|
1654
|
+
- `docs/COMPARISON.md` (basic-memory "when to pick else" entry; four→five), version bump 3.10.0-rc.8 → 3.10.0-rc.9.
|
|
1655
|
+
|
|
1656
|
+
---
|
|
1657
|
+
|
|
5
1658
|
## [3.10.0-rc.8] — 2026-06-02
|
|
6
1659
|
|
|
7
1660
|
> **TL;DR:** **Post-rc.7 audit response — fusion-stage privacy parity (defense-in-depth) + a self-caught vacuous-test correction.** A state-driven audit of the rc.3→rc.7 line (behavioral/threat lens, per the rc.36 meta-audit) found that the two fusion-stage consumers of the RRF `fused` list — pre-existing **graph-boost** (calls `vault.readNote` to parse a candidate's wikilinks → reads its **content**) and the rc.5 **recency re-rank** (stats a candidate's **mtime**) — both run BEFORE the rc.18 L-HYB-1 response-build `isExcluded` guard and don't replicate it. Not exploitable today (every ranker arm already drops excluded paths before `fused`, and the response-build guard drops them from output), so this is a **third, defense-in-depth layer** for a hypothetical future ranker-arm regression — exactly the "RRF fusion trusts ranker inputs; don't" rationale L-HYB-1 was shipped on. Fixed by pruning excluded paths from `fused` once at the source via a new pure `pruneExcludedHits`. **The audit also caught itself overclaiming:** the first test written for this was an *integration* test that **passed with the fix disabled** (vacuous — the per-arm filters prevent an excluded path from ever reaching `fused` through the public API). The revert-verify exposed it; it was replaced with a **pure-helper unit test** that actually fails when the guard is removed. **1072 → 1076 tests.** `src/` change is one fusion-stage filter line + the extracted helper.
|