@oomkapwn/enquire-mcp 3.9.0-rc.8 → 3.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +899 -0
- package/README.md +15 -7
- package/SECURITY.md +19 -15
- package/STABILITY.md +2 -2
- package/assets/social-preview.png +0 -0
- package/dist/bases.d.ts +23 -0
- package/dist/bases.d.ts.map +1 -1
- package/dist/bases.js +29 -4
- package/dist/bases.js.map +1 -1
- package/dist/cli.d.ts +22 -0
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +88 -5
- package/dist/cli.js.map +1 -1
- package/dist/communities.d.ts +19 -1
- package/dist/communities.d.ts.map +1 -1
- package/dist/communities.js +25 -4
- package/dist/communities.js.map +1 -1
- package/dist/doctor.d.ts +12 -0
- package/dist/doctor.d.ts.map +1 -1
- package/dist/doctor.js +35 -2
- package/dist/doctor.js.map +1 -1
- package/dist/dql.d.ts +10 -0
- package/dist/dql.d.ts.map +1 -1
- package/dist/dql.js +13 -1
- package/dist/dql.js.map +1 -1
- package/dist/embed-db.d.ts +29 -1
- package/dist/embed-db.d.ts.map +1 -1
- package/dist/embed-db.js +49 -3
- package/dist/embed-db.js.map +1 -1
- package/dist/embed-pipeline.d.ts +10 -0
- package/dist/embed-pipeline.d.ts.map +1 -1
- package/dist/embed-pipeline.js +22 -1
- package/dist/embed-pipeline.js.map +1 -1
- package/dist/embeddings.d.ts +1 -1
- package/dist/embeddings.js +1 -1
- package/dist/eval.d.ts +14 -0
- package/dist/eval.d.ts.map +1 -1
- package/dist/eval.js +12 -2
- package/dist/eval.js.map +1 -1
- package/dist/hnsw.d.ts.map +1 -1
- package/dist/hnsw.js +5 -1
- package/dist/hnsw.js.map +1 -1
- package/dist/http-transport.d.ts.map +1 -1
- package/dist/http-transport.js +19 -5
- package/dist/http-transport.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/ocr.d.ts +97 -19
- package/dist/ocr.d.ts.map +1 -1
- package/dist/ocr.js +145 -25
- package/dist/ocr.js.map +1 -1
- package/dist/pdf.d.ts.map +1 -1
- package/dist/pdf.js +15 -4
- package/dist/pdf.js.map +1 -1
- package/dist/server.d.ts.map +1 -1
- package/dist/server.js +12 -0
- package/dist/server.js.map +1 -1
- package/dist/tool-registry.d.ts.map +1 -1
- package/dist/tool-registry.js +5 -3
- package/dist/tool-registry.js.map +1 -1
- package/dist/tools/limits.d.ts +14 -0
- package/dist/tools/limits.d.ts.map +1 -0
- package/dist/tools/limits.js +43 -0
- package/dist/tools/limits.js.map +1 -0
- package/dist/tools/meta.d.ts +101 -0
- package/dist/tools/meta.d.ts.map +1 -1
- package/dist/tools/meta.js +602 -1
- package/dist/tools/meta.js.map +1 -1
- package/dist/tools/read.d.ts.map +1 -1
- package/dist/tools/read.js +7 -1
- package/dist/tools/read.js.map +1 -1
- package/dist/tools/search.d.ts.map +1 -1
- package/dist/tools/search.js +14 -4
- package/dist/tools/search.js.map +1 -1
- package/dist/vault.d.ts.map +1 -1
- package/dist/vault.js +26 -11
- package/dist/vault.js.map +1 -1
- package/dist/watcher.d.ts +30 -0
- package/dist/watcher.d.ts.map +1 -1
- package/dist/watcher.js +75 -19
- package/dist/watcher.js.map +1 -1
- package/docs/COMPARISON.md +4 -4
- package/docs/QUICKSTART.md +2 -2
- package/docs/api.md +6 -5
- package/docs/benchmarks.md +50 -7
- package/package.json +6 -4
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,904 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project will be documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
4
4
|
|
|
5
|
+
## [3.9.0] — 2026-06-01
|
|
6
|
+
|
|
7
|
+
> **TL;DR:** **v3.9.0 STABLE — promoted `@rc → @latest` after 37 RCs.** The v3.9.0 minor delivers the last architectural items from the v3.8.0 backlog plus the deepest security/correctness hardening cascade in the project's history. **Headline features:** OCR'd-PDF watcher embed-sync (rc.1), **HNSW in-memory live update** so search reflects vault edits within the watcher debounce window (~250 ms) without a serve restart (rc.2) + close-time disk persistence (rc.6), and **R-10 adaptive HNSW refill** that doubles `k` under heavy privacy-filtering (rc.3). **Hardening cascade (rc.7→rc.37):** the ReDoS guard for `obsidian_open_questions` taken from a single instance to a permanent generative-fuzz CI gate (rc.21/24/25), OCR offline-enforcement actually built to match the docs (rc.10), watcher/HNSW concurrency races closed (rc.11), and a comprehensive in-house audit (rc.36/37) that fixed the open siblings of every recent class **and internalized the missing privacy/DoS lenses as permanent inventory invariants** (erasure-completeness, resource-bound-completeness, orphan-dist). **Promotion basis:** the v3.6.1 ≥2-independent-external-auditor gate is met by the rc.32 deep-audit (Mavis 10-track STRIDE/privacy) + the rc.34 from-scratch audit (different methodology), each re-verified per-item against the pinned commit; the maintainer elected to promote on these two rather than commission a third on the rc.37 commit (the rc.35→37 delta is hardening + docs only). **1039 tests, 89.61% line coverage, all 9 required CI gates green.**
|
|
8
|
+
|
|
9
|
+
**Minor — STABLE promotion of the v3.9.0 line.**
|
|
10
|
+
|
|
11
|
+
### Promoted
|
|
12
|
+
|
|
13
|
+
- npm dist-tag `@rc` (3.9.0-rc.37) → `@latest` (3.9.0). GitHub release marked Latest. The MCP Registry auto-publishes on the stable tag via OIDC (release.yml, gated `dist_tag=='latest'`), reconciling the registry-vs-npm drift the rc.32 advisory tracks (3.8.4 → 3.9.0). No code change vs rc.37 — this is a version bump + this entry.
|
|
14
|
+
|
|
15
|
+
### What shipped across the v3.9.0 line (vs v3.8.8 stable)
|
|
16
|
+
|
|
17
|
+
- **Architectural (closed the v3.8.0 backlog):** OCR'd-PDF watcher embed-sync; HNSW in-memory live update + disk persistence; R-10 adaptive refill; full state-driven self-audit.
|
|
18
|
+
- **Security / correctness:** ReDoS guard + generative-fuzz gate (overlapping-alternation, case/escape aliasing, optional/nullable/variable bodies); OCR offline enforcement (`assertOcrLangsInstalled` + `cacheMethod:"readOnly"` + real `install-ocr-lang`); canvas-OOM clamp; watcher per-file serialization + fail-closed HNSW label zip; bearer ≥16 reconciliation; DQL `like` cap.
|
|
19
|
+
- **Privacy / DoS (the rc.34→rc.37 external + in-house audit line):** HNSW-sidecar right-to-erasure (P-2) + parse-cache `.tmp` erasure (F-2); path-free client errors (P-3/F-3); `find_path`/`communities`/`findSimilar`/`getNoteNeighbors` whole-vault scan caps (R-5/AS#5/F-4/F-5); stale-`dist` ship fix (L-3/F-1).
|
|
20
|
+
- **Apparatus:** OIA grew to 12 checks (+12b orphan-dist); new structural invariants — erasure-completeness, resource-bound-completeness, meta-invariant vacuity closure, ReDoS generative fuzz; `lint` now enforces `noExplicitAny`.
|
|
21
|
+
|
|
22
|
+
### Method note
|
|
23
|
+
|
|
24
|
+
Promotion follows the v3.8.0 precedent: this entry + the version bump only; the **state-driven docs-currency flip** ("stable `@latest` = v3.8.x → v3.9.0" across README/COMPARISON/api.md/ROADMAP) lands as a documented **follow-up patch after `@latest` publishes** (mirroring v3.8.2 after v3.8.0) — those pointers are accurate until the dist-tag actually moves, so flipping them pre-publish would itself be an overclaim. The promotion is the maintainer's call per the v3.6.1 rule; the agent prepared this gated release and the maintainer triggers the `@latest` publish via the stable tag push.
|
|
25
|
+
|
|
26
|
+
### Exit criteria
|
|
27
|
+
|
|
28
|
+
npm `@latest = 3.9.0` · GitHub release v3.9.0 marked Latest · all 37 RCs tagged + on `@rc` · 1039 tests green · 9/9 required CI gates · MCP Registry auto-synced to 3.9.0 via OIDC on the stable tag.
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## [3.9.0-rc.37] — 2026-06-01
|
|
33
|
+
|
|
34
|
+
> **TL;DR:** **Docs/process-drift batch — the deferred (no-behavioral-risk) half of the rc.36 comprehensive audit — plus the structural gate that closes the gate-GAP the headline finding exposed.** The process/contradiction sub-agent found **F1** (MEDIUM): `ROADMAP.md` carried a stale "Process maturity — **1020 tests**" claim (plus a `1002`/`44 tools` stat-pill in a planning bullet) that **NO gate caught** — ROADMAP was absent from both `scope-completeness-audit.mjs` `AUDIT_FILES` and the `docs-consistency` surfaces, and a `[x]` checkbox even *claimed* ROADMAP had already been added to `AUDIT_FILES` (it hadn't — only `AGENTS.md` was). Closed the number AND the gate-gap: ROADMAP is now in `AUDIT_FILES` + the `test-count`/`ci-gate-count` defense scopes + a new `docs-consistency` "ROADMAP test-count" invariant. Plus **F3** (`AGENTS.md` referenced a phantom `src/search.ts` ×2 — the file is `src/rrf.ts` / `src/tools/search.ts`), **F4** (`COMPARISON` "7 stack configs" → "8: 6 full-60-query + 2 HyDE-subset"), **F2** (`lint` didn't enforce the documented "0 warnings / no `any`" claim — `noExplicitAny:"warn"` + plain `biome check` → set to `"error"`, verified zero stray `any`). Documents the meta-audit's durable lesson as a CLAUDE.md anti-pattern. **1038 → 1039 tests.**
|
|
35
|
+
|
|
36
|
+
**Patch — comprehensive-audit docs/process half + apparatus gate-gap closure.**
|
|
37
|
+
|
|
38
|
+
### Fixed (verified-real)
|
|
39
|
+
|
|
40
|
+
- **F1 — `ROADMAP.md` stale test count, caught by no gate** (MEDIUM; the headline finding). Line 21 said "1020 tests" (drifted from canonical 1026→1038); a Tier-2 planning bullet embedded a stale "44 tools / 1002 tests" stat-pill (describing a social-card design rc.29 abandoned). Root cause — a **gate gap**: ROADMAP was in neither `scope-completeness-audit.mjs#AUDIT_FILES` nor any `docs-consistency` surface, and OIA Check 7's `DOCS_FILES_TO_SCAN` omits it. Worse, a `[x]` checkbox claimed "add `ROADMAP.md`/`AGENTS.md` to `AUDIT_FILES`" was DONE — but only `AGENTS` had been added (a false-done — the exact scope-too-narrow recursion the project documents). **Fix:** corrected line 21 → canonical; dropped the stat-pill counts (count-agnostic, per rc.29); added `ROADMAP.md` to `AUDIT_FILES` + the `test-count` and `ci-gate-count` defense scopes; new `docs-consistency` invariant "ROADMAP.md test-count claim matches actual it() count" (3-4-digit pattern pins the maturity total while ignoring the "+15 tests"/"+7 tests" per-RC deltas); made the checkbox honest (AGENTS rc.13, ROADMAP rc.37; OpenSSF Scorecard genuinely deferred).
|
|
41
|
+
- **F3 — `AGENTS.md` phantom `src/search.ts`** (LOW; ×2). The architecture diagram + the "fix a retrieval bug" entrypoint both pointed at `src/search.ts`, which doesn't exist (the orchestration is `src/tools/search.ts`; RRF is `src/rrf.ts`). Corrected both.
|
|
42
|
+
- **F4 — `docs/COMPARISON.md` "7 stack configurations"** (LOW). The benchmarks table has 8 rows (6 on the full 60-query set + 2 HyDE-subset at n=25). Reworded to "8 stack configurations (6 … + 2 HyDE-subset)".
|
|
43
|
+
- **F2 — `lint` did not enforce the documented "0 warnings / no `any`" guarantee** (LOW; claimed-guarantee-vs-reality). CLAUDE.md quality-bar #2 says "biome 0 warnings/errors" and AGENTS says "No `any`", but `biome.json` had `noExplicitAny:"warn"` and the `lint` script was a plain `biome check` (exits 0 on warnings) — a stray `: any` would have passed CI silently. Set `noExplicitAny:"error"` (verified the tree has zero real `: any`, so the gate now matches the claim with no active violation).
|
|
44
|
+
|
|
45
|
+
### Added — durable methodology capture
|
|
46
|
+
|
|
47
|
+
- **CLAUDE.md anti-pattern** documenting the meta-audit conclusion: the internal apparatus is ~85% drift/claim-driven and structurally blind to behavioral/threat classes, so every recent behavioral defect came from an external lens; the rule is to internalize each external lens as an inventory-based invariant (as rc.36 did with erasure + resource-bound + orphan-dist), and the named-but-uncovered dimensions (supply-chain `run:`-download pinning, paired-sink behavior parity, enforcement-verb→code-guard taxonomy) are listed so they're not silently skipped.
|
|
48
|
+
|
|
49
|
+
### Method note
|
|
50
|
+
|
|
51
|
+
This is the no-behavioral-risk docs/process half of the rc.36 comprehensive audit (rc.36 shipped the behavioral fixes + the erasure/resource-bound invariants). The F1 closure follows the project's own "drift findings demand a full-surface sweep + structural defense, not a per-instance fix" rule — the stale number was the symptom; the gate gap (ROADMAP outside the audit set) was the cause. **Deferred (named, not silently skipped):** OpenSSF Scorecard + `dependency-review-action` workflows + an OIA scan for unpinned `run:` downloads (M-9 class), a paired-sink behavior-parity invariant (H-3 class), and a generalized enforcement-verb→code-guard taxonomy (beyond OIA 4d/4e).
|
|
52
|
+
|
|
53
|
+
### Tests (1039)
|
|
54
|
+
|
|
55
|
+
`tests/docs-consistency.test.ts` +1 source `it()` (the ROADMAP test-count invariant). 1038 → 1039; claims synced (README ×4, package.json, llms.txt, AGENTS, COMPARISON, ROADMAP).
|
|
56
|
+
|
|
57
|
+
### Files changed
|
|
58
|
+
|
|
59
|
+
- `ROADMAP.md` (count → 1039, stat-pill dropped, checkbox honesty), `scripts/scope-completeness-audit.mjs` (ROADMAP → AUDIT_FILES + test-count/ci-gate-count scopes), `tests/docs-consistency.test.ts` (+ROADMAP invariant), `AGENTS.md` (rrf.ts/tools/search.ts + count), `docs/COMPARISON.md` (8 configs + count), `biome.json` (`noExplicitAny: error`), `CLAUDE.md` (anti-pattern), test-count claims → 1039.
|
|
60
|
+
- version bump 3.9.0-rc.36 → 3.9.0-rc.37.
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## [3.9.0-rc.36] — 2026-06-01
|
|
65
|
+
|
|
66
|
+
> **TL;DR:** **Comprehensive in-house audit (3 parallel sub-agents — sibling-class hunt · process/contradiction · meta-audit — each cross-checked against my own independent grep sweep) + the structural gates that close each finding's CLASS.** The meta-audit's core result: our home-grown apparatus (12 OIA checks + the invariant suite) is ~85% **drift/claim-driven** (does a CLAIM match reality?) and structurally blind to **behavioral/threat-model** failure — so every recent real bug (P-2 erasure, P-3 path-leak, R-5/AS#5 unbounded-graph, L-3 stale-build) was found by an EXTERNAL privacy/STRIDE lens we don't run ourselves. rc.36 fixes the open siblings of those classes **and internalizes the missing lenses as permanent CI gates.** **Fixed: F-1** (HIGH — `tsc` never purged `dist/`, so the pre-split `dist/tools.{js,d.ts}` (~309 KB stale) SHIPPED to npm, confirmed via `npm pack --dry-run`; the rc.35 fix closed the stale-*import*, this closes the stale-*artifact* ROOT CAUSE), **F-2** (MEDIUM privacy — `clear-cache` left the atomic-write `${cacheFile}.tmp` (full note bodies) on disk — P-2 sibling), **F-3** (LOW — `assertParentInsideVault` leaked an absolute server path to MCP clients — P-3 sibling), **F-4/F-5** (MEDIUM DoS — `findSimilar` + `getNoteNeighbors` did uncapped whole-vault `readNote` fan-out — R-5/AS#5 siblings). **3 new structural gates:** OIA **Check 12b** (orphan-`dist` file detector), `tests/erasure-invariant.test.ts` (writers ⊆ erasers), `tests/resource-bound-invariant.test.ts` (every whole-vault scanner must be CAP-or-EXEMPT classified — ends the AS#5→AS#6 recursion). **1026 → 1038 tests.**
|
|
67
|
+
|
|
68
|
+
**Patch — comprehensive in-house audit response (behavioral-class fixes + apparatus hardening).**
|
|
69
|
+
|
|
70
|
+
### Fixed (verified-real)
|
|
71
|
+
|
|
72
|
+
- **F-1 — `build` never purged `dist/`; stale pre-split artifacts shipped to npm** (HIGH; `package.json`). `"build": "tsc && …"` never cleaned `dist/`, so after the `src/tools.ts → src/tools/` split the 6-week-old pre-split `dist/tools.js` (131 KB) + `dist/tools.d.ts` (30 KB) + maps (~148 KB) lingered and `files:["dist"]` published them — **confirmed via `npm pack --dry-run`**. A consumer importing `…/dist/tools.js` (a valid pre-3.6.0 path) or a resolver picking `dist/tools.d.ts` binds to stale code/types. rc.35's OIA Check 12 caught the stale *import string*; this is the *artifact* root cause. **Fix:** `build` + `prepare` now `rm -rf dist && tsc …`; new `clean` script. **Structural class-closer: OIA Check 12b (`ORPHAN-DIST-FILE`)** flags any `dist/<p>.{js,d.ts}` with no backing `src/<p>.ts` (FLAT 1:1 emit rule — a `src/<p>/` directory emits `dist/<p>/index.js`, never `dist/<p>.js`, so a directory must NOT satisfy the file; this exact false-negative bit my first probe — the rc.24 "analyze the right semantic space" lesson). Detection-power verified (injected orphan flagged; clean build silent; skips when `dist/` absent — the CI `oia` job doesn't build).
|
|
73
|
+
- **F-2 — `clear-cache` left the atomic-write temp on disk** (MEDIUM privacy / right-to-erasure; `src/vault.ts`). `saveDiskCache` writes `${cacheFile}.tmp` (full note bodies, 0600) then renames; a crash/EXDEV between the two leaves the `.tmp`, and `clearDiskCache` only unlinked the final file — so `clear-cache` left raw vault text on disk. Exact P-2 shape (rc.34: HNSW `.meta.json` `text_preview` survived `clear-embeddings`). **Fix:** `clearDiskCache` now erases both the file and its `.tmp` (and always resets in-memory state). **Structural class-closer: `tests/erasure-invariant.test.ts`** — behavioral (`clearDiskCache` erases an injected `.tmp` holding raw text) + the **writers ⊆ erasers** manifest (each on-disk artifact family's eraser source must reference every sidecar suffix: embed-db `-wal`/`-shm`/`.hnsw`/`.bin`/`.meta.json`, fts5 `-wal`/`-shm`, parse-cache `.tmp`), with positive + NEGATIVE controls.
|
|
74
|
+
- **F-3 — `assertParentInsideVault` leaked an absolute path to MCP clients** (LOW info-disclosure; `src/vault.ts:680`). On a write whose parent resolves outside the vault, the thrown message embedded the server-computed absolute dir `${current}` — reaching a `serve-http` client. P-3 / ν-class sibling; the symlink throw 30 lines above already used `path.relative(this.root, abs)`. **Fix:** echo the vault-relative path.
|
|
75
|
+
- **F-4 / F-5 — uncapped whole-vault fan-out in two always-on tools** (MEDIUM DoS defense-in-depth; `src/tools/search.ts`, `src/tools/read.ts`). `findSimilar` builds vault-sized `metas` + `inboundFor` graph maps and scores pairwise; `getNoteNeighbors` does TWO full-vault `readNote` passes building an inbound-count map. Both are always-registered and reachable over bearer-auth `serve-http` — direct siblings of the capped `find_path` (R-5) + `communities` (AS#5). **Fix:** new shared `src/tools/limits.ts` (`MAX_SCAN_NOTES = 50_000` + `capScanEntries`, one-shot stderr warning, graceful truncation) applied to both. 50k ≫ any real vault, so a partial scan only trims the top-K tail.
|
|
76
|
+
|
|
77
|
+
### Added — apparatus hardening (the meta-audit's highest-leverage recommendations)
|
|
78
|
+
|
|
79
|
+
- **`tests/resource-bound-invariant.test.ts`** (P0). Discovers EVERY always-on whole-vault scanner (`export … function` with `listMarkdown(` + `readNote(` + a `for` loop) across `tools/{read,search,meta}.ts` and fails CI unless each is classified **CAP** (builds a vault-sized graph/pairwise structure → must reference `capScanEntries`/`MAX_VISITED`/`MAX_GRAPH_NODES`) or **EXEMPT** (inherent single-pass O(N) — search/aggregation/exhaustive enumeration where capping would corrupt results, documented per tool). A NEW scanner lands unclassified → fails → forces the cap-or-exempt call. This is the structural answer to the rc.34→rc.35 recursion (R-5 fixed `find_path` but no sweep caught the `communities`/`findSimilar`/`getNoteNeighbors` siblings until later auditors did). Mirrors the rc.25 ReDoS-fuzz move: convert "did we bound every scanner?" (undecidable, recursion-prone) into a self-checking gate.
|
|
80
|
+
|
|
81
|
+
### Method note
|
|
82
|
+
|
|
83
|
+
This RC is a response to my OWN comprehensive audit (3 sub-agents + an independent grep cross-check of the same finding-classes), not an external report — the discipline the meta-audit prescribes: stop *depending* on an external auditor remembering to run the privacy/DoS lens, and encode each lens as a permanent gate (erasure-completeness, resource-bound-completeness, orphan-dist). **The resource-bound invariant caught a bug in its OWN implementing patch:** I added the `capScanEntries` import to `read.ts` but forgot to apply it in `getNoteNeighbors`'s body — the cap-token assertion failed in CI-local and I fixed it before ship (the gate works, and this is logged rather than silently corrected). No overclaim introduced. Deferred to rc.37 (docs/process drift, no behavioral risk): ROADMAP stale test count + its missing `AUDIT_FILES` coverage, `AGENTS.md` `src/search.ts`→`src/tools/search.ts`, COMPARISON "7→8 configs", `lint` not enforcing `noExplicitAny`, supply-chain `run:`-download scan.
|
|
84
|
+
|
|
85
|
+
### Tests (1038)
|
|
86
|
+
|
|
87
|
+
`tests/erasure-invariant.test.ts` (+5 source `it()`: behavioral `.tmp` erasure + writers⊆erasers manifest + 3 NEGATIVE/structural controls) · `tests/resource-bound-invariant.test.ts` (+7: classification completeness + cap-token + communities + capScanEntries behavioral + 2 NEGATIVE controls). Source `it()` 1026 → 1038; claims synced (README ×4, package.json, llms.txt, AGENTS, COMPARISON).
|
|
88
|
+
|
|
89
|
+
### Files changed
|
|
90
|
+
|
|
91
|
+
- `package.json` (`build`/`prepare` `rm -rf dist`, new `clean` script), `scripts/oia-walk.mjs` (Check 12b + enumeration 11/12 backfill), `src/vault.ts` (F-2 `.tmp` erasure + F-3 relative path), `src/tools/limits.ts` (new — `MAX_SCAN_NOTES` + `capScanEntries`), `src/tools/search.ts` + `src/tools/read.ts` (apply cap), `tests/erasure-invariant.test.ts` + `tests/resource-bound-invariant.test.ts` (new), test-count claims → 1038.
|
|
92
|
+
- version bump 3.9.0-rc.35 → 3.9.0-rc.36.
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## [3.9.0-rc.35] — 2026-06-01
|
|
97
|
+
|
|
98
|
+
> **TL;DR:** **Full from-scratch external audit (Mavis, 3 methodologies, on the pinned rc.34 commit) re-verification + the 2 genuinely-new code findings.** I commissioned a clean-slate external audit (state-driven + change-driven + adversarial STRIDE) on the exact pinned commit `7a479bb` and re-verified every finding. **Verdict: it graded the correct commit (no staleness this time), confirmed all rc.32→rc.34 P-fixes closed, and recommends promoting v3.9.0 → `@latest`.** Its one **HIGH** ("1024 tests is an overclaim, runtime is 1088") is **rejected** — a methodology error: the canonical metric is *source `it()` = 1024* by design, and the runtime expansion comes from data-driven `for (… ) it(…)` loops the auditor mis-claimed it "found none of" (`redos-guard.test.ts` etc.). Two genuinely-new **LOW** code gaps it surfaced (that our own sweeps missed) are fixed here: **L-3** (`bench.mjs`/`bench-search.mjs` imported the pre-split `../dist/tools.js`) + **AS#5/R-B** (`buildWikilinkGraph` had no node cap — DoS parity gap with the rc.34 `find_path` R-5 cap). **1024 → 1026 tests; +OIA Check 12.**
|
|
99
|
+
|
|
100
|
+
**Patch — external from-scratch audit response.**
|
|
101
|
+
|
|
102
|
+
### Fixed (verified-real, new)
|
|
103
|
+
|
|
104
|
+
- **L-3 — bench scripts import path** (`scripts/bench.mjs`, `scripts/bench-search.mjs`). The `tools.ts → tools/` split means TS emits `dist/tools/index.js`, not `dist/tools.js`; both bench scripts kept the old import, which only "resolved" because a STALE pre-split `dist/tools.js` lingered in the gitignored `dist/` (on a clean build it breaks). CI never runs these (only `bench:retrieval`), so it hid. Fixed both to `../dist/tools/index.js`. **Structural class-closer: new OIA Check 12 (`STALE-DIST-TOOLS-IMPORT`)** fails CI on any `scripts/*.mjs` importing `dist/tools.js` (detection-power verified: injected import flagged, clean after fix). OIA canonical count 11 → 12 (header + AGENTS ×2 + ROADMAP).
|
|
105
|
+
- **AS#5 / R-B — `communities.ts buildWikilinkGraph` node cap** (`MAX_GRAPH_NODES = 50_000`). `obsidian_get_communities` is always-registered and read EVERY `.md` to build the full adjacency map + run Louvain, with only `MAX_PASSES=50` on *iterations* — no bound on graph *size* → unbounded I/O+memory on a pathological/huge vault. Now the node set is capped (graceful truncation), exactly mirroring the rc.34 `find_path` R-5 `MAX_VISITED` cap. +2 tests (cap-triggers via a stubbed oversized vault + below-cap NEGATIVE control).
|
|
106
|
+
|
|
107
|
+
### Rejected / documented (verified)
|
|
108
|
+
|
|
109
|
+
- **H-DOC-1** "test count overclaim: docs 1024 vs runtime 1088" — **REJECTED, methodology error.** Canonical metric is **source `it()` count = 1024** (verified exact); runtime > source because of data-driven loops (`for (const p of catastrophic) { it(…) }` in `redos-guard.test.ts` + ~10 other files), which the auditor incorrectly claimed not to find. A deliberate, internally-consistent metric is not an overclaim. This is the same item the prior Mavis round raised as "L-11" and we already ruled working-as-intended. No change.
|
|
110
|
+
- **M-DOC-1** npm `@latest` 3.8.8 keywords still end `slsa-3` — REAL on the *published stable tag*; HEAD `package.json` keywords are clean (verified, no `slsa-3`). Reconciles when v3.9.0 → `@latest` (clean keywords) or a manual stable re-publish. Not a source-tree bug. **MAINTAINER-GATED.**
|
|
111
|
+
- **M-DOC-2** published `@rc` description (256 chars) shorter than HEAD (567) — registry-side; HEAD is the source of truth; reconciles on next publish. No source change.
|
|
112
|
+
- **L-1 (watcher throttle), L-2 (TF-IDF WeakMap), R-C (per-tool timeout), R-D (TUI password prompt), H-1/H-2/H-3 (branch protection)** — valid-but-non-urgent or maintainer-only; deferred, documented. (L-2 is the WeakMap prior rounds misread as a leak; this auditor correctly downgraded it to LOW.)
|
|
113
|
+
|
|
114
|
+
### Method note
|
|
115
|
+
|
|
116
|
+
Third independent external pass on the v3.9.0 line, and the first to grade the correct pinned commit with no staleness — strong independent confirmation that the rc.25–rc.34 cascade closed everything, plus 2 real LOW gaps our sweeps missed. Combined with the rc.32 deep-audit (different methodology), the v3.6.1 ≥2-independent-auditors-with-different-methodologies gate is now **substantively met** for the v3.9.0 → `@latest` decision (maintainer's call). Its one HIGH was a metric-methodology artifact — reinforcing again: re-verify every external finding's *severity and validity* against current code, never action on faith. Full per-finding verdict in `docs/audits/v3.9.0-rc.34-external-fromscratch-reverification-2026-06-01.md`.
|
|
117
|
+
|
|
118
|
+
### Tests (1026)
|
|
119
|
+
|
|
120
|
+
`tests/communities.test.ts`: +2 (MAX_GRAPH_NODES cap-triggers via stubbed oversized vault + below-cap NEGATIVE control). Test-count claims 1024 → 1026 (README ×4, package.json, llms.txt, AGENTS, COMPARISON).
|
|
121
|
+
|
|
122
|
+
### Files changed
|
|
123
|
+
|
|
124
|
+
- `scripts/bench.mjs` + `scripts/bench-search.mjs` (import path), `scripts/oia-walk.mjs` (Check 12 + canonical count 12), `src/communities.ts` (`MAX_GRAPH_NODES` cap), `tests/communities.test.ts` (+2), `AGENTS.md` ×2 + `ROADMAP.md` (OIA count 11 → 12), `docs/audits/v3.9.0-rc.34-external-fromscratch-reverification-2026-06-01.md` (new), test-count claims → 1026.
|
|
125
|
+
- version bump 3.9.0-rc.34 → 3.9.0-rc.35.
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## [3.9.0-rc.34] — 2026-06-01
|
|
130
|
+
|
|
131
|
+
> **TL;DR:** **Deep-audit (Mavis 10-track on rc.32) re-verification + the genuinely-new privacy/DoS fixes.** A much deeper Mavis audit (type-system, supply-chain, STRIDE, privacy/GDPR, MCP-compliance) was supplied; I re-verified every claim. Its two top non-governance findings (**H-3** extractPdfText, **SC-1** mcp-publisher pin) were **already shipped in rc.33** — the audit graded the older rc.32 commit. But its **new deep-dimension tracks surfaced 3 real, previously-unflagged issues** this RC fixes: **P-2** (`clear-embeddings` left the HNSW `.hnsw.bin`/`.hnsw.meta.json` sidecars on disk — and the `.meta.json` carries `text_preview` raw text → a right-to-erasure gap for `--use-hnsw` users; now erased), **P-3** (the `embeddings_search` "index not found" error echoed the absolute vault + embed-db paths to MCP clients → fingerprinting on `serve-http`; now path-free), **R-5** (`obsidian_find_path` BFS had no explicit visited cap → unbounded I/O on a pathological graph; now capped at 50k). Plus **P-1** doc-honesty (`text_preview` at-rest, in SECURITY.md). **Rejected** SC-4 + M-2 (both auditor misreads — verified). **1022 → 1024 tests.**
|
|
132
|
+
|
|
133
|
+
**Patch — deep-audit response (privacy + DoS hardening).**
|
|
134
|
+
|
|
135
|
+
### Fixed (verified-real, new in the deep tracks)
|
|
136
|
+
|
|
137
|
+
- **P-2 — `clear-embeddings` now erases the HNSW sidecars too** (`src/embed-db.ts` `clearOnDisk`, `src/cli.ts` description). Previously it removed only `.embed.db` + WAL/SHM, leaving `<base>.hnsw.bin` + `<base>.hnsw.meta.json` on disk — and the `.meta.json` stores `text_preview` (raw chunk text), so a `--use-hnsw` user's content survived a "clear". `clearOnDisk()` is now the single authority that erases every embed-derived artifact for the vault. Positive + NEGATIVE (over-deletion) controls.
|
|
138
|
+
- **P-3 — `embeddings_search` "index not found" error no longer leaks filesystem paths** (`src/tools/search.ts`). The throw propagates to the MCP client; on bearer-auth `serve-http` it previously echoed the absolute `embedFile` + `vault.root` (fingerprinting). Reworded to a path-free, still-actionable remediation. (The auditor flagged only `vault.root`; `embedFile` was the bigger leak — both removed. The `server.ts:374` site it also flagged is a server-operator **stderr log**, not client-facing — left intentional, consistent with the startup banner.)
|
|
139
|
+
- **R-5 — `obsidian_find_path` BFS visited-node cap** (`src/tools/meta.ts`, `MAX_VISITED = 50_000`). BFS was already bounded by note-count + `max_depth`, but a very large densely-wikilinked vault meant unbounded per-layer `readNote` I/O for an always-registered tool; the cap bails gracefully (returns not-found) — defense-in-depth.
|
|
140
|
+
- **P-1 — SECURITY.md now documents content-at-rest honestly**: the `text_preview` column (+ `.hnsw.meta.json`) stores raw leading chunk text directly, alongside the existing cosine-reversibility caveat; the `clear-embeddings` purge note updated for the rc.34 sidecar removal.
|
|
141
|
+
|
|
142
|
+
### Rejected / not-actioned (verified, documented)
|
|
143
|
+
|
|
144
|
+
- **SC-4** ("`@huggingface/transformers` dup in dev+optional is a no-op") — **FALSE / would regress**: the `optionalDependencies` entry is how an end-user `npm i -g @oomkapwn/enquire-mcp` gets the embeddings runtime (the error at `embeddings.ts:108` literally says "reinstall … without `--omit=optional`"); the `devDependencies` entry is for the maintainer's own test `npm ci`. Different audiences; the optional entry is load-bearing for users, not a no-op. Removing it would break embeddings for consumers. No change.
|
|
145
|
+
- **H-3 / SC-1** — already shipped in **rc.33** (audit graded rc.32).
|
|
146
|
+
- **M-2** ("TF-IDF WeakMap unbounded") — re-confirmed not-a-leak (WeakMap keyed on live Vault, GC'd); same misread as the v1 round.
|
|
147
|
+
- **H-1/H-2/H-4** branch protection — TRUE, maintainer-only (repo settings).
|
|
148
|
+
- **T-1/T-2** (branded types, z.infer), **M-1/M-4/M-5/M-6/M-7** (logger, refactors, throttle, hnsw flags), **R-1/R-6** etc. — valid-but-non-urgent; deferred, no correctness impact.
|
|
149
|
+
|
|
150
|
+
### Method note
|
|
151
|
+
|
|
152
|
+
Best round yet from this auditor: its **new deep dimensions** (privacy/GDPR, STRIDE) earned their keep by finding P-2/P-3/R-5 that all prior rounds (and my own sweeps) missed — even though it was simultaneously **stale on the code tracks** (re-flagged H-3/SC-1 fixed in rc.33). Lesson holds: per-item re-verification against current code separates the real new findings from the carry-over noise. Full per-finding verdict appended to `docs/audits/v3.9.0-rc.32-external-mavis-reverification-2026-06-01.md`.
|
|
153
|
+
|
|
154
|
+
### Tests (1024)
|
|
155
|
+
|
|
156
|
+
`tests/embed-db.test.ts`: +2 — P-2 HNSW-sidecar erasure (positive) + over-deletion NEGATIVE control. `findPath` R-5 cap covered by the existing `v16.test.ts` BFS suite (regression: cap doesn't alter normal traversal); a 50k-node trigger test is impractical and omitted by design. Test-count claims 1022 → 1024 (README ×4, package.json, llms.txt, AGENTS, COMPARISON).
|
|
157
|
+
|
|
158
|
+
### Files changed
|
|
159
|
+
|
|
160
|
+
- `src/embed-db.ts` (clearOnDisk + HNSW sidecars), `src/cli.ts` (clear-embeddings description), `src/tools/search.ts` (P-3 path-free error), `src/tools/meta.ts` (R-5 cap), `SECURITY.md` (P-1), `tests/embed-db.test.ts` (+2), `docs/audits/…reverification….md` (deep-round appendix), test-count claims → 1024.
|
|
161
|
+
- version bump 3.9.0-rc.33 → 3.9.0-rc.34.
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## [3.9.0-rc.33] — 2026-06-01
|
|
166
|
+
|
|
167
|
+
> **TL;DR:** **External-audit (Mavis on rc.32) re-verification + the 2 genuinely-actionable fixes.** A fresh Mavis audit on rc.32 was supplied; I re-verified every claim against the actual code (treating the report as untrusted). **Verdict**: a broad, mostly-accurate *health* audit (correctly confirms "production-ready, concurrency class closed") — but **materially STALE**: it explicitly carried the rc.24 report forward and re-flagged **4 findings that rc.28 already closed** (MAX_EMBED_CHARS clamp, peekCache LRU, main() TSDoc, bench "p99"→max), plus repeated count errors ("14-check OIA"→11, "22 catch {}"→49/zero-empty, "17 overclaims"→18, "10 floors"→11). Two findings were real and are fixed here: **H-3** (`extractPdfText` silently returned `pages:[]` on an inverted `pageRange` — now throws, matching the OCR sibling) and **M-9** (`mcp-publisher` was downloaded from `releases/latest` — now SHA-tag-pinned). Full per-finding verdict in `docs/audits/v3.9.0-rc.32-external-mavis-reverification-2026-06-01.md`. **1020 → 1022 tests.**
|
|
168
|
+
|
|
169
|
+
**Patch — external-audit re-verification + fixes for the verified-real findings.**
|
|
170
|
+
|
|
171
|
+
### Fixed (verified-real)
|
|
172
|
+
|
|
173
|
+
- **H-3 — `extractPdfText` now fails closed on an inverted/out-of-domain `pageRange`** (`src/pdf.ts`). Previously `{from:50,to:10}` clamped to an empty window and returned `pages:[]` with NO error — a silent caller-error sink and a parity gap with the OCR path (`resolveOcrPageRange` throws on inverted). Now validates `from` is an integer ≥ 1 and ≤ `to`, throwing a clear message ("invalid page range — 'from' must be … ≤ 'to' …") that mirrors the OCR sibling. A valid `from ≤ to` range (incl. clamp-to-doc like `{from:1,to:10}` on a 2-page PDF) is unaffected. Auditor rated this HIGH; re-verified as **LOW** (caller-error → empty, not data loss) but a genuine DX/parity fix.
|
|
174
|
+
- **M-9 — `mcp-publisher` pinned to an exact release tag** (`release.yml`). The rc.32 registry-OIDC step downloaded the CLI from `releases/latest/download` (the official MCP docs' own example does this); pinned to `v1.7.9` via a `MCP_PUBLISHER_TAG` var so a re-tagged/compromised upstream "latest" can't silently change the binary that runs with our OIDC identity. Bump the var deliberately to adopt a newer CLI.
|
|
175
|
+
|
|
176
|
+
### Rejected / not-actioned (documented in the re-verification doc)
|
|
177
|
+
|
|
178
|
+
- **4 STALE carryovers** (M-8 clamp, L-2 peekCache, L-7 bench-p99, main()-TSDoc) — all closed in rc.28; verified still-closed on rc.32, no action.
|
|
179
|
+
- **M-2** ("TF-IDF WeakMap unbounded at 100K notes") — **investigated, not a leak**: `tfidfCache` is a `WeakMap<Vault, …>` (search.ts:336) holding ONE cache per live vault object (overwritten on rebuild, GC'd with the vault), not linear content growth. Auditor misread it. No fix.
|
|
180
|
+
- **M-3** (22 bare `catch {}`) — miscounted (actual 49, **zero** truly-empty, all deliberate fail-soft); same rejection as the rc.24 round.
|
|
181
|
+
- **H-1/H-2/H-4** (branch protection) — TRUE + already ACKNOWLEDGED maintainer-only (repo security settings, out of agent scope).
|
|
182
|
+
- **M-1/M-4/M-5/M-6/M-7/M-10 + L-1/L-4/L-5/L-6/L-8/L-9/L-10** — valid-but-non-urgent hardening/DX/refactor; deferred, no correctness impact.
|
|
183
|
+
|
|
184
|
+
### Method note
|
|
185
|
+
|
|
186
|
+
Mirror-image lesson to the rc.24 Mavis round: there the single auditor **missed** a live CRITICAL; here it went **stale** — re-flagging four fixes that landed 4 RCs earlier because it carried its prior report forward without re-checking current code. Either failure mode is why the v3.6.1 ≥2-independent-auditors gate stands and why every external finding is re-verified per-item against `git`-current source before any action.
|
|
187
|
+
|
|
188
|
+
### Tests (1022)
|
|
189
|
+
|
|
190
|
+
`tests/pdf.test.ts`: the pre-existing "inverted range yields empty pages" test (which asserted the OLD silent-empty behavior) is updated to assert the new throw; +2 source `it()` (from < 1 throw; positive control that a valid `from ≤ to` range still extracts without throwing). Test-count claims 1020 → 1022 (README ×4, package.json, llms.txt, AGENTS, COMPARISON).
|
|
191
|
+
|
|
192
|
+
### Files changed
|
|
193
|
+
|
|
194
|
+
- `src/pdf.ts` (inverted-range guard), `.github/workflows/release.yml` (mcp-publisher SHA-tag pin), `tests/pdf.test.ts` (+2 + 1 updated), `docs/audits/v3.9.0-rc.32-external-mavis-reverification-2026-06-01.md` (new), test-count claims → 1022.
|
|
195
|
+
- version bump 3.9.0-rc.32 → 3.9.0-rc.33.
|
|
196
|
+
|
|
197
|
+
---
|
|
198
|
+
|
|
199
|
+
## [3.9.0-rc.32] — 2026-05-30
|
|
200
|
+
|
|
201
|
+
> **TL;DR:** **Automate MCP Registry publishing + drift backstop (closes the registry-staleness class).** A promotion-channel check found the canonical MCP Registry stuck at **3.8.4** while npm `@latest` is **3.8.8** (~7 versions behind) — and because Glama / mcp.so / smithery **auto-sync from the registry**, that stale entry was silently propagating an outdated "current version" across the whole directory ecosystem. Root cause: the registry was published *manually* (`mcp-publisher publish` after each stable), so it drifted whenever that step was skipped. **Fixed structurally:** `release.yml` now auto-publishes **stable releases** to the registry via GitHub Actions **OIDC** (no secret — the existing `id-token: write` is all it needs), and new **OIA Check 11** is a state-driven advisory that surfaces registry-vs-npm drift on every audit run. **Workflow + audit-script + docs only; 1020 tests unchanged.**
|
|
202
|
+
|
|
203
|
+
**Patch — supply-chain / discoverability automation.**
|
|
204
|
+
|
|
205
|
+
### Added
|
|
206
|
+
|
|
207
|
+
- **Automated MCP Registry publish (OIDC) in `release.yml`** — after the npm publish + GitHub Release steps, a new `Publish to MCP Registry (stable only)` step downloads the official `mcp-publisher`, authenticates via `mcp-publisher login github-oidc` (GitHub Actions OIDC — trusts the repo's identity for the `io.github.oomkapwn/*` namespace; **no dedicated secret**), and runs `mcp-publisher publish`. **Gated to `dist_tag == 'latest'`** so the registry's `isLatest` always reflects what `npm install` gives by default — RCs publish to npm under `@rc` but are deliberately NOT pushed to the registry (else the canonical listing, and every directory that syncs from it, would advertise an `-rc.N` as current). Defensively re-syncs `server.json`'s `version` (+ each `packages[].version`) from `package.json` before publishing.
|
|
208
|
+
- **OIA Check 11 — `MCP-REGISTRY-VERSION-DRIFT`** (`scripts/oia-walk.mjs`, network, `--skip-network`-respecting) — compares the registry's `isLatest` version to npm's `latest` dist-tag and prints a visible **ADVISORY** when they differ. **Non-fatal by design**: remediation (re-publish) is maintainer-gated (runs on a stable tag or a manual login), so a PR author can't fix registry state inside their PR — hard-failing the `oia` gate on it would block unrelated work (same principle as the SLSA network check skipping on infra it doesn't control). Detection-power verified: it flags the live 3.8.4-vs-3.8.8 drift today. OIA canonical count 10 → 11 (header + AGENTS ×2 + ROADMAP, gated by `docs-consistency.test.ts`).
|
|
209
|
+
|
|
210
|
+
### Method note
|
|
211
|
+
|
|
212
|
+
This is the registry analogue of the rc.31 repo-About fix: a promotion surface that lives **outside the repo's files** (here, the canonical registry; there, GitHub metadata) drifted because the publish path was manual and no state-driven check watched it. The durable fix is the same shape — **automate the publish** (OIDC step) **+ add a state-driven drift detector** (OIA Check 11). The advisory will keep flagging 3.8.4-vs-3.8.8 until the next **stable** release runs the new OIDC step (or a maintainer re-publishes manually); on an RC line there is no stable tag to trigger it, so the reconciliation lands with the v3.9.0 → `@latest` promotion.
|
|
213
|
+
|
|
214
|
+
### Files changed
|
|
215
|
+
|
|
216
|
+
- `.github/workflows/release.yml` (registry-publish OIDC step, stable-gated), `scripts/oia-walk.mjs` (Check 11 + canonical count 11), `AGENTS.md` ×2 + `ROADMAP.md` (OIA count 10 → 11; ROADMAP test count 1002 → 1020 stale-fix).
|
|
217
|
+
- version bump 3.9.0-rc.31 → 3.9.0-rc.32; no `src/`, no test change (1020).
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
## [3.9.0-rc.31] — 2026-05-30
|
|
222
|
+
|
|
223
|
+
> **TL;DR:** **Repo-page SLSA overclaim fix + structural guard (residual of overclaim #15).** A state-driven check of the GitHub repo page found the **About description still said "SLSA-3"** — the unenforced claim that overclaim #15 (rc.7) downgraded to "SLSA L2" across README/package.json/llms.txt/COMPARISON/STABILITY, and that rc.18 fixed on the social card. The About string lives ONLY on GitHub (no file → outside OIA Check 4d's scope), so it survived ~23 RCs. **Fixed the live About** (`gh repo edit` → "SLSA L2") and added a structural guard so it can't drift back: `tests/github-metadata-invariant.test.ts` now asserts the About carries no SLSA-level-above-2 claim. **1019 → 1020 tests** (+1 source `it()`, positive + NEGATIVE controls); no `src/` change.
|
|
224
|
+
|
|
225
|
+
**Patch — brand-integrity (repo metadata + invariant). Tests + docs only.**
|
|
226
|
+
|
|
227
|
+
### Fixed
|
|
228
|
+
|
|
229
|
+
- **Repo About "SLSA-3" → "SLSA L2"** (residual instance of overclaim #15). `release.yml` runs `npm publish --provenance` = **SLSA Build L2** (L3 requires the isolated `slsa-framework/slsa-github-generator`). The GitHub About description was the last surface still asserting the higher level — corrected via `gh repo edit`. (Topics verified correct: all `REQUIRED_TOPICS` present incl. `openclaw`; About lead-in "The most advanced Obsidian MCP" intact.)
|
|
230
|
+
|
|
231
|
+
### Added
|
|
232
|
+
|
|
233
|
+
- **`findSlsaOverclaim` analyzer + live assertion** in `tests/github-metadata-invariant.test.ts` — the About-description test now fails if the description claims SLSA-3 / L3 / L4 (tolerant of `SLSA-3`, `SLSA 3`, `SLSA Build L3`, `SLSA Level 3`, `SLSA L3`); `SLSA L2` / `SLSA-2` pass. This is the structural class-closer for the gap that let the overclaim live on GitHub-only metadata: OIA Check 4d guards in-repo claim files + the social SVG, but the repo About string had no guard until now. Positive + NEGATIVE controls + a false-positive guard ("3 transports / L3 caching" must NOT trip).
|
|
234
|
+
|
|
235
|
+
### Method note
|
|
236
|
+
|
|
237
|
+
The change-driven sweep of overclaim #15 (rc.7) fixed every *file*; OIA Check 4d (rc.8) structurally guarded every *file* + the social SVG (rc.18). But the GitHub About/Topics metadata is not a file in the repo — it's reachable only via `gh api` — so it fell outside both. Same root shape the project keeps hitting: **a defense scoped to one surface type misses a sibling surface of a different type**. The fix extends the existing `github-metadata-invariant` (which already pulls live About/Topics) with the SLSA check, so the repo page is now covered by the same fail-loud apparatus as the files.
|
|
238
|
+
|
|
239
|
+
### Files changed
|
|
240
|
+
|
|
241
|
+
- `tests/github-metadata-invariant.test.ts` (+`findSlsaOverclaim` + live assertion + NEGATIVE control), test-count claims 1019 → 1020 (README ×4, package.json, llms.txt, AGENTS, COMPARISON).
|
|
242
|
+
- live GitHub repo About (out-of-band via `gh repo edit`; not a tracked file).
|
|
243
|
+
- version bump 3.9.0-rc.30 → 3.9.0-rc.31.
|
|
244
|
+
|
|
245
|
+
---
|
|
246
|
+
|
|
247
|
+
## [3.9.0-rc.30] — 2026-05-30
|
|
248
|
+
|
|
249
|
+
> **TL;DR:** **Correction patch — overclaim instance #18.** A state-driven post-ship audit (after the multi-hour sandbox outage that interrupted rc.29) caught that the rc.29 CHANGELOG + CLAUDE.md cited social-card asset sizes carried over from the **first design attempt the EPERM outage ate**, not the files actually shipped: SVG claimed "9.7 KB → 11.8 KB" (real **7.3 KB** — it shrank), PNG claimed "188 KB → **49.5 KB**" (real **205 KB** — the 2× density render grew it). No gate catches KB annotations in CHANGELOG prose, so only a state-driven read found it. Corrected to be **size-agnostic** (drop drift-prone KB; keep the verified `1280×640`, which the audit confirmed correct). **Docs-only — zero `src/`, zero asset change, 1019 tests unchanged.**
|
|
250
|
+
|
|
251
|
+
**Patch — claim-vs-reality correction (CHANGELOG + CLAUDE.md only).**
|
|
252
|
+
|
|
253
|
+
### Fixed
|
|
254
|
+
|
|
255
|
+
- **Overclaim #18** — rc.29's two asset-size annotations were inaccurate (numbers from a pre-outage render draft, never re-measured against the shipped files). Removed the wrong KB figures from the rc.29 CHANGELOG entry (SVG + PNG lines) and the CLAUDE.md rc.29 status line. The verifiable `1280×640` dimension (re-confirmed via `sharp().metadata()`) is kept; the volatile byte-size annotations are dropped rather than re-stated, so this stops being a drift surface.
|
|
256
|
+
|
|
257
|
+
### Method note
|
|
258
|
+
|
|
259
|
+
Root cause: I drafted the rc.29 entry **before** the outage forced a re-render, then shipped the draft's numbers without re-measuring the actual artifact. The recovery itself (re-applying the eaten SVG write, deleting a misplaced tag, re-tagging the correct squash SHA) was verified end-to-end — but the *prose numbers* describing the artifact were not re-checked against the final bytes. This is the claim-vs-reality class (overclaims #15/#16/#17): a stated figure the artifact doesn't back. **Lesson reinforced**: after any re-render/re-build, re-measure every quantitative claim in the same commit that ships the artifact — a draft figure is not evidence. Asset byte-sizes are deliberately NOT a tracked claim going forward (volatile, low value); dimensions + content are.
|
|
260
|
+
|
|
261
|
+
### Files changed
|
|
262
|
+
|
|
263
|
+
- `CHANGELOG.md` (rc.29 entry: SVG + PNG size annotations removed), `CLAUDE.md` (rc.29 status line: "49.5 KB" removed).
|
|
264
|
+
- version bump 3.9.0-rc.29 → 3.9.0-rc.30; no `src/`, asset, or test change (1019).
|
|
265
|
+
|
|
266
|
+
---
|
|
267
|
+
|
|
268
|
+
## [3.9.0-rc.29] — 2026-05-29
|
|
269
|
+
|
|
270
|
+
> **TL;DR:** **Social card redesign** — `assets/social-preview.svg` (+ rendered `.png`), the GitHub social-preview / most-shared visual of the repo, was completely redesigned for a more professional, conversion-oriented look. Premium dark treatment (layered gradient + radial glow + subtle dot matrix), an SVG logomark (vault doc → recall rings; no emoji), a value-prop hero (**"Long-term memory for your AI agents"**), the category-differentiator selling line (**"Grounded in the notes you actually wrote — cited, auditable, editable."**), qualitative capability chips (Hybrid + reranked · GraphRAG · Agentic RAG · PDF + OCR), a `claude mcp add enquire-mcp` install CTA pill, an honest trust line (**MIT · SLSA L2 · Claude/Cursor/ChatGPT/Codex/OpenClaw**), and a "vault → knowledge-graph memory" illustration. **Assets only — zero `src/`, 1019 tests unchanged.**
|
|
271
|
+
|
|
272
|
+
**Patch — brand/visual. `assets/social-preview.svg` + `assets/social-preview.png` only.**
|
|
273
|
+
|
|
274
|
+
### Changed
|
|
275
|
+
|
|
276
|
+
- **`assets/social-preview.svg`** fully rewritten with the premium layout above. Deliberately **drops the previous hardcoded count claims** ("44 tools", "19 prompts") — qualitative differentiators replace drift-prone numbers, so the card is no longer a numeric-drift surface (sidesteps the rc.18-deferred "stat-pill needs an invariant" concern entirely). Pure ASCII, no NUL, well-formed.
|
|
277
|
+
- **`assets/social-preview.png`** re-rendered at 2× density → crisp 1280×640.
|
|
278
|
+
- **SLSA honesty preserved**: the trust line reads "SLSA L2" (not "SLSA-3"); OIA Check 4d (which scans `social-preview.svg`) stays green.
|
|
279
|
+
|
|
280
|
+
### Notes
|
|
281
|
+
|
|
282
|
+
- The GitHub repo **Social preview** image is uploaded in repo **Settings → Options → Social preview** (not auto-derived from the file); upload `assets/social-preview.png` there to update the live card.
|
|
283
|
+
- version bump 3.9.0-rc.28 → 3.9.0-rc.29; no package-content change (assets are not in the npm tarball).
|
|
284
|
+
|
|
285
|
+
---
|
|
286
|
+
|
|
287
|
+
## [3.9.0-rc.28] — 2026-05-29
|
|
288
|
+
|
|
289
|
+
> **TL;DR:** **External-audit re-verification response.** An external "Mavis" audit (on rc.24 / commit `d564eb5`) was supplied and re-verified — I treated every claim as untrusted and checked it against the actual code. Verdict: a competent breadth audit, but it **missed a live CRITICAL** (the ReDoS we'd independently found + fixed in rc.25 — its "no critical findings" was false for the commit it graded) and carried several factual errors ("14-check OIA" → it's 10; "22 bare `catch {}`" → 49, and **zero** truly-empty; "12 floors" → 11 files). This RC fixes the audit's **legitimate** code findings (M-2, M-6, M-4, the L-4 residual — each with positive + NEGATIVE controls), **rejects** one (H-4, documented), **defers** one (M-3, documented), and records the full re-verification in `docs/audits/`. The branch-protection findings (H-1/H-2/H-3) are **maintainer-only** (repo settings — out of scope for me to change). **1014 → 1019 tests**; zero behavior change (clamp + cache cap are opt-in/bounding only).
|
|
290
|
+
|
|
291
|
+
**Patch — external-audit re-verification + fixes for the verified-real minor findings.**
|
|
292
|
+
|
|
293
|
+
### Fixed (verified-real audit findings)
|
|
294
|
+
|
|
295
|
+
- **M-2 — `buildEmbedText` assembled text now clamped (`MAX_EMBED_CHARS = 8000`).** A large opt-in `--late-chunk-context` could assemble ~12K chars, far beyond any embedding model's token budget (the model truncates anyway). Now bounded, preserving the core chunk and dropping neighbor context first. The **default path (`contextChars <= 0`) is unaffected** — bit-for-bit identical.
|
|
296
|
+
- **M-6 — `peekCache` is now LRU-bounded (`MAX_PEEK_CACHE_ENTRIES = 512`).** A long-running `serve` over many distinct `.embed.db` paths previously grew the cache without limit. New exported pure `lruMapSet(map, key, value, max)` helper (mirrors rc.15's `boundedSetAdd`) does insert-with-eviction; the cached-hit path bumps recency (true LRU, not FIFO).
|
|
297
|
+
- **M-4 — TSDoc on the CLI entry point.** `main()` (what `dist/index.js` invokes) had zero TSDoc; added `@returns` + `@example` + subcommand overview. Added `@param`/`@returns` to `addAdvancedRetrievalOptions()`.
|
|
298
|
+
- **L-4 residual — `bench.mjs:4` header comment** still said "p99" though the output was relabeled "max" back in rc.8; comment corrected.
|
|
299
|
+
|
|
300
|
+
### Rejected / deferred (documented per CLAUDE.md)
|
|
301
|
+
|
|
302
|
+
- **H-4 (REJECTED as framed)** — "22 bare `catch {}` swallow errors silently." Re-verification: actual bare-catch count is **49** (not 22), and **none** are truly empty — all have deliberate fail-soft bodies (`return null` / `continue` / skip-unreadable). It's a style preference (capture `err` for logging), not silent swallowing; rated HIGH but is LOW. Adding `err` bindings across 49 deliberate fail-soft sites is noise > value.
|
|
303
|
+
- **M-3 (DEFERRED)** — expose hnsw `m`/`efConstruction` as CLI flags. Two new shared CLI flags = disproportionate surface (cli-help.ts + both subcommands + api.md flag table + cli-parity + scope-completeness invariants) for an advanced-vault-only knob. Lighter alternative (document the defaults in `--help`) noted for a future tuning RC.
|
|
304
|
+
- **H-1/H-2/H-3 (MAINTAINER-ONLY)** — branch protection (`docs`+`oia` not in required checks — verified true via `gh api`: 7 enforced, not 9; `enforce_admins:false`; 0 required reviews). Modifying repo security/access settings is out of scope for the agent; left for the maintainer with the exact `gh api` command in the re-verification doc.
|
|
305
|
+
|
|
306
|
+
### Added
|
|
307
|
+
|
|
308
|
+
- `docs/audits/v3.9.0-rc.24-external-mavis-reverification-2026-05-29.md` — the full per-finding re-verification + verdict. Headline: the external audit missed a live CRITICAL that our state-driven + adversarial-fuzz methodology caught — reinforcing the v3.6.1 "≥2 independent external auditors, different methodologies" gate for `@latest`.
|
|
309
|
+
|
|
310
|
+
### Tests (1019)
|
|
311
|
+
|
|
312
|
+
+5 source `it()`: `late-chunking.test.ts` (MAX_EMBED_CHARS clamp positive + NEGATIVE control) + `peek-cache.test.ts` (`lruMapSet` cap/eviction, LRU-recency, + NEGATIVE control proving an unbounded `Map` grows). Test-count claims 1014 → 1019 (README ×4, package.json, llms.txt, AGENTS, COMPARISON).
|
|
313
|
+
|
|
314
|
+
### Files changed
|
|
315
|
+
|
|
316
|
+
- `src/embed-pipeline.ts` (`MAX_EMBED_CHARS` + clamp), `src/embed-db.ts` (`lruMapSet` + `MAX_PEEK_CACHE_ENTRIES` + LRU-bumped peekCache), `src/cli.ts` (TSDoc on `main()` + `addAdvancedRetrievalOptions`), `scripts/bench.mjs` (header comment), `tests/late-chunking.test.ts`, `tests/peek-cache.test.ts`, `docs/audits/…reverification….md`, test-count claims → 1019.
|
|
317
|
+
- version bump 3.9.0-rc.27 → 3.9.0-rc.28.
|
|
318
|
+
|
|
319
|
+
---
|
|
320
|
+
|
|
321
|
+
## [3.9.0-rc.27] — 2026-05-29
|
|
322
|
+
|
|
323
|
+
> **TL;DR:** **Positioning + discoverability** (the rc.18-deferred repo-page work + the ROADMAP's #1 messaging item). Sharpens the core differentiator across README + llms.txt + COMPARISON: enquire-mcp is **grounded** in the markdown you already wrote (verbatim, cited, editable), as opposed to conversation-memory tools (mem0/Zep/Supermemory) that **extract** facts from chat logs into a separate opaque store. Plus a copy-paste `claude mcp add` one-liner promoted into the README hero. Docs only — no code, no test, no numeric-claim change.
|
|
324
|
+
|
|
325
|
+
**Patch — docs/positioning only. Zero `src/` change; test count unchanged (1014).**
|
|
326
|
+
|
|
327
|
+
### Changed
|
|
328
|
+
|
|
329
|
+
- **"Grounded, not extracted" positioning** added to the three canonical surfaces (ROADMAP Tier-2 messaging item): `README.md` (a new paragraph in "The solution"), `llms.txt` (the AI-discovery blockquote), and `docs/COMPARISON.md` (intro). Frames the category distinction vs mem0 / Zep / Supermemory: those *extract* memory from conversations into a store you can't read; enquire-mcp recalls the notes you authored, with citations, auditable and editable in any editor.
|
|
330
|
+
- **README hero `claude mcp add` one-liner.** The Claude Code one-command install (`claude mcp add obsidian -- npx -y @oomkapwn/enquire-mcp serve --vault …`) is now surfaced in the hero (was buried in Quick start), for instant copy-paste onboarding.
|
|
331
|
+
|
|
332
|
+
### Notes / still deferred
|
|
333
|
+
|
|
334
|
+
- Social-preview **stat-pill redesign** (44 tools / 1014 tests / +15.5 NDCG@10) is intentionally NOT in this RC — it introduces a new numeric-drift surface that must ship with its own `docs-consistency` invariant in the same PR (per the v3.9.0-rc.18 deferral rule); deferred to a focused asset RC. `server.json` `categories`/`keywords` likewise deferred pending a schema re-verify (the rc.13 PR #117 server.json schema fix history).
|
|
335
|
+
- No numeric claims were added, so no docs-consistency / scope-completeness invariant change is required; OIA Check 7 currency scan is unaffected (the mem0/Zep/Supermemory mentions are competitor names, not version-currency claims).
|
|
336
|
+
|
|
337
|
+
### Files changed
|
|
338
|
+
|
|
339
|
+
- `README.md`, `llms.txt`, `docs/COMPARISON.md` (positioning + hero one-liner); version bump 3.9.0-rc.26 → 3.9.0-rc.27.
|
|
340
|
+
|
|
341
|
+
---
|
|
342
|
+
|
|
343
|
+
## [3.9.0-rc.26] — 2026-05-29
|
|
344
|
+
|
|
345
|
+
> **TL;DR:** **Closes the pre-stable audit — test-infra rigor (batch 2/3) + docs drift (batch 3/3).** The same 3-agent audit that found the rc.25 CRITICAL flagged two HIGH defense-integrity gaps: (HIGH-1) the **META-invariant** — the enforcer of "every invariant has a real NEGATIVE control" — was itself partly vacuous: it accepted a COMMENTED-OUT `it(` and an EMPTY-BODY `it("NEGATIVE", () => {})`; (HIGH-2) `cli.test.ts` (22 tests incl. the bearer-auth ≥16 checks + the K-1 FTS5-preservation test) used silent `return` skips with **no CI-GUARD**, so the whole file could no-op in CI if a precondition regressed — the rc.23 CI-GUARD sweep missed this file. Plus MED/LOW (github-metadata CI-GUARD, scope-completeness control drove a re-implemented copy, two invariant-shaped files outside the meta-invariant glob, coverage job built dist only via the `prepare` hook) and docs drift (per-file-floor counter under-counted 11→reported 10, ROADMAP stale checkboxes, STABILITY missing `install-ocr-lang`). **1009 → 1014 tests.** Tests/scripts/workflow/docs only — zero `src/` runtime change. With rc.25 (security) this fully closes the pre-stable audit; the `src/` runtime audited exceptionally clean.
|
|
346
|
+
|
|
347
|
+
**Patch — pre-stable audit response, batches 2/3 (test-infra) + 3/3 (docs).**
|
|
348
|
+
|
|
349
|
+
### Fixed — test-infra (batch 2/3)
|
|
350
|
+
|
|
351
|
+
- **HIGH-1 — the META-invariant was itself partly vacuous (meta-recursion).** `checkInvariantHasNegativeCoverage` matched the `NEGATIVE`/`negative-control` token in an `it`/`describe` TITLE but (a) didn't strip comments — a full-line `// it("NEGATIVE", …)` satisfied it — and (b) didn't inspect the body — an empty `it("NEGATIVE", () => {})` satisfied it. The enforcer of "no vacuous invariant" was thus partly vacuous (the exact recursion class CLAUDE.md tracks; rc.23 advertised this fixed but only moved the vacuity from "token anywhere" to "token in a title"). Now: comments are stripped first, and the matched test's **callback body must contain an assertion** (`expect(`/`toThrow`/`rejects`/… via a balanced-brace scan; an expression-bodied arrow `() => expect(…)` is accepted). +3 NEGATIVE controls (commented-out → rejected, empty-body → rejected, expression-body → accepted). Verified against all real invariant files (none false-rejected).
|
|
352
|
+
- **HIGH-2 — `cli.test.ts` silent skips → CI-GUARD + `ctx.skip()`.** 14 `it`-blocks (incl. the rc.9 bearer-auth ≥16 security checks and the v3.6.4 K-1 trigram-preservation correctness test) used bare `if (!distExists()) return;` / `if (!canRunFts5) return;` — silently no-op'ing with zero assertions if dist/FTS5 vanished, with no tripwire. rc.23 added CI-GUARDs to the sibling files (`security`/`fts5`/`e2e-handlers`) but missed this one (incomplete class-sweep). Added a CI-GUARD tripwire (hard-fails in CI if `distExists()`/`canRunFts5` are false) + converted all 22 bare returns to visible `ctx.skip()`.
|
|
353
|
+
- **MED-1 — `github-metadata-invariant.test.ts` CI-GUARD.** Both metadata invariants early-return when `gh` isn't authenticated; added a tripwire that, **when CI provides `GH_TOKEN`**, asserts `gh auth status` actually succeeds (catches a token-scope/CLI regression on the token-bearing job).
|
|
354
|
+
- **MED-3 — scope-completeness NEGATIVE control drove a re-implemented copy.** Extracted the real per-(defense,file) classifier as `classifyDefenseFile` (exported; `runNumericAudit` now calls it) so the negative control drives the REAL code with a synthetic gap, not a hand-copied `wouldFlag` expression that could pass while the script diverged. + a POSITIVE side (in-scope file → no finding).
|
|
355
|
+
- **LOW-1 — two invariant-shaped tests outside the meta-invariant glob.** `k1-version-stamp-consistency.test.ts` + `jsonld.test.ts` assert source/state against canonical values but aren't named `*-invariant.test.ts`; added to `EXTRA_STRUCTURAL_FILES` (count assertion ≥9 → ≥11) so the meta-invariant keeps watching their negative controls.
|
|
356
|
+
- **LOW-2 — coverage CI job now runs `npm run build` explicitly** (matches the `test` job) instead of relying on the `npm ci` `prepare` hook to produce `dist/`; makes the `distExists()` precondition for the new cli.test.ts CI-GUARD explicit.
|
|
357
|
+
|
|
358
|
+
### Fixed — docs drift (batch 3/3)
|
|
359
|
+
|
|
360
|
+
- **F1 — per-file-floor counter under-counted (gate-passes-while-claim-is-wrong).** `docs-consistency.test.ts`'s `countPerFileFloors` regex only matched single-key `{ branches: N }`, skipping rc.23's two-key `"src/ocr.ts": { branches: 60, lines: 40 }` — so it returned 10 while reality was 11, and `AGENTS.md`'s "10 per-file branch floors" passed against a wrong number. Regex broadened to match multi-key floor objects (→ 11); `AGENTS.md` → "11 per-file coverage floors".
|
|
361
|
+
- **F2 — `ROADMAP.md` stale state.** rc.9/rc.12/rc.13 were shown unchecked `[ ]` though shipped → checked with accurate shipped-RC notes; "Updated" date + completed-sprint header refreshed; the unverifiable "MCP Registry entry (currently 404s)" claim reworded to a plain re-verify action item.
|
|
362
|
+
- **F3 — `STABILITY.md` omitted `install-ocr-lang`** from the semver-stable subcommand list (shipped rc.10) → added.
|
|
363
|
+
|
|
364
|
+
### Tests (1014)
|
|
365
|
+
|
|
366
|
+
+5 source `it()`: 3 META-invariant NEGATIVE controls + 1 cli.test.ts CI-GUARD + 1 github-metadata CI-GUARD (the 22 cli.test.ts skip conversions and the scope-completeness rewrite are net-zero `it()`). Test-count claims 1009 → 1014 (README ×4, package.json, llms.txt, AGENTS, COMPARISON).
|
|
367
|
+
|
|
368
|
+
### Method note
|
|
369
|
+
|
|
370
|
+
Both HIGH findings are the project's signature **incomplete-class-sweep** anti-pattern: rc.23 hardened the meta-invariant + added CI-GUARDs to security test files, but left a vacuity hole in the meta-invariant itself AND skipped `cli.test.ts`. The fresh independent audit (not the gates) caught both — reinforcing the v3.6.1 "≥2 external auditors" discipline. With rc.25 this closes the pre-stable audit end-to-end.
|
|
371
|
+
|
|
372
|
+
### Files changed
|
|
373
|
+
|
|
374
|
+
- `tests/meta-invariant-coverage.test.ts` (stripComments + callbackBody + ASSERTION_RE + 3 controls + EXTRA_STRUCTURAL_FILES +2 + ≥11), `tests/cli.test.ts` (CI-GUARD + 22 ctx.skip), `tests/github-metadata-invariant.test.ts` (CI-GUARD), `tests/scope-completeness-invariant.test.ts` (real-classifier control), `scripts/scope-completeness-audit.mjs` (`classifyDefenseFile` export), `tests/docs-consistency.test.ts` (per-file-floor regex), `.github/workflows/ci.yml` (coverage build), `AGENTS.md` / `ROADMAP.md` / `STABILITY.md` (docs), test-count claims → 1014.
|
|
375
|
+
- version bump 3.9.0-rc.25 → 3.9.0-rc.26.
|
|
376
|
+
|
|
377
|
+
---
|
|
378
|
+
|
|
379
|
+
## [3.9.0-rc.25] — 2026-05-29
|
|
380
|
+
|
|
381
|
+
> **TL;DR:** **Security — close the 3rd ReDoS recursion + add the fuzz harness that ends the treadmill.** A fresh independent pre-stable audit (3 agents: code · docs · tests) reproduced a **CRITICAL** the rc.21/rc.24 guard still missed: `(a?b|b)+$` (9 chars) hangs V8 >5s on bearer-auth `serve-http`. An OPTIONAL leading atom (`a?`/`a*`/`a{0,n}`) makes a branch's leading set overlap another branch, and a NULLABLE or VARIABLE-LENGTH body under an unbounded quantifier (`(a?){25}`, `(a{2,5})+`) partitions a long run exponentially — three shapes the leading-atom analysis couldn't see. Rather than chase shapes a 4th time, this RC fixes them with the *general* conditions (leading-SET intersection, nullable-body, variable-body) **and** adds `tests/redos-fuzz.test.ts`: it runs every SAFE-classified pattern from a 2000-pattern corpus through a real timed `exec` in a worker, so the NEXT missed shape fails CI empirically (the class is now structurally self-checking). **993 → 1009 tests** (+16: the fuzz + decode-helper direct tests + new-shape regression cases).
|
|
382
|
+
|
|
383
|
+
**Patch — pre-stable security audit response, batch 1/3 (the @latest-gate blocker). `src/tools/meta.ts` + tests only.**
|
|
384
|
+
|
|
385
|
+
### Fixed
|
|
386
|
+
|
|
387
|
+
- **ReDoS C-1 (CRITICAL) — optional / nullable / variable bodies (overclaim #18, 3rd recursion).** The rc.21 TSDoc claimed `alternationBodyAmbiguous` "never under-flags a real overlap"; rc.24 made that true for case/escape aliases, but it was STILL false for quantifier-induced shapes:
|
|
388
|
+
- **Optional leading atom** — `(a?b|b)+$`, `(a*b|b)+`, `(a{0,5}b|b)+`: `a?b` can start with `a` OR `b`, overlapping the `b` branch. The single-token `leadingAtomToken` read the literal and ignored the quantifier. Replaced with `leadingAtomSet` — a precise leading-**set** per branch — so `alternationBodyAmbiguous` checks real set INTERSECTION. (This also REMOVES the over-flag on disjoint cases like `(a?b|c)+` in the alternation analysis.)
|
|
389
|
+
- **Nullable body** — `(a?){25}`, `(\s*)*`, `()+`: a body that can match empty under an unbounded quantifier loops ambiguously. New `branchIsNullable` (recurses into nested groups) adds the `bodyNullable` term.
|
|
390
|
+
- **Variable-length body** — `(a{2,5})+`, `(\w[ba]{0,3})+`, `(a[ab]?)+`: a variable-length body partitions a long run super-linearly. `readUnboundedQuantifier`'s amplify-threshold treated bounded ranges like `{2,5}` as "not unbounded", so a whole class slipped. New `bodyHasVariableQuantifier` adds the `bodyVariable` term, **gated on the OUTER quantifier being unbounded** so a bounded `(.+){2,5}` (≤5 reps) stays accepted.
|
|
391
|
+
|
|
392
|
+
All three reproduced multi-second V8 hangs at ≤12 chars on the always-registered `obsidian_open_questions` tool (remote DoS on bearer-auth `serve-http`). Now rejected before compile. The guard stays sound-but-conservative: it may over-flag `(a?b)+` / `(\w+\s)+` (variable but anchored) — per the guard's documented stance, a rare false positive beats a hung event loop. Realistic capture-group overrides (`^(Q|TODO|Open question):\s*(.+)$`) stay accepted (verified).
|
|
393
|
+
- **`decodeEscapedChar` now returns `{char, length}`** (was `string | null`) — the SINGLE source of truth for escape spans, so `leadingAtomSet` and `branchIsNullable` locate atom ends without a divergent re-parser. Exported for direct unit tests.
|
|
394
|
+
|
|
395
|
+
### Added — the durable structural defense
|
|
396
|
+
|
|
397
|
+
- **`tests/redos-fuzz.test.ts`** — the class has recurred 3× (rc.21/rc.24/rc.25) because unit tests of *known* shapes can't catch the *next* missed shape. This fuzz generates a deterministic 2000-pattern ReDoS-prone corpus, and for every pattern `isCatastrophicRegex` classifies SAFE, runs it through a real `exec` in a worker (700 ms timeout, alphabet-matched adversarial inputs). A SAFE-classified pattern that HANGS is an under-flag → CI fails. Includes a NEGATIVE control proving the harness detects a real hang (non-vacuous). This turns the undecidable "did I catch every shape?" into an empirical check — the treadmill ends here.
|
|
398
|
+
|
|
399
|
+
### Tests (1009)
|
|
400
|
+
|
|
401
|
+
`tests/redos-guard.test.ts`: +new-shape catastrophic cases (optional-leading-atom, nullable-body, variable-body) + disjoint/fixed-body safe regression guards + a direct `decodeEscapedChar` block (5 `it()`, covering the control-escape / invalid-escape branches the test audit flagged as untested). `tests/redos-fuzz.test.ts`: +2 `it()` (fuzz + NEGATIVE control). Net +7 source `it()` (993 → 1009; the rest were array-loop entries). The 5000-candidate fuzz I ran during development found 0 under-flags after the fix; the CI fuzz (2000) is the permanent guard.
|
|
402
|
+
|
|
403
|
+
### Method note
|
|
404
|
+
|
|
405
|
+
This is the project's deepest anti-pattern on display — the **incomplete class-sweep**: the ReDoS class recurred a 3rd time, found by an INDEPENDENT audit (not the gates, not my own rc.24 self-fuzz). The lesson applied: when a heuristic security detector's class keeps recurring, stop enumerating shapes and add an **empirical fuzz** that compares the static verdict against real execution. Batches 2/3 (test-infra: meta-invariant vacuity, `cli.test.ts` CI-GUARD) and 3/3 (docs drift) follow as rc.26/rc.27.
|
|
406
|
+
|
|
407
|
+
### Files changed
|
|
408
|
+
|
|
409
|
+
- `src/tools/meta.ts` (`leadingAtomSet` + `branchIsNullable` + `bodyHasVariableQuantifier` + `quantifierMinZero` + `classEnd`/`groupEnd` helpers; `decodeEscapedChar` → `{char,length}`; `)` handler adds `bodyNullable`+`bodyVariable`; TSDoc documents all 4 shapes + the conservative over-flag), `tests/redos-guard.test.ts`, `tests/redos-fuzz.test.ts` (new), test-count claims 1002 → 1009 (README ×4, package.json, llms.txt, AGENTS.md, COMPARISON).
|
|
410
|
+
- version bump 3.9.0-rc.24 → 3.9.0-rc.25.
|
|
411
|
+
|
|
412
|
+
---
|
|
413
|
+
|
|
414
|
+
## [3.9.0-rc.24] — 2026-05-29
|
|
415
|
+
|
|
416
|
+
> **TL;DR:** **Security — close a ReDoS recursion the rc.21 fix itself introduced.** A re-audit (adversarial fresh-eyes review of the rc.21–rc.23 diffs) reproduced **two CRITICAL false-negatives in rc.21's own ReDoS detector** — the exact "audit-driven fix ships a fresh instance of the class it fixed" recursion this project most fears. `isCatastrophicRegex` compared **surface syntax**, not the matched character: (1) `(a|A)+` slipped through because the tool compiles `/i` (so `a`/`A` overlap) but `leadingAtomToken` was case-*sensitive*; (2) `(\x61|a)+` (= `(a|a)+`, also `a`/`\u{61}`) slipped because the helper returned the raw byte after `\` (`"x"`), not the decoded char. Both reproduced ~16s V8 hangs at ≤12 chars on bearer-auth `serve-http`. Fixed by case-folding + decoding escapes (unresolved escapes conservatively over-flag → the soundness claim is now true). Also closed the MEDIUM the same re-audit found (rc.23's two-key `{branches,lines}` floor objects fell out of OIA Check 6's drift regex). **1002 tests** (+6 detector cases via the data-driven loops; count unchanged — array entries, not new `it()`).
|
|
417
|
+
|
|
418
|
+
**Patch — security recursion fix (re-audit response). `src/tools/meta.ts` + tests + audit-script only.**
|
|
419
|
+
|
|
420
|
+
### Fixed
|
|
421
|
+
|
|
422
|
+
- **ReDoS recursion in `isCatastrophicRegex` (CRITICAL ×2).** rc.21's alternation-overlap analysis compared *surface syntax* (case-sensitive literal / first byte after `\`) instead of the matched char under the `/i` compile flag, so two real-overlap classes were ACCEPTED:
|
|
423
|
+
- **Case-fold**: `(a|A)+` / `(A|a)+` — the tool always compiles `new RegExp(pattern, "i")`, so `a` and `A` match the same input. `leadingAtomToken` now case-folds literals (`foldCase`).
|
|
424
|
+
- **Escape-alias**: `(\x61|a)+`, `(a|\x61)+`, `(a|a)+`, `(\u{61}|a)+` — all `(a|a)+` in disguise. `leadingAtomToken` now resolves the escape to its real char via a new `decodeEscapedChar` (`\xHH`/`\uHHHH`/`\u{H+}`/control/punctuation escapes); octal/backrefs/unknown escapes return `LEADING_ANY` (conservative over-flag — the safe direction). Both reproduced (~16s V8 hangs); both now rejected. Disjoint escapes (`(\.|a)+`) and disjoint literals (`(a|b|c)+`, `(cat|dog)+`) stay accepted (regression-guarded).
|
|
425
|
+
- With unresolved escapes over-flagging, the helper's "never under-flags a real first-char overlap" soundness claim is now **true** (rc.21's TSDoc asserted it while the code didn't — the claimed-guarantee-vs-reality anti-pattern; now matched).
|
|
426
|
+
- **Sentinel NUL byte (tooling).** `LEADING_ANY` was `"\0ANY"` — a literal NUL byte in source, which made `grep` treat `meta.ts` as binary (silently breaking text tooling/audits that shell-grep it). Changed to plain-ASCII `"<<ANY>>"`.
|
|
427
|
+
- **OIA Check 6 dropped the rc.23 two-key floors (MEDIUM).** Check 6 (coverage-comment drift) regex only matched single-key `{ branches: N } // current X%`; rc.23's `{ branches, lines }` + `// current branches X% / lines Y%` form silently fell out of drift-checking — the very gap Check 6 exists to prevent. Regex broadened to tolerate extra floor keys + an optional `branches ` word; `vault.ts` got an inline `// current 78.03%` so it's tracked too. Detection-power verified on both `vault.ts` + `ocr.ts`.
|
|
428
|
+
- **LOW**: per-file-coverage success message "all N per-file *branch* floors met" → "coverage floors met" (rc.23 added a non-branch `lines` floor).
|
|
429
|
+
|
|
430
|
+
### Tests (1002)
|
|
431
|
+
|
|
432
|
+
`tests/redos-guard.test.ts`: +6 catastrophic bypass cases (`(a|A)+`, `(A|a)+`, `(\x61|a)+`, `(a|\x61)+`, `(a|a)+`, `(\u{61}|a)+`) + 1 disjoint-escape POSITIVE control (`(\.|a)+` stays safe — guards the decoder against over-flagging). Array entries, so the canonical `it()` count is unchanged (1002). End-to-end probe: all 6 bypasses now flagged, all regressions preserved.
|
|
433
|
+
|
|
434
|
+
### Method note
|
|
435
|
+
|
|
436
|
+
This is exactly the recursion the project's own anti-pattern log warns about (audit-driven fix → fresh same-class instance) — **overclaim instance #17**, and a claimed-guarantee-vs-reality case too (rc.21's TSDoc asserted the analysis "never under-flags a real overlap" while it did). Caught by an **adversarial re-review of my own fix** rather than the gates (which can't fuzz a detector). Root cause — "the detector compared surface syntax, not the character set matched at the sink (case-folded + escape-resolved)" — is now a durable CLAUDE.md anti-pattern; the helper over-flags on any uncertainty, so it errs toward the safe (flag) direction.
|
|
437
|
+
|
|
438
|
+
### Files changed
|
|
439
|
+
|
|
440
|
+
- `src/tools/meta.ts` (`foldCase` + `decodeEscapedChar` + `leadingAtomToken` rewrite + de-NUL sentinel + TSDoc), `tests/redos-guard.test.ts` (+7 array cases), `scripts/oia-walk.mjs` (Check 6 regex), `scripts/check-per-file-coverage.mjs` (vault.ts inline comment + message wording).
|
|
441
|
+
- version bump 3.9.0-rc.23 → 3.9.0-rc.24 (7 surfaces); test count unchanged (1002).
|
|
442
|
+
|
|
443
|
+
---
|
|
444
|
+
|
|
445
|
+
## [3.9.0-rc.23] — 2026-05-29
|
|
446
|
+
|
|
447
|
+
> **TL;DR:** **Test-infra rigor (full-audit batch 3/3 — closes the audit).** The test auditor found the project's structural-enforcement apparatus was weaker than CLAUDE.md claimed: (HIGH) the META-invariant — the enforcer of the "every invariant has a NEGATIVE control" rule — passed if the token `NEGATIVE` appeared **anywhere, including a TODO comment** (reproduced), and its `*-invariant.test.ts` glob **silently excluded** real structural invariants (`no-internal-imports`, `lint`) and even itself; (MED) `security.test.ts` + `fts5.test.ts` had silent `return`-skips (the exact T1 anti-pattern rc.8 fixed) on security surfaces with no CI-GUARD; (LOW) `vault.ts` — the most security-critical module — had **no per-file coverage floor**, and `ocr.ts` floored only branches while its line coverage rotted to 44%. All closed. **1002 tests** (+5).
|
|
448
|
+
|
|
449
|
+
**Patch — test-infrastructure rigor (full-audit batch 3/3). Tests/scripts only; no `src/` runtime change.**
|
|
450
|
+
|
|
451
|
+
### Fixed
|
|
452
|
+
|
|
453
|
+
- **META-invariant comment-bypass (HIGH).** `checkInvariantHasNegativeCoverage` accepted the `NEGATIVE`/`negative-control` token **anywhere in the file** — so `// TODO: add a negative-control later` + a vacuous test satisfied the rule (reproduced by the auditor). Path (a) now requires the token inside an actual **test declaration** (`it`/`test`/`describe` title) — a real inline negative control; a bare comment no longer counts. Files whose coverage lives in siblings or that delegate to a tool use the explicit `META-INVARIANT-EXEMPT` marker (path b).
|
|
454
|
+
- **META-invariant glob-miss (HIGH).** The scan only walked `tests/*-invariant.test.ts`, silently excluding real structural invariants (`no-internal-imports.test.ts`, `lint.test.ts`) and the meta file itself — a dev could dodge the rule by filename. The scan now also covers a curated `EXTRA_STRUCTURAL_FILES` set (`docs-consistency`, `cli-parity`, `lint`, `no-internal-imports`, `meta-invariant-coverage`). `no-internal-imports` got a **real inline NEGATIVE control** (extracted a pure `restrictedImportViolations` matcher; a synthetic restricted-import is flagged, an allowed one isn't); `lint` + `k1-class` carry `META-INVARIANT-EXEMPT` markers (delegation-to-biome / sibling-coverage).
|
|
455
|
+
- **Silent-skip security surfaces (MED).** `security.test.ts` (symlink-escape privacy) + `fts5.test.ts` (FTS5 injection-escaping) `return`ed silently with zero assertions when their precondition (symlink support / better-sqlite3) was absent — green-passing a security surface. Added **CI-GUARD tripwires** (fail loud in CI if the precondition vanishes) + converted the 3 `security.test.ts` symlink skips to visible `ctx.skip()`. Also added a CI-GUARD to `e2e-handlers.test.ts` (the 401-no-bearer auth E2E). Same fix as rc.8's T1.
|
|
456
|
+
- **Per-file FLOORS gaps (LOW).** Added `src/vault.ts` (`branches: 75`; actual 78.03%) — the most security-critical module (path-traversal/symlink/privacy), previously the lone critical module with no per-file gate. Added an `ocr.ts` `lines: 40` floor (actual 44.44%) so the #16 offline-enforcement surface's line coverage can't silently rot under a branches-only floor.
|
|
457
|
+
|
|
458
|
+
### Tests (1002, +5)
|
|
459
|
+
|
|
460
|
+
+1 META-invariant self-test proving the comment/TODO-only bypass is now **rejected** (positive: the EXEMPT path still works); +1 `no-internal-imports` NEGATIVE control; +3 CI-GUARD tripwires (`security`, `fts5`, `e2e-handlers`).
|
|
461
|
+
|
|
462
|
+
### Files changed
|
|
463
|
+
|
|
464
|
+
- `tests/meta-invariant-coverage.test.ts` (tightened path-a check + broadened scan + bypass-rejected self-test), `tests/no-internal-imports.test.ts` (pure matcher + NEGATIVE control), `tests/lint.test.ts` + `tests/k1-class-invariant.test.ts` (EXEMPT markers), `tests/security.test.ts` (CI-GUARD + 3 `ctx.skip()`), `tests/fts5.test.ts` + `tests/e2e-handlers.test.ts` (CI-GUARDs), `scripts/check-per-file-coverage.mjs` (vault.ts + ocr.ts floors).
|
|
465
|
+
- version bump 3.9.0-rc.22 → 3.9.0-rc.23 (7 surfaces); test count 997 → 1002.
|
|
466
|
+
|
|
467
|
+
### Full-audit closure
|
|
468
|
+
|
|
469
|
+
This completes the 3-batch response to the multi-agent state-driven audit: **rc.21** (security — verified ReDoS), **rc.22** (docs drift + structural guards), **rc.23** (test-infra rigor). The automated 10-gate baseline was clean throughout; the `src/` runtime audited exceptionally clean (the only `src/` finding was the rc.21 ReDoS). Net: 4 HIGH + 1 security-MED + several LOW closed, each with a structural defense where one was missing.
|
|
470
|
+
|
|
471
|
+
---
|
|
472
|
+
|
|
473
|
+
## [3.9.0-rc.22] — 2026-05-29
|
|
474
|
+
|
|
475
|
+
> **TL;DR:** **Docs-drift + structural guards (full-audit batch 2/3).** The docs auditor found 2 claim-vs-reality drifts the gates didn't catch: (HIGH) `STABILITY.md` stated the `--reranker-model` default alias is `rerank-multilingual` — but the code default is `rerank-bge` (the **3rd instance** of the exact α-class drift fixed in rc.15 TSDoc + rc.16 CLI help, now in a *packaged semver-contract doc*); (MED) `ROADMAP.md` said "**8** state-driven OIA drift checks" when the canonical count is **10** (Check 9 rc.14, Check 10 rc.20). Both fixed AND each gets a structural guard in `tests/docs-consistency.test.ts` so the class can't recur. **997 tests** (+2 docs-consistency guards).
|
|
476
|
+
|
|
477
|
+
**Patch — docs-drift + structural defense (full-audit batch 2/3). Docs/tests only.**
|
|
478
|
+
|
|
479
|
+
### Fixed
|
|
480
|
+
|
|
481
|
+
- **`STABILITY.md` reranker default α-drift (HIGH).** The "Default models" bullet named `rerank-multilingual` as the `--reranker-model` default; `src/embeddings.ts` defines `DEFAULT_RERANKER_ALIAS = "rerank-bge"` (`rerank-multilingual` is a *valid* catalog alias but NOT the default). Same drift rc.15 fixed in `loadReranker`'s TSDoc and rc.16 in the CLI `--enable-reranker` help — this 3rd instance lived on the packaged semver-contract doc. → `rerank-bge`.
|
|
482
|
+
- **`ROADMAP.md` OIA-check undercount (MED).** "8 state-driven OIA drift checks" → **10** (the lone count straggler; AGENTS/CLAUDE/CHANGELOG were already correct).
|
|
483
|
+
|
|
484
|
+
### Changed (structural defenses — close both classes)
|
|
485
|
+
|
|
486
|
+
- **`tests/docs-consistency.test.ts` (+2 invariants):**
|
|
487
|
+
- **reranker-default α-guard** — reads `DEFAULT_RERANKER_ALIAS` from `src/embeddings.ts` and asserts STABILITY's "Default models" bullet names it AND does not present `rerank-multilingual` as the default. Pins the 3rd-instance class structurally.
|
|
488
|
+
- **OIA-count consistency** — derives the canonical count from `scripts/oia-walk.mjs`'s self-declared `canonical count is "N"` (cross-checked it's ≥10) and asserts every count-stating surface (`AGENTS.md` ×2, `ROADMAP.md`) matches it — so adding an OIA check forces a docs sync.
|
|
489
|
+
|
|
490
|
+
### Tests (997)
|
|
491
|
+
|
|
492
|
+
+2 `it()` in `tests/docs-consistency.test.ts` (the two guards above). Test count 995 → 997 across surfaces.
|
|
493
|
+
|
|
494
|
+
### Files changed
|
|
495
|
+
|
|
496
|
+
- `STABILITY.md` (reranker default → rerank-bge), `ROADMAP.md` (OIA 8→10 + test-count 997), `tests/docs-consistency.test.ts` (+2 guards), test-count surfaces (README/COMPARISON/llms.txt/AGENTS/package.json) → 997.
|
|
497
|
+
- version bump 3.9.0-rc.21 → 3.9.0-rc.22 (7 surfaces).
|
|
498
|
+
|
|
499
|
+
### Deferred to rc.23 (same audit, batch 3/3)
|
|
500
|
+
|
|
501
|
+
Test-infra rigor: meta-invariant comment-bypass + glob-miss (HIGH×2), silent-`return`→`ctx.skip()`+CI-GUARD propagation (security.test.ts/fts5.test.ts), `vault.ts`/`ocr.ts` per-file FLOORS.
|
|
502
|
+
|
|
503
|
+
---
|
|
504
|
+
|
|
505
|
+
## [3.9.0-rc.21] — 2026-05-29
|
|
506
|
+
|
|
507
|
+
> **TL;DR:** **Security — close a verified ReDoS hole the rc.9 guard missed (full-audit response, batch 1/3).** A fresh multi-agent state-driven audit (code + docs + tests, all green on the 10-gate baseline) reproduced ONE genuine exploit: `obsidian_open_questions`'s `isCatastrophicRegex` (rc.9) catches *nested* quantifiers (`(a+)+`) but **not overlapping-alternation** (`(a|a)+`) — the auditor hung V8 >8s with a 200-char-cap-legal pattern, and the tool is always-registered, so any bearer-authenticated `serve-http` client could freeze the event loop (remote DoS). The guard now also rejects **unbounded-quantified AMBIGUOUS alternations** via leading-atom overlap analysis — catching `(a|a)+`, `(a|ab)*`, `(.|a)+`, `((a|a))+`, `(a|)+` while keeping DISJOINT ones like `(a|b|c)+` / `(cat|dog)+` accepted (they match linearly) and the unquantified default-pattern alternation unaffected. **995 tests** (+2 integration; +13 detector cases via the existing data-driven loops). **No CRITICAL/HIGH code findings otherwise — the codebase audited exceptionally clean.**
|
|
508
|
+
|
|
509
|
+
**Patch — security (full-audit batch 1/3). `src/tools/meta.ts` + tests only.**
|
|
510
|
+
|
|
511
|
+
### Fixed
|
|
512
|
+
|
|
513
|
+
- **ReDoS via overlapping-alternation in `obsidian_open_questions` (verified remote DoS).** `isCatastrophicRegex` modelled only "star height ≥ 2"; an unbounded-quantified ambiguous alternation (`(a|a)+`) slipped past it AND the 200-char `MAX_QUESTION_PATTERN_LEN` cap (the exploit pattern is 7 chars). Since the tool is always-registered, a `serve-http` client could peg the single-threaded event loop. The guard now additionally rejects an **unbounded-quantified group whose top-level branches can match a common starting input** (or a nullable branch), decided by a new dependency-free `alternationBodyAmbiguous` (leading-atom overlap — a *sound over-approximation*: it never under-flags a real overlap, and may over-flag a shared-first-char-but-divergent group like `(cat|car)+`, which is the safe direction for a security guard). Ambiguity **bubbles up** through nesting (`((a|a))+`). Disjoint alternations (`(a|b|c)+`, `(cat|dog)+`) and the unquantified default-pattern shape stay accepted. The error message + TSDoc updated to describe both rejected classes.
|
|
514
|
+
|
|
515
|
+
### Tests (995, positive + NEGATIVE controls)
|
|
516
|
+
|
|
517
|
+
- `tests/redos-guard.test.ts` — +10 catastrophic alternation cases (`(a|a)+`, `(a|ab)*`, `(.|a)+`, `(\w|x)+`, `(a|)+`, `((a|a))+`, `(?:a|a)+`, `(cat|car)+`, …), +3 safe-disjoint POSITIVE controls (`(cat|dog)+`, unquantified `(a|b|c)`, the `(?:open|q|todo)\s*` default shape), +2 standalone integration `it()` (the tool rejects a runtime-built `(a|a)+` pattern; accepts a disjoint `(open question|todo)` override). The pre-existing `(a|b|c)+`-is-safe control is the key regression guard — the conservative fix must NOT over-reject disjoint alternations.
|
|
518
|
+
|
|
519
|
+
### Audit baseline (this batch)
|
|
520
|
+
|
|
521
|
+
The full audit's automated baseline was clean: lint, `tsc` strict, version-consistency (7), **all tests + coverage 89.46% lines / 76.02% branches**, OIA (10 checks). The 3 fresh-eyes auditors confirmed the `src/` codebase clean apart from this finding; the remaining audit findings (docs drift, test-infra rigor) ship in rc.22 + rc.23.
|
|
522
|
+
|
|
523
|
+
### Files changed
|
|
524
|
+
|
|
525
|
+
- `src/tools/meta.ts` (`isCatastrophicRegex` + new `splitTopLevelAlternation` / `leadingAtomToken` / `alternationBodyAmbiguous` helpers + error message + TSDoc), `tests/redos-guard.test.ts`.
|
|
526
|
+
- version bump 3.9.0-rc.20 → 3.9.0-rc.21 (7 surfaces); test count 993 → 995.
|
|
527
|
+
|
|
528
|
+
### Deferred to rc.22 / rc.23 (same audit)
|
|
529
|
+
|
|
530
|
+
rc.22: `STABILITY.md` reranker-default α-drift (HIGH) + `ROADMAP.md` OIA-count 8→10 + structural guards. rc.23: meta-invariant comment-bypass + glob-miss (HIGH×2) + silent-skip → `ctx.skip()`+CI-GUARD propagation + `vault.ts`/`ocr.ts` per-file FLOORS.
|
|
531
|
+
|
|
532
|
+
---
|
|
533
|
+
|
|
534
|
+
## [3.9.0-rc.20] — 2026-05-29
|
|
535
|
+
|
|
536
|
+
> **TL;DR:** **CI hardening — kill the recurring `npm ci` flake that just failed a release (sprint RC 12).** The rc.19 release **failed at the assert-CI gate** because the squash-merge commit's `test (24)` leg flaked: `npm ci` → `onnxruntime-node` postinstall → CDN `ETIMEDOUT` (same transient flake as rc.9; the rc.19 PR was all-green, only the main-push re-run flaked). Re-running the job published rc.19 — but a transient network blip should never fail a release. All **10 `npm ci` steps** across the 3 workflows are now wrapped in a **dependency-free bash retry loop** (3 attempts, 15s backoff — no marketplace retry action, so nothing new to SHA-pin per rc.14's supply-chain posture). New **OIA Check 10** fails CI if any bare `- run: npm ci` reappears (detection-power verified: injected one → flags `publish-docs.yml`; clean after). **993 tests unchanged** (workflows + audit-script + docs only).
|
|
537
|
+
|
|
538
|
+
**Patch — CI/supply-chain hardening (sprint RC 12). Workflows/audit-script/docs only; no `src/` runtime change.**
|
|
539
|
+
|
|
540
|
+
### Fixed
|
|
541
|
+
|
|
542
|
+
- **Recurring `npm ci` release-failing flake.** `onnxruntime-node`'s postinstall (`node ./script/install`) downloads its native binary from a CDN that intermittently times out; a bare `- run: npm ci` then fails the whole job — and when it hits the squash-merge commit's CI, `release.yml`'s "assert required CI checks passed" gate correctly refuses to publish (it did, on rc.19). All **10** `npm ci` invocations (`ci.yml` ×8, `release.yml`, `publish-docs.yml`) now run inside:
|
|
543
|
+
```bash
|
|
544
|
+
for n in 1 2 3; do
|
|
545
|
+
npm ci && break
|
|
546
|
+
[ "$n" -eq 3 ] && { echo "::error::npm ci failed after 3 attempts"; exit 1; }
|
|
547
|
+
echo "::warning::npm ci attempt $n failed (transient — e.g. onnxruntime postinstall CDN ETIMEDOUT); retrying in 15s"
|
|
548
|
+
sleep 15
|
|
549
|
+
done
|
|
550
|
+
```
|
|
551
|
+
Dependency-free (a bash loop, not a marketplace retry action) so it adds **no new action to SHA-pin** — consistent with rc.14's pinned-dependencies posture.
|
|
552
|
+
|
|
553
|
+
### Changed (structural defense — close the flake class)
|
|
554
|
+
|
|
555
|
+
- **OIA Check 10 (`NPM-CI-NOT-RETRY-WRAPPED`)** — scans `.github/workflows/*.yml` and fails CI on any line that is exactly a bare `- run: npm ci`. Makes the retry-wrap self-enforcing: a future PR that adds an unwrapped `npm ci` trips the `oia` gate. **Detection power verified non-vacuously**: injecting a bare `npm ci` flags `publish-docs.yml:<line>`; the wrapped form (`npm ci && break` inside `run: |`) is silent. OIA count synced **9 → 10** (oia-walk header + AGENTS.md ×2).
|
|
556
|
+
|
|
557
|
+
### Method note
|
|
558
|
+
|
|
559
|
+
The rc.19 release failure is the *first time* this known flake (documented since rc.9) actually **blocked a publish** rather than just a PR check — which is exactly the signal that "re-run by hand" was no longer an acceptable response. Fixed the class (all 10 steps) + a structural guard (Check 10), not the instance.
|
|
560
|
+
|
|
561
|
+
### Files changed
|
|
562
|
+
|
|
563
|
+
- `.github/workflows/{ci,release,publish-docs}.yml` (10 `npm ci` → retry loop), `scripts/oia-walk.mjs` (Check 10 + header 9→10 / 13→14 walks / marker order), `AGENTS.md` (OIA count 9→10 ×2).
|
|
564
|
+
- version bump 3.9.0-rc.19 → 3.9.0-rc.20 (7 surfaces); test count unchanged (993).
|
|
565
|
+
|
|
566
|
+
---
|
|
567
|
+
|
|
568
|
+
## [3.9.0-rc.19] — 2026-05-29
|
|
569
|
+
|
|
570
|
+
> **TL;DR:** **LongMemEval retrieval harness (sprint RC 11 — the v3.10 credibility lever, engineering half).** [LongMemEval](https://github.com/xiaowu0162/LongMemEval) (Wu et al. 2024) is the long-term-memory benchmark Mem0/Zep publish against; no Obsidian-MCP has any LongMemEval-derived number. New [`scripts/bench-longmemeval.mjs`](https://github.com/oomkapwn/enquire-mcp/blob/main/scripts/bench-longmemeval.mjs) materializes each question's haystack sessions into a throwaway vault, indexes with FTS5, runs `searchHybrid`, and scores **`recall@k` / `MRR` / `NDCG@k` of the answer-bearing session(s)** (reusing `src/eval.ts`), aggregated per `question_type`. It measures **retrieval quality, NOT end-to-end QA accuracy** — enquire is a retriever, not an answerer; claiming a QA number would be an overclaim. The dataset is **not** committed (size + licensing); the **headline numbers are intentionally NOT published** — they're maintainer-gated (a full reference-hardware run + methodology review, per the project's "measured, reproducible, reviewed — never a placeholder" bar). **982 → 993 tests** (+11 pure-function tests, positive + NEGATIVE controls).
|
|
571
|
+
|
|
572
|
+
**Patch — discoverability/credibility infrastructure (sprint RC 11). Scripts/tests/docs only; no `src/` runtime change.**
|
|
573
|
+
|
|
574
|
+
### Added
|
|
575
|
+
|
|
576
|
+
- **`scripts/bench-longmemeval.mjs`** — LongMemEval **retrieval** benchmark harness. Per question: materialize haystack sessions → one note each in a temp vault → `syncFtsIndex` → `searchHybrid` → score `recall@k`/`MRR`/`NDCG@k` of the answer session(s) (the same `src/eval.ts` metrics as the rest of `docs/benchmarks.md`), aggregated overall + per `question_type`; abstention (`*_abs`) questions counted separately. Pure helpers (`sessionToMarkdown`, `sessionNotePath`, `relevantSessionPaths`, `isAbstention`, `aggregateByType`) exported for unit testing; CLI guarded by `isEntrypoint`. `--dataset <path> [--limit N] [--k 10] [--embeddings]`. Missing dataset → exit 2 with download guidance (it's not committed).
|
|
577
|
+
- **`npm run bench:longmemeval`** script.
|
|
578
|
+
- **`docs/benchmarks.md` → "LongMemEval retrieval (external benchmark)"** section: the retrieval-vs-QA framing, the run command, and an explicit **"numbers pending a full maintainer run"** status (no fabricated/placeholder figures — the LongMemEval headline is the credibility centerpiece and goes through the same measured-and-reviewed bar as every other number).
|
|
579
|
+
- **`.gitignore`** guard (`longmemeval*.json`, `longmemeval_*/`) so a maintainer's dataset download can't be accidentally committed.
|
|
580
|
+
|
|
581
|
+
### Tests added (+11 new it() blocks, positive + NEGATIVE controls) — 982 → 993
|
|
582
|
+
|
|
583
|
+
- `tests/longmemeval-harness.test.ts` (new) — `sessionNotePath` (safe-id + **path-traversal NEGATIVE control**), `sessionToMarkdown` (role-labelled turns + **malformed/empty-session NEGATIVE control**), `relevantSessionPaths` (explicit `answer_session_ids` + `has_answer` fallback + **empty-on-abstention NEGATIVE control**), `isAbstention` (`_abs` + NEGATIVE), `aggregateByType` (per-type averages + hit-rate + **empty-input NEGATIVE control**). The full benchmark run (needs the uncommitted dataset + heavy compute) is intentionally not a CI gate; the *logic that decides what's scored and how it aggregates* is.
|
|
584
|
+
|
|
585
|
+
### Scope note — what ships vs. what's gated
|
|
586
|
+
|
|
587
|
+
The **harness + tests ship now** (verifiable engineering). The **published LongMemEval score**, forgetting-aware staleness, and "grounded in your knowledge, not extracted" messaging remain **v3.10** — the score specifically is maintainer-gated (download + full run + review) so the credibility centerpiece is never an unreviewed auto-publish.
|
|
588
|
+
|
|
589
|
+
### Files changed
|
|
590
|
+
|
|
591
|
+
- `scripts/bench-longmemeval.mjs` (new), `tests/longmemeval-harness.test.ts` (new, +11), `docs/benchmarks.md` (LongMemEval section), `package.json` (`bench:longmemeval` script), `.gitignore` (dataset guard).
|
|
592
|
+
- version bump 3.9.0-rc.18 → 3.9.0-rc.19 (7 surfaces); test count 982 → 993.
|
|
593
|
+
|
|
594
|
+
---
|
|
595
|
+
|
|
596
|
+
## [3.9.0-rc.18] — 2026-05-29
|
|
597
|
+
|
|
598
|
+
> **TL;DR:** **Brand-integrity: the social card stopped lying about SLSA (sprint RC 10).** State-driven read of `assets/social-preview.svg` — the GitHub social card, the single most-shared visual of the repo — caught a **`SLSA-3`** trust badge (line 137). That's a **residual instance of overclaim #15** (rc.7 downgraded SLSA-3 → SLSA Build L2 everywhere because `release.yml` only does `npm publish --provenance`); rc.7's sweep AND OIA Check 4d's original file scope both missed the SVG, so the card advertised a false security level for 11 RCs. Fixed the badge (`SLSA-3` → `SLSA L2`), re-rendered the PNG, and **extended OIA Check 4d's `claimFiles` to include `assets/social-preview.svg`** so the surface is permanently guarded. Detection power verified (injected `SLSA-3` → Check 4d flags `social-preview.svg:137`; clean after fix). **982 tests unchanged** (assets + audit-script only).
|
|
599
|
+
|
|
600
|
+
**Patch — brand-integrity + structural defense (sprint RC 10). Assets/audit-script only; no `src/` runtime change.**
|
|
601
|
+
|
|
602
|
+
### Fixed
|
|
603
|
+
|
|
604
|
+
- **`assets/social-preview.svg` claimed `SLSA-3` (overclaim #15, residual instance).** The bottom trust-signal row badge said `SLSA-3` — a level the build doesn't earn (`npm publish --provenance` = SLSA Build **L2**; L3 needs an isolated builder via `slsa-framework/slsa-github-generator`). This is the same overclaim rc.7 retracted across README/package.json/llms.txt/COMPARISON/STABILITY, but the **social card was outside both rc.7's sweep and OIA Check 4d's scope**, so it persisted on the most externally-visible surface. → `SLSA L2`. `assets/social-preview.png` re-rendered from the corrected SVG via `scripts/render-social-preview.mjs`.
|
|
605
|
+
- **`src/pdf.ts:13` asserted pdfjs-dist is "SLSA-3 published" (unverified third-party claim).** A repo-wide sweep for the SLSA-3 class (triggered by the SVG find, per the root-cause-sweep rule) surfaced a source comment claiming the **pdfjs-dist dependency** ships SLSA-3 provenance — something we never verified. Per the project rule ("any SLSA-level claim must point to backing evidence, else downgrade"), the unverified clause was removed (the comment's real point — pure-JS, no native deps, Apache-2.0, optional — is unchanged). All other repo `SLSA-3` hits are legitimate: CLAUDE.md/ROADMAP history + the "earn real L3" roadmap target, `oia-walk.mjs`'s own detector regex, and `docs/audits/*` point-in-time audit artifacts (excluded from OIA currency + npm — rewriting them would falsify the historical record).
|
|
606
|
+
|
|
607
|
+
### Changed (structural defense — close the recursion)
|
|
608
|
+
|
|
609
|
+
- **OIA Check 4d (`SLSA-LEVEL-OVERCLAIM`) `claimFiles` now includes `assets/social-preview.svg`.** Root-cause: the SLSA-level check guarded the doc surfaces but not the rendered-asset surface. Adding the SVG makes the social-card SLSA badge self-enforcing (CI fails if it ever drifts to L3/SLSA-3 again). **Detection power verified non-vacuously**: with `SLSA-3` injected the check flags `assets/social-preview.svg:137`; with `SLSA L2` it's silent. Mirrors the v3.8.0-rc.11 "drift findings demand a full-surface sweep + structural defense" rule.
|
|
610
|
+
|
|
611
|
+
### Method note
|
|
612
|
+
|
|
613
|
+
This is a textbook **state-driven** catch: a change-driven sweep (rc.7) fixed the class on the files it was looking at; reading *every* file as it exists on disk — including a rendered-asset source — surfaced the one instance it missed. The fix isn't just the instance (SVG badge) but the **defense-scope gap** (Check 4d file list), so the class is closed, not just the symptom.
|
|
614
|
+
|
|
615
|
+
### Files changed
|
|
616
|
+
|
|
617
|
+
- `assets/social-preview.svg` (SLSA-3 → SLSA L2), `assets/social-preview.png` (re-rendered), `scripts/oia-walk.mjs` (Check 4d `claimFiles` += social-preview.svg), `src/pdf.ts` (comment-only: dropped unverified pdfjs-dist "SLSA-3 published" — dist output byte-identical, no runtime change).
|
|
618
|
+
- version bump 3.9.0-rc.17 → 3.9.0-rc.18 (7 surfaces); test count unchanged (982).
|
|
619
|
+
|
|
620
|
+
### Deferred (repo-page polish, lower priority)
|
|
621
|
+
|
|
622
|
+
Social-preview stat-pill redesign (would add new numeric-claim drift surface — needs a docs-consistency invariant in the same PR), README hero `claude mcp add` one-liner, `server.json` `categories`/`websiteUrl` (verify against the 2025-12-11 schema first). Then **v3.10 LongMemEval** (the #1 credibility lever).
|
|
623
|
+
|
|
624
|
+
---
|
|
625
|
+
|
|
626
|
+
## [3.9.0-rc.17] — 2026-05-29
|
|
627
|
+
|
|
628
|
+
> **TL;DR:** **AI-search discoverability: Schema.org `@graph` structured data (sprint RC 9).** The single biggest lever for getting cited by Google AI Overviews / Perplexity / Bing Copilot is machine-readable structured data, and the highest-citation type is **FAQPage**. `scripts/inject-jsonld.mjs` (run at GH-Pages publish time) is upgraded from a lone `SoftwareApplication` node to a Schema.org **`@graph`** with three cross-linked nodes: an enriched **SoftwareApplication** (now with `featureList` + `maintainer`), a **SoftwareSourceCode** node (`codeRepository`/`runtimePlatform`/`targetProduct` → the app), and a **FAQPage** carrying the README's 6 Q&A pairs. Plus a `glama.json` (`maintainers: [oomkapwn]`) so the Glama.ai crawler can attribute + index the server instead of withholding it from search. The builder is refactored into a pure, exported `buildJsonLdGraph(pkg)` so it's unit-tested (deterministic — no dates/RNG). **975 → 982 tests** (+7, positive + NEGATIVE controls).
|
|
629
|
+
|
|
630
|
+
**Patch — discoverability (sprint RC 9). Docs/scripts/config only; no `src/` runtime change.**
|
|
631
|
+
|
|
632
|
+
### Added
|
|
633
|
+
|
|
634
|
+
- **Schema.org `@graph` JSON-LD** (`scripts/inject-jsonld.mjs`, expanded). Three nodes:
|
|
635
|
+
- **SoftwareApplication** — now includes `featureList` (8 differentiators), `maintainer`, `applicationSubCategory: "Model Context Protocol (MCP) server"`, stable `@id`.
|
|
636
|
+
- **SoftwareSourceCode** — `codeRepository` (cleaned of `git+`/`.git`), `runtimePlatform`, `programmingLanguage`, `targetProduct` cross-referencing the SoftwareApplication `@id`.
|
|
637
|
+
- **FAQPage** — the README "## ❓ FAQ" Q&A as `Question`/`acceptedAnswer` pairs (highest AI-citation structured-data type).
|
|
638
|
+
- **`glama.json`** at repo root (`$schema` + `maintainers: ["oomkapwn"]`) — lets the Glama.ai MCP directory attribute the server to its maintainer and index it (claimed servers move from "withheld from search" to discoverable for Glama's user base).
|
|
639
|
+
|
|
640
|
+
### Changed
|
|
641
|
+
|
|
642
|
+
- `scripts/inject-jsonld.mjs` refactored: `buildJsonLdGraph(pkg)` + `FAQ_ENTRIES` are now **exported pure** functions/data (CLI behavior guarded behind an `isEntrypoint` check), so the JSON-LD is unit-testable. The injected `<script type="application/ld+json">` now carries a `@graph`; the idempotency marker (`application/ld+json`) is unchanged, so `publish-docs.yml` needs no edit.
|
|
643
|
+
|
|
644
|
+
### Tests added (+7 new it() blocks, positive + NEGATIVE controls) — 975 → 982
|
|
645
|
+
|
|
646
|
+
- `tests/jsonld.test.ts` (new) — `buildJsonLdGraph`: `@graph` has exactly the 3 expected `@type`s; SoftwareApplication carries `softwareVersion === package.json` + `featureList` + `maintainer`; `SoftwareSourceCode.targetProduct["@id"]` cross-refs the app `@id` + repo URL is clean (no `git+`/`.git`); FAQPage mirrors `FAQ_ENTRIES` with **non-empty Q + A (NEGATIVE control on empty answers)**; the graph is JSON-serializable. Plus a **README-FAQ-count drift guard**: `FAQ_ENTRIES.length` must equal the README FAQ bold-question count (so a 7th README FAQ that's not mirrored into the JSON-LD fails CI), and every entry is well-formed (`q` ends with `?`, `a` non-empty).
|
|
647
|
+
|
|
648
|
+
### Files changed
|
|
649
|
+
|
|
650
|
+
- `scripts/inject-jsonld.mjs` (expanded + exported builder), `glama.json` (new), `tests/jsonld.test.ts` (new, +7).
|
|
651
|
+
- version bump 3.9.0-rc.16 → 3.9.0-rc.17 (7 surfaces); test count 975 → 982.
|
|
652
|
+
|
|
653
|
+
### Deferred to rc.18 (repo-page polish)
|
|
654
|
+
|
|
655
|
+
Social-preview regen (`scripts/render-social-preview.mjs` → stat-pill design: 44 tools / 982 tests / +15.5 NDCG@10), README hero `claude mcp add` one-liner + canonical-URL comments, `server.json` `categories`/`keywords`, then **v3.10 LongMemEval** harness (the #1 credibility lever).
|
|
656
|
+
|
|
657
|
+
---
|
|
658
|
+
|
|
659
|
+
## [3.9.0-rc.16] — 2026-05-29
|
|
660
|
+
|
|
661
|
+
> **TL;DR:** **Correctness batch 2 (sprint RC 8) — user-facing correctness + honesty.** Clears the rc.15-deferred backlog plus the rc.15 post-ship self-audit. (1) `doctor` now actually applies the privacy filter it claimed (`--exclude-glob`/`--read-paths` were never wired — it counted all files yet labeled the count "privacy filter applied" — **P2-12**). (2) `eval` distinguishes an *errored* query from a genuine zero-relevance one (new `query_errors` count + per-query `error` flag + a banner warning — a benchmark's means were silently deflatable by infra hiccups). (3) The stateless HTTP handler now wires its per-request cleanup **before** `connect()`, so a connect failure no longer leaks the McpServer + transport (parity with the stateful path's close discipline). (4) `--ocr-pdfs` warns instead of silently no-op'ing when `--watch` or the embed-db is absent. (5) rc.15's `converged` flag is now actually **surfaced** to MCP callers, and a stale "`+5-10 NDCG@10`" reranker undersell in CLI `--help` (missed by rc.12's docs-only sweep) is corrected to the measured **+15.5 / +24.7**. The deferred `tools/search.ts` "citation mis-attribution" item was **investigated and found to be a non-issue** (snippet/line/chunk/kind all follow one consistent `bm25 ?? embeddings ?? tfidf` precedence). **970 → 975 tests** (+5, positive + NEGATIVE controls).
|
|
662
|
+
|
|
663
|
+
**Patch — audit-driven correctness, batch 2 (sprint RC 8).**
|
|
664
|
+
|
|
665
|
+
### Fixed
|
|
666
|
+
|
|
667
|
+
- **`doctor` ignored the privacy filter while claiming to apply it (P2-12).** `runDoctor` built `new Vault(opts.vault)` with no `excludeGlobs`/`readPaths` — so it walked the *unfiltered* vault, counted every file, and labeled the count `"(privacy filter applied)"`. A privacy-conscious user verifying setup got false reassurance. Now `RunDoctorOptions` accepts `excludeGlobs`/`readPaths`, the CLI `doctor` command exposes `--exclude-glob`/`--read-paths`, the count is honest (`"(after privacy filter)"` only when one is set), and a new `privacy` check reports the active pattern counts — or surfaces a config **error** (instead of crashing) on an empty-after-trim glob.
|
|
668
|
+
- **`eval` conflated errored queries with zero-relevance hits.** A query that threw in `searchHybrid` was pushed to `per_query` with all-zero scores and counted in the means — indistinguishable from a genuine miss, silently deflating published NDCG/Recall/MRR. New `EvalResult.query_errors` count + per-query `error?: true` flag + a `formatEvalResult` banner warning ("re-run before publishing"). Means still include the zeros (you don't get to drop hard queries that crashed) but the deflation is now **visible**.
|
|
669
|
+
- **Stateless HTTP per-request cleanup leaked on connect failure (parity).** `handleStatelessRequest` registered `res.on("close", cleanup)` *after* `await server.connect(transport)`, so a connect throw skipped straight to the catch and the freshly-built McpServer + transport were never closed. Cleanup is now wired **before** connect, made idempotent (`cleanedUp` guard) + error-safe (`.catch`), and also invoked in the catch — matching the stateful path's close discipline (P2-10).
|
|
670
|
+
- **`--ocr-pdfs` was a silent no-op in two cases.** Passed without `--watch` (the flag only acts on the watcher path) → now warns + ignores. Passed with `--watch` but no embed-db (OCR'd text has nowhere to be indexed) → now warns + continues FTS5-only, instead of the block being skipped inside `if (existsSync(embedFile))` with zero feedback.
|
|
671
|
+
|
|
672
|
+
### rc.15 post-ship self-audit (same-class re-sweep)
|
|
673
|
+
|
|
674
|
+
- **`converged` was computed but never surfaced.** rc.15 added `CommunityResult.converged` "so callers can surface this" — but the `obsidian_get_communities` handler dropped it. Now in the tool output; tool description corrected ("`iterations` until convergence" → "`iterations` (greedy passes run) and `converged`").
|
|
675
|
+
- **α-class comment drift (bases.ts).** The v3.6.2 HN-2 comment still framed the unbounded warn-Set as "fine" ("one log line each") right next to rc.15's `MAX_WARNED_PREDICATES` cap that exists *because* a distinct-predicate stream broke that reasoning. Comment corrected.
|
|
676
|
+
- **Reranker undersell in CLI `--help` (missed instance).** `--enable-reranker` help still said the generic "+5-10 NDCG@10 typical"; rc.12's "corrected everywhere" sweep covered `docs/` but not `src/` CLI strings. → measured **≈+15.5 NDCG@10 / +24.7 MRR (60-query ablation)**.
|
|
677
|
+
|
|
678
|
+
### Investigated — no change (empirical rejection)
|
|
679
|
+
|
|
680
|
+
- **`tools/search.ts` "citation line/kind mis-attribution across rankers"** (rc.15-deferred hypothesis): traced the final-hit assembly — `snippet`, `line_start`/`line_end`, `chunk_index`, and `kind` all derive from the same `bm25 ?? embeddings ?? tfidf` precedence (TF-IDF carries no line/kind, so a TF-IDF-only hit reports `line: undefined` + `kind: "md"`, never a *cross-ranker mix*). `kind` is a file-level property and can't conflict across signals. Current `main` is consistent; no fix warranted.
|
|
681
|
+
|
|
682
|
+
### Tests added (+5 new it() blocks, positive + NEGATIVE controls) — 970 → 975
|
|
683
|
+
|
|
684
|
+
- `tests/eval.test.ts` — errored-query: `query_errors === 1`, per-query `error === true`, banner contains "errored", successful query `error` undefined (NEGATIVE); + an all-success NEGATIVE control (`query_errors === 0`, no banner). `makeResult()` literal updated for the new field.
|
|
685
|
+
- `tests/doctor.test.ts` — privacy-active (ok check + "after privacy filter" count), no-filter NEGATIVE control (no `privacy` check, no false claim), empty-glob error path (`ready === false`).
|
|
686
|
+
- `tests/http-transport.test.ts` — 6 sequential stateless requests each 200 (exercises per-request build→connect→cleanup repeatedly).
|
|
687
|
+
- `tests/e2e-handlers.test.ts` — `converged` surfaced in the `obsidian_get_communities` MCP output.
|
|
688
|
+
|
|
689
|
+
### Files changed
|
|
690
|
+
|
|
691
|
+
- `src/doctor.ts` (privacy opts + check + honest count), `src/cli.ts` (doctor `--exclude-glob`/`--read-paths`; reranker help number), `src/eval.ts` (`query_errors` + `error` + banner), `src/http-transport.ts` (stateless cleanup parity), `src/server.ts` (two `--ocr-pdfs` warnings), `src/tool-registry.ts` (`converged` surfaced + description), `src/bases.ts` (HN-2 comment).
|
|
692
|
+
- tests: eval (+2 + literal), doctor (+3), http-transport (+1), e2e-handlers (+1 assertion). `scripts/check-per-file-coverage.mjs` bases.ts comment 73.17% → 74.71%.
|
|
693
|
+
- version bump 3.9.0-rc.15 → 3.9.0-rc.16 (7 surfaces); test count 970 → 975.
|
|
694
|
+
|
|
695
|
+
---
|
|
696
|
+
|
|
697
|
+
## [3.9.0-rc.15] — 2026-05-29
|
|
698
|
+
|
|
699
|
+
> **TL;DR:** **Correctness cleanup (sprint RC 7).** Three MEDIUM/LOW findings from the audit: `bases.ts`'s warn-once dedup `Set` grew without bound on a stream of distinct malformed `.base` predicates (slow memory leak on a long-lived `serve`); `detectCommunities` gave no signal when Louvain hit the `MAX_PASSES=50` cap without converging (callers couldn't tell a sub-optimal partition); and `loadReranker`'s TSDoc claimed the default alias is `rerank-multilingual` when it's actually `rerank-bge` (α-class drift). **966 → 970 tests** (+4, positive + NEGATIVE controls).
|
|
700
|
+
|
|
701
|
+
**Patch — audit-driven correctness (sprint RC 7).**
|
|
702
|
+
|
|
703
|
+
### Fixed
|
|
704
|
+
|
|
705
|
+
- **`bases.ts` unbounded warn-Set (memory growth).** `warnedUnknownPredicates` `.add()`ed every distinct unevaluated predicate forever. A `.base`/DQL query with many unique malformed predicates (attacker- or agent-controlled) grew it without bound for the process lifetime. New exported `boundedSetAdd(set, value, max)` caps it at `MAX_WARNED_PREDICATES`=1000 (past the cap a distinct predicate may re-warn once — acceptable vs. unbounded memory).
|
|
706
|
+
- **`communities.ts` convergence signal.** `CommunityResult` gains **`converged: boolean`** — true when Louvain reached a stable partition (a pass made no moves), false when it exited on the `MAX_PASSES` cap with moves pending (valid but possibly sub-optimal). Derived from the loop's final `!changed`; the edgeless short-circuit reports `converged: true, iterations: 0`.
|
|
707
|
+
- **`embeddings.ts` reranker-default TSDoc.** `loadReranker`'s `@param` said `default: "rerank-multilingual"`; the real `DEFAULT_RERANKER_ALIAS` is `"rerank-bge"`. Corrected (published TypeDoc/IDE-hover was lying — α-class).
|
|
708
|
+
|
|
709
|
+
### Tests added (+4, positive + NEGATIVE controls)
|
|
710
|
+
|
|
711
|
+
- `tests/bases.test.ts` — `boundedSetAdd`: adds under cap (POSITIVE), no-grow on duplicate, **refuses to grow past the cap (NEGATIVE control)**, `MAX_WARNED_PREDICATES` sanity.
|
|
712
|
+
- `tests/communities.test.ts` — `converged` asserted on the edgeless path (`true`, `iterations === 0`) + a clustered graph (`true`, `iterations < 50`).
|
|
713
|
+
|
|
714
|
+
### Deferred to rc.16 (correctness batch 2)
|
|
715
|
+
|
|
716
|
+
`tools/search.ts` citation line/kind mis-attribution across rankers, `eval.ts` `query_errors` count, `doctor` privacy-glob flags (P2-12), `http-transport.ts` stateless-handler cleanup parity, `server.ts` `--ocr-pdfs`-no-embed-db warning — each needs heavier integration-test setup; batched next.
|
|
717
|
+
|
|
718
|
+
### Files changed
|
|
719
|
+
|
|
720
|
+
- `src/bases.ts` (`boundedSetAdd` + cap), `src/communities.ts` (`converged`), `src/embeddings.ts` (TSDoc).
|
|
721
|
+
- `tests/bases.test.ts` (+4), `tests/communities.test.ts` (+assertions).
|
|
722
|
+
- test count 966 → 970 across README/COMPARISON/llms.txt/AGENTS/package.json/ROADMAP.
|
|
723
|
+
- version bump 3.9.0-rc.14 → 3.9.0-rc.15 (7 surfaces).
|
|
724
|
+
|
|
725
|
+
---
|
|
726
|
+
|
|
727
|
+
## [3.9.0-rc.14] — 2026-05-29
|
|
728
|
+
|
|
729
|
+
> **TL;DR:** **Supply-chain: SHA-pin every GitHub Action + a structural guard so they can't drift back (sprint RC 6).** Floating action tags (`uses: actions/checkout@v6`) can be silently retagged to malicious code — the OpenSSF "Pinned-Dependencies" check + this project's supply-chain brand (SLSA L2 + signed provenance) call for commit-SHA pins. All **28 action refs across the 4 workflows** are now pinned to their exact current 40-hex commit SHA (behavior identical) with a `# vN` comment for humans + Dependabot. New **OIA Check 9** fails CI if any third-party action ever uses a floating tag again — making the pin self-enforcing. **Workflows + audit-script + docs only; 966 tests unchanged.**
|
|
730
|
+
|
|
731
|
+
**Patch — audit-driven supply-chain (sprint RC 6).**
|
|
732
|
+
|
|
733
|
+
### Fixed
|
|
734
|
+
|
|
735
|
+
- **SHA-pin all GitHub Actions (28 refs / 4 workflows).** `actions/checkout@v6`, `actions/setup-node@v6`, `actions/upload-artifact@v7`, `actions/configure-pages@v6`, `actions/upload-pages-artifact@v5`, `actions/deploy-pages@v5` → each pinned to the exact commit SHA the tag currently resolves to (resolved via `gh api repos/actions/<x>/commits/<tag>`), with a trailing `# vN` comment. Identical behavior today; immune to tag-moving supply-chain attacks. Spans `ci.yml` (19), `publish-docs.yml` (5), `release.yml` (2), `dist-tag-cleanup.yml` (2).
|
|
736
|
+
|
|
737
|
+
### Structural defense
|
|
738
|
+
|
|
739
|
+
- **OIA Check 9 — Actions SHA-pin.** Scans every `.github/workflows/*.yml` `uses:` line; flags any third-party action NOT pinned to a 40-hex commit SHA (local `./.github/...` reusable refs exempt). **Verified non-vacuous** (all 28 current refs pass — silent for the right reason) **and with detection power** (a floating `@v6` / `@main` would flag). Makes the pin permanent: a future unpinned action fails CI. This is the 9th numbered OIA walk (header + AGENTS + CLAUDE counts synced 8 → 9).
|
|
740
|
+
|
|
741
|
+
### Deferred (tracked)
|
|
742
|
+
|
|
743
|
+
OpenSSF Scorecard workflow + `dependency-review-action` on PRs — additive new workflows (each itself SHA-pinned) → a follow-up supply-chain RC. SHA-pinning is the highest-value item (the concrete hardening + the Scorecard "Pinned-Dependencies" win) and ships here first.
|
|
744
|
+
|
|
745
|
+
### Files changed
|
|
746
|
+
|
|
747
|
+
- `.github/workflows/{ci,publish-docs,release,dist-tag-cleanup}.yml` — 28 action refs SHA-pinned.
|
|
748
|
+
- `scripts/oia-walk.mjs` — Check 9 + header enumeration (8 → 9 numbered, 12 → 13 blocks).
|
|
749
|
+
- `AGENTS.md`, `CLAUDE.md` — OIA check count 8 → 9.
|
|
750
|
+
- version bump 3.9.0-rc.13 → 3.9.0-rc.14 (7 surfaces). **966 tests unchanged.**
|
|
751
|
+
|
|
752
|
+
---
|
|
753
|
+
|
|
754
|
+
## [3.9.0-rc.13] — 2026-05-29
|
|
755
|
+
|
|
756
|
+
> **TL;DR:** **State-driven docs hygiene (sprint RC 5).** Clears the deferred-from-rc.12 backlog of stale-fragment fixes the file-by-file audit found — none CI-blocking, all honesty/credibility: CITATION.cff named the wrong default models; a script comment still credited the retracted "Cursor external audit" (overclaim #11); AGENTS.md said the version gate checks "5 surfaces" (it's 7) and listed a phantom `bench` CLI subcommand; several **packaged docs** (README, docs/api.md, docs/benchmarks.md — all ship in the npm tarball) linked to repo paths that **don't** ship (`../tests/`, `../src/`, `../bench/`, `./AGENTS.md`, `./ROADMAP.md`, `./llms.txt`, `.github/…`) → 404 for npm-page readers; and the rc.7 CHANGELOG entry's forward-claim ("#16 → rc.8, H1 → rc.9") was left stale after the rc.8 pivot re-sequenced them to rc.10/rc.11. **Docs/metadata/script only; 966 tests unchanged.**
|
|
757
|
+
|
|
758
|
+
**Patch — audit-driven docs hygiene (sprint RC 5).**
|
|
759
|
+
|
|
760
|
+
### Fixed
|
|
761
|
+
|
|
762
|
+
- **CITATION.cff model names.** Said "enquire-mcp uses bge-multilingual-gemma2 and bge-reranker-base" — `bge-multilingual-gemma2` isn't even in the model catalog. Corrected to the actual defaults: `paraphrase-multilingual-MiniLM-L12-v2` (embeddings) + `bge-reranker-base` (reranker). (Consumed by Zenodo/OpenAlex/Scholar — a factually wrong metadata claim.)
|
|
763
|
+
- **Retracted-Cursor-audit comment.** `scripts/check-version-consistency.mjs` header still credited the server.json gate to a "Cursor external audit on rc.15" — that attribution was retracted as overclaim #11 (the doc was for a different project). Re-credited to the M-REG-1 external-audit finding.
|
|
764
|
+
- **AGENTS.md drift.** "version sync across 5 surfaces" → **7** (×4 incl. the hyphenated "5-surface" + the surface list, which now names server.json version + packages[0]); dropped the phantom `bench` CLI subcommand from the architecture comment (no such subcommand) and listed the real `install-ocr-lang` instead.
|
|
765
|
+
- **Broken packaged-doc links → absolute GitHub URLs.** README (`llms.txt`, `AGENTS.md`, `ROADMAP.md`, `publish-docs.yml`), docs/api.md (`../scripts/bench-search.mjs`), docs/benchmarks.md (`../tests/…`, `../src/eval.ts` ×2, `../bench/benchmarks.json`, `./api-reference/` → the GH Pages URL) — all 404'd in the npm tarball (those paths aren't in `package.json#files`). Now absolute `github.com/.../blob/main/…` links that resolve everywhere.
|
|
766
|
+
- **CHANGELOG rc.7 forward-claim.** Added an inline "re-sequenced" note: #16 actually shipped in rc.10 and H1 in rc.11 (the rc.8 integrity-batch pivot pushed both back two RCs); the original "ships in rc.8 / rc.9" lines are preserved as history.
|
|
767
|
+
|
|
768
|
+
### Deferred (tracked)
|
|
769
|
+
|
|
770
|
+
`ROADMAP`/`AGENTS` into `scope-completeness-audit.mjs` AUDIT_FILES (needs a coordinated docs-consistency assertion so the numbers are actually verified, not just "claimed covered") + extending OIA Check 3's CLI-subcommand scan to AGENTS.md → a later structural RC. Supply-chain (SHA-pin Actions + OpenSSF Scorecard) → rc.14. Correctness cleanup (bases Set leak, search citation, eval errors, doctor globs, stateless-HTTP cleanup) → rc.15.
|
|
771
|
+
|
|
772
|
+
### Files changed
|
|
773
|
+
|
|
774
|
+
`CITATION.cff`, `scripts/check-version-consistency.mjs`, `AGENTS.md`, `README.md`, `docs/api.md`, `docs/benchmarks.md`, `CHANGELOG.md` (rc.7 note); version bump 3.9.0-rc.12 → 3.9.0-rc.13 (7 surfaces). **966 tests unchanged.**
|
|
775
|
+
|
|
776
|
+
---
|
|
777
|
+
|
|
778
|
+
## [3.9.0-rc.12] — 2026-05-29
|
|
779
|
+
|
|
780
|
+
> **TL;DR:** **Claim-accuracy: a structural RC-level currency guard + the stale-doc instances it surfaces (sprint RC 4).** The audit's root-cause theme — "the stale-claim findings stem from a defense gap" — gets its second structural fix (the first was rc.10's OIA Check 4e for OCR). OIA Check 7 only compared **major.minor** (so `v3.9.0-rc.3` read as "current" because `3.9 == 3.9`), letting a pinned "currently v3.9.0-rc.N" drift every release. New **RC-level sub-check** compares the **full** version: a "currently / valid as of vX.Y.Z-rc.N" claim must match the exact current version. It immediately caught 3 stale instances (README, api.md, benchmarks.md, all pinned to rc.3/rc.6); all rephrased to version-agnostic. Also closes the **reranker-number undersell** that rc.7's "corrected everywhere" sweep missed (4 sites still said the generic "+5-10 NDCG@10" vs the measured **+15.5 NDCG@10 / +24.7 MRR**). **Docs + audit-script only; 966 tests unchanged.**
|
|
781
|
+
|
|
782
|
+
**Patch — audit-driven claim-accuracy (sprint RC 4).**
|
|
783
|
+
|
|
784
|
+
### Fixed
|
|
785
|
+
|
|
786
|
+
- **RC-level currency drift (structural + instances).** `scripts/oia-walk.mjs` Check 7 gains an RC sub-check: `/(?:currently|(?:still )?valid as of) vX.Y.Z-rc.N/` must equal the exact `package.json` version (a tombstone-verb-after-version skip avoids flagging "vX shipped" history; bare "As of vX, <feature> ships" is excluded as a *since* claim). **Detection-power verified**: with the instances still stale it flagged README:280, docs/api.md:5, docs/benchmarks.md:3; after rephrasing to version-agnostic ("the latest release candidate — see CHANGELOG", "still valid through the v3.9.0-rc cascade", `3.9.0-rc.N` placeholder) it's silent. The api.md RC feature-list (rc.1/rc.2/rc.3) — already incomplete (missing rc.10/rc.11) and unmaintainable — was replaced with a CHANGELOG pointer.
|
|
787
|
+
- **Reranker number undersell (brand credibility).** 4 surfaces (docs/api.md ×2, docs/QUICKSTART.md, docs/COMPARISON.md) still claimed the generic literature figure "+5-10 NDCG@10" for our BGE reranker; corrected to the **measured +15.5 NDCG@10 / +24.7 MRR (60-query ablation)** that COMPARISON's headline + benchmarks.md already report. (benchmarks.md:396's "+5-10 across BEIR" is a legitimate *literature* citation about rerankers in general, not our self-claim — left as-is.)
|
|
788
|
+
|
|
789
|
+
### Deferred to rc.13 (state-driven backlog, batched with the correctness cleanup)
|
|
790
|
+
|
|
791
|
+
CITATION.cff model names, the retracted-Cursor-audit comment in `check-version-consistency.mjs`, AGENTS.md "5 surfaces"→7 + the phantom `bench` subcommand, broken packaged-doc relative links → absolute URLs, the rc.7↔rc.8 CHANGELOG sequencing note, `ROADMAP`/`AGENTS` into `scope-completeness-audit.mjs` AUDIT_FILES, and **SHA-pinning GitHub Actions + OpenSSF Scorecard** (a separable supply-chain batch).
|
|
792
|
+
|
|
793
|
+
### Files changed
|
|
794
|
+
|
|
795
|
+
- `scripts/oia-walk.mjs` — Check 7 RC-level currency sub-check + header note.
|
|
796
|
+
- `README.md`, `docs/api.md`, `docs/QUICKSTART.md`, `docs/benchmarks.md` — RC-currency → version-agnostic.
|
|
797
|
+
- `docs/api.md` (×2), `docs/QUICKSTART.md`, `docs/COMPARISON.md` — reranker "+5-10" → measured +15.5/+24.7.
|
|
798
|
+
- version bump 3.9.0-rc.11 → 3.9.0-rc.12 (7 surfaces).
|
|
799
|
+
|
|
800
|
+
---
|
|
801
|
+
|
|
802
|
+
## [3.9.0-rc.11] — 2026-05-28
|
|
803
|
+
|
|
804
|
+
> **TL;DR:** **Watcher / HNSW live-update correctness (sprint RC 3).** Two HIGH concurrency/integrity findings from the audit: **H1** — the watcher's file-change handler was fire-and-forget, so concurrent saves to the *same* file could interleave their embed-db upsert + HNSW `applyDiff` + the shared `rowsByLabel` mutation → silent index drift (ghost labels live in HNSW but absent from the embed-db → stale search hits). Now a **per-file promise queue** serializes same-file events (different files stay parallel), and `close()` drains in-flight handlers before the HNSW flush. **`-1` sentinel-label corruption** — the HNSW add-zip used `newIds[i] ?? -1`, which on any row/id length mismatch inserted a vector under label `-1`, corrupting the index + `rowsByLabel` + the persisted sidecar; the new `zipHnswAddPoints` throws fail-closed instead. Plus **M1** (`saveTo` persists the live `getCurrentCount()`, not the stale build-time `size`) and **L2** (correct `kind` on PDF unlink). **959 → 966 tests** (+7, positive + NEGATIVE controls). No API breaks.
|
|
805
|
+
|
|
806
|
+
**Patch — audit-driven correctness (sprint RC 3).**
|
|
807
|
+
|
|
808
|
+
### Fixed
|
|
809
|
+
|
|
810
|
+
- **H1 — watcher per-file serialization (HIGH, race).** `onChange` chained each event onto `this.handle(...).catch(...)` fire-and-forget; chokidar can dispatch overlapping events, and `handle()` has multiple `await` points between reading `oldIds` and applying the HNSW diff. Two concurrent edits to one file could interleave so a stale `applyDiff` left labels live in HNSW + `rowsByLabel` but absent from the embed-db (search then returns ghost hits, masked by `applyDiff`'s silent missing-label skip). Fix: a `fileQueues: Map<absPath, Promise>` chains same-file events sequentially (different files keep independent chains → still parallel); the map self-evicts when a file's chain drains. `close()` now `await Promise.allSettled([...fileQueues.values()])` before `flushHnswToDisk()` so a pending update completes before the flush.
|
|
811
|
+
- **`-1` sentinel-label corruption (HIGH).** `result.rows.map((r, i) => ({ id: newIds[i] ?? -1, … }))` at both the md and PDF zip sites silently inserted a vector under sentinel label `-1` if `newIds.length < rows.length` — corrupting the in-memory index, the shared `rowsByLabel`, and the flushed `.hnsw.bin`. New exported **`zipHnswAddPoints(rows, newIds)`** asserts equal length and throws (fail-closed) — caught by the watcher's per-event try/catch (logs + skips HNSW for that file; signature guard rebuilds a correct index next serve). No corrupt label is ever inserted.
|
|
812
|
+
- **M1 — HNSW `saveTo` live count.** `hnsw.ts` persisted the build-time `size` closure into `.meta.json`; after live updates that's stale. Now persists `hasLiveUpdate ? ctor.getCurrentCount() : size` (the same source the `size` getter uses).
|
|
813
|
+
- **L2 — unlink kind for PDFs.** The unlink branch hardcoded `kind: "md"` in its `syncHnswForFile` call; now passes `isPdf ? "pdf" : "md"`. Cosmetic on today's pure-delete diff (no rows are set) but correct + future-proof.
|
|
814
|
+
|
|
815
|
+
### Tests added (+7, positive + NEGATIVE controls)
|
|
816
|
+
|
|
817
|
+
- `tests/zip-hnsw-points.test.ts` (NEW) — `zipHnswAddPoints`: matched zip (POSITIVE), empty case, too-few-ids + too-many-ids throw (NEGATIVE — the `-1` guard), never-emits-`-1`.
|
|
818
|
+
- `tests/hnsw.test.ts` — M1: build → `applyDiff` add 1 → `saveTo` → persisted `meta.size` equals the live count, **not** the build-time size (NEGATIVE control).
|
|
819
|
+
- `tests/watcher.test.ts` — H1: after `close()` drains an edit, the invariant holds — no `-1` sentinel in `rowsByLabel`, no ghost label (every tracked label exists in the embed-db). (chokidar's 250ms `awaitWriteFinish` coalesces writes, so this asserts the serialization+drain invariant rather than forcing the exact race.)
|
|
820
|
+
|
|
821
|
+
### Files changed
|
|
822
|
+
|
|
823
|
+
- `src/watcher.ts` — `zipHnswAddPoints` helper + `EmbedRowLike`; `fileQueues` field + serialized `onChange`; `close()` drain; both zip sites use the helper; unlink kind.
|
|
824
|
+
- `src/hnsw.ts` — `saveTo` persists the live `getCurrentCount()`.
|
|
825
|
+
- `tests/zip-hnsw-points.test.ts` (new), `tests/hnsw.test.ts`, `tests/watcher.test.ts`.
|
|
826
|
+
- test count 959 → 966 across README/COMPARISON/llms.txt/AGENTS/package.json/ROADMAP.
|
|
827
|
+
- version bump 3.9.0-rc.10 → 3.9.0-rc.11 (7 surfaces).
|
|
828
|
+
|
|
829
|
+
---
|
|
830
|
+
|
|
831
|
+
## [3.9.0-rc.10] — 2026-05-28
|
|
832
|
+
|
|
833
|
+
> **TL;DR:** **Closes overclaim #16 — OCR offline enforcement is now REAL (CRITICAL), plus the OCR canvas-OOM DoS.** The TSDoc/CLI-help/SECURITY.md all claimed `serve` "makes zero outbound network calls" / "no runtime CDN download" / "throws if a language isn't installed" and referenced an `install-ocr-lang` subcommand — but the code did none of it (`createWorker` silently CDN-fetched; the subcommand didn't exist). This RC builds the guards the docs promised: a **pre-flight cache check that throws fail-closed before the worker is created**, a real **`install-ocr-lang <code>` subcommand**, a worker pinned read-only to the local tessdata cache, an **absolute canvas-dimension clamp** (the `scale` cap was a false OOM guard for giant MediaBoxes), page-range validation, and **OIA Check 4e** which fails CI if any doc claims the offline guarantee while a code guard is absent (regression-proofs the #16 class, like Check 4d did for SLSA). **+15 tests (positive + NEGATIVE controls), all CI-runnable without the OCR optional deps. 944 → 959 tests.**
|
|
834
|
+
|
|
835
|
+
**Patch — audit-driven security (sprint RC 2): #16 + DoS.**
|
|
836
|
+
|
|
837
|
+
### Fixed
|
|
838
|
+
|
|
839
|
+
- **#16 OCR offline enforcement (CRITICAL — claimed-guarantee vs code-guard).** `src/ocr.ts`: `extractPdfWithOcr` now calls **`assertOcrLangsInstalled(langs, langPath)`** BEFORE loading any optional dep — it `existsSync`-checks every requested `<lang>.traineddata` in the local tessdata cache and throws (fail-closed), naming the exact `install-ocr-lang` command, if any is missing. The Tesseract worker is created with `langPath` + `cachePath` at the local cache and **`cacheMethod: "readOnly"`** (never writes/refetches). New **`resolveTessdataDir()`** (`$ENQUIRE_TESSDATA_DIR` → `$XDG_CACHE_HOME/enquire-mcp/tessdata` → `~/.cache/enquire-mcp/tessdata`). New CLI **`install-ocr-lang <code>`** subcommand (mirrors `install-model`) downloads `<code>.traineddata` from tessdata_fast into that dir — the ONLY OCR network call, explicit + opt-in, with strict `^[a-z0-9_]+$` code validation (no path-traversal / URL-injection). `serve` now makes **zero** outbound calls for OCR.
|
|
840
|
+
- **OCR canvas-OOM DoS (HIGH).** The `scale ∈ [0.5,4]` clamp bounds the multiplier, not the absolute pixel count — a PDF with a giant MediaBox (spec allows 14400×14400 pt) rendered to a multi-GB single-page canvas → OOM. New **`clampOcrScale(w, h, scale)`** lowers the effective scale so the larger rendered side never exceeds **`MAX_OCR_CANVAS_DIM`** (5000 px).
|
|
841
|
+
- **Inverted page range (LOW).** **`resolveOcrPageRange`** throws on an empty/inverted range (e.g. `pages:[5,2]`) instead of silently returning zero pages (which a caller could misread as "image-only scan").
|
|
842
|
+
- **Docs corrected to the enforced reality.** Rewrote SECURITY.md "OCR network posture" (was: `install-ocr-lang` "Deferred" + "the only outbound call in serve mode" — both now false) with the code-guard list + a stable `<a id="ocr-network-posture">` anchor; fixed the api.md broken anchor; updated `--ocr-pdfs`/`--ocr-langs` CLI help + api.md to cite `install-ocr-lang`.
|
|
843
|
+
|
|
844
|
+
### Structural defense (closes the #16 class)
|
|
845
|
+
|
|
846
|
+
- **OIA Check 4e** (`scripts/oia-walk.mjs`) — the "claimed-guarantee vs code-guard" pattern applied to OCR (parallel to rc.8's SLSA Check 4d). If any of README/SECURITY.md/COMPARISON/api.md/llms.txt claims "zero outbound / no runtime CDN / install-ocr-lang" (non-roadmap), it asserts `src/ocr.ts` calls `assertOcrLangsInstalled` + sets `cacheMethod:"readOnly"` AND `src/cli.ts` registers `install-ocr-lang` — failing CI otherwise. **Verified non-vacuous** (all 3 guards detected present → silent for the right reason) **and with detection power** (would flag 4+ claim lines if a guard were removed). The generalized enforcement-verb grep remains a tracked ROADMAP item (this is the #16-specific guard, mirroring how 4d was #15-specific).
|
|
847
|
+
|
|
848
|
+
### Tests added (+15, positive + NEGATIVE controls)
|
|
849
|
+
|
|
850
|
+
`tests/ocr-offline.test.ts` (NEW) — `resolveTessdataDir` precedence (3), `ocrLangIsInstalled`/`assertOcrLangsInstalled` incl. multi-lang + missing-pack throw (5), `extractPdfWithOcr` pre-flight throw before any dep loads (1, the load-bearing #16 guard), `clampOcrScale` normal-unchanged + huge-MediaBox-shrinks (3), `resolveOcrPageRange` clamp + inverted-throws (3). All run without `tesseract.js`/`canvas`/`pdfjs` because the guards execute before those load.
|
|
851
|
+
|
|
852
|
+
### Files changed
|
|
853
|
+
|
|
854
|
+
- `src/ocr.ts` — `resolveTessdataDir`/`ocrLangIsInstalled`/`assertOcrLangsInstalled`/`clampOcrScale`/`resolveOcrPageRange`/`MAX_OCR_CANVAS_DIM`; pre-flight + readOnly worker + canvas clamp + page-range in `extractPdfWithOcr`; TSDoc corrected to the enforced behavior.
|
|
855
|
+
- `src/cli.ts` — `install-ocr-lang` subcommand; `--ocr-pdfs`/`--ocr-langs` help cite it.
|
|
856
|
+
- `scripts/oia-walk.mjs` — Check 4e + header enumeration (11 → 12 blocks).
|
|
857
|
+
- `SECURITY.md`, `docs/api.md` — OCR posture rewrite + stable anchor + subcommand row.
|
|
858
|
+
- `tests/ocr-offline.test.ts` (new).
|
|
859
|
+
- test count 944 → 959 across README/COMPARISON/llms.txt/AGENTS/package.json/ROADMAP.
|
|
860
|
+
- version bump 3.9.0-rc.9 → 3.9.0-rc.10 (7 surfaces).
|
|
861
|
+
|
|
862
|
+
---
|
|
863
|
+
|
|
864
|
+
## [3.9.0-rc.9] — 2026-05-28
|
|
865
|
+
|
|
866
|
+
> **TL;DR:** **First RC of the post-audit sprint — input-validation security.** A second, five-agent comprehensive audit (core-retrieval code · server/transport/CLI code · docs/workflows/config · competitor landscape · repo-page/discoverability) ran against rc.8; `ROADMAP.md` is rewritten around its findings + the competitive read (we are capability-ahead of every Obsidian-MCP peer; the gap to the memory leaders is published benchmarks + discoverability, not tech). This RC ships the **P0 input-validation** findings: a real **ReDoS guard** on `obsidian_open_questions` (an always-registered tool that compiled a caller-supplied `pattern` straight into V8's backtracking engine and ran it over every line of every note — a remote DoS on `serve-http`), a defensive length cap on DQL `like`, and reconciliation of the bearer-token min-length check between the CLI and the transport. **No behavior change for legitimate callers. 927 → 944 tests** (+17, all with positive + negative controls).
|
|
867
|
+
|
|
868
|
+
**Patch — audit-driven security (sprint RC 1 of N).**
|
|
869
|
+
|
|
870
|
+
### The audit (sprint kickoff)
|
|
871
|
+
|
|
872
|
+
Five parallel agents re-read the project end-to-end on rc.8. Net: **zero CRITICAL beyond the already-tracked #16 OCR overclaim**; the codebase's path-safety, FTS5 escaping, int8 quantization, RRF/IR-metric, bearer-compare, CORS, and P2-10/11 session-lifecycle layers were all re-confirmed solid. New actionable findings were sequenced into a phased sprint (see `ROADMAP.md` Tier 1): **rc.9 input-validation (this RC) → rc.10 OCR offline enforcement + canvas-OOM → rc.11 watcher/HNSW correctness → rc.12 structural defenses + state-driven docs + supply-chain → rc.13 remaining correctness → rc.14 discoverability**. Audit checkpoint after each RC.
|
|
873
|
+
|
|
874
|
+
### Fixed (input-validation security)
|
|
875
|
+
|
|
876
|
+
- **ReDoS in `obsidian_open_questions` (HIGH).** `tools/meta.ts` compiled `args.pattern` (zod `z.string().optional()`, no constraint) into a `RegExp` and ran it per-line across the whole vault. The tool is **always registered** (not gated), so any stdio or bearer-authenticated `serve-http` client could submit a catastrophic-backtracking pattern (`(a+)+$`, `(.*)*`) and freeze the single-threaded event loop. Fix: a dependency-free **`isCatastrophicRegex`** guard that rejects "star height ≥ 2" patterns (an unbounded/amplifying quantifier applied to a group whose body also has one — honoring char-classes + escapes) **before** compile, plus a hard **`MAX_QUESTION_PATTERN_LEN` = 200** cap mirrored on the zod schema. The safe default pattern is unaffected (regression-guarded in tests).
|
|
877
|
+
- **DQL `like` length cap (defensive).** `dql.ts`'s `likeToRegex` is catastrophic-backtracking-**safe by construction** (it only ever emits `.*`, never a nested quantifier — re-confirmed by the audit), so this is **not** a ReDoS fix; it just bounds regex-compile/match CPU on an absurdly long user-supplied LIKE value via **`MAX_LIKE_PATTERN_LEN` = 512** (throws above it).
|
|
878
|
+
- **Bearer min-length reconciliation.** `cli.ts` accepted any non-empty `--bearer-token` while `startHttpServer` independently threw on `< 16` — so a short token passed the CLI gate then failed deeper with a less-friendly error. The `≥16` check now also fires in the CLI action (before any server setup), giving the user the `gen-token` hint + a clean `exit(1)`. The transport-layer check stays as defense-in-depth.
|
|
879
|
+
|
|
880
|
+
### ROADMAP refresh
|
|
881
|
+
|
|
882
|
+
`ROADMAP.md` rewritten after the second audit + competitive/discoverability survey: sharpened "#1 in our spheres" thesis, the phased rc.9→rc.14 sprint, a Tier-3 push to **publish LongMemEval scores** (the #1 credibility lever — no Obsidian MCP has any) + a "forgetting-aware" note-staleness signal (a frontier every memory competitor fails), and a "Requires the maintainer" section for the account/OAuth-gated discoverability actions (Glama claim, MCP Registry re-submit, forum post).
|
|
883
|
+
|
|
884
|
+
### Tests added (+17, all positive + negative controls)
|
|
885
|
+
|
|
886
|
+
- `tests/redos-guard.test.ts` (NEW) — 13 catastrophic patterns flagged (NEGATIVE), 11 safe patterns accepted incl. the production default (POSITIVE regression guard), `readUnboundedQuantifier` unit cases, + 4 `getOpenQuestions` integration cases (rejects catastrophic/over-long; accepts safe/default). The catastrophic *integration* fixture is built at runtime (`String.fromCharCode`) so CodeQL's `js/redos` static pass doesn't flag a regex literal that the guard rejects before compile — keeps "0 new CodeQL alerts" true (caught by the advisory CodeQL gate on the first PR push).
|
|
887
|
+
- `tests/dql.test.ts` — `likeToRegex` cap: normal pattern matches (POSITIVE), boundary at the cap passes, over-long throws (NEGATIVE).
|
|
888
|
+
- `tests/cli.test.ts` — `serve-http` short-token → `exit(1)` + "≥16 chars" hint (NEGATIVE); no-token → "required" with the length error explicitly NOT firing (contrast control).
|
|
889
|
+
|
|
890
|
+
### Files changed
|
|
891
|
+
|
|
892
|
+
- `src/tools/meta.ts` — `isCatastrophicRegex` + `readUnboundedQuantifier` + `MAX_QUESTION_PATTERN_LEN` + guarded compile in `getOpenQuestions`.
|
|
893
|
+
- `src/tool-registry.ts` — `.max(MAX_QUESTION_PATTERN_LEN)` on the `pattern` schema + import.
|
|
894
|
+
- `src/dql.ts` — `MAX_LIKE_PATTERN_LEN` + cap in `likeToRegex` (exported for tests).
|
|
895
|
+
- `src/cli.ts` — bearer `≥16` check in the `serve-http` action.
|
|
896
|
+
- `ROADMAP.md` — full rewrite (post-audit).
|
|
897
|
+
- `tests/redos-guard.test.ts` (new), `tests/dql.test.ts`, `tests/cli.test.ts`.
|
|
898
|
+
- test count 927 → 944 across README/COMPARISON/llms.txt/AGENTS/package.json; README suite-timing ~5s → ~12s (audit LOW).
|
|
899
|
+
- version bump 3.9.0-rc.8 → 3.9.0-rc.9 (7 surfaces).
|
|
900
|
+
|
|
901
|
+
---
|
|
902
|
+
|
|
5
903
|
## [3.9.0-rc.8] — 2026-05-28
|
|
6
904
|
|
|
7
905
|
> **TL;DR:** **Integrity-batch #2 from the exhaustive file-by-file audit** (every `src/` module, every doc, every workflow, every script re-read on Opus 4.8). Closes the cheap-but-real drift the audit surfaced and adds the FIRST structural defense for the "claimed-guarantee vs code-guard" class introduced in rc.7: a new **OIA Check 4d** that reads `.github/workflows/release.yml`, computes the SLSA Build Level it actually earns, and fails CI if any doc claims a higher level. Also: a bench-harness honesty fix (a 5-sample "p99" that always returned the max — relabeled `max`), determinism fix (`Date.now()` tag → stable), the privacy-test soft-skips made VISIBLE via `ctx.skip()` + a CI tripwire that fails loudly if the native deps that gate them ever go missing in CI, two stale test-title positioning claims, a benchmarks rounding drift, a biome binary/schema unification (2.4.14/2.4.15 → 2.4.16), and a stale Node placeholder in the bug template. **Docs/tests/scripts/config only — zero `src/` runtime logic changed. 926 → 927 tests (+1 CI tripwire).**
|
|
@@ -71,6 +969,7 @@ Three parallel passes:
|
|
|
71
969
|
|
|
72
970
|
- **rc.8 — #16 OCR offline enforcement (HIGH, "implement" decision).** SECURITY.md claims "zero outbound network calls in serve mode" and `ocr.ts` TSDoc claims a pre-flight "throws if language not installed" check, but `extractPdfWithOcr` only warns then `createWorker` silently CDN-fetches; `install-ocr-lang` is referenced in 4 files but never existed. Implement: pre-flight cache check + `langPath` wiring + real `install-ocr-lang` subcommand + env-gated integration test.
|
|
73
971
|
- **rc.9 — H1 watcher per-file serialization (HIGH).** Fire-and-forget `handle()` lets concurrent saves to one file interleave `applyDiff` + the shared `rowsByLabel` mutation → in-memory HNSW drift. Add a per-relPath promise queue + concurrent-event test. Plus M1 (HNSW `saveTo` live count), L2 (unlink kind).
|
|
972
|
+
- _**Re-sequenced after this entry** (rc.13 doc fix): the rc.8 integrity-batch pivot pushed both items back two RCs — **#16 OCR offline enforcement actually shipped in v3.9.0-rc.10**, **H1 watcher serialization in v3.9.0-rc.11** (see those entries). The "ships in rc.8 / rc.9" lines above are the original rc.7 plan, preserved as history._
|
|
74
973
|
|
|
75
974
|
### Files changed
|
|
76
975
|
|