@wei840222/qmd 2026.8.23

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (94) hide show
  1. package/CHANGELOG.md +1373 -0
  2. package/LICENSE +45 -0
  3. package/README.md +1439 -0
  4. package/THIRD_PARTY_NOTICES.md +31 -0
  5. package/bin/qmd +192 -0
  6. package/dist/ast.d.ts +65 -0
  7. package/dist/ast.js +334 -0
  8. package/dist/bench/bench.d.ts +35 -0
  9. package/dist/bench/bench.js +338 -0
  10. package/dist/bench/cjk-baseline.d.ts +36 -0
  11. package/dist/bench/cjk-baseline.js +111 -0
  12. package/dist/bench/fixture.d.ts +2 -0
  13. package/dist/bench/fixture.js +84 -0
  14. package/dist/bench/score.d.ts +38 -0
  15. package/dist/bench/score.js +107 -0
  16. package/dist/bench/types.d.ts +110 -0
  17. package/dist/bench/types.js +8 -0
  18. package/dist/cli/build-info.json +4 -0
  19. package/dist/cli/embed-lock.d.ts +24 -0
  20. package/dist/cli/embed-lock.js +94 -0
  21. package/dist/cli/embedding-owner.d.ts +10 -0
  22. package/dist/cli/embedding-owner.js +20 -0
  23. package/dist/cli/formatter.d.ts +120 -0
  24. package/dist/cli/formatter.js +355 -0
  25. package/dist/cli/mcp-pid.d.ts +25 -0
  26. package/dist/cli/mcp-pid.js +86 -0
  27. package/dist/cli/qmd.d.ts +72 -0
  28. package/dist/cli/qmd.js +4806 -0
  29. package/dist/cli/version.d.ts +42 -0
  30. package/dist/cli/version.js +80 -0
  31. package/dist/collections.d.ts +200 -0
  32. package/dist/collections.js +433 -0
  33. package/dist/db.d.ts +65 -0
  34. package/dist/db.js +143 -0
  35. package/dist/diagnostics.d.ts +62 -0
  36. package/dist/diagnostics.js +260 -0
  37. package/dist/embedding/config.d.ts +52 -0
  38. package/dist/embedding/config.js +229 -0
  39. package/dist/embedding/identity.d.ts +58 -0
  40. package/dist/embedding/identity.js +321 -0
  41. package/dist/embedding/local-identity.d.ts +1 -0
  42. package/dist/embedding/local-identity.js +15 -0
  43. package/dist/embedding/local.d.ts +34 -0
  44. package/dist/embedding/local.js +290 -0
  45. package/dist/embedding/openai.d.ts +79 -0
  46. package/dist/embedding/openai.js +477 -0
  47. package/dist/embedding/owner.d.ts +13 -0
  48. package/dist/embedding/owner.js +36 -0
  49. package/dist/embedding/provider.d.ts +68 -0
  50. package/dist/embedding/provider.js +16 -0
  51. package/dist/embedding/remote-chunking.d.ts +22 -0
  52. package/dist/embedding/remote-chunking.js +83 -0
  53. package/dist/embedding/remote-embedding.d.ts +15 -0
  54. package/dist/embedding/remote-embedding.js +77 -0
  55. package/dist/hybrid-llm.d.ts +18 -0
  56. package/dist/hybrid-llm.js +53 -0
  57. package/dist/index.d.ts +244 -0
  58. package/dist/index.js +418 -0
  59. package/dist/llm.d.ts +566 -0
  60. package/dist/llm.js +1847 -0
  61. package/dist/maintenance.d.ts +33 -0
  62. package/dist/maintenance.js +52 -0
  63. package/dist/mcp/origin-guard.d.ts +67 -0
  64. package/dist/mcp/origin-guard.js +137 -0
  65. package/dist/mcp/server.d.ts +116 -0
  66. package/dist/mcp/server.js +919 -0
  67. package/dist/paths.d.ts +1 -0
  68. package/dist/paths.js +4 -0
  69. package/dist/remote-llm.d.ts +52 -0
  70. package/dist/remote-llm.js +464 -0
  71. package/dist/search/cjk-analyzer.d.ts +33 -0
  72. package/dist/search/cjk-analyzer.js +158 -0
  73. package/dist/search/cjk-index.d.ts +104 -0
  74. package/dist/search/cjk-index.js +1031 -0
  75. package/dist/search/jieba-loader.d.ts +23 -0
  76. package/dist/search/jieba-loader.js +79 -0
  77. package/dist/search/query-expansion.d.ts +23 -0
  78. package/dist/search/query-expansion.js +43 -0
  79. package/dist/search/zh-dict.txt +624013 -0
  80. package/dist/store.d.ts +1218 -0
  81. package/dist/store.js +6076 -0
  82. package/dist/trust.d.ts +152 -0
  83. package/dist/trust.js +249 -0
  84. package/package.json +139 -0
  85. package/scripts/build.mjs +83 -0
  86. package/scripts/check-package-grammars.mjs +29 -0
  87. package/scripts/package-smoke.mjs +205 -0
  88. package/scripts/sync-zh-dict.mjs +187 -0
  89. package/scripts/test-all.mjs +45 -0
  90. package/skills/qmd/SKILL.md +324 -0
  91. package/skills/qmd/references/mcp-setup.md +119 -0
  92. package/skills/release/SKILL.md +141 -0
  93. package/skills/release/scripts/install-hooks.sh +38 -0
  94. package/skills/release/scripts/release-context.sh +129 -0
package/CHANGELOG.md ADDED
@@ -0,0 +1,1373 @@
1
+ # Changelog
2
+
3
+ ## [Unreleased]
4
+
5
+ ### Added
6
+
7
+ - Added Oxlint lint fence.
8
+ - Added a bundled Traditional Chinese Jieba dictionary that deterministically
9
+ combines the node-rs base dictionary, APCLab's Taiwan dictionary, and
10
+ zhtw-mcp technical terms. `pnpm dict:sync` verifies pinned source hashes;
11
+ `pnpm dict:sync --update-pins` refreshes the upstream pins.
12
+ - Added CJK-aware lexical retrieval with independent character, Jieba word, and
13
+ bigram FTS5 channels, versioned rank fusion and explain traces. The bundled
14
+ reviewed Traditional Chinese technical dictionary records its pinned source
15
+ and MIT attribution in `THIRD_PARTY_NOTICES.md`.
16
+ - Added typed local/OpenAI embedding providers, provider-qualified vector
17
+ identities, resumable chunk persistence, build leases, remote preflight and
18
+ acknowledgement, request-purpose guards, safe OpenAI errors, and additive
19
+ diagnostics in CLI, SDK, and MCP. Local embedding remains the default.
20
+ - Added `OPENAI_BASE_URL` for routing OpenAI-compatible embedding requests
21
+ through a self-hosted proxy while retaining the existing remote-consent flow.
22
+ - Added the native `@node-rs/jieba` dependency and package smoke coverage for its
23
+ Node/Bun runtime loading and packaged dictionary data.
24
+
25
+ ### Changed
26
+
27
+ - Chat-completions reranking now prioritizes explicit query constraints such as
28
+ entities, locations, versions, time, and negations. Valid empty rerank results
29
+ no longer fall back to neutral scores, and parsed chat rerank output is sorted
30
+ by descending relevance score.
31
+ - Remote query expansion and chat-completions reranking now use a consistent
32
+ Gemini-style XML prompt structure in both system and user messages, separating
33
+ role, instructions, constraints, output format, context, task, and final
34
+ instruction. Their OpenAI-compatible chat requests omit provider-specific
35
+ reasoning controls so each configured model uses its default behavior.
36
+ - Remote expansion and chat reranking now escape untrusted prompt data, use
37
+ leaner instructions without requiring a reasoning mode, preserve established
38
+ cross-language technical terms, and discard chat rerank scores below `0.1`.
39
+ - Remote query expansion now requests concise, backend-specific variations with
40
+ a short example and conservative handling for ambiguous queries. Remote output
41
+ is capped at one expansion per type, and `includeLexical: false` suppresses
42
+ `lex:` output both in the request and parser.
43
+ - Provider failures no longer retain native error causes, preventing nested
44
+ runtime errors from exposing document content, headers, credentials, or stacks.
45
+ - CLI and SDK local embedding now share the same provider-qualified identity.
46
+ CLI and SDK shutdown now drain active scoped local/OpenAI embedding, expansion,
47
+ and reranking work before disposing llama.cpp; CLI database closure is ordered
48
+ last and concurrent cleanup requests share one idempotent operation. An
49
+ instance-local atomic acquisition barrier also prevents scoped session leases
50
+ from racing inactivity unload while it disposes contexts. Full llama.cpp
51
+ disposal now closes session admission synchronously, drains default and
52
+ per-instance sessions, and shares one completion promise; failed inactivity
53
+ unloads no longer prevent best-effort full cleanup.
54
+ - CJK search now omits both analyzed lexical channels (word and bigram) when the
55
+ Jieba capability is unavailable, preserving character-only fallback even when
56
+ a persisted analyzed index exists.
57
+ - OpenAI embedding requests now preserve single-request ordering while allowing
58
+ queued aborts and deadlines, enforce per-attempt fetch/body timeouts, validate
59
+ UTF-8 budgets and usage metadata, and return usage with each operation result.
60
+ - Destructive remote embedding rebuilds now verify provider capability with a
61
+ fixed sentinel before clearing vectors, then atomically revalidate consent,
62
+ document generation, embedding identity, and build authorization before reset.
63
+ - Opening or updating an index now creates versioned CJK shadow tables, mutation
64
+ journal/build state, and embedding identity state. Document and collection
65
+ mutations keep all lexical channels synchronized; raw SQLite writes mark the
66
+ analyzed channels dirty and force character-only fallback until rebuilt.
67
+ - CJK queries skip automatic expansion by default. CLI, SDK, and MCP now share
68
+ the `auto | force | skip` expansion policy and explicit override semantics;
69
+ forced empty expansions are reported separately from provider failures.
70
+ - Structured typed queries now apply the documented double weight to every
71
+ result list from the first supplied sub-query, regardless of lexical/vector
72
+ execution order. CLI query help also documents standalone `lex:` policy input.
73
+ - Collection and full embedding-fingerprint filters now apply before lexical and
74
+ vector candidate truncation, including vector indexes larger than SQLite Vec's
75
+ 4,096-result KNN cap. CLI, SDK, and MCP queries honor every requested
76
+ collection, and query embedding is skipped unless a matching vector identity
77
+ is published as ready.
78
+ - Embedding identity state now keeps published vectors searchable during
79
+ same-identity incremental work, fences every vector write with the active
80
+ unexpired generation lease, requires that lease for destructive vector clears,
81
+ persists unverifiable legacy identities as incompatible across schema upgrades, and
82
+ rejects collection-scoped identity changes before any global reset. Local
83
+ embeddings now publish the same canonical formatter/chunk identity contract
84
+ as provider-backed builds; missing identity state fails closed, and orphan or
85
+ collection cleanup rolls back atomically if either vector table update fails.
86
+ - `qmd status`, `qmd doctor`, and MCP startup, status, and remote preflight now use read-only
87
+ index composition. Invalid configuration is diagnosed without creating or
88
+ mutating the SQLite index. Legacy vector schemas are reported conservatively
89
+ without lazy migration, persisted database embedding configuration is restored
90
+ when YAML is absent, and read-only search skips optional cache and CJK repair writes.
91
+ - Every remote provider request and build transition now fails closed without a
92
+ Store authorization hook. SDK and MCP still permit lexical-only fallback when
93
+ no matching ready vector identity exists. Remote no-op checks include the
94
+ requested chunking strategy, SDK preflight uses the canonical collection config
95
+ resolver, and the physical OpenAI request boundary requires the complete active
96
+ identity fingerprint instead of falling back to provider-only identity. Remote
97
+ acknowledgement diagnostics now consume the canonical consent schema and full
98
+ build identity rather than duplicating stale consent columns, and remote
99
+ authorization rejects local identity fingerprints before transport.
100
+ - Config-first collection and context mutations reconcile SQLite through one
101
+ atomic path, persist diagnostics for `qmd doctor`, recover YAML/SQLite drift on
102
+ reopen, keep CLI update/include settings synchronized, and use atomic durable
103
+ YAML replacement.
104
+
105
+ ### Compatibility
106
+
107
+ - `vectors_vec` still supports one embedding dimension at a time. Switching
108
+ provider, model, dimension, formatter, or chunk profile requires a full vector
109
+ rebuild with `qmd embed --force`.
110
+ - Back up an index before opening it with this version if it must later be used by
111
+ an older QMD release. Schema migration has no in-place downgrade, and once an
112
+ authorized vector reset starts there is no rollback to the deleted vectors;
113
+ lexical search remains available while embeddings are rebuilt.
114
+
115
+ ## [2.8.3] - 2026-08-16
116
+
117
+ ### Security
118
+
119
+ - `qmd update` no longer runs a project-local `.qmd/index.yml`'s `update:`
120
+ commands without approval (#886). That file arrives with a `git clone` and is
121
+ adopted automatically for any command run inside the tree, so cloning a
122
+ repository and running `qmd update` executed shell commands chosen by whoever
123
+ wrote it. On a terminal QMD now lists the commands and asks; with nobody to
124
+ ask it skips them and keeps indexing. Approvals are recorded per config file
125
+ and per command set in `<config dir>/trusted.json`, so editing a command — or
126
+ a `git pull` that rewrites one — asks again. New `qmd trust`,
127
+ `qmd trust list` and `qmd trust revoke` manage approvals, and
128
+ `QMD_TRUST_UPDATE_HOOKS=1` opts unattended runs back in. Commands in your own
129
+ `~/.config/qmd/*.yml`, including anything `qmd collection update-cmd` writes,
130
+ are unaffected.
131
+
132
+ - The same project-local trust gate now covers collection `path` values that
133
+ resolve outside the project and non-default `models.embed` / `models.rerank`
134
+ / `models.generate` URIs (#889). In-project paths still index unattended;
135
+ out-of-project directories are skipped until `qmd trust`, and custom model
136
+ URIs are not loaded or downloaded. `QMD_TRUST_LOCAL_CONFIG=1` opts unattended
137
+ runs back in (and `QMD_TRUST_UPDATE_HOOKS=1` still does). `qmd collection
138
+ add` records trust as it writes, the same way `update-cmd` does.
139
+
140
+ - Indexing no longer follows file symlinks or glob `../` / absolute patterns
141
+ out of the collection directory. `fast-glob` already skipped symlinked
142
+ directories, but a file symlink (or a mask like `../**/*.md`) still resolved
143
+ via `realpath` and ingested the target. `qmd://` filesystem resolution uses
144
+ the same containment check, so `qmd://collection/../../../etc/passwd` no
145
+ longer produces a path outside the collection.
146
+
147
+ - `qmd mcp --http` now validates the `Origin` and `Host` headers on every
148
+ request and answers `403` when they name anything but a loopback address
149
+ (#881). Binding to localhost is no defence against the user's own browser:
150
+ a page can re-point its hostname at `127.0.0.1` (DNS rebinding) and read
151
+ the indexed corpus through `POST /query` or `POST /mcp`. Requests with no
152
+ `Origin` (curl, MCP clients, editors) are unaffected. Extend the allowlists
153
+ with `QMD_ALLOWED_ORIGINS` / `QMD_ALLOWED_HOSTS`, or set
154
+ `QMD_ALLOWED_ORIGINS=*` behind your own authenticating proxy. A wildcard
155
+ bind (`--host 0.0.0.0`) skips the host check and warns at startup.
156
+
157
+ ### Changed
158
+
159
+ - Dependencies: `node-llama-cpp` 3.18.1 → **3.20.0** (llama.cpp b8390 → b10361, 2026-08-11). Also safe patch/minors: `picomatch` 4.0.4 → 4.0.5, `web-tree-sitter` 0.26.8 → 0.26.12, `tsx` 4.21.0 → 4.23.12, `vitest` 3.2.4 → 3.2.7. No zod/vitest major; `@modelcontextprotocol/server` stays 2.0.0 (no 2.x patch). `flake.nix` FOD hashes are not updated here.
160
+ - `generate` and query expansion now await `LlamaContextSequence.dispose()` before disposing the parent context. node-llama-cpp 3.20 made sequence dispose async; the library's context-onDispose path does not wait.
161
+
162
+ - MCP server now speaks protocol revision **2026-07-28** via the official
163
+ TypeScript SDK 2.x (`@modelcontextprotocol/server`). HTTP is sessionless
164
+ (no `Mcp-Session-Id`, no initialize handshake, no idle-session TTL / #816
165
+ reaper). Clients send version and capabilities in `_meta`; `server/discover`
166
+ is implemented; Streamable HTTP POST requires `Mcp-Method` / `Mcp-Name`
167
+ (mismatch → `-32020`); `tools/list` is deterministic and carries `ttlMs` /
168
+ `cacheScope`. 2025-era stdio clients still work (`serveStdio` dual-speak);
169
+ 2025-era HTTP `initialize` is answered per-request without minting a
170
+ session. Existing tools (`query` / `get` / `multi_get` / `status`), stdio
171
+ EOF shutdown, and named-index daemon PIDs are unchanged. No release.
172
+
173
+ - `qmd pull` (and implicit model downloads in `embed`/`query`) no longer print
174
+ node-llama-cpp's download progress bar. The bar redraws every few kilobytes
175
+ and flooded agent transcripts with thousands of tokens (#776). Pass
176
+ `qmd pull --progress` to show it on an interactive terminal.
177
+ - `--full-path` no longer degrades silently when a result cannot be resolved on
178
+ disk (#785). A fallback there means the file moved or was deleted since the
179
+ last index, so `search`, `query`, `get` and `multi-get` now print a notice to
180
+ stderr naming how many results fell back and suggesting `qmd update`; stdout
181
+ stays machine-readable.
182
+ - `search`/`query` now decide per result whether to show the docid under
183
+ `--full-path`, matching `multi-get` and `get`: a result that resolved shows
184
+ its on-disk path and no docid, one that did not keeps its `qmd://` URI *and*
185
+ its docid, so it is still addressable. Previously the docid was dropped for
186
+ every row whenever the flag was set, leaving unresolved rows with neither a
187
+ usable path nor an identifier.
188
+ - `search --format csv` always emits the `docid` column, empty for rows that
189
+ resolved to an on-disk path. Under `--full-path` the header previously
190
+ dropped the column entirely — which also disagreed with the empty-result
191
+ header, always printed with `docid`. Column positions are now stable across
192
+ runs and formats.
193
+
194
+ ### Fixed
195
+
196
+ - Concurrent first-open of a cold index no longer fails with
197
+ `table documents_fts already exists` on Bun/macOS. FTS5
198
+ `CREATE VIRTUAL TABLE IF NOT EXISTS` is not atomic across WAL
199
+ connections: two processes can both see a missing table on their
200
+ schema snapshot and the loser throws. Table create and legacy-schema
201
+ repair now use the same `BEGIN IMMEDIATE` + double-check as the FTS
202
+ sync triggers, and treat a concurrent "already exists" as success
203
+ when the table is present.
204
+
205
+ - Nix flake `qmd-node-modules` FOD hashes updated for x86_64-linux and
206
+ aarch64-darwin after the MCP SDK 2.0 bump. `nix build` / Nix GHA was
207
+ failing with a fixed-output hash mismatch.
208
+
209
+ - CJK FTS rebuild no longer skips leftover `fts5(name, body, content='documents')`
210
+ tables when `fts_cjk_normalized_version` is already stamped, and schema
211
+ repair now checks live FTS columns (`PRAGMA table_info`) as well as
212
+ `sqlite_master.sql`. The MCP HTTP test helper still seeds that legacy
213
+ table; `startMcpHttpServer` / `createStore` on it must not throw
214
+ `no such column: T.name` (#792 regression).
215
+
216
+ - `qmd collection add --glob` is no longer silently ignored. parseArgs ran
217
+ with `strict: false`, so OpenClaw's `--glob memory.md` (and any other
218
+ `--glob`) fell through, the default `**/*.md` was used, and a second
219
+ collection on the same path collided as a duplicate instead of indexing
220
+ the requested mask (#536). `--glob` is now an alias for `--mask`.
221
+
222
+ - The `bin/qmd` trampoline now execs `process.execPath` instead of
223
+ re-resolving `node` from PATH. Native addons (`better-sqlite3`) are
224
+ compiled for the Node that installed qmd; a version manager (nvm, fnm,
225
+ mise) selecting a different major in the working directory used to spawn
226
+ that other binary and fail with `NODE_MODULE_VERSION` / `ERR_DLOPEN_FAILED`
227
+ (#577 leftover; #319). `bun bin/qmd` still resolves `node` from PATH so
228
+ Node-ABI addons are not loaded into bun.
229
+
230
+ - `qmd update` / `qmd collection add` no longer swallow unreadable files
231
+ silently (#460). `readFileSync` failures (ETIMEDOUT on APFS compressed
232
+ files, EAGAIN, EACCES, …) still skip the file so the rest of the collection
233
+ indexes, but the CLI now warns with the path and error code and reports
234
+ the skip count. The SDK `update()` result includes `skipped`.
235
+
236
+ - The architecture diagram no longer draws Vec expansions into BM25 search
237
+ (#680). `lex` expansions are FTS-only; `vec` and `hyde` expansions are
238
+ vector-only. The original query still goes to both backends.
239
+
240
+ - `qmd bench` no longer runs to a wall of 0.00 when the fixture collection is
241
+ missing or empty (#716). It errors up front with the same "Collection not
242
+ found" / index hint as `qmd search -c`, and if every backend still scores
243
+ zero it warns on stderr to check `qmd ls`.
244
+
245
+ - `qmd cleanup` now reclaims the content and FTS space left behind after a
246
+ wrong-directory `qmd update`. Deactivating files (the next update in the
247
+ right directory) only tombstoned the `documents` rows; cleanup deleted those
248
+ rows and vacuumed, but never dropped the unreferenced `content` hashes and
249
+ never ran FTS5 `optimize`, so `documents_fts_data` kept the old bodies
250
+ (#550). Cleanup now deletes inactive docs, then orphaned content, then
251
+ compact FTS, then vacuum. `--dry-run` reports the content hashes too.
252
+
253
+ - Concurrent `query` calls with `rerank: true` on a cold MCP server no longer
254
+ race `ensureRerankContexts()`. Embed already serialized context creation;
255
+ rerank did not, so two overlapping first queries both saw an empty pool,
256
+ both created ranking contexts, and the inactivity timer disposed the loser
257
+ (`Object is disposed`, #682). Callers now await the in-flight create.
258
+
259
+ - Embedding-context pool size no longer assumes every GGUF costs 150 MB of
260
+ VRAM (the nomic-embed figure). Larger models such as Qwen3-Embedding-0.6B
261
+ are ~1190 MB per 2048-token context; opening 8 of those exhausted an 8 GB
262
+ card so `qmd query` failed with `Failed to create any rerank context` even
263
+ though the reranker itself was fine. The pool is now sized from the weight
264
+ file, and 1 GB is reserved for the reranker (#799). Default
265
+ embeddinggemma/nomic throughput is unchanged. `QMD_EMBED_PARALLELISM` still
266
+ overrides.
267
+
268
+ - Multi-collection `-c A -c B` (and SDK/MCP `collections: [A, B]`) no longer
269
+ searches globally then post-filters. A large unrelated collection could fill
270
+ the FTS/ANN top-k so the requested collections vanished, yielding false-empty
271
+ results even though each collection matched on its own. `searchFTS` /
272
+ `searchVec` now search each requested collection, then merge by score
273
+ (#775). Single-collection exact-scan (#791, #803) is unchanged.
274
+
275
+ - Query expansion no longer consumes caller `intent`, and a cached expansion
276
+ whose sub-queries all miss is dropped instead of replaying forever (#818).
277
+ Intent still steers reranking and snippet/chunk selection; it just no longer
278
+ enters the expansion prompt or cache key, where the model copied meta-language
279
+ ("so I can compare spend settings") into lex/vec terms that matched nothing.
280
+
281
+ - Files whose names differ only in the characters the legacy slug collapsed to
282
+ `-` (spaces, underscores) no longer evict each other from the index (#717).
283
+ The handalized-path migration now skips any row whose path is still owned by
284
+ a file in the current scan, so it only adopts genuinely stale pre-2.6 rows.
285
+ `qmd get` and `qmd ls <prefix>` also match `_` and `%` in paths literally
286
+ instead of as SQL `LIKE` wildcards, so `qmd get 2026_06_16.md` no longer
287
+ returns a sibling `2026-06-16.md`.
288
+
289
+ - CLI `multi-get` and SDK/MCP `multi_get` now share one comma-list resolver.
290
+ Collection-prefixed paths (`qmd/docs/SYNTAX.md`) work in both transports,
291
+ unanchored `LIKE '%name'` no longer silently fetches a different document
292
+ for a filename fragment (`NTAX.md` ≠ `SYNTAX.md`), and ambiguous names
293
+ across collections error with the candidate list instead of `LIMIT 1` (#759).
294
+
295
+ - The Nix flake wrapper now seeds the same pre-import env as `bin/qmd`.
296
+ Nix installs exec `bun src/cli/qmd.ts` directly, so they previously skipped
297
+ the launcher: `qmd mcp` could leak llama/ggml native logs onto JSON-RPC
298
+ stdio, and Darwin CLI exits dumped a ggml Metal residency-set stack trace
299
+ after an otherwise successful query. The wrapper now quiets those logs for
300
+ `mcp` and sets `GGML_METAL_NO_RESIDENCY=1` on Darwin unless
301
+ `QMD_METAL_KEEP_RESIDENCY=1` (#723).
302
+
303
+ - Rerank context creation no longer swallows the real failure. A VRAM OOM or
304
+ corrupt model previously produced only `Reranker unavailable — skipping
305
+ reranking` (and a dead identical retry whose comment claimed it disabled
306
+ flash attention, which ranking contexts never supported). The warning now
307
+ includes the underlying message so the two cases are distinguishable (#782).
308
+
309
+ - The Nix flake package now ships `skills/` next to `src/` in `$out/lib/qmd/`.
310
+ `findPackageRoot()` walks up from the wrapped `src/cli/qmd.ts` looking for a
311
+ sibling `skills/` directory; without it, `qmd skill show` and `qmd skills list`
312
+ always failed with "QMD skill not found" on Nix-installed binaries (#722).
313
+
314
+ - `/release` step 1 no longer points at a missing script. `skills/release/scripts/release-context.sh`
315
+ now exists: it silently installs git hooks and prints version info, working-tree
316
+ status, commits and files since the last tag, `[Unreleased]`, and the previous
317
+ changelog entry. The skill's process list also drops the duplicate step 7 and
318
+ checks dependency updates before cutting the release (#796).
319
+
320
+ - `store.searchVec()` (and SDK `searchVector()`) now embed the query with the
321
+ store's pinned embed model instead of the global `QMD_EMBED_MODEL`. A store
322
+ created with a non-default `models.embed` previously failed with
323
+ `Dimension mismatch ... Expected N ... received M` and loaded the wrong
324
+ (often much larger) model at query time. Hybrid/precomputed/session search
325
+ paths were unaffected and stay unchanged (#690).
326
+
327
+ - `qmd collection add --mask "a.md,*.txt"` now indexes the union of each
328
+ pattern. The comma-separated form was documented and commonly guessed, but
329
+ the joined string was passed to fast-glob as one literal glob, so it
330
+ matched zero files with no error. Brace form `{a.md,*.txt}` is unchanged.
331
+ The same split applies on `qmd update` for stored comma-list masks (#557).
332
+
333
+ - NixOS / immutable-root installs no longer crash `qmd embed` with EACCES
334
+ when node-llama-cpp tries to compile llama.cpp into a read-only
335
+ `node_modules`. The flake wrapper puts Nix's glibc and libstdc++ on
336
+ `LD_LIBRARY_PATH` so prebuilt binaries can `dlopen` them, and
337
+ `getLlama()` uses `build: "never"` when the llama directory is not
338
+ writable (#574).
339
+
340
+ - CJK FTS rebuild no longer raises "database is busy" when flushing insert
341
+ batches. The streaming `.iterate()` cursor stayed open across
342
+ `BEGIN` on the same connection; the scan now uses keyset-paginated
343
+ `LIMIT` batches so each SELECT finalizes before the insert transaction
344
+ starts (#797).
345
+
346
+ - Quoted FTS phrases containing dotted tokens (e.g. `"1.0.21"`) now match the
347
+ indexed document. The porter unicode61 tokenizer stores dotted strings as
348
+ adjacent parts, but phrase sanitization stripped the dots into a single
349
+ token (`1021`) that could never hit. Dotted tokens inside quotes are split
350
+ into adjacent phrase terms, matching the bare-term rewrite from #563 (#757).
351
+
352
+ - `scripts/build.mjs` no longer passes `shell: true` to `spawnSync` on Windows.
353
+ With the default Node install path (`C:\Program Files\nodejs\node.exe`),
354
+ `cmd.exe` split the unquoted `process.execPath` at the space, the `tsc`
355
+ spawn failed, and `prepare` could still report success with no `dist/` —
356
+ leaving `bin/qmd` at "not built". The helper always receives a real binary
357
+ path plus an args array, so no shell is needed. A spawn error now prints
358
+ the missing binary path instead of failing silently. (#681)
359
+
360
+ - Case-sensitive collections no longer collapse distinct document identities that
361
+ differ only by path casing. The implicit `COLLATE NOCASE` legacy migration was
362
+ unsafe for filesystems that contain both `README.md` and `readme.md`; case-only
363
+ legacy migrations must now be explicit and operator-reviewed (#801).
364
+
365
+ - `cleanupOrphanedVectors` now runs its orphan count and both DELETEs in a
366
+ single immediate transaction. An interruption between the two DELETEs
367
+ (crash, `SQLITE_BUSY`) could desync `vectors_vec` from `content_vectors`,
368
+ leaving stale metadata rows that make a later reactivation of the same
369
+ content hash look already-embedded — so `qmd embed` skips it and the
370
+ document becomes silently unsearchable by vector, with no orphan left to
371
+ clean up (#766).
372
+
373
+ - `qmd embed` no longer splits a UTF-16 surrogate pair (emoji, etc.) across a
374
+ chunk boundary. A chunk ending or starting mid-pair produced an unpaired
375
+ surrogate in the chunk text, which some remote embedding APIs reject as
376
+ invalid JSON — permanently failing that chunk on every retry, since the
377
+ boundary calculation is deterministic. This covers both the character-based
378
+ chunker and `chunkDocumentByTokens`'s recursive re-splitting for
379
+ astral-plane-dense content, which could previously drive the char budget
380
+ low enough to reproduce the same split (#777).
381
+
382
+ - `insertContext` looks up `store_collections` by name. #754 retargeted the
383
+ query from the dropped `collections` table but left `WHERE id = ?`, and
384
+ `store_collections` has `name TEXT PRIMARY KEY` with no `id` column — the
385
+ call threw `no such column: id`. Matches `deleteContext` /
386
+ `updateStoreContext` (#853).
387
+
388
+ - `qmd collection add` with no path argument now errors with usage instead of
389
+ silently indexing the current working directory (#684). Pass `.` to index
390
+ CWD, matching the documented examples.
391
+
392
+ - `qmd status` reports orphaned embedding chunks, `qmd update` hints when they
393
+ exceed 10% of vectors, and `qmd cleanup --dry-run` previews what would be
394
+ removed. Incremental update still does not auto-prune vectors (a transient
395
+ empty mount would otherwise force a full re-embed) (#768).
396
+
397
+ - `qmd --index <name> mcp --http --daemon` now scopes PID/log files per index
398
+ (`mcp-<name>.pid`) and passes the resolved database path to the child, so a
399
+ named-index daemon no longer collides with the default `mcp.pid` or opens
400
+ the default store (#772).
401
+
402
+ - Opening a store no longer throws `SQLiteError: no such column: T.name` when
403
+ `documents_fts` is still the legacy `fts5(name, body, content='documents')`
404
+ schema. `CREATE VIRTUAL TABLE IF NOT EXISTS` left that table in place, and
405
+ the CJK FTS rebuild's `DELETE FROM documents_fts` compiled against
406
+ `documents.name`, which does not exist (#792).
407
+
408
+ - Rerank cache keys now include the resolved `models.rerank` URI, so swapping
409
+ the configured reranker no longer serves the previous model's cached scores
410
+ (#764).
411
+
412
+ - `vsearch -c <collection>` no longer returns empty results for small
413
+ collections crowded out of the global ANN candidate pool. `searchVec` now
414
+ exact-scans the collection's vectors with `vec_distance_cosine` when the
415
+ set is within 20k rows (ANN + post-filter cannot see collections that never
416
+ enter global top-k, and sqlite-vec caps `k` at 4096 so a larger multiplier
417
+ alone is not enough). Larger collections still use capped ANN over-fetch
418
+ (#791, #803).
419
+
420
+ - `multi-get --format files` now emits the docid as its own CSV field
421
+ (`#docid,path,...`) instead of prepending it into the path field with a
422
+ space (`#docid path,...`), matching `search --format files` and keeping
423
+ naive comma-splitting usable (#760).
424
+
425
+ - `qmd embed` now takes an exclusive process lock (`.qmd-embed.lock` next to
426
+ the index DB) so concurrent invocations no longer race on `vectors_vec`
427
+ and fail with `UNIQUE constraint failed: vectors_vec.hash_seq`. A second
428
+ embed exits early with `Another embed process is already running. Skipping.`
429
+ Stale locks from crashed processes are recovered via PID identity checks
430
+ (#825).
431
+
432
+ - windows prepare fix #778 keeps dist build #824
433
+
434
+ - `qmd mcp` (stdio) now shuts down gracefully when stdin reaches EOF instead
435
+ of orphaning to PID 1 when the parent MCP client dies (#751): the server
436
+ closes its transport, gives in-flight request handlers a bounded window to
437
+ settle, closes the store (which disposes its llama.cpp instance), and lets
438
+ the process drain via `process.exitCode` (no forced `process.exit()`, which
439
+ has caused exit-time native crashes before).
440
+
441
+ - `qmd --version` no longer reports an unrelated repository's commit (#787).
442
+ The commit was discovered at runtime with `git -C <installDir> rev-parse`,
443
+ and `git -C` walks *up*, so any install nested inside another checkout
444
+ reported that checkout's HEAD — a global npm install under a git-managed
445
+ prefix such as Homebrew's `/opt/homebrew` claimed Homebrew's commit as
446
+ qmd's. Identical tarballs reported different "commits" depending only on
447
+ where they were installed, which is why one issue can collect three distinct
448
+ hashes for the same published build. `scripts/build.mjs` now stamps the
449
+ commit it built from into `dist/cli/build-info.json` (suffixed `-dirty` when
450
+ built from a modified tree), and the runtime prefers that. Source checkouts
451
+ still resolve their own HEAD, but only after confirming the enclosing
452
+ repository is qmd's; anything else reports no commit rather than a
453
+ misleading one. The lookup also no longer interpolates the install path into
454
+ a shell string, so a path containing a space stops silently dropping the
455
+ commit, and `--version` from a build runs no subprocess at all.
456
+
457
+ - `qmd doctor` no longer false-positives `.etag` HTTP sidecars (written by
458
+ `qmd pull` next to each download) as invalid GGUF models. The model-cache
459
+ check now only inspects real `.gguf` files, so a sidecar that happens to
460
+ sort before the blob no longer poisons the report. #812
461
+
462
+ - `qmd mcp stop` and `qmd mcp --http --daemon` now verify that a pidfile PID still belongs to a qmd process before signalling it or refusing to start. Recycled PIDs (common after reboot) are treated as stale: the pidfile is unlinked instead of SIGTERM'ing an unrelated process or blocking daemon start with a false "Already running" error (#806).
463
+
464
+ - `qmd collection add` now rejects missing paths and regular files before
465
+ creating collection configuration or index state. The error reports both the
466
+ received and resolved path so malformed shell arguments can be corrected.
467
+
468
+ - Claude Code plugin: scope the plugin `source` to `./skills` so installs
469
+ copy just the skills (~50 KB) instead of the entire repository. Previously a
470
+ canonical install materialized ~230 MB / 9,000+ items into
471
+ `~/.claude/plugins/cache/` — including a full npm dependency install
472
+ triggered by the repo-root `package.json`. Both skills (`qmd` and `release`)
473
+ still ship, unchanged. (#790)
474
+
475
+ - Claude Code plugin: releases now bump the plugin version in
476
+ `.claude-plugin/marketplace.json` in lockstep with `package.json`. The
477
+ plugin cache is keyed on this version, and it had been stuck at `0.1.0`
478
+ since February — so installed plugins never received skill updates
479
+ (users who installed in February are still being served that snapshot
480
+ today, despite the qmd skill nearly tripling in size since). Also bumps
481
+ the plugin to `2.6.3` as a one-time catch-up so existing installs pick
482
+ up the current skill on their next `claude plugin update`. (#789)
483
+
484
+ ### Changed
485
+
486
+ - `--full-path` no longer degrades silently when a result cannot be resolved on
487
+ disk (#785). A fallback there means the file moved or was deleted since the
488
+ last index, so `search`, `query`, `get` and `multi-get` now print a notice to
489
+ stderr naming how many results fell back and suggesting `qmd update`; stdout
490
+ stays machine-readable.
491
+ - `search`/`query` now decide per result whether to show the docid under
492
+ `--full-path`, matching `multi-get` and `get`: a result that resolved shows
493
+ its on-disk path and no docid, one that did not keeps its `qmd://` URI *and*
494
+ its docid, so it is still addressable. Previously the docid was dropped for
495
+ every row whenever the flag was set, leaving unresolved rows with neither a
496
+ usable path nor an identifier.
497
+ - `search --format csv` always emits the `docid` column, empty for rows that
498
+ resolved to an on-disk path. Under `--full-path` the header previously
499
+ dropped the column entirely — which also disagreed with the empty-result
500
+ header, always printed with `docid`. Column positions are now stable across
501
+ runs and formats.
502
+
503
+ ## [2.6.3] - 2026-06-24
504
+
505
+ ### Added
506
+
507
+ - `qmd embed --timeout <minutes>` overrides the embed session's max duration
508
+ (previously hardcoded to 30 minutes). Use a larger value to let a big index
509
+ finish in one run, or `--timeout 0` to remove the cap entirely. When the cap is
510
+ reached, remaining document batches are skipped as before, so re-running
511
+ `qmd embed` continues where it left off.
512
+
513
+ ### Documentation
514
+
515
+ - README: added a "Configuring `index.yml`" section documenting the full config
516
+ schema — `global_context`, `editor_uri`, the `models.embed`/`rerank`/`generate`
517
+ overrides, and per-collection `path`/`pattern`/`ignore`/`update`/
518
+ `includeByDefault`/`context` — with file-location rules (`XDG_CONFIG_HOME`,
519
+ `QMD_CONFIG_DIR`, named `{name}.yml`, project-local `.qmd/index.yml`). Every key
520
+ is verified against `src/collections.ts` and its consumers. Documents behavior
521
+ that previously existed only in code, where the absence of docs led contributors
522
+ to repeatedly re-submit already-shipped model-resolution fixes (#502, #559, #564)
523
+ and to request config that already works (#645, #678). Added
524
+ `XDG_CONFIG_HOME`/`QMD_CONFIG_DIR` to the environment-variable table and noted
525
+ the `index.yml` `models:` / `QMD_EMBED_MODEL` override path in the Model
526
+ Configuration section.
527
+ - README: expanded the per-collection `update` field into an "Automatic update
528
+ commands" subsection — the feature the maintainer publicly called under-documented
529
+ — covering execution via `bash -c` in the collection's directory, the run-then-
530
+ reindex order, the non-zero-exit abort behavior, and the `qmd collection
531
+ update-cmd` set/clear shortcut. The `ignore` key now states it is YAML-only (no
532
+ CLI command) and additive with the un-overridable built-in exclusions.
533
+ - `example-index.yml`: overhauled from three near-identical collections into a
534
+ fully-commented starter template where each collection demonstrates a distinct
535
+ feature (hierarchical context, auto-`update`, `ignore` patterns, non-markdown
536
+ globs, `includeByDefault: false`, and an all-fields example), plus commented
537
+ `editor_uri`/`models` stubs. README now links to it. Model URIs are intentionally
538
+ left as placeholders so the template can't drift from the defaults.
539
+ - README: documented collection filtering (`-c` semantics), the `collection
540
+ show`/`include`/`exclude`/`update-cmd` subcommands, the `--intent`/`--no-rerank`/
541
+ `-C`/`--full-path` search flags, the `--format <kind>` output selector (with the
542
+ legacy `--json`/`--csv`/`--md`/`--xml`/`--files` booleans noted as aliases),
543
+ `vector-search`/`deep-search` aliases, embed
544
+ memory flags (`--max-docs-per-batch`/`--max-batch-mb`), a sample `--explain`
545
+ score trace, the `qmd doctor`/`qmd init` commands, the `get` `:from:count`
546
+ suffix and `--no-line-numbers`, an MCP tool parameter reference, and a
547
+ Benchmarking section for `qmd bench`.
548
+ - docs/SYNTAX.md: removed the non-existent `q` MCP parameter example (the `query`
549
+ tool and REST endpoint accept only the `searches` array) and added a Scoping
550
+ section.
551
+ - README: removed the misleading `qmd update --pull` example. The `--pull` flag is
552
+ parsed but never consumed (`updateCollections()` ignores it); the real mechanism
553
+ for running `git pull` before re-indexing is a per-collection `update` command,
554
+ set via `qmd collection update-cmd`.
555
+
556
+ ### Fixed
557
+
558
+ - `qmd update` now rebuilds and publishes the CJK word and bigram indexes after
559
+ re-indexing collections instead of leaving both analyzed channels stale.
560
+ - MCP server instructions now tell agents to scope with the plural `collections`
561
+ parameter (matching the schema). The previous singular `collection` hint led
562
+ agents to pass a parameter that Zod silently strips, producing unscoped results.
563
+ The `get` instruction line also now documents the full `file.md:from:count`
564
+ range suffix instead of only the single-line `file.md:100` offset.
565
+
566
+ - Filesystem paths with special characters (`#`, `&`, spaces, `[]`, `()`, etc.)
567
+ now round-trip correctly through index → search → get. Previously
568
+ `reindexCollection` called `handelize()` on relative paths before storing
569
+ them, turning `# Meeting - 234232 3432 __ 5.md` into
570
+ `Meeting-234232-3432-5.md` and making `qmd get <actual-path>`,
571
+ `qmd get --full-path`, and `qmd ls` return dead or garbled paths. Paths are
572
+ now stored verbatim. Existing indexes auto-migrate on the next `qmd update`.
573
+
574
+ - FTS5 search now correctly matches dotted version strings like `2026.4.10`. The
575
+ `porter unicode61` tokenizer splits on dots (storing `2026`, `4`, `10` as
576
+ separate tokens), but the query sanitizer was stripping dots and producing
577
+ `2026410` which never matched. Dotted terms are now split and ANDed together
578
+ so version-string searches work as expected (#563).
579
+ - HTTP REST endpoints `/query` and `/search` now return `qmd://collection/path`
580
+ URIs in the `file` field, matching the output format used by the CLI and MCP
581
+ resource URIs. Previously the raw `displayPath` (`collection/path`) was
582
+ returned without the scheme prefix (#576).
583
+ - The embed session `maxDuration` is now env-configurable via
584
+ `QMD_EMBED_MAX_DURATION_MS` (default: 30 min). This prevents large-corpus
585
+ embeddings from being aborted by the hardcoded 30-minute ceiling (#673).
586
+ - `qmd query`, `qmd update`, and other commands no longer fail with
587
+ `SQLiteError: database is locked` when multiple processes run against the
588
+ same index in parallel (e.g. an `update` racing a long `embed`, the
589
+ first-open schema migration racing a routine command, or an agent fanning
590
+ out searches). `openDatabase` now sets `PRAGMA busy_timeout = 120000` on
591
+ every connection, so a writer that loses the race queues at batch
592
+ boundaries instead of throwing immediately. WAL handles read/write
593
+ concurrency but does not serialise concurrent writers, and `bun:sqlite`
594
+ and `better-sqlite3` both default the timeout to 0, so the loser
595
+ previously failed on the first DDL statement in `initializeDatabase`.
596
+ Override the default with `QMD_SQLITE_BUSY_TIMEOUT` (milliseconds; `0`
597
+ restores fail-fast). Two more crashes on the same concurrent-open
598
+ path are fixed: `trigger documents_ai already exists` (the FTS sync
599
+ triggers were dropped and recreated as separate statements on every
600
+ open, so two processes interleaved between the `DROP` and the
601
+ `CREATE`; `busy_timeout` serialises individual statements but not the
602
+ pair) and `database is locked` while migrating a cold database to WAL
603
+ (the `journal_mode` switch needs a brief exclusive lock and does not
604
+ invoke the busy handler). FTS trigger setup is now gated behind
605
+ `PRAGMA user_version` inside one `IMMEDIATE` transaction, and the WAL
606
+ migration retries within the busy-timeout budget.
607
+
608
+ ## [2.5.3] - 2026-05-28
609
+
610
+ ### Features
611
+
612
+ - `qmd get` now accepts a `:from:count` suffix on a path or docid (e.g.
613
+ `qmd get "#abc123:120:40"` reads 40 lines starting at line 120). Explicit
614
+ `--from`/`-l` flags still override the suffix. The MCP `get` tool accepts the
615
+ same suffix.
616
+ - `qmd get` and `qmd multi-get` are now **line-numbered by default** and print
617
+ the document's `#docid` and `qmd://` path in the output header. Disable line
618
+ numbers with `--no-line-numbers`. The MCP `get`/`multi_get` tools default
619
+ `lineNumbers` to `true` to match.
620
+ - `qmd multi-get` now includes the `#docid` in every output format
621
+ (`--md`, `--json`, `--csv`, `--xml`, `--files`, and the default CLI view),
622
+ consistent with `qmd search`.
623
+ - `qmd get` and `qmd multi-get` accept `--full-path`, which replaces the
624
+ `qmd://` path + `#docid` with the document's on-disk filesystem path (handy for
625
+ piping into `Read`/`Edit`/an editor). Falls back to the canonical `qmd://` +
626
+ docid header when the file no longer exists on disk.
627
+ - `qmd search` / `qmd query` now show a clearer hit identifier: the default CLI
628
+ view (and the new `**file:**` line in `--md` output) always prints the full
629
+ `qmd://collection/path` URI so you can pipe it straight back into `qmd get`.
630
+ - `qmd search` / `qmd query` accept `--full-path` with the same semantics as
631
+ `qmd get`: the result label becomes the file's on-disk path — `./`-prefixed
632
+ relative path when the file lives in a subfolder of `$PWD`, absolute realpath
633
+ otherwise — and the per-result `#docid` is dropped because the path is the
634
+ identifier. The leading `./` is intentional so the output is unambiguously a
635
+ filesystem path. Applies to all output formats.
636
+ - `qmd get` and `qmd multi-get` now also use the `./`-prefixed convention when
637
+ `--full-path` renders a path under `$PWD`, matching `search`/`query`.
638
+ - New `--format <kind>` flag selects the output format (`cli` | `json` | `csv` |
639
+ `md` | `xml` | `files`) for `search`, `query`, and `multi-get`. The legacy
640
+ boolean aliases (`--json`/`--csv`/`--md`/`--xml`/`--files`) still work but are
641
+ no longer in `--help`; prefer `--format`.
642
+
643
+ ### Fixes
644
+
645
+ - Launcher: source-mode runner selection now prefers Node + tsx over Bun when
646
+ both `package-lock.json` and `bun.lock` are present in the package root,
647
+ mirroring the dist-mode "npm priority" rule. Fixes pnpm-global installs that
648
+ copy the entire working tree (including `.git` and `bun.lock`) into the
649
+ install dir and previously routed through Bun, causing ABI mismatches with
650
+ the Node-built `better-sqlite3` / `sqlite-vec` native modules.
651
+ - Darwin Metal: llama-using commands (`query`, `vsearch`, `embed`) no longer
652
+ dump a multi-kB GGML/Metal backtrace at process exit even when output
653
+ succeeded. The libggml-metal static `ggml_metal_device` destructor asserts
654
+ `[rsets->data count] == 0` during `__cxa_finalize_ranges`, but the
655
+ buffer-free path never calls the symmetric `ggml_metal_device_rsets_rm`
656
+ to remove released rsets from the device collection (upstream
657
+ ggml-org/llama.cpp#22593, one-line fix open as PR #22595). The assertion
658
+ only fires when `process.exit()` skips Node's `beforeExit` hook, which is
659
+ what node-llama-cpp uses to auto-dispose Metal contexts. Primary fix:
660
+ `finishSuccessfulCliCommand` now sets `process.exitCode = 0` and returns
661
+ instead of calling `process.exit(0)`, so `beforeExit` fires and the native
662
+ binding cleans up before libc's static destructor runs. Defense-in-depth:
663
+ the launcher (`bin/qmd`) and the npm test driver (`scripts/test-all.mjs`
664
+ + the `test:bun` / `test:unit` package.json scripts) also set
665
+ `GGML_METAL_NO_RESIDENCY=1` on darwin before spawning node/bun, covering
666
+ error paths and tests that still terminate via `process.exit()`. The env
667
+ var must be set before node/bun start — libggml-metal reads it via libc
668
+ `getenv` at module-load time, and Bun does not propagate `process.env`
669
+ mutations to libc `setenv` — so it lives in the launcher rather than in
670
+ test-preload. Residency sets give no measurable speedup for QMD's
671
+ short-lived CLI workflow (benchmarked on M3 Pro). Opt back in with
672
+ `QMD_METAL_KEEP_RESIDENCY=1` for long-lived qmd processes (e.g. the MCP
673
+ daemon may benefit on hot reload) or to triage the upstream fix.
674
+ `qmd doctor` reports the mitigation state. Minimal reproduction:
675
+ `scripts/repro-metal-rsets-crash.mjs`.
676
+
677
+ ### Docs
678
+
679
+ - qmd skill: emphasize reading line ranges with `get`'s built-in
680
+ `:from:count` suffix / `--from`/`-l` flags instead of piping through
681
+ `sed`/`head`/`tail`; cite the docid and line numbers now present in retrieval
682
+ output; and author structured `intent:`/`lex:`/`vec:`/`hyde:` queries yourself
683
+ rather than relying on built-in query expansion.
684
+
685
+ ## [2.5.2] - 2026-05-22
686
+
687
+ ### Fixes
688
+
689
+ - Launcher: Rewrite `bin/qmd` as a Node-based shebang polyglot to fix global npm installation execution failures on Windows (#668 / #452), while supporting seamless fallback to Bun in Node-less environments.
690
+
691
+ ## [2.5.1] - 2026-05-20
692
+
693
+ ### Changes
694
+
695
+ - Release: publish from GitHub Actions via npm Trusted Publishing/OIDC instead of a long-lived `NPM_TOKEN` secret.
696
+
697
+ ## [2.5.0] - 2026-05-19
698
+
699
+ ### Changes
700
+
701
+ - Dependencies: update core SQLite/config/chunking packages (`better-sqlite3`, `yaml`, `web-tree-sitter`, `tree-sitter-go`, and `tree-sitter-python`) while keeping incompatible `zod`, `tsx`, and `vitest` majors pinned.
702
+ - Agent skills: add `qmd skills list|get|path` to serve version-matched runtime skill instructions from the installed CLI, and make `qmd skill install` write a stable discovery stub so installed agent skills do not go stale after QMD upgrades.
703
+ - CLI: add `qmd doctor` for index/runtime diagnostics, including SQLite/sqlite-vec versions, embedding fingerprint freshness, mixed-fingerprint detection, safe legacy fingerprint adoption, and content-hash sampling.
704
+
705
+ ### Fixes
706
+
707
+ - Launcher: prefer runnable TypeScript source in git checkouts even when ignored `dist/` artifacts exist, while packaged installs continue to run `dist/`.
708
+ - GPU: keep node-llama-cpp's documented `gpu: "auto"` initialization as the primary path, then perform no-build packaged CUDA/Vulkan/Metal probes only if auto falls back to CPU.
709
+ - CLI: move GPU/CPU runtime diagnostics out of `qmd status`; use `qmd doctor` for device probing and related environment guidance.
710
+ - CLI: point unexpected command/setup failures toward `qmd doctor` so diagnostics are the default next step when QMD behaves incorrectly.
711
+ - Doctor: explicitly warn when `content_vectors` contains multiple non-empty embedding fingerprint names, with the per-fingerprint document/chunk breakdown.
712
+ - Embed: make the TTY progress line label byte-based input progress explicitly, show embedded chunks as a count, and shorten the displayed model name.
713
+ - Embed: retain per-chunk failure details, retry failed chunks after later successful embeds and again when no other chunks remain, clear recovered errors, and cap retries to avoid endless loops.
714
+ - Tests: expand the container smoke harness to cover npm-global, npx-style, and Bun-global install scenarios, always checking auto and `QMD_FORCE_CPU=1` doctor modes, with opt-in tiny `qmd embed` and GPU probe runs for supported container runtimes.
715
+ - Embedding: fingerprint vector metadata using the active embedding model and formatting/chunking parameters so stale vectors are treated as pending after search semantics change. Legacy `content_vectors` columns are migrated lazily on first vector-health/write use to preserve fast QMD startup.
716
+
717
+ - Skill: expand the packaged QMD skill with retrieval-first workflows, structured query examples, wiki/source collection guidance, and safe fallbacks when model-backed search is unavailable.
718
+ - Tests: make `bun run test` execute the local unit suite under both Node/Vitest and Bun (`test:node` + `test:bun`) so runtime-specific regressions are caught before CI.
719
+ - Model config: centralize embedding/rerank/generation model resolution so `qmd embed`, `status`, `query`, `vsearch`, `pull`, SDK vector search, and `bench` use the same active `.qmd/index.yaml` model hints and environment fallbacks.
720
+ - GPU/status: `qmd status` now uses the same embedding model identity as `qmd embed` when computing pending embeddings, so URI-backed embeddings are not incorrectly reported as pending under the legacy `embeddinggemma` alias.
721
+ - GPU status: `qmd status` now always shows GPU mode/configuration without unsafe native probing, and CPU-fallback warnings point to `QMD_STATUS_DEVICE_PROBE=1 qmd status` for an actual backend probe. The no-GPU warning is emitted once per process instead of once per LLM instance during benchmarks.
722
+ - GPU: add `QMD_FORCE_CPU=1` / `--no-gpu` to bypass CUDA/Vulkan/Metal probing entirely, and route native llama.cpp stdout noise to stderr so JSON output stays parseable during search/query commands.
723
+ - Snippet line numbers: `qmd_query` (MCP), HTTP `/query`, and `qmd query`
724
+ (CLI JSON output and snippet headers) now return absolute source-file
725
+ line numbers instead of chunk-local ones, so the `line` field can be
726
+ passed back to `qmd_get` as `fromLine` without a separate lookup.
727
+ Snippet selection remains scoped to the best matching chunk
728
+ (preserves #149).
729
+ - CLI: `qmd query --full` now emits the full document body in all output
730
+ formats (json, csv, md, xml), restoring the documented behavior of the
731
+ flag. Previously it returned only the best matching chunk (~3.6KB max
732
+ per result). Output payload for `--full` queries is now proportional
733
+ to total document size.
734
+ - macOS Metal: `qmd query --json` now flushes successful JSON output and uses a safe immediate-exit path on Darwin to avoid ggml Metal finalizer aborts; other commands still dispose LLM contexts/models before the llama runtime. #368
735
+ - Embedding: require complete chunk coverage before treating a document as
736
+ embedded, remove partial vectors when chunk/session failures leave a
737
+ document incomplete, and keep `qmd status` pending counts honest after
738
+ interrupted long embed runs. #637 #378
739
+ - Embedding: `qmd embed -c <collection>` now scopes pending-doc selection
740
+ to the requested collection instead of embedding global pending work.
741
+ Scoped `--force` clears only collection-owned vectors, preserves shared
742
+ hashes referenced by sibling collections, and drops `vectors_vec` only
743
+ when the scoped clear empties all vectors.
744
+ - Hybrid search: weight RRF lists by query type so original FTS and original vector evidence get the intended 2x boost, instead of accidentally boosting the first lexical expansion. #591
745
+ - MCP: seed llama.cpp/GGML quiet env vars before launching `qmd mcp` so native logs cannot pollute stdio JSON-RPC framing. #593
746
+ - CLI: remove CommonJS `require()` calls from ESM index path normalization so `qmd --index <path>` no longer crashes with `ERR_AMBIGUOUS_MODULE_SYNTAX` on Node 22+. #634
747
+ - Windows CUDA: serialize llama.cpp embedding/reranking contexts by default to avoid intermittent `ggml-cuda.cu:98` crashes in `qmd query`; set `QMD_EMBED_PARALLELISM` to opt back into parallel contexts if your driver is stable. #519
748
+ - MCP: make `qmd mcp --index <name>` use the selected index for both foreground and daemon HTTP servers instead of falling back to the default store. #343
749
+ - Embedding: respect `QMD_EMBED_MODEL` consistently for vector indexing and vector-backed search, with default-model fallback when unset.
750
+ - Config: use one home-directory resolver for YAML config and the default SQLite cache path, avoiding Windows CLI/MCP split-brain when `HOME` is unset.
751
+ - GPU: respect explicit `QMD_LLAMA_GPU=metal|vulkan|cuda` backend overrides instead of always using auto GPU selection. #529
752
+ - Fix: preserve original filename case in `handelize()`. The previous
753
+ `.toLowerCase()` call made indexed paths unreachable on case-sensitive
754
+ filesystems (Linux). `qmd update` automatically migrates legacy
755
+ lowercase paths without re-embedding.
756
+ - CLI: make `qmd status` skip native `node-llama-cpp` device probing by
757
+ default so status stays safe on machines with broken or unsupported GPU
758
+ drivers. Set `QMD_STATUS_DEVICE_PROBE=1` to opt in.
759
+ - CLI: lazy-load `node-llama-cpp` so lightweight commands such as
760
+ `qmd status` do not import native ML dependencies or trigger llama.cpp
761
+ builds on ARM/no-GPU machines. #491
762
+ - Store: keep content rows referenced by inactive documents during orphan
763
+ cleanup so `qmd update` preserves soft-deleted tombstones for removed
764
+ files. #585
765
+ - Packaging: install AST grammar WASM packages as required dependencies so
766
+ Bun global installs include TypeScript/TSX/JavaScript grammars, and add a
767
+ `smoke:package-grammars` verification command. #595
768
+ - Launcher: add wrapper smoke coverage for scoped package, npm/npx,
769
+ Homebrew/Linuxbrew, Bun global symlink layouts, and `$BUN_INSTALL`
770
+ false-positive runtime selection regressions. #351 #353 #354 #356 #358 #359
771
+
772
+ ## [2.1.0] - 2026-04-05
773
+
774
+ Code files now chunk at function and class boundaries via tree-sitter,
775
+ clickable editor links land you at the right line from search results,
776
+ and per-collection model configuration means you can point different
777
+ collections at different embedding models. 25+ community PRs fix
778
+ embedding stability, BM25 accuracy, and cross-platform launcher issues.
779
+
780
+ ### Changes
781
+
782
+ - AST-aware chunking for code files via `web-tree-sitter`. Supported
783
+ languages: TypeScript/JavaScript, Python, Go, and Rust. Code files
784
+ are chunked at function, class, and import boundaries instead of
785
+ arbitrary text positions. Markdown and unknown file types are unchanged.
786
+ `--chunk-strategy <auto|regex>` flag on `qmd embed` and `qmd query`
787
+ (default `regex`). SDK: `chunkStrategy` option on `embed()` and
788
+ `search()`. `qmd status` shows grammar availability.
789
+ - `qmd bench <fixture.json>` command for search quality benchmarks.
790
+ Measures precision@k, recall, MRR, and F1 across BM25, vector, hybrid,
791
+ and full pipeline backends. Ships with an example fixture against
792
+ the eval-docs test collection. #470 (thanks @jmilinovich)
793
+ - `models:` section in `index.yml` lets you configure `embed`, `rerank`,
794
+ and `generate` model URIs per collection. Resolution order is
795
+ config > env var (`QMD_EMBED_MODEL`, `QMD_RERANK_MODEL`,
796
+ `QMD_GENERATE_MODEL`) > built-in default. #502
797
+ (thanks @JohnRichardEnders)
798
+ - CLI search output now emits clickable OSC 8 terminal hyperlinks when
799
+ stdout is a TTY. Links resolve `qmd://` paths to absolute filesystem
800
+ paths and open in editors via URI templates (default:
801
+ `vscode://file/{path}:{line}:{col}`). Configure with `QMD_EDITOR_URI`
802
+ or `editor_uri` in the YAML config. #508 (thanks @danmackinlay)
803
+ - `--no-rerank` flag skips the reranking step in `qmd query` — useful
804
+ when you want fast results or don't have a GPU. Also exposed as
805
+ `rerank: false` on the MCP `query` tool. #370 (thanks @mvanhorn),
806
+ #478 (thanks @zestyboy)
807
+ - ONNX conversion script for deploying embedding models via
808
+ Transformers.js. #399 (thanks @shreyaskarnik)
809
+ - GitHub Actions workflow to build the Nix flake on Linux and macOS.
810
+
811
+ ### Fixes
812
+
813
+ - Embedding: prevent `qmd embed` from running indefinitely when the
814
+ embedding loop stalls. #458 (thanks @ccc-fff)
815
+ - Embedding: truncate oversized text before embedding to prevent GGML
816
+ crash, and bound memory usage during batch embedding. #393
817
+ (thanks @lskun), #395 (thanks @ProgramCaiCai)
818
+ - Embedding: set explicit embed context size (default 2048, configurable
819
+ via `QMD_EMBED_CONTEXT_SIZE`) instead of using the model's full
820
+ window. #500
821
+ - Embedding: error on dimension mismatch instead of silently rebuilding
822
+ the vec0 table. #501
823
+ - Embedding: handle vec0 `OR REPLACE` limitation in `insertEmbedding`.
824
+ #456 (thanks @antonio-mello-ai)
825
+ - Embedding: fix model selection when multiple models are configured.
826
+ #494
827
+ - BM25: correct field weights to include all 3 FTS columns — title,
828
+ body, and path were not weighted correctly. #462 (thanks @goldsr09)
829
+ - BM25: handle hyphenated tokens in FTS5 lex queries so terms like
830
+ "real-time" match correctly. #463 (thanks @goldsr09)
831
+ - BM25: preserve underscores in search terms instead of stripping them.
832
+ #404
833
+ - BM25: use CTE in `searchFTS` to prevent query planner regression with
834
+ collection filter.
835
+ - Reranker: increase default context size 2048→4096 and make
836
+ configurable via `QMD_RERANK_CONTEXT_SIZE`. Fix template overhead
837
+ underestimate 200→512. #453 (thanks @builderjarvis)
838
+ - GPU: catch initialization failures and fall back to CPU instead of
839
+ crashing.
840
+ - MCP: read version from `package.json` instead of hardcoding. #431
841
+ - MCP: include collection name in status output. #416
842
+ - Multi-get: support brace expansion patterns in glob matching. #424
843
+ - Launcher: prioritize `package-lock.json` to prevent Bun false
844
+ positive. #385 (thanks @rymalia)
845
+ - Launcher: remove `$BUN_INSTALL` check that caused false Bun detection.
846
+ #362 (thanks @syedair)
847
+ - Launcher: skip Git Bash path detection on WSL. #371
848
+ (thanks @oysteinkrog)
849
+ - Model cache: respect `XDG_CACHE_HOME` for model cache directory. #457
850
+ (thanks @antonio-mello-ai)
851
+ - SQLite: add macOS Homebrew SQLite support for Bun and restore
852
+ actionable errors. #377 (thanks @serhii12)
853
+ - Pin zod to exact 4.2.1 to fix `tsc` build failure. #382
854
+ (thanks @rymalia)
855
+ - Preserve dots and original case in `handelize()` — filenames like
856
+ `MEMORY.md` no longer become `memory-md`. #475 (thanks @alexei-led)
857
+ - Include `line` in `--json` search output so editor integrations can
858
+ jump directly to `file:line`. #506 (thanks @danmackinlay)
859
+ - Nix: fix paths in flake and make Bun dependency a fixed-output
860
+ derivation so sandboxed Linux builds work offline. #479
861
+ (thanks @surma-dump)
862
+ - Sync stale `bun.lock` (`better-sqlite3` 11.x → 12.x). CI and release
863
+ script now use `--frozen-lockfile` to prevent recurrence. #386
864
+ (thanks @Mic92)
865
+ - Approve native build scripts in pnpm so `better-sqlite3` and
866
+ tree-sitter modules compile correctly. Update vitest ^3.0.0 → ^3.2.4.
867
+
868
+ ## [2.0.1] - 2026-03-10
869
+
870
+ ### Changes
871
+
872
+ - `qmd skill install` copies the packaged QMD skill into
873
+ `~/.claude/commands/` for one-command setup. #355 (thanks @nibzard)
874
+
875
+ ### Fixes
876
+
877
+ - Fix Qwen3-Embedding GGUF filename case — HuggingFace filenames are
878
+ case-sensitive, the lowercase variant returned 404. #349 (thanks @byheaven)
879
+ - Resolve symlinked global launcher path so `qmd` works correctly when
880
+ installed via `npm i -g`. #352 (thanks @nibzard)
881
+
882
+ ## [2.0.0] - 2026-03-10
883
+
884
+ QMD 2.0 declares a stable library API. The SDK is now the primary interface —
885
+ the MCP server is a clean consumer of it, and the source is organized into
886
+ `src/cli/` and `src/mcp/`. Also: Node 25 support and a runtime-aware bin wrapper
887
+ for bun installs.
888
+
889
+ ### Changes
890
+
891
+ - Stable SDK API with `QMDStore` interface — search, retrieval, collection/context
892
+ management, indexing, lifecycle
893
+ - Unified `search()`: pass `query` for auto-expansion or `queries` for
894
+ pre-expanded lex/vec/hyde — replaces the old query/search/structuredSearch split
895
+ - New `getDocumentBody()`, `getDefaultCollectionNames()`, `Maintenance` class
896
+ - MCP server rewritten as a clean SDK consumer — zero internal store access
897
+ - CLI and MCP organized into `src/cli/` and `src/mcp/` subdirectories
898
+ - Runtime-aware `bin/qmd` wrapper detects bun vs node to avoid ABI mismatches.
899
+ Closes #319
900
+ - `better-sqlite3` bumped to ^12.4.5 for Node 25 support. Closes #257
901
+ - Utility exports: `extractSnippet`, `addLineNumbers`, `DEFAULT_MULTI_GET_MAX_BYTES`
902
+
903
+ ### Fixes
904
+
905
+ - Remove unused `import { resolve }` in store.ts that shadowed local export
906
+
907
+ ## [1.1.6] - 2026-03-09
908
+
909
+ QMD can now be used as a library. `import { createStore } from '@tobilu/qmd'`
910
+ gives you the full search and indexing API — hybrid query, BM25, structured
911
+ search, collection/context management — without shelling out to the CLI.
912
+
913
+ ### Changes
914
+
915
+ - **SDK / library mode**: `createStore({ dbPath, config })` returns a
916
+ `QMDStore` with `query()`, `search()`, `structuredSearch()`, `get()`,
917
+ `multiGet()`, and collection/context management methods. Supports inline
918
+ config (no files needed) or a YAML config path.
919
+ - **Package exports**: `package.json` now declares `main`, `types`, and
920
+ `exports` so bundlers and TypeScript resolve `@tobilu/qmd` correctly.
921
+
922
+ ## [1.1.5] - 2026-03-07
923
+
924
+ Ambiguous queries like "performance" now produce dramatically better results
925
+ when the caller knows what they mean. The new `intent` parameter steers all
926
+ five pipeline stages — expansion, strong-signal bypass, chunk selection,
927
+ reranking, and snippet extraction — without searching on its own. Design and
928
+ original implementation by Ilya Grigorik (@vyalamar) in #180.
929
+
930
+ ### Changes
931
+
932
+ - **Intent parameter**: optional `intent` string disambiguates queries across
933
+ the entire search pipeline. Available via CLI (`--intent` flag or `intent:`
934
+ line in query documents), MCP (`intent` field on the query tool), and
935
+ programmatic API. Adapted from PR #180 (thanks @vyalamar).
936
+ - **Query expansion**: when intent is provided, the expansion LLM prompt
937
+ includes `Query intent: {intent}`, matching the finetune training data
938
+ format for better-aligned expansions.
939
+ - **Reranking**: intent is prepended to the rerank query so Qwen3-Reranker
940
+ scores with domain context.
941
+ - **Chunk selection**: intent terms scored at 0.5× weight alongside query
942
+ terms (1.0×) when selecting the best chunk per document for reranking.
943
+ - **Snippet extraction**: intent terms scored at 0.3× weight to nudge
944
+ snippets toward intent-relevant lines without overriding query anchoring.
945
+ - **Strong-signal bypass disabled with intent**: when intent is provided, the
946
+ BM25 strong-signal shortcut is skipped — the obvious keyword match may not
947
+ be what the caller wants.
948
+ - **MCP instructions**: callers are now guided to provide `intent` on every
949
+ search call for disambiguation.
950
+ - **Query document syntax**: `intent:` recognized as a line type. At most one
951
+ per document, cannot appear alone. Grammar updated in `docs/SYNTAX.md`.
952
+
953
+ ## [1.1.2] - 2026-03-07
954
+
955
+ 13 community PRs merged. GPU initialization replaced with node-llama-cpp's
956
+ built-in `autoAttempt` — deleting ~220 lines of manual fallback code and
957
+ fixing GPU issues reported across 10+ PRs in one shot. Reranking is faster
958
+ through chunk deduplication and a parallelism cap that prevents VRAM
959
+ exhaustion.
960
+
961
+ ### Changes
962
+
963
+ - **GPU init**: use node-llama-cpp's `build: "autoAttempt"` instead of manual
964
+ GPU backend detection. Automatically tries Metal/CUDA/Vulkan and falls back
965
+ gracefully. #310 (thanks @giladgd — the node-llama-cpp author)
966
+ - **Query `--explain`**: `qmd query --explain` exposes retrieval score traces
967
+ — backend scores, per-list RRF contributions, top-rank bonus, reranker
968
+ score, and final blended score. Works in JSON and CLI output. #242
969
+ (thanks @vyalamar)
970
+ - **Collection ignore patterns**: `ignore: ["Sessions/**", "*.tmp"]` in
971
+ collection config to exclude files from indexing. #304 (thanks @sebkouba)
972
+ - **Multilingual embeddings**: `QMD_EMBED_MODEL` env var lets you swap in
973
+ models like Qwen3-Embedding for non-English collections. #273 (thanks
974
+ @daocoding)
975
+ - **Configurable expansion context**: `QMD_EXPAND_CONTEXT_SIZE` env var
976
+ (default 2048) — previously used the model's full 40960-token window,
977
+ wasting VRAM. #313 (thanks @0xble)
978
+ - **`candidateLimit` exposed**: `-C` / `--candidate-limit` flag and MCP
979
+ parameter to tune how many candidates reach the reranker. #255 (thanks
980
+ @pandysp)
981
+ - **MCP multi-session**: HTTP transport now supports multiple concurrent
982
+ client sessions, each with its own server instance. #286 (thanks @joelev)
983
+
984
+ ### Fixes
985
+
986
+ - **Reranking performance**: cap parallel rerank contexts at 4 to prevent
987
+ VRAM exhaustion on high-core machines. Deduplicate identical chunk texts
988
+ before reranking — same content from different files now shares a single
989
+ reranker call. Cache scores by content hash instead of file path.
990
+ - Deactivate stale docs when all files are removed from a collection and
991
+ `qmd update` is run. #312 (thanks @0xble)
992
+ - Handle emoji-only filenames (`🐘.md` → `1f418.md`) instead of crashing.
993
+ #308 (thanks @debugerman)
994
+ - Skip unreadable files during indexing (e.g. iCloud-evicted files returning
995
+ EAGAIN) instead of crashing. #253 (thanks @jimmynail)
996
+ - Suppress progress bar escape sequences when stderr is not a TTY. #230
997
+ (thanks @dgilperez)
998
+ - Emit format-appropriate empty output (`[]` for JSON, CSV header for CSV,
999
+ etc.) instead of plain text "No results." #228 (thanks @amsminn)
1000
+ - Correct Windows sqlite-vec package name (`sqlite-vec-windows-x64`) and add
1001
+ `sqlite-vec-linux-arm64`. #225 (thanks @ilepn)
1002
+ - Fix claude plugin setup CLI commands in README. #311 (thanks @gi11es)
1003
+
1004
+ ## [1.1.1] - 2026-03-06
1005
+
1006
+ ### Fixes
1007
+
1008
+ - Reranker: truncate documents exceeding the 2048-token context window
1009
+ instead of silently producing garbage scores. Long chunks (e.g. from
1010
+ PDF ingestion) now get a fair ranking.
1011
+ - Nix: add python3 and cctools to build dependencies. #214 (thanks
1012
+ @pcasaretto)
1013
+
1014
+ ## [1.1.0] - 2026-02-20
1015
+
1016
+ QMD now speaks in **query documents** — structured multi-line queries where every line is typed (`lex:`, `vec:`, `hyde:`), combining keyword precision with semantic recall. A single plain query still works exactly as before (it's treated as an implicit `expand:` and auto-expanded by the LLM). Lex now supports quoted phrases and negation (`"C++ performance" -sports -athlete`), making intent-aware disambiguation practical. The formal query grammar is documented in `docs/SYNTAX.md`.
1017
+
1018
+ The npm package now uses the standard `#!/usr/bin/env node` bin convention, replacing the custom bash wrapper. This fixes native module ABI mismatches when installed via bun and works on any platform with node >= 22 on PATH.
1019
+
1020
+ ### Changes
1021
+
1022
+ - **Query document format**: multi-line queries with typed sub-queries (`lex:`, `vec:`, `hyde:`). Plain queries remain the default (`expand:` implicit, but not written inside the document). First sub-query gets 2× fusion weight — put your strongest signal first. Formal grammar in `docs/SYNTAX.md`.
1023
+ - **Lex syntax**: full BM25 operator support. `"exact phrase"` for verbatim matching; `-term` and `-"phrase"` for exclusions. Essential for disambiguation when a term is overloaded across domains (e.g. `performance -sports -athlete`).
1024
+ - **`expand:` shortcut**: send a single plain query (or start the document with `expand:` on its only line) to auto-expand via the local LLM. Query documents themselves are limited to `lex`, `vec`, and `hyde` lines.
1025
+ - **MCP `query` tool** (renamed from `structured_search`): rewrote the tool description to fully teach AI agents the query document format, lex syntax, and combination strategy. Includes worked examples with intent-aware lex.
1026
+ - **HTTP `/query` endpoint** (renamed from `/search`; `/search` kept as silent alias).
1027
+ - **`collections` array filter**: filter by multiple collections in a single query (`collections: ["notes", "brain"]`). Removed the single `collection` string param — array only.
1028
+ - **Collection `include`/`exclude`**: `includeByDefault: false` hides a collection from all queries unless explicitly named via `collections`. CLI: `qmd collection exclude <name>` / `qmd collection include <name>`.
1029
+ - **Collection `update-cmd`**: attach a shell command that runs before every `qmd update` (e.g. `git stash && git pull --rebase --ff-only && git stash pop`). CLI: `qmd collection update-cmd <name> '<cmd>'`.
1030
+ - **`qmd status` tips**: shows actionable tips when collections lack context descriptions or update commands.
1031
+ - **`qmd collection` subcommands**: `show`, `update-cmd`, `include`, `exclude`. Bare `qmd collection` now prints help.
1032
+ - **Packaging**: replaced custom bash wrapper with standard `#!/usr/bin/env node` shebang on `dist/qmd.js`. Fixes native module ABI mismatches when installed via bun, and works on any platform where node >= 22 is on PATH.
1033
+ - **Removed MCP tools** `search`, `vector_search`, `deep_search` — all superseded by `query`.
1034
+ - **Removed** `qmd context check` command.
1035
+ - **CLI timing**: each LLM step (expand, embed, rerank) prints elapsed time inline (`Expanding query... (4.2s)`).
1036
+
1037
+ ### Fixes
1038
+
1039
+ - `qmd collection list` shows `[excluded]` tag for collections with `includeByDefault: false`.
1040
+ - Default searches now respect `includeByDefault` — excluded collections are skipped unless explicitly named.
1041
+ - Fix main module detection when installed globally via npm/bun (symlink resolution).
1042
+
1043
+ ## [1.0.7] - 2026-02-18
1044
+
1045
+ ### Changes
1046
+
1047
+ - LLM: add LiquidAI LFM2-1.2B as an alternative base model for query
1048
+ expansion fine-tuning. LFM2's hybrid architecture (convolutions + attention)
1049
+ is 2x faster at decode/prefill vs standard transformers — good fit for
1050
+ on-device inference.
1051
+ - CLI: support multiple `-c` flags to search across several collections at
1052
+ once (e.g. `qmd search -c notes -c journals "query"`). #191 (thanks
1053
+ @openclaw)
1054
+
1055
+ ### Fixes
1056
+
1057
+ - Return empty JSON array `[]` instead of no output when `--json` search
1058
+ finds no results.
1059
+ - Resolve relative paths passed to `--index` so they don't produce malformed
1060
+ config entries.
1061
+ - Respect `XDG_CONFIG_HOME` for collection config path instead of always
1062
+ using `~/.config`. #190 (thanks @openclaw)
1063
+ - CLI: empty-collection hint now shows the correct `collection add` command.
1064
+ #200 (thanks @vincentkoc)
1065
+
1066
+ ## [1.0.6] - 2026-02-16
1067
+
1068
+ ### Changes
1069
+
1070
+ - CLI: `qmd status` now shows models with full HuggingFace links instead of
1071
+ static names in `--help`. Model info is derived from the actual configured
1072
+ URIs so it stays accurate if models change.
1073
+ - Release tooling: pre-push hook handles non-interactive shells (CI, editors)
1074
+ gracefully — warnings auto-proceed instead of hanging on a tty prompt.
1075
+ Annotated tags now resolve correctly for CI checks.
1076
+
1077
+ ## [1.0.5] - 2026-02-16
1078
+
1079
+ The npm package now ships compiled JavaScript instead of raw TypeScript,
1080
+ removing the `tsx` runtime dependency. A new `/release` skill automates the
1081
+ full release workflow with changelog validation and git hook enforcement.
1082
+
1083
+ ### Changes
1084
+
1085
+ - Build: compile TypeScript to `dist/` via `tsc` so the npm package no longer
1086
+ requires `tsx` at runtime. The `qmd` shell wrapper now runs `dist/qmd.js`
1087
+ directly.
1088
+ - Release tooling: new `/release` skill that manages the full release
1089
+ lifecycle — validates changelog, installs git hooks, previews release notes,
1090
+ and cuts the release. Auto-populates `[Unreleased]` from git history when
1091
+ empty.
1092
+ - Release tooling: `scripts/extract-changelog.sh` extracts cumulative notes
1093
+ for the full minor series (e.g. 1.0.0 through 1.0.5) for GitHub releases.
1094
+ Includes `[Unreleased]` content in previews.
1095
+ - Release tooling: `scripts/release.sh` renames `[Unreleased]` to a versioned
1096
+ heading and inserts a fresh empty `[Unreleased]` section automatically.
1097
+ - Release tooling: pre-push git hook blocks `v*` tag pushes unless
1098
+ `package.json` version matches the tag, a changelog entry exists, and CI
1099
+ passed on GitHub.
1100
+ - Publish workflow: GitHub Actions now builds TypeScript, creates a GitHub
1101
+ release with cumulative notes extracted from the changelog, and publishes
1102
+ to npm with provenance.
1103
+
1104
+ ## [1.0.0] - 2026-02-15
1105
+
1106
+ QMD now runs on both Node.js and Bun, with up to 2.7x faster reranking
1107
+ through parallel GPU contexts. GPU auto-detection replaces the unreliable
1108
+ `gpu: "auto"` with explicit CUDA/Metal/Vulkan probing.
1109
+
1110
+ ### Changes
1111
+
1112
+ - Runtime: support Node.js (>=22) alongside Bun via a cross-runtime SQLite
1113
+ abstraction layer (`src/db.ts`). `bun:sqlite` on Bun, `better-sqlite3` on
1114
+ Node. The `qmd` wrapper auto-detects a suitable Node.js install via PATH,
1115
+ then falls back to mise, asdf, nvm, and Homebrew locations.
1116
+ - Performance: parallel embedding & reranking via multiple LlamaContext
1117
+ instances — up to 2.7x faster on multi-core machines.
1118
+ - Performance: flash attention for ~20% less VRAM per reranking context,
1119
+ enabling more parallel contexts on GPU.
1120
+ - Performance: right-sized reranker context (40960 → 2048 tokens, 17x less
1121
+ memory) since chunks are capped at ~900 tokens.
1122
+ - Performance: adaptive parallelism — context count computed from available
1123
+ VRAM (GPU) or CPU math cores rather than hardcoded.
1124
+ - GPU: probe for CUDA, Metal, Vulkan explicitly at startup instead of
1125
+ relying on node-llama-cpp's `gpu: "auto"`. `qmd status` shows device info.
1126
+ - Tests: reorganized into flat `test/` directory with vitest for Node.js and
1127
+ bun test for Bun. New `eval-bm25` and `store.helpers.unit` suites.
1128
+
1129
+ ### Fixes
1130
+
1131
+ - Prevent VRAM waste from duplicate context creation during concurrent
1132
+ `embedBatch` calls — initialization lock now covers the full path.
1133
+ - Collection-aware FTS filtering so scoped keyword search actually restricts
1134
+ results to the requested collection.
1135
+
1136
+ ## [0.9.0] - 2026-02-15
1137
+
1138
+ First published release on npm as `@tobilu/qmd`. MCP HTTP transport with
1139
+ daemon mode cuts warm query latency from ~16s to ~10s by keeping models
1140
+ loaded between requests.
1141
+
1142
+ ### Changes
1143
+
1144
+ - MCP: HTTP transport with daemon lifecycle — `qmd mcp --http --daemon`
1145
+ starts a background server, `qmd mcp stop` shuts it down. Models stay warm
1146
+ in VRAM between queries. #149 (thanks @igrigorik)
1147
+ - Search: type-routed query expansion preserves lex/vec/hyde type info and
1148
+ routes to the appropriate backend. Eliminates ~4 wasted backend calls per
1149
+ query (10.0 → 6.0 calls, 1278ms → 549ms). #149 (thanks @igrigorik)
1150
+ - Search: unified pipeline — extracted `hybridQuery()` and
1151
+ `vectorSearchQuery()` to `store.ts` so CLI and MCP share identical logic.
1152
+ Fixes a class of bugs where results differed between the two. #149 (thanks
1153
+ @igrigorik)
1154
+ - MCP: dynamic instructions generated at startup from actual index state —
1155
+ LLMs see collection names, doc counts, and content descriptions. #149
1156
+ (thanks @igrigorik)
1157
+ - MCP: tool renames (vsearch → vector_search, query → deep_search) with
1158
+ rewritten descriptions for better tool selection. #149 (thanks @igrigorik)
1159
+ - Integration: Claude Code plugin with inline status checks and MCP
1160
+ integration. #99 (thanks @galligan)
1161
+
1162
+ ### Fixes
1163
+
1164
+ - BM25 score normalization — formula was inverted (`1/(1+|x|)` instead of
1165
+ `|x|/(1+|x|)`), so strong matches scored *lowest*. Broke `--min-score`
1166
+ filtering and made the "strong signal" short-circuit dead code. #76 (thanks
1167
+ @dgilperez)
1168
+ - Normalize Unicode paths to NFC for macOS compatibility. #82 (thanks
1169
+ @c-stoeckl)
1170
+ - Handle dense content (code) that tokenizes beyond expected chunk size.
1171
+ - Proper cleanup of Metal GPU resources on process exit.
1172
+ - SQLite-vec readiness verification after extension load.
1173
+ - Reactivate deactivated documents on re-index instead of creating duplicates.
1174
+ - Bun UTF-8 path corruption workaround for non-ASCII filenames.
1175
+ - Disable following symlinks in glob.scan to avoid infinite loops.
1176
+
1177
+ ## [0.8.0] - 2026-01-28
1178
+
1179
+ Fine-tuned query expansion model trained with GRPO replaces the stock Qwen3
1180
+ 0.6B. The training pipeline scores expansions on named entity preservation,
1181
+ format compliance, and diversity — producing noticeably better lexical
1182
+ variations and HyDE documents.
1183
+
1184
+ ### Changes
1185
+
1186
+ - LLM: deploy GRPO-trained (Group Relative Policy Optimization) query
1187
+ expansion model, hosted on HuggingFace and auto-downloaded on first use.
1188
+ Better preservation of proper nouns and technical terms in expansions.
1189
+ - LLM: `/only:lex` mode for single-type expansions — useful when you know
1190
+ which search backend will help.
1191
+ - LLM: HyDE output moved to first position so vector search can start
1192
+ embedding while other expansions generate.
1193
+ - LLM: session lifecycle management via `withLLMSession()` pattern — ensures
1194
+ cleanup even on failure, similar to database transactions.
1195
+ - Integration: org-mode title extraction support. #50 (thanks @sh54)
1196
+ - Integration: SQLite extension loading in Nix devshell. #48 (thanks @sh54)
1197
+ - Integration: AI agent discovery via skills.sh. #64 (thanks @Algiras)
1198
+
1199
+ ### Fixes
1200
+
1201
+ - Use sequential embedding on CPU-only systems — parallel contexts caused a
1202
+ race condition where contexts competed for CPU cores, making things slower.
1203
+ #54 (thanks @freeman-jiang)
1204
+ - Fix `collectionName` column in vector search SQL (was still using old
1205
+ `collectionId` from before YAML migration). #61 (thanks @jdvmi00)
1206
+ - Fix Qwen3 sampling params to prevent repetition loops — stock
1207
+ temperature/top-p caused occasional infinite repeat patterns.
1208
+ - Add `--index` option to CLI argument parser (was documented but not wired
1209
+ up). #84 (thanks @Tritlo)
1210
+ - Fix DisposedError during slow batch embedding. #41 (thanks @wuhup)
1211
+
1212
+ ## [0.7.0] - 2026-01-09
1213
+
1214
+ First community contributions. The project gained external contributors,
1215
+ surfacing bugs that only appear in diverse environments — Homebrew sqlite-vec
1216
+ paths, case-sensitive model filenames, and sqlite-vec JOIN incompatibilities.
1217
+
1218
+ ### Changes
1219
+
1220
+ - Indexing: native `realpathSync()` replaces `readlink -f` subprocess spawn
1221
+ per file. On a 5000-file collection this eliminates 5000 shell spawns,
1222
+ ~15% faster. #8 (thanks @burke)
1223
+ - Indexing: single-pass tokenization — chunking algorithm tokenized each
1224
+ document twice (count then split); now tokenizes once and reuses. #9
1225
+ (thanks @burke)
1226
+
1227
+ ### Fixes
1228
+
1229
+ - Fix `vsearch` and `query` hanging — sqlite-vec's virtual table doesn't
1230
+ support the JOIN pattern used; rewrote to subquery. #23 (thanks @mbrendan)
1231
+ - Fix MCP server exiting immediately after startup — process had no active
1232
+ handles keeping the event loop alive. #29 (thanks @mostlydev)
1233
+ - Fix collection filter SQL to properly restrict vector search results.
1234
+ - Support non-ASCII filenames in collection filter.
1235
+ - Skip empty files during indexing instead of crashing on zero-length content.
1236
+ - Fix case sensitivity in Qwen3 model filename resolution. #15 (thanks
1237
+ @gavrix)
1238
+ - Fix sqlite-vec loading on macOS with Homebrew (`BREW_PREFIX` detection).
1239
+ #42 (thanks @komsit37)
1240
+ - Fix Nix flake to use correct `src/qmd.ts` path. #7 (thanks @burke)
1241
+ - Fix docid lookup with quotes support in get command. #36 (thanks
1242
+ @JoshuaLelon)
1243
+ - Fix query expansion model size in documentation. #38 (thanks @odysseus0)
1244
+
1245
+ ## [0.6.0] - 2025-12-28
1246
+
1247
+ Replaced Ollama HTTP API with node-llama-cpp for all LLM operations. Ollama
1248
+ adds convenience but also a running server dependency. node-llama-cpp loads
1249
+ GGUF models directly in-process — zero external dependencies. Models
1250
+ auto-download from HuggingFace on first use.
1251
+
1252
+ ### Changes
1253
+
1254
+ - LLM: structured query expansion via JSON schema grammar constraints.
1255
+ Model produces typed expansions — **lexical** (BM25 keywords), **vector**
1256
+ (semantic rephrasings), **HyDE** (hypothetical document excerpts) — so each
1257
+ routes to the right backend instead of sending everything everywhere.
1258
+ - LLM: lazy model loading with 2-minute inactivity auto-unload. Keeps memory
1259
+ low when idle while avoiding ~3s model load on every query.
1260
+ - Search: conditional query expansion — when BM25 returns strong results, the
1261
+ expensive LLM expansion is skipped entirely.
1262
+ - Search: multi-chunk reranking — documents with multiple relevant chunks
1263
+ scored by aggregating across all chunks rather than best single chunk.
1264
+ - Search: cosine distance for vector search (was L2).
1265
+ - Search: embeddinggemma nomic-style prompt formatting.
1266
+ - Testing: evaluation harness with synthetic test documents and Hit@K metrics
1267
+ for BM25, vector, and hybrid RRF.
1268
+
1269
+ ## [0.5.0] - 2025-12-13
1270
+
1271
+ Collections and contexts moved from SQLite tables to YAML at
1272
+ `~/.config/qmd/index.yml`. SQLite was overkill for config — you can't share
1273
+ it, and it's opaque. YAML is human-readable and version-controllable. The
1274
+ migration was extensive (35+ commits) because every part of the system that
1275
+ touched collections or contexts had to be updated.
1276
+
1277
+ ### Changes
1278
+
1279
+ - Config: YAML-based collections and contexts replace SQLite tables.
1280
+ `collections` and `path_contexts` tables dropped from schema. Collections
1281
+ support an optional `update:` command (e.g., `git pull`) before re-index.
1282
+ - CLI: `qmd collection add/list/remove/rename` commands with `--name` and
1283
+ `--mask` glob pattern support.
1284
+ - CLI: `qmd ls` virtual file tree — list collections, files in a collection,
1285
+ or files under a path prefix.
1286
+ - CLI: `qmd context add/list/check/rm` with hierarchical context inheritance.
1287
+ A query to `qmd://notes/2024/jan/` inherits context from `notes/`,
1288
+ `notes/2024/`, and `notes/2024/jan/`.
1289
+ - CLI: `qmd context add / "text"` for global context across all collections.
1290
+ - CLI: `qmd context check` audit command to find paths without context.
1291
+ - Paths: `qmd://` virtual URI scheme for portable document references.
1292
+ `qmd://notes/ideas.md` works regardless of where the collection lives on
1293
+ disk. Works in `get`, `multi-get`, `ls`, and context commands.
1294
+ - CLI: document IDs (docid) — first 6 chars of content hash for stable
1295
+ references. Shown as `#abc123` in search results, usable with `get` and
1296
+ `multi-get`.
1297
+ - CLI: `--line-numbers` flag for get command output.
1298
+
1299
+ ## [0.4.0] - 2025-12-10
1300
+
1301
+ MCP server for AI agent integration. Without it, agents had to shell out to
1302
+ `qmd search` and parse CLI output. The monolithic `qmd.ts` (1840 lines) was
1303
+ split into focused modules with the project's first test suite (215 tests).
1304
+
1305
+ ### Changes
1306
+
1307
+ - MCP: stdio server with tools for search, vector search, hybrid query,
1308
+ document retrieval, and status. Runs over stdio transport for Claude
1309
+ Desktop and MCP clients.
1310
+ - MCP: spec-compliant with June 2025 MCP specification — removed non-spec
1311
+ `mimeType`, added `isError: true` to errors, `structuredContent` for
1312
+ machine-readable results, proper URI encoding.
1313
+ - MCP: simplified tool naming (`qmd_search` → `search`) since MCP already
1314
+ namespaces by server.
1315
+ - Architecture: extract `store.ts` (1221 LOC), `llm.ts` (539 LOC),
1316
+ `formatter.ts` (359 LOC), `mcp.ts` (503 LOC) from monolithic `qmd.ts`.
1317
+ - Testing: 215 tests (store: 96, llm: 60, mcp: 59) with mocked Ollama for
1318
+ fast, deterministic runs. Before this: zero tests.
1319
+
1320
+ ## [0.3.0] - 2025-12-08
1321
+
1322
+ Document chunking for vector search. A 5000-word document about many topics
1323
+ gets a single embedding that averages everything together, matching poorly for
1324
+ specific queries. Chunking produces one embedding per ~900-token section with
1325
+ focused semantic signal.
1326
+
1327
+ ### Changes
1328
+
1329
+ - Search: markdown-aware chunking — prefers heading boundaries, then paragraph
1330
+ breaks, then sentence boundaries. 15% overlap between chunks ensures
1331
+ cross-boundary queries still match.
1332
+ - Search: multi-chunk scoring bonus (+0.02 per additional chunk, capped at
1333
+ +0.1 for 5+ chunks). Documents relevant in multiple sections rank higher.
1334
+ - CLI: display paths show collection-relative paths and extracted titles
1335
+ (from H1 headings or YAML frontmatter) instead of raw filesystem paths.
1336
+ - CLI: `--all` flag returns all matches (use with `--min-score` to filter).
1337
+ - CLI: byte-based progress bar with ETA for `embed` command.
1338
+ - CLI: human-readable time formatting ("15m 4s" instead of "904.2s").
1339
+ - CLI: documents >64KB truncated with warning during embedding.
1340
+
1341
+ ## [0.2.0] - 2025-12-08
1342
+
1343
+ ### Changes
1344
+
1345
+ - CLI: `--json`, `--csv`, `--files`, `--md`, `--xml` output format flags.
1346
+ `--json` for programmatic access, `--files` for piping, `--md`/`--xml` for
1347
+ LLM consumption, `--csv` for spreadsheets.
1348
+ - CLI: `qmd status` shows index health — document count, size, embedding
1349
+ coverage, time since last update.
1350
+ - Search: weighted RRF — original query gets 2x weight relative to expanded
1351
+ queries since the user's actual words are a more reliable signal.
1352
+
1353
+ ## [0.1.0] - 2025-12-07
1354
+
1355
+ Initial implementation. Built in a single day for searching personal markdown
1356
+ notes, journals, and meeting transcripts.
1357
+
1358
+ ### Changes
1359
+
1360
+ - Search: SQLite FTS5 with BM25 ranking. Chose SQLite over Elasticsearch
1361
+ because QMD is a personal tool — single binary, no server dependencies.
1362
+ - Search: sqlite-vec for vector similarity. Same rationale: in-process, no
1363
+ external vector database.
1364
+ - Search: Reciprocal Rank Fusion to combine BM25 and vector results. RRF is
1365
+ parameter-free and handles missing signals gracefully.
1366
+ - LLM: Ollama for embeddings, reranking, and query expansion. Later replaced
1367
+ with node-llama-cpp in 0.6.0.
1368
+ - CLI: `qmd add`, `qmd embed`, `qmd search`, `qmd vsearch`, `qmd query`,
1369
+ `qmd get`. ~1800 lines of TypeScript in a single `qmd.ts` file.
1370
+
1371
+ [Unreleased]: https://github.com/tobi/qmd/compare/v1.0.0...HEAD
1372
+ [1.0.0]: https://github.com/tobi/qmd/releases/tag/v1.0.0
1373
+ [0.9.0]: https://github.com/tobi/qmd/compare/v0.8.0...v0.9.0