@tobilu/qmd 1.1.6 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,45 @@
2
2
 
3
3
  ## [Unreleased]
4
4
 
5
+ ## [2.0.1] - 2026-03-10
6
+
7
+ ### Changes
8
+
9
+ - `qmd skill install` copies the packaged QMD skill into
10
+ `~/.claude/commands/` for one-command setup. #355 (thanks @nibzard)
11
+
12
+ ### Fixes
13
+
14
+ - Fix Qwen3-Embedding GGUF filename case — HuggingFace filenames are
15
+ case-sensitive, the lowercase variant returned 404. #349 (thanks @byheaven)
16
+ - Resolve symlinked global launcher path so `qmd` works correctly when
17
+ installed via `npm i -g`. #352 (thanks @nibzard)
18
+
19
+ ## [2.0.0] - 2026-03-10
20
+
21
+ QMD 2.0 declares a stable library API. The SDK is now the primary interface —
22
+ the MCP server is a clean consumer of it, and the source is organized into
23
+ `src/cli/` and `src/mcp/`. Also: Node 25 support and a runtime-aware bin wrapper
24
+ for bun installs.
25
+
26
+ ### Changes
27
+
28
+ - Stable SDK API with `QMDStore` interface — search, retrieval, collection/context
29
+ management, indexing, lifecycle
30
+ - Unified `search()`: pass `query` for auto-expansion or `queries` for
31
+ pre-expanded lex/vec/hyde — replaces the old query/search/structuredSearch split
32
+ - New `getDocumentBody()`, `getDefaultCollectionNames()`, `Maintenance` class
33
+ - MCP server rewritten as a clean SDK consumer — zero internal store access
34
+ - CLI and MCP organized into `src/cli/` and `src/mcp/` subdirectories
35
+ - Runtime-aware `bin/qmd` wrapper detects bun vs node to avoid ABI mismatches.
36
+ Closes #319
37
+ - `better-sqlite3` bumped to ^12.4.5 for Node 25 support. Closes #257
38
+ - Utility exports: `extractSnippet`, `addLineNumbers`, `DEFAULT_MULTI_GET_MAX_BYTES`
39
+
40
+ ### Fixes
41
+
42
+ - Remove unused `import { resolve }` in store.ts that shadowed local export
43
+
5
44
  ## [1.1.6] - 2026-03-09
6
45
 
7
46
  QMD can now be used as a library. `import { createStore } from '@tobilu/qmd'`
package/README.md CHANGED
@@ -74,12 +74,10 @@ qmd get "docs/api-reference.md" --full
74
74
  Although the tool works perfectly fine when you just tell your agent to use it on the command line, it also exposes an MCP (Model Context Protocol) server for tighter integration.
75
75
 
76
76
  **Tools exposed:**
77
- - `qmd_search` - Fast BM25 keyword search (supports collection filter)
78
- - `qmd_vector_search` - Semantic vector search (supports collection filter)
79
- - `qmd_deep_search` - Deep search with query expansion and reranking (supports collection filter)
80
- - `qmd_get` - Retrieve document by path or docid (with fuzzy matching suggestions)
81
- - `qmd_multi_get` - Retrieve multiple documents by glob pattern, list, or docids
82
- - `qmd_status` - Index health and collection info
77
+ - `query` Search with typed sub-queries (`lex`/`vec`/`hyde`), combined via RRF + reranking
78
+ - `get` Retrieve a document by path or docid (with fuzzy matching suggestions)
79
+ - `multi_get` Batch retrieve by glob pattern, comma-separated list, or docids
80
+ - `status` Index health and collection info
83
81
 
84
82
  **Claude Desktop configuration** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
85
83
 
@@ -139,78 +137,239 @@ Point any MCP client at `http://localhost:8181/mcp` to connect.
139
137
 
140
138
  ### SDK / Library Usage
141
139
 
142
- Use QMD as a library in your own Node.js or Bun applications:
140
+ Use QMD as a library in your own Node.js or Bun applications.
141
+
142
+ #### Installation
143
143
 
144
144
  ```sh
145
145
  npm install @tobilu/qmd
146
146
  ```
147
147
 
148
+ #### Quick Start
149
+
148
150
  ```typescript
149
151
  import { createStore } from '@tobilu/qmd'
150
152
 
151
- // Create a store with inline config (no config file needed)
152
- const store = createStore({
153
+ const store = await createStore({
153
154
  dbPath: './my-index.sqlite',
154
155
  config: {
155
156
  collections: {
156
157
  docs: { path: '/path/to/docs', pattern: '**/*.md' },
157
- notes: { path: '/path/to/notes', pattern: '**/*.md' },
158
158
  },
159
159
  },
160
160
  })
161
161
 
162
- // Or reference a YAML config file
163
- const store2 = createStore({
164
- dbPath: './my-index.sqlite',
162
+ const results = await store.search({ query: "authentication flow" })
163
+ console.log(results.map(r => `${r.title} (${Math.round(r.score * 100)}%)`))
164
+
165
+ await store.close()
166
+ ```
167
+
168
+ #### Store Creation
169
+
170
+ `createStore()` accepts three modes:
171
+
172
+ ```typescript
173
+ import { createStore } from '@tobilu/qmd'
174
+
175
+ // 1. Inline config — no files needed besides the DB
176
+ const store = await createStore({
177
+ dbPath: './index.sqlite',
178
+ config: {
179
+ collections: {
180
+ docs: { path: '/path/to/docs', pattern: '**/*.md' },
181
+ notes: { path: '/path/to/notes' },
182
+ },
183
+ },
184
+ })
185
+
186
+ // 2. YAML config file — collections defined in a file
187
+ const store2 = await createStore({
188
+ dbPath: './index.sqlite',
165
189
  configPath: './qmd.yml',
166
190
  })
191
+
192
+ // 3. DB-only — reopen a previously configured store
193
+ const store3 = await createStore({ dbPath: './index.sqlite' })
167
194
  ```
168
195
 
169
- **Search & retrieval:**
196
+ #### Search
197
+
198
+ The unified `search()` method handles both simple queries and pre-expanded structured queries:
170
199
 
171
200
  ```typescript
172
- // Hybrid search: BM25 + vector + query expansion + LLM reranking (best quality)
173
- const results = await store.query("authentication flow", { limit: 5 })
201
+ // Simple query auto-expanded via LLM, then BM25 + vector + reranking
202
+ const results = await store.search({ query: "authentication flow" })
203
+
204
+ // With options
205
+ const results2 = await store.search({
206
+ query: "rate limiting",
207
+ intent: "API throttling and abuse prevention",
208
+ collection: "docs",
209
+ limit: 5,
210
+ minScore: 0.3,
211
+ explain: true,
212
+ })
213
+
214
+ // Pre-expanded queries — skip auto-expansion, control each sub-query
215
+ const results3 = await store.search({
216
+ queries: [
217
+ { type: 'lex', query: '"connection pool" timeout -redis' },
218
+ { type: 'vec', query: 'why do database connections time out under load' },
219
+ ],
220
+ collections: ["docs", "notes"],
221
+ })
174
222
 
175
- // Fast BM25 keyword search (no LLM, synchronous)
176
- const keywords = store.search("auth middleware", { limit: 10 })
223
+ // Skip reranking for faster results
224
+ const fast = await store.search({ query: "auth", rerank: false })
225
+ ```
177
226
 
178
- // Structured search with pre-expanded queries (for LLM callers)
179
- const structured = await store.structuredSearch([
180
- { type: 'lex', query: 'authentication' },
181
- { type: 'vec', query: 'how users log in' },
182
- ], { limit: 5 })
227
+ For direct backend access:
183
228
 
229
+ ```typescript
230
+ // BM25 keyword search (fast, no LLM)
231
+ const lexResults = await store.searchLex("auth middleware", { limit: 10 })
232
+
233
+ // Vector similarity search (embedding model, no reranking)
234
+ const vecResults = await store.searchVector("how users log in", { limit: 10 })
235
+
236
+ // Manual query expansion for full control
237
+ const expanded = await store.expandQuery("auth flow", { intent: "user login" })
238
+ const results4 = await store.search({ queries: expanded })
239
+ ```
240
+
241
+ #### Retrieval
242
+
243
+ ```typescript
184
244
  // Get a document by path or docid
185
- const doc = store.get("docs/readme.md")
186
- const byId = store.get("#abc123")
245
+ const doc = await store.get("docs/readme.md")
246
+ const byId = await store.get("#abc123")
187
247
 
188
- // Get multiple documents by glob
189
- const { docs, errors } = store.multiGet("docs/**/*.md")
248
+ if (!("error" in doc)) {
249
+ console.log(doc.title, doc.displayPath, doc.context)
250
+ }
251
+
252
+ // Get document body with line range
253
+ const body = await store.getDocumentBody("docs/readme.md", {
254
+ fromLine: 50,
255
+ maxLines: 100,
256
+ })
257
+
258
+ // Batch retrieve by glob or comma-separated list
259
+ const { docs, errors } = await store.multiGet("docs/**/*.md", {
260
+ maxBytes: 20480,
261
+ })
190
262
  ```
191
263
 
192
- **Collection & context management:**
264
+ #### Collections
193
265
 
194
266
  ```typescript
195
267
  // Add a collection
196
- store.addCollection("myapp", { path: "/src/myapp", pattern: "**/*.ts" })
268
+ await store.addCollection("myapp", {
269
+ path: "/src/myapp",
270
+ pattern: "**/*.ts",
271
+ ignore: ["node_modules/**", "*.test.ts"],
272
+ })
273
+
274
+ // List collections with document stats
275
+ const collections = await store.listCollections()
276
+ // => [{ name, pwd, glob_pattern, doc_count, active_count, last_modified, includeByDefault }]
277
+
278
+ // Get names of collections included in queries by default
279
+ const defaults = await store.getDefaultCollectionNames()
197
280
 
198
- // Add context (improves search relevance)
199
- store.addContext("myapp", "/auth", "Authentication and session management")
200
- store.setGlobalContext("Internal engineering documentation")
281
+ // Remove / rename
282
+ await store.removeCollection("myapp")
283
+ await store.renameCollection("old-name", "new-name")
284
+ ```
285
+
286
+ #### Context
287
+
288
+ Context adds descriptive metadata that improves search relevance and is returned alongside results:
289
+
290
+ ```typescript
291
+ // Add context for a path within a collection
292
+ await store.addContext("docs", "/api", "REST API reference documentation")
293
+
294
+ // Set global context (applies to all collections)
295
+ await store.setGlobalContext("Internal engineering documentation")
201
296
 
202
- // List everything
203
- store.listCollections()
204
- store.listContexts()
297
+ // List all contexts
298
+ const contexts = await store.listContexts()
299
+ // => [{ collection, path, context }]
300
+
301
+ // Remove context
302
+ await store.removeContext("docs", "/api")
303
+ await store.setGlobalContext(undefined) // clear global
304
+ ```
305
+
306
+ #### Indexing
307
+
308
+ ```typescript
309
+ // Re-index collections by scanning the filesystem
310
+ const result = await store.update({
311
+ collections: ["docs"], // optional — defaults to all
312
+ onProgress: ({ collection, file, current, total }) => {
313
+ console.log(`[${collection}] ${current}/${total} ${file}`)
314
+ },
315
+ })
316
+ // => { collections, indexed, updated, unchanged, removed, needsEmbedding }
317
+
318
+ // Generate vector embeddings
319
+ const embedResult = await store.embed({
320
+ force: false, // true to re-embed everything
321
+ onProgress: ({ current, total, collection }) => {
322
+ console.log(`Embedding ${current}/${total}`)
323
+ },
324
+ })
325
+ ```
326
+
327
+ #### Types
328
+
329
+ Key types exported for SDK consumers:
330
+
331
+ ```typescript
332
+ import type {
333
+ QMDStore, // The store interface
334
+ SearchOptions, // Options for search()
335
+ LexSearchOptions, // Options for searchLex()
336
+ VectorSearchOptions, // Options for searchVector()
337
+ HybridQueryResult, // Search result with score, snippet, context
338
+ SearchResult, // Result from searchLex/searchVector
339
+ ExpandedQuery, // Typed sub-query { type: 'lex'|'vec'|'hyde', query }
340
+ DocumentResult, // Document metadata + body
341
+ DocumentNotFound, // Error with similarFiles suggestions
342
+ MultiGetResult, // Batch retrieval result
343
+ UpdateProgress, // Progress callback info for update()
344
+ UpdateResult, // Aggregated update result
345
+ EmbedProgress, // Progress callback info for embed()
346
+ EmbedResult, // Embedding result
347
+ StoreOptions, // createStore() options
348
+ CollectionConfig, // Inline config shape
349
+ IndexStatus, // From getStatus()
350
+ IndexHealthInfo, // From getIndexHealth()
351
+ } from '@tobilu/qmd'
352
+ ```
353
+
354
+ Utility exports:
355
+
356
+ ```typescript
357
+ import {
358
+ extractSnippet, // Extract a relevant snippet from text
359
+ addLineNumbers, // Add line numbers to text
360
+ DEFAULT_MULTI_GET_MAX_BYTES, // Default max file size for multiGet (10KB)
361
+ Maintenance, // Database maintenance operations
362
+ } from '@tobilu/qmd'
205
363
  ```
206
364
 
207
- **Lifecycle:**
365
+ #### Lifecycle
208
366
 
209
367
  ```typescript
210
- store.close()
368
+ // Close the store — disposes LLM models and DB connection
369
+ await store.close()
211
370
  ```
212
371
 
213
- The SDK requires explicit `dbPath` and config — no defaults are assumed. This makes it safe to embed in any application without side effects.
372
+ The SDK requires explicit `dbPath` — no defaults are assumed. This makes it safe to embed in any application without side effects.
214
373
 
215
374
  ## Architecture
216
375
 
@@ -341,7 +500,7 @@ This is useful for multilingual corpora (e.g. Chinese, Japanese, Korean) where
341
500
 
342
501
  ```sh
343
502
  # Use Qwen3-Embedding-0.6B for better multilingual (CJK) support
344
- export QMD_EMBED_MODEL="hf:Qwen/Qwen3-Embedding-0.6B-GGUF/qwen3-embedding-0.6b-q8_0.gguf"
503
+ export QMD_EMBED_MODEL="hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf"
345
504
 
346
505
  # After changing the model, re-embed all collections:
347
506
  qmd embed -f
package/bin/qmd ADDED
@@ -0,0 +1,23 @@
1
+ #!/bin/sh
2
+ # Resolve symlinks so global installs (npm link / npm install -g) can find the
3
+ # actual package directory instead of the global bin directory.
4
+ SOURCE="$0"
5
+ while [ -L "$SOURCE" ]; do
6
+ SOURCE_DIR="$(cd -P "$(dirname "$SOURCE")" && pwd)"
7
+ TARGET="$(readlink "$SOURCE")"
8
+ case "$TARGET" in
9
+ /*) SOURCE="$TARGET" ;;
10
+ *) SOURCE="$SOURCE_DIR/$TARGET" ;;
11
+ esac
12
+ done
13
+
14
+ # Detect the runtime used to install this package and use the matching one
15
+ # to avoid native module ABI mismatches (e.g., better-sqlite3 compiled for bun vs node)
16
+ DIR="$(cd -P "$(dirname "$SOURCE")/.." && pwd)"
17
+
18
+ # Check if we were installed with bun (look for bun.lock or bun-lockb)
19
+ if [ -f "$DIR/bun.lock" ] || [ -f "$DIR/bun.lockb" ] || [ -n "$BUN_INSTALL" ]; then
20
+ exec bun "$DIR/dist/cli/qmd.js" "$@"
21
+ else
22
+ exec node "$DIR/dist/cli/qmd.js" "$@"
23
+ fi
@@ -4,7 +4,7 @@
4
4
  * Provides methods to format search results and documents into various output formats:
5
5
  * JSON, CSV, XML, Markdown, files list, and CLI (colored terminal output).
6
6
  */
7
- import type { SearchResult, MultiGetResult, DocumentResult } from "./store.js";
7
+ import type { SearchResult, MultiGetResult, DocumentResult } from "../store.js";
8
8
  export type { SearchResult, MultiGetResult, DocumentResult };
9
9
  export type MultiGetFile = {
10
10
  filepath: string;
@@ -4,7 +4,7 @@
4
4
  * Provides methods to format search results and documents into various output formats:
5
5
  * JSON, CSV, XML, Markdown, files list, and CLI (colored terminal output).
6
6
  */
7
- import { extractSnippet } from "./store.js";
7
+ import { extractSnippet } from "../store.js";
8
8
  // =============================================================================
9
9
  // Helper Functions
10
10
  // =============================================================================