@tobilu/qmd 1.1.6 → 2.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +39 -0
- package/README.md +198 -39
- package/bin/qmd +23 -0
- package/dist/{formatter.d.ts → cli/formatter.d.ts} +1 -1
- package/dist/{formatter.js → cli/formatter.js} +1 -1
- package/dist/{qmd.js → cli/qmd.js} +266 -154
- package/dist/embedded-skills.d.ts +6 -0
- package/dist/embedded-skills.js +14 -0
- package/dist/index.d.ts +129 -38
- package/dist/index.js +175 -41
- package/dist/llm.d.ts +6 -0
- package/dist/llm.js +24 -1
- package/dist/maintenance.d.ts +23 -0
- package/dist/maintenance.js +37 -0
- package/dist/{mcp.js → mcp/server.js} +41 -61
- package/dist/store.d.ts +83 -19
- package/dist/store.js +561 -84
- package/package.json +12 -11
- /package/dist/{qmd.d.ts → cli/qmd.d.ts} +0 -0
- /package/dist/{mcp.d.ts → mcp/server.d.ts} +0 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,45 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [2.0.1] - 2026-03-10
|
|
6
|
+
|
|
7
|
+
### Changes
|
|
8
|
+
|
|
9
|
+
- `qmd skill install` copies the packaged QMD skill into
|
|
10
|
+
`~/.claude/commands/` for one-command setup. #355 (thanks @nibzard)
|
|
11
|
+
|
|
12
|
+
### Fixes
|
|
13
|
+
|
|
14
|
+
- Fix Qwen3-Embedding GGUF filename case — HuggingFace filenames are
|
|
15
|
+
case-sensitive, the lowercase variant returned 404. #349 (thanks @byheaven)
|
|
16
|
+
- Resolve symlinked global launcher path so `qmd` works correctly when
|
|
17
|
+
installed via `npm i -g`. #352 (thanks @nibzard)
|
|
18
|
+
|
|
19
|
+
## [2.0.0] - 2026-03-10
|
|
20
|
+
|
|
21
|
+
QMD 2.0 declares a stable library API. The SDK is now the primary interface —
|
|
22
|
+
the MCP server is a clean consumer of it, and the source is organized into
|
|
23
|
+
`src/cli/` and `src/mcp/`. Also: Node 25 support and a runtime-aware bin wrapper
|
|
24
|
+
for bun installs.
|
|
25
|
+
|
|
26
|
+
### Changes
|
|
27
|
+
|
|
28
|
+
- Stable SDK API with `QMDStore` interface — search, retrieval, collection/context
|
|
29
|
+
management, indexing, lifecycle
|
|
30
|
+
- Unified `search()`: pass `query` for auto-expansion or `queries` for
|
|
31
|
+
pre-expanded lex/vec/hyde — replaces the old query/search/structuredSearch split
|
|
32
|
+
- New `getDocumentBody()`, `getDefaultCollectionNames()`, `Maintenance` class
|
|
33
|
+
- MCP server rewritten as a clean SDK consumer — zero internal store access
|
|
34
|
+
- CLI and MCP organized into `src/cli/` and `src/mcp/` subdirectories
|
|
35
|
+
- Runtime-aware `bin/qmd` wrapper detects bun vs node to avoid ABI mismatches.
|
|
36
|
+
Closes #319
|
|
37
|
+
- `better-sqlite3` bumped to ^12.4.5 for Node 25 support. Closes #257
|
|
38
|
+
- Utility exports: `extractSnippet`, `addLineNumbers`, `DEFAULT_MULTI_GET_MAX_BYTES`
|
|
39
|
+
|
|
40
|
+
### Fixes
|
|
41
|
+
|
|
42
|
+
- Remove unused `import { resolve }` in store.ts that shadowed local export
|
|
43
|
+
|
|
5
44
|
## [1.1.6] - 2026-03-09
|
|
6
45
|
|
|
7
46
|
QMD can now be used as a library. `import { createStore } from '@tobilu/qmd'`
|
package/README.md
CHANGED
|
@@ -74,12 +74,10 @@ qmd get "docs/api-reference.md" --full
|
|
|
74
74
|
Although the tool works perfectly fine when you just tell your agent to use it on the command line, it also exposes an MCP (Model Context Protocol) server for tighter integration.
|
|
75
75
|
|
|
76
76
|
**Tools exposed:**
|
|
77
|
-
- `
|
|
78
|
-
- `
|
|
79
|
-
- `
|
|
80
|
-
- `
|
|
81
|
-
- `qmd_multi_get` - Retrieve multiple documents by glob pattern, list, or docids
|
|
82
|
-
- `qmd_status` - Index health and collection info
|
|
77
|
+
- `query` — Search with typed sub-queries (`lex`/`vec`/`hyde`), combined via RRF + reranking
|
|
78
|
+
- `get` — Retrieve a document by path or docid (with fuzzy matching suggestions)
|
|
79
|
+
- `multi_get` — Batch retrieve by glob pattern, comma-separated list, or docids
|
|
80
|
+
- `status` — Index health and collection info
|
|
83
81
|
|
|
84
82
|
**Claude Desktop configuration** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
|
|
85
83
|
|
|
@@ -139,78 +137,239 @@ Point any MCP client at `http://localhost:8181/mcp` to connect.
|
|
|
139
137
|
|
|
140
138
|
### SDK / Library Usage
|
|
141
139
|
|
|
142
|
-
Use QMD as a library in your own Node.js or Bun applications
|
|
140
|
+
Use QMD as a library in your own Node.js or Bun applications.
|
|
141
|
+
|
|
142
|
+
#### Installation
|
|
143
143
|
|
|
144
144
|
```sh
|
|
145
145
|
npm install @tobilu/qmd
|
|
146
146
|
```
|
|
147
147
|
|
|
148
|
+
#### Quick Start
|
|
149
|
+
|
|
148
150
|
```typescript
|
|
149
151
|
import { createStore } from '@tobilu/qmd'
|
|
150
152
|
|
|
151
|
-
|
|
152
|
-
const store = createStore({
|
|
153
|
+
const store = await createStore({
|
|
153
154
|
dbPath: './my-index.sqlite',
|
|
154
155
|
config: {
|
|
155
156
|
collections: {
|
|
156
157
|
docs: { path: '/path/to/docs', pattern: '**/*.md' },
|
|
157
|
-
notes: { path: '/path/to/notes', pattern: '**/*.md' },
|
|
158
158
|
},
|
|
159
159
|
},
|
|
160
160
|
})
|
|
161
161
|
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
162
|
+
const results = await store.search({ query: "authentication flow" })
|
|
163
|
+
console.log(results.map(r => `${r.title} (${Math.round(r.score * 100)}%)`))
|
|
164
|
+
|
|
165
|
+
await store.close()
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
#### Store Creation
|
|
169
|
+
|
|
170
|
+
`createStore()` accepts three modes:
|
|
171
|
+
|
|
172
|
+
```typescript
|
|
173
|
+
import { createStore } from '@tobilu/qmd'
|
|
174
|
+
|
|
175
|
+
// 1. Inline config — no files needed besides the DB
|
|
176
|
+
const store = await createStore({
|
|
177
|
+
dbPath: './index.sqlite',
|
|
178
|
+
config: {
|
|
179
|
+
collections: {
|
|
180
|
+
docs: { path: '/path/to/docs', pattern: '**/*.md' },
|
|
181
|
+
notes: { path: '/path/to/notes' },
|
|
182
|
+
},
|
|
183
|
+
},
|
|
184
|
+
})
|
|
185
|
+
|
|
186
|
+
// 2. YAML config file — collections defined in a file
|
|
187
|
+
const store2 = await createStore({
|
|
188
|
+
dbPath: './index.sqlite',
|
|
165
189
|
configPath: './qmd.yml',
|
|
166
190
|
})
|
|
191
|
+
|
|
192
|
+
// 3. DB-only — reopen a previously configured store
|
|
193
|
+
const store3 = await createStore({ dbPath: './index.sqlite' })
|
|
167
194
|
```
|
|
168
195
|
|
|
169
|
-
|
|
196
|
+
#### Search
|
|
197
|
+
|
|
198
|
+
The unified `search()` method handles both simple queries and pre-expanded structured queries:
|
|
170
199
|
|
|
171
200
|
```typescript
|
|
172
|
-
//
|
|
173
|
-
const results = await store.query
|
|
201
|
+
// Simple query — auto-expanded via LLM, then BM25 + vector + reranking
|
|
202
|
+
const results = await store.search({ query: "authentication flow" })
|
|
203
|
+
|
|
204
|
+
// With options
|
|
205
|
+
const results2 = await store.search({
|
|
206
|
+
query: "rate limiting",
|
|
207
|
+
intent: "API throttling and abuse prevention",
|
|
208
|
+
collection: "docs",
|
|
209
|
+
limit: 5,
|
|
210
|
+
minScore: 0.3,
|
|
211
|
+
explain: true,
|
|
212
|
+
})
|
|
213
|
+
|
|
214
|
+
// Pre-expanded queries — skip auto-expansion, control each sub-query
|
|
215
|
+
const results3 = await store.search({
|
|
216
|
+
queries: [
|
|
217
|
+
{ type: 'lex', query: '"connection pool" timeout -redis' },
|
|
218
|
+
{ type: 'vec', query: 'why do database connections time out under load' },
|
|
219
|
+
],
|
|
220
|
+
collections: ["docs", "notes"],
|
|
221
|
+
})
|
|
174
222
|
|
|
175
|
-
//
|
|
176
|
-
const
|
|
223
|
+
// Skip reranking for faster results
|
|
224
|
+
const fast = await store.search({ query: "auth", rerank: false })
|
|
225
|
+
```
|
|
177
226
|
|
|
178
|
-
|
|
179
|
-
const structured = await store.structuredSearch([
|
|
180
|
-
{ type: 'lex', query: 'authentication' },
|
|
181
|
-
{ type: 'vec', query: 'how users log in' },
|
|
182
|
-
], { limit: 5 })
|
|
227
|
+
For direct backend access:
|
|
183
228
|
|
|
229
|
+
```typescript
|
|
230
|
+
// BM25 keyword search (fast, no LLM)
|
|
231
|
+
const lexResults = await store.searchLex("auth middleware", { limit: 10 })
|
|
232
|
+
|
|
233
|
+
// Vector similarity search (embedding model, no reranking)
|
|
234
|
+
const vecResults = await store.searchVector("how users log in", { limit: 10 })
|
|
235
|
+
|
|
236
|
+
// Manual query expansion for full control
|
|
237
|
+
const expanded = await store.expandQuery("auth flow", { intent: "user login" })
|
|
238
|
+
const results4 = await store.search({ queries: expanded })
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
#### Retrieval
|
|
242
|
+
|
|
243
|
+
```typescript
|
|
184
244
|
// Get a document by path or docid
|
|
185
|
-
const doc = store.get("docs/readme.md")
|
|
186
|
-
const byId = store.get("#abc123")
|
|
245
|
+
const doc = await store.get("docs/readme.md")
|
|
246
|
+
const byId = await store.get("#abc123")
|
|
187
247
|
|
|
188
|
-
|
|
189
|
-
|
|
248
|
+
if (!("error" in doc)) {
|
|
249
|
+
console.log(doc.title, doc.displayPath, doc.context)
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
// Get document body with line range
|
|
253
|
+
const body = await store.getDocumentBody("docs/readme.md", {
|
|
254
|
+
fromLine: 50,
|
|
255
|
+
maxLines: 100,
|
|
256
|
+
})
|
|
257
|
+
|
|
258
|
+
// Batch retrieve by glob or comma-separated list
|
|
259
|
+
const { docs, errors } = await store.multiGet("docs/**/*.md", {
|
|
260
|
+
maxBytes: 20480,
|
|
261
|
+
})
|
|
190
262
|
```
|
|
191
263
|
|
|
192
|
-
|
|
264
|
+
#### Collections
|
|
193
265
|
|
|
194
266
|
```typescript
|
|
195
267
|
// Add a collection
|
|
196
|
-
store.addCollection("myapp", {
|
|
268
|
+
await store.addCollection("myapp", {
|
|
269
|
+
path: "/src/myapp",
|
|
270
|
+
pattern: "**/*.ts",
|
|
271
|
+
ignore: ["node_modules/**", "*.test.ts"],
|
|
272
|
+
})
|
|
273
|
+
|
|
274
|
+
// List collections with document stats
|
|
275
|
+
const collections = await store.listCollections()
|
|
276
|
+
// => [{ name, pwd, glob_pattern, doc_count, active_count, last_modified, includeByDefault }]
|
|
277
|
+
|
|
278
|
+
// Get names of collections included in queries by default
|
|
279
|
+
const defaults = await store.getDefaultCollectionNames()
|
|
197
280
|
|
|
198
|
-
//
|
|
199
|
-
store.
|
|
200
|
-
store.
|
|
281
|
+
// Remove / rename
|
|
282
|
+
await store.removeCollection("myapp")
|
|
283
|
+
await store.renameCollection("old-name", "new-name")
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
#### Context
|
|
287
|
+
|
|
288
|
+
Context adds descriptive metadata that improves search relevance and is returned alongside results:
|
|
289
|
+
|
|
290
|
+
```typescript
|
|
291
|
+
// Add context for a path within a collection
|
|
292
|
+
await store.addContext("docs", "/api", "REST API reference documentation")
|
|
293
|
+
|
|
294
|
+
// Set global context (applies to all collections)
|
|
295
|
+
await store.setGlobalContext("Internal engineering documentation")
|
|
201
296
|
|
|
202
|
-
// List
|
|
203
|
-
store.
|
|
204
|
-
|
|
297
|
+
// List all contexts
|
|
298
|
+
const contexts = await store.listContexts()
|
|
299
|
+
// => [{ collection, path, context }]
|
|
300
|
+
|
|
301
|
+
// Remove context
|
|
302
|
+
await store.removeContext("docs", "/api")
|
|
303
|
+
await store.setGlobalContext(undefined) // clear global
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
#### Indexing
|
|
307
|
+
|
|
308
|
+
```typescript
|
|
309
|
+
// Re-index collections by scanning the filesystem
|
|
310
|
+
const result = await store.update({
|
|
311
|
+
collections: ["docs"], // optional — defaults to all
|
|
312
|
+
onProgress: ({ collection, file, current, total }) => {
|
|
313
|
+
console.log(`[${collection}] ${current}/${total} ${file}`)
|
|
314
|
+
},
|
|
315
|
+
})
|
|
316
|
+
// => { collections, indexed, updated, unchanged, removed, needsEmbedding }
|
|
317
|
+
|
|
318
|
+
// Generate vector embeddings
|
|
319
|
+
const embedResult = await store.embed({
|
|
320
|
+
force: false, // true to re-embed everything
|
|
321
|
+
onProgress: ({ current, total, collection }) => {
|
|
322
|
+
console.log(`Embedding ${current}/${total}`)
|
|
323
|
+
},
|
|
324
|
+
})
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
#### Types
|
|
328
|
+
|
|
329
|
+
Key types exported for SDK consumers:
|
|
330
|
+
|
|
331
|
+
```typescript
|
|
332
|
+
import type {
|
|
333
|
+
QMDStore, // The store interface
|
|
334
|
+
SearchOptions, // Options for search()
|
|
335
|
+
LexSearchOptions, // Options for searchLex()
|
|
336
|
+
VectorSearchOptions, // Options for searchVector()
|
|
337
|
+
HybridQueryResult, // Search result with score, snippet, context
|
|
338
|
+
SearchResult, // Result from searchLex/searchVector
|
|
339
|
+
ExpandedQuery, // Typed sub-query { type: 'lex'|'vec'|'hyde', query }
|
|
340
|
+
DocumentResult, // Document metadata + body
|
|
341
|
+
DocumentNotFound, // Error with similarFiles suggestions
|
|
342
|
+
MultiGetResult, // Batch retrieval result
|
|
343
|
+
UpdateProgress, // Progress callback info for update()
|
|
344
|
+
UpdateResult, // Aggregated update result
|
|
345
|
+
EmbedProgress, // Progress callback info for embed()
|
|
346
|
+
EmbedResult, // Embedding result
|
|
347
|
+
StoreOptions, // createStore() options
|
|
348
|
+
CollectionConfig, // Inline config shape
|
|
349
|
+
IndexStatus, // From getStatus()
|
|
350
|
+
IndexHealthInfo, // From getIndexHealth()
|
|
351
|
+
} from '@tobilu/qmd'
|
|
352
|
+
```
|
|
353
|
+
|
|
354
|
+
Utility exports:
|
|
355
|
+
|
|
356
|
+
```typescript
|
|
357
|
+
import {
|
|
358
|
+
extractSnippet, // Extract a relevant snippet from text
|
|
359
|
+
addLineNumbers, // Add line numbers to text
|
|
360
|
+
DEFAULT_MULTI_GET_MAX_BYTES, // Default max file size for multiGet (10KB)
|
|
361
|
+
Maintenance, // Database maintenance operations
|
|
362
|
+
} from '@tobilu/qmd'
|
|
205
363
|
```
|
|
206
364
|
|
|
207
|
-
|
|
365
|
+
#### Lifecycle
|
|
208
366
|
|
|
209
367
|
```typescript
|
|
210
|
-
store
|
|
368
|
+
// Close the store — disposes LLM models and DB connection
|
|
369
|
+
await store.close()
|
|
211
370
|
```
|
|
212
371
|
|
|
213
|
-
The SDK requires explicit `dbPath`
|
|
372
|
+
The SDK requires explicit `dbPath` — no defaults are assumed. This makes it safe to embed in any application without side effects.
|
|
214
373
|
|
|
215
374
|
## Architecture
|
|
216
375
|
|
|
@@ -341,7 +500,7 @@ This is useful for multilingual corpora (e.g. Chinese, Japanese, Korean) where
|
|
|
341
500
|
|
|
342
501
|
```sh
|
|
343
502
|
# Use Qwen3-Embedding-0.6B for better multilingual (CJK) support
|
|
344
|
-
export QMD_EMBED_MODEL="hf:Qwen/Qwen3-Embedding-0.6B-GGUF/
|
|
503
|
+
export QMD_EMBED_MODEL="hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf"
|
|
345
504
|
|
|
346
505
|
# After changing the model, re-embed all collections:
|
|
347
506
|
qmd embed -f
|
package/bin/qmd
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
#!/bin/sh
|
|
2
|
+
# Resolve symlinks so global installs (npm link / npm install -g) can find the
|
|
3
|
+
# actual package directory instead of the global bin directory.
|
|
4
|
+
SOURCE="$0"
|
|
5
|
+
while [ -L "$SOURCE" ]; do
|
|
6
|
+
SOURCE_DIR="$(cd -P "$(dirname "$SOURCE")" && pwd)"
|
|
7
|
+
TARGET="$(readlink "$SOURCE")"
|
|
8
|
+
case "$TARGET" in
|
|
9
|
+
/*) SOURCE="$TARGET" ;;
|
|
10
|
+
*) SOURCE="$SOURCE_DIR/$TARGET" ;;
|
|
11
|
+
esac
|
|
12
|
+
done
|
|
13
|
+
|
|
14
|
+
# Detect the runtime used to install this package and use the matching one
|
|
15
|
+
# to avoid native module ABI mismatches (e.g., better-sqlite3 compiled for bun vs node)
|
|
16
|
+
DIR="$(cd -P "$(dirname "$SOURCE")/.." && pwd)"
|
|
17
|
+
|
|
18
|
+
# Check if we were installed with bun (look for bun.lock or bun-lockb)
|
|
19
|
+
if [ -f "$DIR/bun.lock" ] || [ -f "$DIR/bun.lockb" ] || [ -n "$BUN_INSTALL" ]; then
|
|
20
|
+
exec bun "$DIR/dist/cli/qmd.js" "$@"
|
|
21
|
+
else
|
|
22
|
+
exec node "$DIR/dist/cli/qmd.js" "$@"
|
|
23
|
+
fi
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
* Provides methods to format search results and documents into various output formats:
|
|
5
5
|
* JSON, CSV, XML, Markdown, files list, and CLI (colored terminal output).
|
|
6
6
|
*/
|
|
7
|
-
import type { SearchResult, MultiGetResult, DocumentResult } from "
|
|
7
|
+
import type { SearchResult, MultiGetResult, DocumentResult } from "../store.js";
|
|
8
8
|
export type { SearchResult, MultiGetResult, DocumentResult };
|
|
9
9
|
export type MultiGetFile = {
|
|
10
10
|
filepath: string;
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
* Provides methods to format search results and documents into various output formats:
|
|
5
5
|
* JSON, CSV, XML, Markdown, files list, and CLI (colored terminal output).
|
|
6
6
|
*/
|
|
7
|
-
import { extractSnippet } from "
|
|
7
|
+
import { extractSnippet } from "../store.js";
|
|
8
8
|
// =============================================================================
|
|
9
9
|
// Helper Functions
|
|
10
10
|
// =============================================================================
|