@dikolab/kbdb 0.2.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (120) hide show
  1. package/README.md +59 -24
  2. package/dist/cli.cjs +462 -73
  3. package/dist/cli.cjs.map +4 -4
  4. package/dist/cli.mjs +454 -65
  5. package/dist/cli.mjs.map +4 -4
  6. package/dist/kbdb-worker.cjs +395 -60
  7. package/dist/kbdb-worker.cjs.map +4 -4
  8. package/dist/mod.cjs +18 -1
  9. package/dist/mod.cjs.map +2 -2
  10. package/dist/mod.mjs +18 -1
  11. package/dist/mod.mjs.map +2 -2
  12. package/dist/src/shared/cli/functions/dispatch-command.function.d.ts +3 -1
  13. package/dist/src/shared/cli/functions/install-default-skills.function.d.ts +12 -0
  14. package/dist/src/shared/cli/functions/run-check.function.d.ts +47 -5
  15. package/dist/src/shared/cli/functions/run-export.function.d.ts +34 -0
  16. package/dist/src/shared/cli/functions/run-gc.function.d.ts +6 -0
  17. package/dist/src/shared/cli/functions/run-learn.function.d.ts +35 -6
  18. package/dist/src/shared/cli/functions/run-recall.function.d.ts +45 -0
  19. package/dist/src/shared/cli/functions/run-search.function.d.ts +52 -15
  20. package/dist/src/shared/cli/index.d.ts +4 -0
  21. package/dist/src/shared/cli/typings/cli-options.interface.d.ts +13 -0
  22. package/dist/src/shared/cli/typings/learn-result.model.d.ts +19 -0
  23. package/dist/src/shared/cli/typings/search-display.model.d.ts +18 -0
  24. package/dist/src/shared/mcp/classes/mcp-server.class.d.ts +8 -0
  25. package/dist/src/shared/mcp/typings/mcp-client.interface.d.ts +43 -0
  26. package/dist/src/shared/version/constants/version.constant.d.ts +1 -1
  27. package/dist/src/shared/wasm-codec/functions/decode-add-section-result.function.d.ts +48 -0
  28. package/dist/src/shared/wasm-codec/functions/decode-scored-results.function.d.ts +58 -0
  29. package/dist/src/shared/wasm-codec/functions/decode-section-records.function.d.ts +33 -0
  30. package/dist/src/shared/wasm-codec/functions/encode-add-section-params.function.d.ts +52 -0
  31. package/dist/src/shared/wasm-codec/index.d.ts +2 -0
  32. package/dist/src/shared/worker-client/classes/worker-client.class.d.ts +16 -0
  33. package/dist/src/shared/worker-client/index.d.ts +1 -1
  34. package/dist/src/shared/worker-client/typings/add-section-result.model.d.ts +29 -15
  35. package/dist/src/shared/worker-client/typings/export-result.model.d.ts +14 -0
  36. package/dist/src/shared/worker-client/typings/gc-result.model.d.ts +6 -0
  37. package/dist/src/shared/worker-client/typings/search-result.model.d.ts +12 -0
  38. package/dist/src/shared/worker-client/typings/section.model.d.ts +16 -0
  39. package/dist/src/shared/worker-daemon/classes/worker-daemon.class.d.ts +35 -0
  40. package/dist/wasm/fs-database/kbdb_fs_database.d.ts +22 -1
  41. package/dist/wasm/fs-database/kbdb_fs_database.js +28 -1
  42. package/dist/wasm/fs-database/kbdb_fs_database_bg.wasm +0 -0
  43. package/dist/wasm/fs-database/kbdb_fs_database_bg.wasm.d.ts +1 -0
  44. package/dist/wasm/kb-worker/kbdb_worker.d.ts +48 -1
  45. package/dist/wasm/kb-worker/kbdb_worker.js +60 -1
  46. package/dist/wasm/kb-worker/kbdb_worker_bg.wasm +0 -0
  47. package/dist/wasm/kb-worker/kbdb_worker_bg.wasm.d.ts +2 -0
  48. package/dist/wasm/query-parser/kbdb_query_parser_bg.wasm +0 -0
  49. package/dist/worker.mjs +396 -60
  50. package/dist/worker.mjs.map +4 -4
  51. package/docs/details/cli.md +102 -15
  52. package/docs/details/library-api.md +55 -6
  53. package/docs/details/mcp-server.md +8 -4
  54. package/docs/details/search-and-ranking.md +63 -5
  55. package/docs/details/storage.md +2 -0
  56. package/docs/goals/cli.md +255 -14
  57. package/docs/goals/database.md +244 -17
  58. package/docs/goals/document.md +1 -1
  59. package/docs/goals/mcp.md +83 -14
  60. package/docs/goals/query-parser.md +98 -4
  61. package/docs/goals/query-result.md +177 -10
  62. package/docs/goals/skills.md +72 -0
  63. package/docs/goals/worker-daemon.md +54 -7
  64. package/docs/modules/rust/fs-database/functions/integrity.md +11 -0
  65. package/docs/modules/rust/fs-database/functions/section.md +23 -6
  66. package/docs/modules/rust/fs-database/module.md +1 -0
  67. package/docs/modules/rust/fs-database/types/section.md +27 -1
  68. package/docs/modules/rust/overview.md +2 -1
  69. package/docs/modules/rust/query-parser/functions/module.md +2 -2
  70. package/docs/modules/rust/query-parser/functions/pipeline.md +44 -6
  71. package/docs/modules/rust/query-parser/module.md +5 -4
  72. package/docs/modules/rust/query-parser/types/keyphrase.md +11 -0
  73. package/docs/modules/rust/query-parser/types/module.md +3 -3
  74. package/docs/modules/typescript/shared/cli/functions.md +68 -3
  75. package/package.json +1 -1
  76. package/src/shared/cli/functions/dispatch-command.function.ts +53 -1
  77. package/src/shared/cli/functions/format-output.function.ts +36 -12
  78. package/src/shared/cli/functions/generate-help.function.ts +56 -1
  79. package/src/shared/cli/functions/install-default-skills.function.ts +296 -0
  80. package/src/shared/cli/functions/parse-args.function.ts +13 -0
  81. package/src/shared/cli/functions/run-check.function.ts +87 -6
  82. package/src/shared/cli/functions/run-export.function.ts +40 -0
  83. package/src/shared/cli/functions/run-gc.function.ts +6 -0
  84. package/src/shared/cli/functions/run-init.function.ts +2 -0
  85. package/src/shared/cli/functions/run-learn.function.ts +68 -16
  86. package/src/shared/cli/functions/run-recall.function.ts +66 -0
  87. package/src/shared/cli/functions/run-search.function.ts +58 -15
  88. package/src/shared/cli/index.ts +4 -0
  89. package/src/shared/cli/typings/cli-options.interface.ts +13 -0
  90. package/src/shared/cli/typings/learn-result.model.ts +16 -0
  91. package/src/shared/cli/typings/search-display.model.ts +18 -0
  92. package/src/shared/mcp/classes/mcp-server.class.ts +30 -1
  93. package/src/shared/mcp/functions/create-tool-definitions.function.ts +46 -5
  94. package/src/shared/mcp/functions/handle-tool-call.function.ts +70 -2
  95. package/src/shared/mcp/typings/mcp-client.interface.ts +45 -0
  96. package/src/shared/version/constants/version.constant.ts +1 -1
  97. package/src/shared/wasm-codec/functions/decode-add-section-result.function.ts +112 -0
  98. package/src/shared/wasm-codec/functions/decode-scored-results.function.ts +86 -0
  99. package/src/shared/wasm-codec/functions/decode-section-records.function.ts +87 -0
  100. package/src/shared/wasm-codec/functions/encode-add-section-params.function.ts +68 -0
  101. package/src/shared/wasm-codec/index.ts +5 -0
  102. package/src/shared/worker-client/classes/worker-client.class.ts +19 -0
  103. package/src/shared/worker-client/index.ts +4 -1
  104. package/src/shared/worker-client/typings/add-section-result.model.ts +30 -15
  105. package/src/shared/worker-client/typings/export-result.model.ts +14 -0
  106. package/src/shared/worker-client/typings/gc-result.model.ts +6 -0
  107. package/src/shared/worker-client/typings/search-result.model.ts +12 -0
  108. package/src/shared/worker-client/typings/section.model.ts +16 -0
  109. package/src/shared/worker-daemon/classes/worker-daemon.class.ts +315 -46
  110. package/src/shared/worker-daemon/functions/create-host-imports.function.ts +19 -4
  111. package/src/shared/worker-daemon/functions/create-ipc-server.function.ts +5 -1
  112. package/src/wasm/fs-database/kbdb_fs_database.d.ts +22 -1
  113. package/src/wasm/fs-database/kbdb_fs_database.js +28 -1
  114. package/src/wasm/fs-database/kbdb_fs_database_bg.wasm +0 -0
  115. package/src/wasm/fs-database/kbdb_fs_database_bg.wasm.d.ts +1 -0
  116. package/src/wasm/kb-worker/kbdb_worker.d.ts +48 -1
  117. package/src/wasm/kb-worker/kbdb_worker.js +60 -1
  118. package/src/wasm/kb-worker/kbdb_worker_bg.wasm +0 -0
  119. package/src/wasm/kb-worker/kbdb_worker_bg.wasm.d.ts +2 -0
  120. package/src/wasm/query-parser/kbdb_query_parser_bg.wasm +0 -0
package/README.md CHANGED
@@ -6,7 +6,7 @@ required.
6
6
  [GitLab](https://gitlab.com/tech-slaves/knowledge-base-db)
7
7
  | [NPM](https://www.npmjs.com/package/@dikolab/kbdb)
8
8
  | [JSR](https://jsr.io/@dikolab/kbdb)
9
- | [License: ISC](LICENSE)
9
+ | [License: ISC](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/LICENSE)
10
10
 
11
11
  ---
12
12
 
@@ -26,11 +26,36 @@ It runs anywhere Node.js or Deno runs. Import your
26
26
  Markdown files, and kbdb indexes them automatically.
27
27
  Search returns ranked results instantly.
28
28
 
29
+ kbdb uses **lexical search** (keyword matching with
30
+ stemming and synonym expansion), not semantic
31
+ embeddings. This is a deliberate tradeoff: fast,
32
+ offline, deterministic, and zero embedding cost.
33
+ When AI agents are the primary consumers, lexical
34
+ search works well -- agents can issue multiple
35
+ keyword queries, use boolean/phrase syntax, and
36
+ expand vocabulary themselves. When an exact-match
37
+ query returns nothing, kbdb automatically retries
38
+ with relaxed OR matching so you always get the
39
+ best available results. Write clear headings
40
+ (they carry 2x ranking weight) and use specific
41
+ terms for best recall.
42
+
43
+ kbdb tracks **knowledge freshness** automatically.
44
+ Re-learn a file and supersession replaces the old
45
+ version instead of duplicating it. Near-duplicate
46
+ detection warns you before content drifts.
47
+ Integrity checks catch contradictions between
48
+ versions of the same source. Confidence scores
49
+ let agents tell strong matches from weak tail
50
+ results.
51
+
29
52
  As a bonus, kbdb also works as an MCP server --
30
53
  giving AI agents like Claude a searchable second
31
54
  brain. Store skills, agent profiles, and
32
55
  documentation that your AI can query across
33
- sessions.
56
+ sessions. Default skills (`query-guidance` and
57
+ `memory-protocol`) teach agents how to search
58
+ effectively and what to store.
34
59
 
35
60
  ---
36
61
 
@@ -80,7 +105,9 @@ kbdb learn ./docs --db ./my-kb
80
105
 
81
106
  Point it at a folder of Markdown files. kbdb reads
82
107
  them, breaks them into sections, and builds a
83
- search index.
108
+ search index. Add `--tags design,v2` to tag
109
+ sections for scoping, or `--replace` to supersede
110
+ existing sections from the same source.
84
111
 
85
112
  **3. Search**
86
113
 
@@ -88,9 +115,11 @@ search index.
88
115
  kbdb search "how does auth work" --db ./my-kb
89
116
  ```
90
117
 
91
- Results are ranked by relevance with snippets
92
- showing matched terms in context. Use `--offset`
93
- to page through large result sets.
118
+ Results are ranked by relevance with confidence
119
+ scores and snippets showing matched terms in
120
+ context. Each result includes `matched_fields`
121
+ (heading, body, code) and `created_at` timestamps.
122
+ Use `--offset` to page through large result sets.
94
123
 
95
124
  **4. Recall context**
96
125
 
@@ -109,21 +138,27 @@ referenced sections.
109
138
 
110
139
  ## What Can It Do?
111
140
 
112
- - **Import** Markdown files and other documents
113
- - **Search** with ranked results, snippets, and
114
- paginated output
141
+ - **Import** Markdown files and other documents,
142
+ with tags and source tracking
143
+ - **Supersede** outdated sections automatically
144
+ when re-learning from the same source
145
+ - **Search** with ranked results, confidence
146
+ scores, snippets, and paginated output
147
+ - **Relaxed fallback** -- automatic OR retry when
148
+ exact AND queries return nothing
115
149
  - **Recall** sections with progressive context --
116
150
  from a quick summary to full related content
151
+ - **Export** the database as a portable snapshot
117
152
  - **Skills** -- store reusable prompt templates
118
153
  with placeholder arguments
119
154
  - **Agents** -- create AI agent profiles that
120
155
  combine a persona with skills
121
156
  - **Manage** documents, sections, and metadata
122
- - **Verify** database integrity and clean up
123
- unused data
157
+ - **Verify** database integrity, detect
158
+ contradictions, and clean up superseded data
124
159
  - **Rebuild** indexes if anything goes wrong
125
160
 
126
- See the [CLI Reference](docs/details/cli.md)
161
+ See the [CLI Reference](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/details/cli.md)
127
162
  for the full command list and examples.
128
163
 
129
164
  ---
@@ -132,10 +167,10 @@ for the full command list and examples.
132
167
 
133
168
  kbdb runs as an
134
169
  [MCP](https://modelcontextprotocol.io/) server
135
- with 18 tools, so AI agents can search your docs,
136
- recall context progressively, store reusable
137
- skills, and build agent profiles -- a second brain
138
- that persists between sessions.
170
+ with 19 tools, so AI agents can search your docs,
171
+ recall context progressively, export snapshots,
172
+ store reusable skills, and build agent profiles --
173
+ a second brain that persists between sessions.
139
174
 
140
175
  ### Claude Code (CLI)
141
176
 
@@ -203,7 +238,7 @@ Add to your `claude_desktop_config.json`:
203
238
  ### Other AI Agents
204
239
 
205
240
  Any MCP-compatible agent can connect. See the
206
- [MCP Server Guide](docs/details/mcp-server.md) for
241
+ [MCP Server Guide](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/details/mcp-server.md) for
207
242
  setup instructions, Deno variants, environment
208
243
  variable configuration, and the full list of
209
244
  available tools.
@@ -230,13 +265,13 @@ const results = await client.search({
230
265
  offset: 0,
231
266
  });
232
267
 
233
- // results = { items, total, offset, limit, has_more }
268
+ // results = { items, total, offset, limit, has_more, relaxed }
234
269
  console.log(results.items);
235
270
  client.disconnect();
236
271
  ```
237
272
 
238
273
  See the
239
- [Library API Reference](docs/details/library-api.md)
274
+ [Library API Reference](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/details/library-api.md)
240
275
  for the full API.
241
276
 
242
277
  ### Development Setup
@@ -257,7 +292,7 @@ A Docker setup is included with all build tools
257
292
  HOST_UMASK=$(umask) docker compose run --rm dev bash
258
293
  ```
259
294
 
260
- See the [Makefile](Makefile) for all available
295
+ See the [Makefile](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/Makefile) for all available
261
296
  build targets.
262
297
 
263
298
  ### Contributing
@@ -272,17 +307,17 @@ build targets.
272
307
 
273
308
  ## Documentation
274
309
 
275
- - [Detailed Guides](docs/details/) -- CLI
310
+ - [Detailed Guides](https://gitlab.com/tech-slaves/knowledge-base-db/-/tree/main/docs/details) -- CLI
276
311
  reference, MCP setup, search internals, storage
277
312
  format, library API
278
- - [Release Notes](docs/release-notes/)
279
- - [Architecture Overview](docs/overview.md)
313
+ - [Release Notes](https://gitlab.com/tech-slaves/knowledge-base-db/-/tree/main/docs/release-notes)
314
+ - [Architecture Overview](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/overview.md)
280
315
 
281
316
  ---
282
317
 
283
318
  ## License
284
319
 
285
- ISC License -- see [LICENSE](LICENSE) for details.
320
+ ISC License -- see [LICENSE](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/LICENSE) for details.
286
321
 
287
322
  ---
288
323