@dikolab/kbdb 0.1.6 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (193) hide show
  1. package/README.md +30 -9
  2. package/dist/cli.cjs +181 -50
  3. package/dist/cli.cjs.map +3 -3
  4. package/dist/cli.mjs +181 -50
  5. package/dist/cli.mjs.map +3 -3
  6. package/dist/kbdb-worker.cjs +283 -29
  7. package/dist/kbdb-worker.cjs.map +4 -4
  8. package/dist/mod.cjs +13 -2
  9. package/dist/mod.cjs.map +2 -2
  10. package/dist/mod.mjs +13 -2
  11. package/dist/mod.mjs.map +2 -2
  12. package/dist/src/shared/cli/functions/format-output.function.d.ts +7 -7
  13. package/dist/src/shared/cli/functions/run-search.function.d.ts +25 -16
  14. package/dist/src/shared/cli/index.d.ts +1 -1
  15. package/dist/src/shared/cli/typings/cli-options.interface.d.ts +2 -0
  16. package/dist/src/shared/cli/typings/search-display.model.d.ts +25 -7
  17. package/dist/src/shared/mcp/functions/handle-recall.function.d.ts +29 -0
  18. package/dist/src/shared/mcp/typings/mcp-client.interface.d.ts +30 -11
  19. package/dist/src/shared/version/constants/version.constant.d.ts +1 -1
  20. package/dist/src/shared/wasm-codec/functions/decode-recall-result.function.d.ts +28 -0
  21. package/dist/src/shared/wasm-codec/functions/decode-scored-results.function.d.ts +14 -7
  22. package/dist/src/shared/wasm-codec/functions/encode-recall-params.function.d.ts +21 -0
  23. package/dist/src/shared/wasm-codec/functions/encode-search-params.function.d.ts +1 -1
  24. package/dist/src/shared/wasm-codec/index.d.ts +5 -2
  25. package/dist/src/shared/worker-client/classes/worker-client.class.d.ts +13 -3
  26. package/dist/src/shared/worker-client/index.d.ts +2 -1
  27. package/dist/src/shared/worker-client/typings/recall-result.model.d.ts +94 -0
  28. package/dist/src/shared/worker-client/typings/search-params.model.d.ts +2 -0
  29. package/dist/src/shared/worker-client/typings/search-result.model.d.ts +23 -7
  30. package/dist/src/shared/worker-daemon/classes/worker-daemon.class.d.ts +1 -0
  31. package/dist/wasm/fs-database/kbdb_fs_database.d.ts +18 -0
  32. package/dist/wasm/fs-database/kbdb_fs_database.js +30 -0
  33. package/dist/wasm/fs-database/kbdb_fs_database_bg.wasm +0 -0
  34. package/dist/wasm/fs-database/kbdb_fs_database_bg.wasm.d.ts +2 -0
  35. package/dist/wasm/kb-worker/kbdb_worker.d.ts +38 -0
  36. package/dist/wasm/kb-worker/kbdb_worker.js +56 -0
  37. package/dist/wasm/kb-worker/kbdb_worker_bg.wasm +0 -0
  38. package/dist/wasm/kb-worker/kbdb_worker_bg.wasm.d.ts +3 -0
  39. package/dist/worker.mjs +283 -29
  40. package/dist/worker.mjs.map +4 -4
  41. package/docs/details/README.md +26 -0
  42. package/docs/details/cli.md +608 -0
  43. package/docs/details/library-api.md +406 -0
  44. package/docs/details/mcp-server.md +430 -0
  45. package/docs/details/search-and-ranking.md +457 -0
  46. package/docs/details/storage.md +529 -0
  47. package/docs/goals/agents.md +751 -0
  48. package/docs/goals/architecture.svg +198 -0
  49. package/docs/goals/cli.md +2308 -0
  50. package/docs/goals/content-composer.md +609 -0
  51. package/docs/goals/content-parser.md +279 -0
  52. package/docs/goals/database.md +1452 -0
  53. package/docs/goals/document.md +368 -0
  54. package/docs/goals/mcp.md +1467 -0
  55. package/docs/goals/overview.md +124 -0
  56. package/docs/goals/query-parser.md +271 -0
  57. package/docs/goals/query-result.md +693 -0
  58. package/docs/goals/skills.md +738 -0
  59. package/docs/goals/worker-client.md +944 -0
  60. package/docs/goals/worker-daemon.md +1447 -0
  61. package/docs/modules/overview.md +298 -0
  62. package/docs/modules/rust/fs-database/functions/document.md +48 -0
  63. package/docs/modules/rust/fs-database/functions/integrity.md +68 -0
  64. package/docs/modules/rust/fs-database/functions/io.md +87 -0
  65. package/docs/modules/rust/fs-database/functions/module.md +22 -0
  66. package/docs/modules/rust/fs-database/functions/query-exec.md +44 -0
  67. package/docs/modules/rust/fs-database/functions/section.md +59 -0
  68. package/docs/modules/rust/fs-database/indexes/btree.md +35 -0
  69. package/docs/modules/rust/fs-database/indexes/corpus.md +61 -0
  70. package/docs/modules/rust/fs-database/indexes/inverted.md +60 -0
  71. package/docs/modules/rust/fs-database/indexes/module.md +64 -0
  72. package/docs/modules/rust/fs-database/indexes/positional.md +39 -0
  73. package/docs/modules/rust/fs-database/module.md +149 -0
  74. package/docs/modules/rust/fs-database/types/catalog.md +53 -0
  75. package/docs/modules/rust/fs-database/types/module.md +12 -0
  76. package/docs/modules/rust/fs-database/types/section.md +101 -0
  77. package/docs/modules/rust/kb-worker/functions/compose.md +57 -0
  78. package/docs/modules/rust/kb-worker/functions/module.md +20 -0
  79. package/docs/modules/rust/kb-worker/functions/retrieve.md +194 -0
  80. package/docs/modules/rust/kb-worker/functions/search.md +173 -0
  81. package/docs/modules/rust/kb-worker/functions/write.md +172 -0
  82. package/docs/modules/rust/kb-worker/module.md +171 -0
  83. package/docs/modules/rust/kb-worker/types/cache.md +90 -0
  84. package/docs/modules/rust/kb-worker/types/context.md +26 -0
  85. package/docs/modules/rust/kb-worker/types/module.md +16 -0
  86. package/docs/modules/rust/kb-worker/types/status.md +29 -0
  87. package/docs/modules/rust/kb-worker/types/token.md +50 -0
  88. package/docs/modules/rust/overview.md +495 -0
  89. package/docs/modules/rust/query-parser/functions/extract.md +56 -0
  90. package/docs/modules/rust/query-parser/functions/module.md +16 -0
  91. package/docs/modules/rust/query-parser/functions/parse.md +33 -0
  92. package/docs/modules/rust/query-parser/functions/pipeline.md +103 -0
  93. package/docs/modules/rust/query-parser/functions/rank.md +85 -0
  94. package/docs/modules/rust/query-parser/module.md +136 -0
  95. package/docs/modules/rust/query-parser/types/cache.md +39 -0
  96. package/docs/modules/rust/query-parser/types/keyphrase.md +60 -0
  97. package/docs/modules/rust/query-parser/types/module.md +17 -0
  98. package/docs/modules/rust/query-parser/types/token.md +42 -0
  99. package/docs/modules/rust/shared/content-composer/functions.md +114 -0
  100. package/docs/modules/rust/shared/content-composer/module.md +147 -0
  101. package/docs/modules/rust/shared/content-composer/types.md +93 -0
  102. package/docs/modules/rust/shared/content-parser/functions.md +72 -0
  103. package/docs/modules/rust/shared/content-parser/module.md +99 -0
  104. package/docs/modules/rust/shared/content-parser/types.md +71 -0
  105. package/docs/modules/rust/shared/corpus/module.md +24 -0
  106. package/docs/modules/rust/shared/corpus/types.md +141 -0
  107. package/docs/modules/rust/shared/document/module.md +25 -0
  108. package/docs/modules/rust/shared/document/types.md +154 -0
  109. package/docs/modules/rust/shared/encode/module.md +22 -0
  110. package/docs/modules/rust/shared/encode/traits.md +304 -0
  111. package/docs/modules/rust/shared/error/module.md +28 -0
  112. package/docs/modules/rust/shared/error/types.md +302 -0
  113. package/docs/modules/rust/shared/kbid/module.md +24 -0
  114. package/docs/modules/rust/shared/kbid/types.md +147 -0
  115. package/docs/modules/rust/shared/memory/functions.md +209 -0
  116. package/docs/modules/rust/shared/memory/module.md +24 -0
  117. package/docs/modules/rust/shared/module.md +168 -0
  118. package/docs/modules/rust/shared/query/module.md +27 -0
  119. package/docs/modules/rust/shared/query/types.md +236 -0
  120. package/docs/modules/rust/shared/section/module.md +25 -0
  121. package/docs/modules/rust/shared/section/types.md +294 -0
  122. package/docs/modules/rust/shared/term/module.md +25 -0
  123. package/docs/modules/rust/shared/term/types.md +139 -0
  124. package/docs/modules/typescript/cli.md +129 -0
  125. package/docs/modules/typescript/kbdb-worker.md +135 -0
  126. package/docs/modules/typescript/overview.md +378 -0
  127. package/docs/modules/typescript/shared/cli/constants.md +23 -0
  128. package/docs/modules/typescript/shared/cli/functions.md +630 -0
  129. package/docs/modules/typescript/shared/cli/module.md +125 -0
  130. package/docs/modules/typescript/shared/cli/typings.md +172 -0
  131. package/docs/modules/typescript/shared/hash/functions.md +20 -0
  132. package/docs/modules/typescript/shared/hash/module.md +21 -0
  133. package/docs/modules/typescript/shared/mcp/classes.md +179 -0
  134. package/docs/modules/typescript/shared/mcp/functions.md +386 -0
  135. package/docs/modules/typescript/shared/mcp/module.md +160 -0
  136. package/docs/modules/typescript/shared/mcp/typings.md +522 -0
  137. package/docs/modules/typescript/shared/module.md +94 -0
  138. package/docs/modules/typescript/shared/platform/functions.md +42 -0
  139. package/docs/modules/typescript/shared/platform/module.md +25 -0
  140. package/docs/modules/typescript/shared/runtime/functions.md +26 -0
  141. package/docs/modules/typescript/shared/runtime/module.md +27 -0
  142. package/docs/modules/typescript/shared/runtime/typings.md +13 -0
  143. package/docs/modules/typescript/shared/version/module.md +27 -0
  144. package/docs/modules/typescript/shared/wasm-codec/functions.md +391 -0
  145. package/docs/modules/typescript/shared/wasm-codec/module.md +98 -0
  146. package/docs/modules/typescript/shared/worker-client/classes.md +264 -0
  147. package/docs/modules/typescript/shared/worker-client/functions.md +175 -0
  148. package/docs/modules/typescript/shared/worker-client/module.md +84 -0
  149. package/docs/modules/typescript/shared/worker-client/typings.md +508 -0
  150. package/docs/modules/typescript/shared/worker-daemon/classes.md +239 -0
  151. package/docs/modules/typescript/shared/worker-daemon/constants.md +37 -0
  152. package/docs/modules/typescript/shared/worker-daemon/functions.md +221 -0
  153. package/docs/modules/typescript/shared/worker-daemon/module.md +112 -0
  154. package/docs/modules/typescript/shared/worker-daemon/typings.md +126 -0
  155. package/docs/overview.md +188 -0
  156. package/docs/release-notes/0.1.0.md +189 -0
  157. package/docs/release-notes/0.1.1.md +46 -0
  158. package/docs/release-notes/0.1.2.md +38 -0
  159. package/docs/release-notes/0.1.3.md +42 -0
  160. package/docs/release-notes/0.2.0.md +147 -0
  161. package/docs/release-notes/README.md +9 -0
  162. package/package.json +8 -2
  163. package/src/shared/cli/functions/dispatch-command.function.ts +3 -3
  164. package/src/shared/cli/functions/format-output.function.ts +41 -34
  165. package/src/shared/cli/functions/parse-args.function.ts +7 -0
  166. package/src/shared/cli/functions/run-search.function.ts +53 -27
  167. package/src/shared/cli/index.ts +4 -1
  168. package/src/shared/cli/typings/cli-options.interface.ts +2 -0
  169. package/src/shared/cli/typings/search-display.model.ts +26 -7
  170. package/src/shared/mcp/functions/create-tool-definitions.function.ts +64 -4
  171. package/src/shared/mcp/functions/handle-recall.function.ts +66 -0
  172. package/src/shared/mcp/functions/handle-tool-call.function.ts +11 -1
  173. package/src/shared/mcp/typings/mcp-client.interface.ts +33 -15
  174. package/src/shared/version/constants/version.constant.ts +1 -1
  175. package/src/shared/wasm-codec/functions/decode-recall-result.function.ts +267 -0
  176. package/src/shared/wasm-codec/functions/decode-scored-results.function.ts +81 -27
  177. package/src/shared/wasm-codec/functions/encode-recall-params.function.ts +68 -0
  178. package/src/shared/wasm-codec/functions/encode-search-params.function.ts +4 -1
  179. package/src/shared/wasm-codec/index.ts +11 -3
  180. package/src/shared/worker-client/classes/worker-client.class.ts +24 -4
  181. package/src/shared/worker-client/index.ts +11 -1
  182. package/src/shared/worker-client/typings/recall-result.model.ts +98 -0
  183. package/src/shared/worker-client/typings/search-params.model.ts +2 -0
  184. package/src/shared/worker-client/typings/search-result.model.ts +24 -7
  185. package/src/shared/worker-daemon/classes/worker-daemon.class.ts +25 -4
  186. package/src/wasm/fs-database/kbdb_fs_database.d.ts +18 -0
  187. package/src/wasm/fs-database/kbdb_fs_database.js +30 -0
  188. package/src/wasm/fs-database/kbdb_fs_database_bg.wasm +0 -0
  189. package/src/wasm/fs-database/kbdb_fs_database_bg.wasm.d.ts +2 -0
  190. package/src/wasm/kb-worker/kbdb_worker.d.ts +38 -0
  191. package/src/wasm/kb-worker/kbdb_worker.js +56 -0
  192. package/src/wasm/kb-worker/kbdb_worker_bg.wasm +0 -0
  193. package/src/wasm/kb-worker/kbdb_worker_bg.wasm.d.ts +3 -0
@@ -0,0 +1,944 @@
1
+ # Worker Client
2
+
3
+ *A shared TypeScript module that discovers or spawns
4
+ the worker daemon and provides a typed API for all
5
+ knowledge base operations over IPC.*
6
+
7
+ ## Goal
8
+
9
+ `src/cli.ts` needs to communicate with the worker
10
+ daemon (`src/worker.ts`) to perform database
11
+ operations. The worker client module is the single
12
+ shared entry point for that communication. It
13
+ handles:
14
+
15
+ - Resolving the database directory path.
16
+ - Computing the context ID.
17
+ - Discovering an existing daemon or spawning a new one.
18
+ - Connecting to the daemon's IPC socket.
19
+ - Exposing every daemon IPC method as a typed async
20
+ function.
21
+ - Managing connection lifecycle (connect, reconnect,
22
+ disconnect).
23
+
24
+ No other module in the TypeScript layer communicates
25
+ with the daemon directly. `src/cli.ts` imports the
26
+ worker client and calls its methods -- it never
27
+ constructs IPC messages, manages PID files, or
28
+ spawns processes itself.
29
+
30
+ ```
31
+ src/cli.ts ──→ worker-client ──→ src/worker.ts
32
+ ```
33
+
34
+ ## Module Location
35
+
36
+ ```
37
+ src/shared/worker-client/
38
+ ├── mod.ts # public API: createWorkerClient
39
+ ├── resolve.ts # database directory resolution
40
+ ├── context.ts # context ID computation
41
+ ├── discovery.ts # PID file lookup, process check
42
+ ├── spawn.ts # daemon spawning (Node + Deno)
43
+ ├── connection.ts # IPC socket connect / reconnect
44
+ └── types.ts # request/response type defs
45
+ ```
46
+
47
+ The module is internal to the package. It is not
48
+ exported from the library's public API (`mod.ts` at
49
+ the project root). Only `src/cli.ts` imports it.
50
+
51
+ ## When the Daemon Is Needed
52
+
53
+ The worker client is only created when the caller needs
54
+ to perform operations that involve the `.kbdb` database
55
+ directory:
56
+
57
+ - **Search and query** -- Looking up indexes, ranking
58
+ results, retrieving sections or documents.
59
+ - **Content management** -- `kbdb learn` (import,
60
+ index) and `kbdb unlearn` (remove, re-index).
61
+ - **Status and retrieval** -- Database metadata,
62
+ content retrieval by ID.
63
+
64
+ Operations that do not touch the database do not create
65
+ a worker client and do not require a daemon:
66
+
67
+ - `kbdb --help`, `kbdb --version`
68
+ - Invalid commands or argument validation errors
69
+ - Any subcommand that exits before reaching the
70
+ database
71
+
72
+ ## Database Directory Resolution
73
+
74
+ The worker client resolves the database directory path
75
+ before connecting to the daemon:
76
+
77
+ 1. If the caller provides a `targetDir` (e.g. from a
78
+ CLI `--db <path>` flag), use it.
79
+ 2. Otherwise, use the current working directory.
80
+
81
+ The database directory is always
82
+ `${targetDir}/.kbdb`. The worker client checks that
83
+ this directory exists before proceeding. If `.kbdb`
84
+ does not exist, the factory throws -- the caller is
85
+ responsible for database initialization (e.g. the CLI
86
+ prompts the user to create it).
87
+
88
+ ## Context ID
89
+
90
+ The worker client computes the context ID during
91
+ `createWorkerClient()` and includes it in every IPC
92
+ request. The context ID is a **SHA-256 hash of the
93
+ database directory's absolute path**, truncated to 16
94
+ lowercase hexadecimal characters.
95
+
96
+ ```
97
+ SHA-256(absolutePath) → truncate to 16 hex chars → ctx
98
+ ```
99
+
100
+ See [Worker Daemon -- Context ID](
101
+ worker-daemon.md#context-id) for the full
102
+ specification, including how the daemon validates the
103
+ context ID on every request.
104
+
105
+ ## Daemon Discovery
106
+
107
+ When the worker client is created, it follows this
108
+ discovery sequence:
109
+
110
+ 1. Compute the context ID from the database directory
111
+ path.
112
+ 2. Look for `kb-worker-{ctx}.pid` in the OS temporary
113
+ directory (e.g., `/tmp` on Linux/macOS, `%TEMP%` on
114
+ Windows).
115
+ 3. If the PID file exists, read the process ID and
116
+ check whether the process is alive (see
117
+ Platform-Specific PID Handling below).
118
+ 4. If the process is alive, connect to the IPC socket
119
+ at `kb-worker-{ctx}.sock` (or the named pipe on
120
+ Windows).
121
+ 5. If the PID file is missing or the process is dead
122
+ (stale PID file), remove the stale file and spawn
123
+ a new daemon (see Spawning below).
124
+
125
+ The PID file contains only the process ID number as
126
+ plain text (e.g., `12345`). No other data.
127
+
128
+ ### Platform-Specific PID Handling
129
+
130
+ Both Linux/macOS and Windows use integer process IDs.
131
+ The PID file format is identical across platforms -- a
132
+ single integer as plain text. The differences are in
133
+ how the worker client resolves the temporary directory
134
+ and checks whether the process is alive.
135
+
136
+ **Temporary directory:**
137
+
138
+ | Platform | Resolution |
139
+ |---------------|---------------------------------------|
140
+ | Linux / macOS | `$TMPDIR` if set, otherwise `/tmp` |
141
+ | Windows | `%TEMP%` (typically `C:\Users\{user}\AppData\Local\Temp`) |
142
+
143
+ In Node.js, `os.tmpdir()` returns the correct path on
144
+ all platforms. In Deno, `Deno.env.get("TMPDIR")` (Unix)
145
+ or `Deno.env.get("TEMP")` (Windows) provides the same.
146
+
147
+ **Checking if a process is alive:**
148
+
149
+ The worker client must verify that the PID read from
150
+ the file refers to a running daemon, not a dead process
151
+ whose PID has been recycled by the OS.
152
+
153
+ **Linux / macOS** -- Use a zero signal via `kill()`:
154
+
155
+ ```ts
156
+ // Node.js
157
+ try {
158
+ process.kill(pid, 0); // signal 0 = no signal sent
159
+ // process exists
160
+ } catch {
161
+ // process does not exist (ESRCH) or
162
+ // not permitted (EPERM -- process exists
163
+ // but is owned by another user)
164
+ }
165
+ ```
166
+
167
+ `kill(pid, 0)` sends no actual signal. It returns
168
+ successfully if the process exists and the caller has
169
+ permission to signal it. It throws `ESRCH` if the
170
+ process does not exist. An `EPERM` error means the
171
+ process exists but belongs to another user -- this
172
+ should be treated as "alive" since the PID is in use.
173
+
174
+ **Windows** -- `process.kill(pid, 0)` is supported in
175
+ Node.js on Windows and behaves the same way: it checks
176
+ process existence without sending a signal. This is the
177
+ preferred cross-platform method.
178
+
179
+ For Deno on Windows, where `Deno.kill()` does not
180
+ support signal 0, use a subprocess check:
181
+
182
+ ```ts
183
+ // Deno (Windows fallback)
184
+ const cmd = new Deno.Command('tasklist', {
185
+ args: ['/FI', `PID eq ${pid}`, '/NH', '/FO', 'CSV'],
186
+ stdout: 'piped',
187
+ stderr: 'null',
188
+ });
189
+ const { stdout } = await cmd.output();
190
+ const output = new TextDecoder().decode(stdout);
191
+ const alive = output.includes(`"${pid}"`);
192
+ ```
193
+
194
+ `tasklist /FI "PID eq {pid}"` filters the process list
195
+ by PID. If the output contains the PID, the process is
196
+ alive. The `/NH` flag suppresses the header and `/FO
197
+ CSV` outputs in CSV format for reliable parsing.
198
+
199
+ **Cross-platform wrapper:**
200
+
201
+ The worker client implements a single
202
+ `isProcessAlive(pid)` function that selects the correct
203
+ method based on `process.platform` (Node.js) or
204
+ `Deno.build.os` (Deno):
205
+
206
+ | Runtime | Linux / macOS | Windows |
207
+ |-----------|------------------------|--------------------------|
208
+ | Node.js | `process.kill(pid, 0)` | `process.kill(pid, 0)` |
209
+ | Deno | `Deno.kill(pid, "SIGCONT")` | `tasklist` subprocess |
210
+
211
+ This function returns `true` if the process is alive,
212
+ `false` if it is not. The daemon discovery sequence
213
+ calls it after reading the PID file to decide whether
214
+ to connect or spawn.
215
+
216
+ ## Spawning
217
+
218
+ The worker client spawns the daemon as a **detached
219
+ child process** so it outlives the parent. Unlike a
220
+ typical fire-and-forget detach, the worker client
221
+ **captures the daemon's stderr** during the spawn
222
+ window so that startup failures can be relayed to the
223
+ caller.
224
+
225
+ **Node.js:**
226
+
227
+ ```js
228
+ const child = spawn(
229
+ 'node',
230
+ ['kb-worker.js', contextPath],
231
+ {
232
+ detached: true,
233
+ stdio: ['ignore', 'ignore', 'pipe'],
234
+ },
235
+ );
236
+ ```
237
+
238
+ **Deno:**
239
+
240
+ ```ts
241
+ const cmd = new Deno.Command('deno', {
242
+ args: [
243
+ 'run', '--allow-all',
244
+ 'src/worker.ts', contextPath,
245
+ ],
246
+ stdin: 'null',
247
+ stdout: 'null',
248
+ stderr: 'piped',
249
+ });
250
+ const child = cmd.spawn();
251
+ ```
252
+
253
+ Note: `stderr` is `'pipe'` / `'piped'`, not
254
+ `'ignore'`. The worker client reads from it during
255
+ the spawn window. Once the PID file appears
256
+ (indicating successful startup), the worker client
257
+ closes the stderr pipe and unrefs the child so the
258
+ daemon can outlive the parent.
259
+
260
+ The spawning sequence:
261
+
262
+ 1. Worker client ensures `worker.toml` exists in the
263
+ context directory (writes defaults if missing).
264
+ 2. Worker client spawns `src/worker.ts` (or its
265
+ compiled `.js` equivalent) with the context path as
266
+ the sole command-line argument. stderr is piped to
267
+ the worker client.
268
+ 3. Worker client polls for the PID file to appear (with
269
+ a short timeout, default 10 000 ms) to confirm the
270
+ daemon started.
271
+ 4. **On PID file found (success):** worker client
272
+ closes the stderr pipe, detaches the child, unrefs
273
+ it, and connects to the IPC socket.
274
+ 5. **On timeout or child exit (failure):** worker
275
+ client reads all buffered stderr from the daemon,
276
+ includes it in the thrown error (see Startup
277
+ Failure Handling below).
278
+
279
+ There is no fallback to in-process WASM loading -- the
280
+ daemon is required for all database operations.
281
+
282
+ ### Startup Failure Handling
283
+
284
+ When the daemon fails to start -- either because a
285
+ startup precondition failed (see
286
+ [Worker Daemon -- Startup Preconditions](
287
+ worker-daemon.md#startup-preconditions)) or the
288
+ process exited unexpectedly -- the worker client
289
+ detects the failure by one of:
290
+
291
+ - The child process exits before the PID file appears.
292
+ - The PID file poll times out.
293
+
294
+ In either case, the worker client:
295
+
296
+ 1. Reads the daemon's buffered stderr output.
297
+ 2. Throws a `WorkerClientError` that includes the
298
+ daemon's stderr message in the error's `message`
299
+ property.
300
+
301
+ ```ts
302
+ interface WorkerClientError extends Error {
303
+ code: number;
304
+ /** Daemon stderr output, if available. */
305
+ daemonStderr?: string;
306
+ }
307
+ ```
308
+
309
+ The error code is `-32003` (daemon unavailable).
310
+
311
+ The caller (CLI or MCP) is responsible for printing
312
+ the error. In practice, the CLI prints the error to
313
+ its own stderr and exits non-zero. This means the
314
+ daemon's error message flows through to the user's
315
+ terminal:
316
+
317
+ ```
318
+ kbdb search "query"
319
+ -> worker client spawns daemon
320
+ -> daemon checks preconditions
321
+ -> daemon prints to its stderr:
322
+ "error: database version 3 is newer than kbdb
323
+ version 2 -- upgrade kbdb to open this database"
324
+ -> daemon exits with code 1
325
+ -> worker client reads daemon stderr
326
+ -> worker client throws WorkerClientError with
327
+ daemonStderr containing the message
328
+ -> src/cli.ts catches error, prints to stderr:
329
+ "error: daemon failed to start: database version
330
+ 3 is newer than kbdb version 2 -- upgrade kbdb
331
+ to open this database"
332
+ -> src/cli.ts sets process.exitCode = 1
333
+ ```
334
+
335
+ This relay ensures the user sees the daemon's specific
336
+ error reason even though the daemon runs as a separate
337
+ detached process. The daemon's stderr is the single
338
+ source of truth for startup failure diagnostics.
339
+
340
+ ## `createWorkerClient(options?)`
341
+
342
+ Factory function. Returns a connected `WorkerClient`
343
+ instance.
344
+
345
+ ```ts
346
+ interface WorkerClientOptions {
347
+ targetDir?: string;
348
+ }
349
+
350
+ function createWorkerClient(
351
+ options?: WorkerClientOptions,
352
+ ): Promise<WorkerClient>;
353
+ ```
354
+
355
+ **Resolution order for `targetDir`:**
356
+
357
+ 1. If `options.targetDir` is provided, use it.
358
+ 2. Otherwise, use the current working directory.
359
+
360
+ The factory then:
361
+
362
+ 1. Resolves the absolute path: `${targetDir}/.kbdb`.
363
+ 2. Computes the context ID (SHA-256 of the absolute
364
+ path, truncated to 16 lowercase hex characters).
365
+ 3. Checks whether `.kbdb` exists at the resolved path.
366
+ If not, throws -- the caller is responsible for
367
+ prompting the user or initializing the database
368
+ before creating a client.
369
+ 4. Looks for `kb-worker-{ctx}.pid` in the OS temp
370
+ directory and checks whether the daemon is alive
371
+ (see Daemon Discovery above).
372
+ 5. If no daemon is running, spawns one (see Spawning
373
+ above).
374
+ 6. Connects to the IPC socket
375
+ (`kb-worker-{ctx}.sock` on Unix,
376
+ `\\.\pipe\kb-worker-{ctx}` on Windows).
377
+ 7. Returns the connected `WorkerClient`.
378
+
379
+ If the daemon fails to start or the IPC connection
380
+ cannot be established, the factory throws.
381
+
382
+ ## `WorkerClient`
383
+
384
+ All methods are async. High-level methods (`search`,
385
+ `readSections`, `addSection`, `updateSection`,
386
+ `removeSection`, `groupSections`, `removeGrouping`,
387
+ `retrieveDocument`, `content`) are composed from
388
+ the low-level IPC primitives (`query` →
389
+ `get_result_status` → `get_result`) documented in
390
+ [Worker Daemon -- IPC API](worker-daemon.md). The
391
+ daemon routes these queries to `kb-worker.wasm`,
392
+ which orchestrates `fs-database.wasm` and
393
+ `query-parser.wasm`.
394
+
395
+ `content()` takes one or more kbid or docid
396
+ identifiers (mixed freely) and returns a composed
397
+ Markdown document. It delegates to the daemon's
398
+ `compose` IPC method, which triggers `kb-worker.wasm`
399
+ to resolve identifiers, fetch sections, extract
400
+ markers recursively, and call the `content-composer`
401
+ module in `kbdb-shared`. See
402
+ [Content Composer](content-composer.md) for the full
403
+ composition specification.
404
+
405
+ Maintenance methods (`integrityCheck`, `gc`,
406
+ `rebuildIndexes`) delegate to the corresponding
407
+ daemon IPC methods (`integrity_check`, `gc`,
408
+ `rebuild_indexes`). These return results directly
409
+ (not via the token pattern) because maintenance
410
+ operations are infrequent and their results are not
411
+ cached.
412
+
413
+ `dbStatus()` returns data from the initialization
414
+ response cached at connect time -- it does not
415
+ issue a separate IPC call.
416
+
417
+ ```ts
418
+ interface WorkerClient {
419
+ readonly ctx: string;
420
+ readonly targetDir: string;
421
+
422
+ // Lifecycle
423
+ disconnect(): Promise<void>;
424
+
425
+ // Status
426
+ status(): Promise<DaemonStatus>;
427
+ idleAt(): Promise<number>;
428
+ config(key: ConfigKey): Promise<number>;
429
+ dbStatus(): Promise<DbStatus>;
430
+
431
+ // Search
432
+ search(params: SearchParams):
433
+ Promise<PagedSearchResult>;
434
+
435
+ // Context expansion
436
+ recall(params: {
437
+ kbid?: string;
438
+ kbids?: string[];
439
+ depth?: number;
440
+ }): Promise<RecallResult | RecallResult[]>;
441
+
442
+ // Sections
443
+ readSections(kbIds: string[]):
444
+ Promise<KbdbResult<Section[]>>;
445
+ addSection(params: AddSectionParams):
446
+ Promise<AddSectionResult>;
447
+ updateSection(params: UpdateSectionParams):
448
+ Promise<UpdateSectionResult>;
449
+ removeSection(kbId: string):
450
+ Promise<RemoveSectionResult>;
451
+
452
+ // Documents
453
+ groupSections(params: GroupSectionsParams):
454
+ Promise<GroupSectionsResult>;
455
+ removeGrouping(docid: string):
456
+ Promise<RemoveGroupingResult>;
457
+ retrieveDocument(docid: string):
458
+ Promise<KbdbResult<DocumentResult>>;
459
+
460
+ // Content composition
461
+ content(ids: string[]):
462
+ Promise<ContentResult>;
463
+
464
+ // Maintenance
465
+ integrityCheck(): Promise<IntegrityCheckResult>;
466
+ gc(): Promise<GcResult>;
467
+ rebuildIndexes(): Promise<RebuildResult>;
468
+
469
+ // Cache
470
+ invalidate(keys: string[]): Promise<number>;
471
+ invalidateAll(): Promise<number>;
472
+
473
+ // Low-level query (token-based)
474
+ query(kbId: string): Promise<string>;
475
+ getResultStatus(token: string):
476
+ Promise<ResultStatus>;
477
+ getResult<T extends ResultType>(
478
+ token: string,
479
+ type: T,
480
+ ): Promise<ResultData<T> | null>;
481
+ }
482
+ ```
483
+
484
+ ## Type Definitions
485
+
486
+ **Status types:**
487
+
488
+ ```ts
489
+ type DaemonStatus = 'busy' | 'idle';
490
+
491
+ type ConfigKey =
492
+ | 'timeout_ms'
493
+ | 'item_ttl_ms'
494
+ | 'item_limit'
495
+ | 'index_limit';
496
+
497
+ interface DbStatus {
498
+ target_dir: string;
499
+ document_count: number;
500
+ section_count: number;
501
+ index_size_bytes: number;
502
+ last_modified: string;
503
+ cache_stats: {
504
+ items_cached: number;
505
+ hit_rate: number;
506
+ };
507
+ }
508
+ ```
509
+
510
+ **Search types:**
511
+
512
+ `search()` now returns a `PagedSearchResult` envelope.
513
+ Individual `SectionMatch` items include `heading`,
514
+ `type`, `docids`, `snippet`, and `matched_terms`
515
+ alongside `score`. The MCP layer strips `score`; the
516
+ CLI and IPC layers retain it.
517
+
518
+ ```ts
519
+ interface SearchParams {
520
+ query: string;
521
+ mode: 'sections' | 'documents' | 'stats';
522
+ limit?: number;
523
+ offset?: number;
524
+ }
525
+
526
+ interface PagedSearchResult {
527
+ items: SectionMatch[];
528
+ total: number;
529
+ offset: number;
530
+ limit: number;
531
+ has_more: boolean;
532
+ }
533
+
534
+ type SearchResult =
535
+ | SectionSearchResult
536
+ | DocumentSearchResult
537
+ | StatsSearchResult;
538
+
539
+ interface SectionSearchResult {
540
+ type: 'section';
541
+ data: SectionMatch[];
542
+ references: (Section | DocumentResult)[];
543
+ total: number;
544
+ query_time_ms: number;
545
+ query: string;
546
+ }
547
+
548
+ interface DocumentSearchResult {
549
+ type: 'document';
550
+ data: DocumentMatch[];
551
+ references: (Section | DocumentResult)[];
552
+ total: number;
553
+ query_time_ms: number;
554
+ query: string;
555
+ }
556
+
557
+ interface StatsSearchResult {
558
+ total_matches: number;
559
+ query_time_ms: number;
560
+ top_terms: TermStat[];
561
+ query: string;
562
+ }
563
+ ```
564
+
565
+ **Match types:**
566
+
567
+ ```ts
568
+ interface SectionMatch {
569
+ kbid: string;
570
+ docids: string[];
571
+ type: string;
572
+ heading: string | null;
573
+ score: number;
574
+ snippet: string;
575
+ matched_terms: string[];
576
+ }
577
+
578
+ interface DocumentMatch {
579
+ docid: string;
580
+ title: string;
581
+ score: number;
582
+ matching_sections: Array<{
583
+ kbid: string;
584
+ type: string;
585
+ heading: string;
586
+ score: number;
587
+ snippet: string;
588
+ }>;
589
+ }
590
+
591
+ interface TermStat {
592
+ term: string;
593
+ frequency: number;
594
+ sections_matched: number;
595
+ }
596
+ ```
597
+
598
+ **Recall types:**
599
+
600
+ `recall()` returns progressive context expansion for
601
+ one or more sections. Returns `RecallResult` for a
602
+ single kbid or `RecallResult[]` for a batch. The
603
+ `depth` parameter controls how much context is
604
+ included (0--3).
605
+
606
+ ```ts
607
+ interface RecallResult {
608
+ kbid: string;
609
+ heading: string | null;
610
+ type: string;
611
+ content: string;
612
+ docids: string[];
613
+ // Present at depth >= 1:
614
+ documents?: RecallDocument[];
615
+ back_references?: RecallReference[];
616
+ // Present at depth >= 2:
617
+ siblings?: RecallResult[];
618
+ // Present at depth >= 3:
619
+ references?: RecallResult[];
620
+ }
621
+
622
+ interface RecallDocument {
623
+ docid: string;
624
+ title: string;
625
+ sections: Array<{
626
+ kbid: string;
627
+ heading: string | null;
628
+ type: string;
629
+ }>;
630
+ }
631
+
632
+ interface RecallReference {
633
+ kbid: string;
634
+ heading: string | null;
635
+ type: string;
636
+ marker_type: string;
637
+ context_snippet: string;
638
+ direction: 'back';
639
+ }
640
+ ```
641
+
642
+ Maps to the daemon's `recall` IPC method, which
643
+ calls `worker_recall` in the WASM engine.
644
+
645
+ **Section types:**
646
+
647
+ ```ts
648
+ interface Section {
649
+ kbid: string;
650
+ docids: string[];
651
+ type: string;
652
+ title?: string;
653
+ description?: string;
654
+ token_count: number;
655
+ heading_token_count: number;
656
+ body_token_count: number;
657
+ code_token_count: number;
658
+ content: string;
659
+ }
660
+
661
+ interface AddSectionParams {
662
+ content: string;
663
+ type: string;
664
+ title?: string;
665
+ description?: string;
666
+ docid?: string;
667
+ }
668
+
669
+ interface AddSectionResult {
670
+ kbid: string;
671
+ type: string;
672
+ title?: string;
673
+ description?: string;
674
+ token_count: number;
675
+ indexed_terms_count: number;
676
+ elapsed_ms: number;
677
+ attached: boolean;
678
+ }
679
+
680
+ interface UpdateSectionParams {
681
+ kbid: string;
682
+ content: string;
683
+ }
684
+
685
+ interface UpdateSectionResult {
686
+ old_kbid: string;
687
+ new_kbid: string;
688
+ reindexed_terms_count: number;
689
+ elapsed_ms: number;
690
+ }
691
+
692
+ interface RemoveSectionResult {
693
+ removed: boolean;
694
+ elapsed_ms: number;
695
+ }
696
+ ```
697
+
698
+ **Document types:**
699
+
700
+ ```ts
701
+ interface GroupSectionsParams {
702
+ docid: string;
703
+ title: string;
704
+ kbids: string[];
705
+ }
706
+
707
+ interface GroupSectionsResult {
708
+ docid: string;
709
+ section_count: number;
710
+ elapsed_ms: number;
711
+ }
712
+
713
+ interface RemoveGroupingResult {
714
+ removed: boolean;
715
+ sections_ungrouped: number;
716
+ elapsed_ms: number;
717
+ }
718
+
719
+ interface DocumentResult {
720
+ docid: string;
721
+ title: string;
722
+ created_at: string;
723
+ sections: Array<{
724
+ kbid: string;
725
+ docids: string[];
726
+ type: string;
727
+ title?: string;
728
+ description?: string;
729
+ content: string;
730
+ position: number;
731
+ }>;
732
+ }
733
+ ```
734
+
735
+ **Unified result shape:**
736
+
737
+ ```ts
738
+ interface KbdbResult<T> {
739
+ type: 'section' | 'document';
740
+ data: T;
741
+ references: (Section | DocumentResult)[];
742
+ total: number;
743
+ }
744
+ ```
745
+
746
+ `KbdbResult<T>` wraps all retrieval and search
747
+ responses. The `references` array contains unique,
748
+ recursively resolved records for every marker found
749
+ in the returned content (see
750
+ [Document](document.md) for marker types and
751
+ [Content Parser](content-parser.md) for extraction).
752
+ `total` is the full count of matching entities --
753
+ `data.length` may be smaller when `limit` is applied.
754
+
755
+ `SearchResult` (`SectionSearchResult` and
756
+ `DocumentSearchResult`) follows the same shape with
757
+ additional search-specific fields (`query_time_ms`,
758
+ `query`). `StatsSearchResult` does not use the
759
+ unified shape since it returns aggregate statistics,
760
+ not entities.
761
+
762
+ **Content composition result:**
763
+
764
+ ```ts
765
+ interface ContentResult {
766
+ type: 'text/markdown';
767
+ data: string;
768
+ total: number;
769
+ }
770
+ ```
771
+
772
+ `ContentResult` wraps the output of the content
773
+ composition engine. `data` is the composed Markdown
774
+ string. `total` is the character count of `data`
775
+ (not byte count). See
776
+ [Content Composer](content-composer.md) for the full
777
+ composition specification.
778
+
779
+ **Maintenance types:**
780
+
781
+ ```ts
782
+ interface IntegrityCheckResult {
783
+ ok: boolean;
784
+ errors: IntegrityError[];
785
+ sections_checked: number;
786
+ documents_checked: number;
787
+ references_checked: number;
788
+ elapsed_ms: number;
789
+ }
790
+
791
+ interface IntegrityError {
792
+ type:
793
+ | 'checksum_mismatch'
794
+ | 'orphan_section'
795
+ | 'orphan_document'
796
+ | 'broken_reference'
797
+ | 'type_mismatch'
798
+ | 'circular_reference';
799
+ entity: string;
800
+ detail: string;
801
+ }
802
+
803
+ interface GcResult {
804
+ removed_sections: number;
805
+ removed_bytes: number;
806
+ elapsed_ms: number;
807
+ }
808
+
809
+ interface RebuildResult {
810
+ sections_reindexed: number;
811
+ terms_indexed: number;
812
+ elapsed_ms: number;
813
+ }
814
+ ```
815
+
816
+ `IntegrityCheckResult` reports all issues found in
817
+ the database. Error types match
818
+ [Database -- Integrity and Maintenance](database.md).
819
+ `GcResult` reports sections removed and bytes freed.
820
+ `RebuildResult` reports sections and terms
821
+ re-indexed.
822
+
823
+ **Low-level query types:**
824
+
825
+ ```ts
826
+ type ResultType =
827
+ | 'kbid'
828
+ | 'docid'
829
+ | 'section'
830
+ | 'content'
831
+ | 'kbid[]'
832
+ | 'docid[]'
833
+ | 'section[]'
834
+ | 'content[]';
835
+
836
+ interface ResultStatus {
837
+ status: 'busy' | 'success' | 'fail';
838
+ reason: string;
839
+ }
840
+ ```
841
+
842
+ ## Connection Lifecycle
843
+
844
+ **Short-lived (CLI):**
845
+
846
+ ```
847
+ src/cli.ts
848
+
849
+ ├─ parse args, resolve targetDir
850
+ ├─ createWorkerClient({ targetDir })
851
+ │ ├─ find or spawn daemon
852
+ │ └─ connect to IPC socket
853
+ ├─ client.search(...) / client.addSection(...)
854
+ ├─ print result to stdout
855
+ └─ client.disconnect()
856
+ ```
857
+
858
+ The CLI creates the client, makes one or a few calls,
859
+ and disconnects. The daemon stays alive for subsequent
860
+ invocations.
861
+
862
+ **Long-lived (MCP):**
863
+
864
+ ```
865
+ src/cli.ts
866
+
867
+ ├─ createWorkerClient({ targetDir })
868
+ │ ├─ find or spawn daemon
869
+ │ └─ connect to IPC socket
870
+ ├─ listen for MCP requests
871
+ │ ├─ MCP search → client.search(...)
872
+ │ ├─ MCP add → client.addSection(...)
873
+ │ ├─ MCP remove → client.removeSection(...)
874
+ │ └─ ...
875
+ └─ on shutdown: client.disconnect()
876
+ ```
877
+
878
+ The MCP server holds the connection open. All incoming
879
+ MCP requests route through the same `WorkerClient`.
880
+
881
+ ## Reconnection
882
+
883
+ If the IPC connection drops (daemon crashed, idle
884
+ timeout, OS killed the process), the client does not
885
+ throw immediately. On the next method call, it:
886
+
887
+ 1. Detects the broken connection.
888
+ 2. Runs the discovery sequence again (PID file check,
889
+ spawn if needed).
890
+ 3. Reconnects to the new daemon's IPC socket.
891
+ 4. Retries the method call once.
892
+
893
+ If reconnection fails, the method throws. The caller
894
+ handles the error.
895
+
896
+ This makes the MCP server resilient to daemon restarts
897
+ without manual reconnection logic. The CLI benefits
898
+ too -- if a daemon dies between the spawn check and
899
+ the first method call, the client recovers.
900
+
901
+ ## Error Handling
902
+
903
+ All client methods throw on:
904
+
905
+ - **Connection failure** -- Cannot connect to the
906
+ daemon after discovery and spawn attempts.
907
+ - **Spawn failure** -- Daemon did not start within
908
+ the timeout.
909
+ - **IPC error** -- JSON-RPC 2.0 error response from
910
+ the daemon (context mismatch, invalid token, method
911
+ not found).
912
+ - **Reconnection failure** -- Broken connection and
913
+ the retry also failed.
914
+
915
+ Errors are thrown as typed error objects with a `code`
916
+ property matching the JSON-RPC 2.0 error codes defined
917
+ in [Worker Daemon -- Error Responses](
918
+ worker-daemon.md#error-responses):
919
+
920
+ ```ts
921
+ interface WorkerClientError extends Error {
922
+ code: number;
923
+ }
924
+ ```
925
+
926
+ ## Design Constraints
927
+
928
+ - The worker client is the **sole IPC client** for the
929
+ daemon. No other TypeScript module sends messages to
930
+ `src/worker.ts`.
931
+ - The client does not cache data. Caching is the
932
+ daemon's responsibility (via `kb-worker.wasm`).
933
+ - The client does not perform database I/O. All
934
+ storage operations go through the daemon.
935
+ - The client does not perform text processing. That is
936
+ `query-parser.wasm`'s job inside the daemon.
937
+ - The module works on both Deno and Node.js. Platform
938
+ differences (IPC sockets, process spawning, PID
939
+ checks) are abstracted inside internal files.
940
+ - The client's public API mirrors the IPC wire format.
941
+ Types match the JSON shapes defined in the Endpoints
942
+ section of [Database](database.md) and the IPC API
943
+ sections of [Worker Daemon](worker-daemon.md). No
944
+ transformation layer.