@dikolab/kbdb 0.4.3 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLA.md +68 -0
- package/LICENSE +676 -12
- package/LICENSING.md +58 -0
- package/README.md +50 -27
- package/README.md.bak +320 -0
- package/dist/README.md +320 -0
- package/dist/{chunk-BJXEVALU.mjs → chunk-QLJ33C74.mjs} +44 -44
- package/dist/chunk-QLJ33C74.mjs.map +7 -0
- package/dist/cli.cjs +550 -232
- package/dist/cli.cjs.map +4 -4
- package/dist/cli.d.ts +0 -8
- package/dist/cli.mjs +483 -154
- package/dist/cli.mjs.map +4 -4
- package/dist/kbdb-worker.cjs +36 -5
- package/dist/kbdb-worker.cjs.map +3 -3
- package/dist/mod.cjs +89 -30
- package/dist/mod.cjs.map +4 -4
- package/dist/mod.mjs +88 -30
- package/dist/mod.mjs.map +4 -4
- package/dist/src/cli.d.ts +0 -8
- package/dist/src/shared/cli/constants/defaults.constant.d.ts +1 -1
- package/dist/src/shared/cli/constants/recfile-command-field-map.constant.d.ts +8 -0
- package/dist/src/shared/cli/constants/recfile-field-maps.constant.d.ts +21 -0
- package/dist/src/shared/cli/functions/compute-directory-level.function.d.ts +14 -0
- package/dist/src/shared/cli/functions/dispatch-command.function.d.ts +2 -28
- package/dist/src/shared/cli/functions/format-command-output.function.d.ts +14 -0
- package/dist/src/shared/cli/functions/format-db-not-found.function.d.ts +20 -0
- package/dist/src/shared/cli/functions/format-output.function.d.ts +1 -0
- package/dist/src/shared/cli/functions/format-recfile-output.function.d.ts +11 -0
- package/dist/src/shared/cli/functions/render-recfile-value.function.d.ts +11 -0
- package/dist/src/shared/cli/functions/resolve-db-dir.function.d.ts +24 -0
- package/dist/src/shared/cli/functions/resolve-recfile-key.function.d.ts +9 -0
- package/dist/src/shared/cli/functions/run-db-path.function.d.ts +10 -0
- package/dist/src/shared/cli/functions/run-export.function.d.ts +8 -4
- package/dist/src/shared/cli/functions/run-import.function.d.ts +4 -0
- package/dist/src/shared/cli/functions/run-learn.function.d.ts +12 -0
- package/dist/src/shared/cli/index.d.ts +4 -1
- package/dist/src/shared/cli/typings/cli-options.interface.d.ts +9 -2
- package/dist/src/shared/cli/typings/db-resolution.interface.d.ts +9 -0
- package/dist/src/shared/cli/typings/recfile-field-map.type.d.ts +10 -0
- package/dist/src/shared/version/constants/version.constant.d.ts +1 -1
- package/dist/src/shared/wasm-codec/functions/decode-option-u8.function.d.ts +9 -0
- package/dist/src/shared/wasm-codec/functions/decode-section-records.function.d.ts +4 -0
- package/dist/src/shared/wasm-codec/functions/encode-add-section-params.function.d.ts +2 -45
- package/dist/src/shared/wasm-codec/functions/encode-update-section-params.function.d.ts +10 -1
- package/dist/src/shared/wasm-codec/index.d.ts +1 -1
- package/dist/src/shared/wasm-codec/typings/add-section-input.model.d.ts +6 -0
- package/dist/src/shared/wasm-codec/typings/section-record-trailing.model.d.ts +5 -0
- package/dist/src/shared/wasm-codec/typings/section-record.model.d.ts +5 -0
- package/dist/src/shared/worker-client/functions/build-deno-permissions.function.d.ts +11 -0
- package/dist/src/shared/worker-client/functions/read-crash-reason.function.d.ts +9 -0
- package/dist/src/shared/worker-client/functions/resolve-spawn-script.function.d.ts +6 -0
- package/dist/src/shared/worker-client/functions/spawn-daemon.function.d.ts +1 -1
- package/dist/src/shared/worker-client/typings/section.model.d.ts +5 -0
- package/dist/src/shared/worker-client/typings/worker-client-options.interface.d.ts +0 -6
- package/dist/src/shared/worker-daemon/functions/count-files.function.d.ts +12 -0
- package/dist/src/shared/worker-daemon/functions/handle-import.function.d.ts +2 -0
- package/dist/src/shared/worker-daemon/functions/handle-read-ops.function.d.ts +8 -0
- package/dist/src/shared/worker-daemon/functions/validate-daemon-path.function.d.ts +15 -0
- package/dist/src/worker.d.ts +0 -7
- package/dist/wasm/default-embedding/kbdb_default_embedding_bg.wasm +0 -0
- package/dist/wasm/default-embedding/package.json +1 -1
- package/dist/wasm/fs-database/kbdb_fs_database_bg.wasm +0 -0
- package/dist/wasm/fs-database/package.json +1 -1
- package/dist/wasm/fs-migration/kbdb_fs_migration_bg.wasm +0 -0
- package/dist/wasm/fs-migration/package.json +1 -1
- package/dist/wasm/kb-worker/kbdb_worker_bg.wasm +0 -0
- package/dist/wasm/kb-worker/package.json +1 -1
- package/dist/wasm/query-parser/kbdb_query_parser_bg.wasm +0 -0
- package/dist/wasm/query-parser/package.json +1 -1
- package/dist/worker.d.ts +0 -7
- package/dist/worker.mjs +37 -6
- package/dist/worker.mjs.map +3 -3
- package/package.json +13 -12
- package/dist/chunk-BJXEVALU.mjs.map +0 -7
- package/dist/src/shared/cli/functions/resolve-raw-db-path.function.d.ts +0 -10
- package/docs/details/README.md +0 -30
- package/docs/details/agent-tooling.md +0 -132
- package/docs/details/cli.md +0 -777
- package/docs/details/knowledge-base.md +0 -180
- package/docs/details/library-api.md +0 -495
- package/docs/details/mcp-server.md +0 -482
- package/docs/details/search-and-ranking.md +0 -575
- package/docs/details/storage.md +0 -531
- package/docs/goals/agents.md +0 -751
- package/docs/goals/architecture.svg +0 -198
- package/docs/goals/auto-capture.md +0 -378
- package/docs/goals/benchmarking.md +0 -296
- package/docs/goals/cli.md +0 -2654
- package/docs/goals/concurrency.md +0 -259
- package/docs/goals/content-composer.md +0 -609
- package/docs/goals/content-parser.md +0 -279
- package/docs/goals/database.md +0 -1679
- package/docs/goals/document.md +0 -368
- package/docs/goals/hybrid-search.md +0 -465
- package/docs/goals/mcp.md +0 -1541
- package/docs/goals/overview.md +0 -124
- package/docs/goals/query-parser.md +0 -373
- package/docs/goals/query-result.md +0 -860
- package/docs/goals/semantic-dedup.md +0 -233
- package/docs/goals/skills.md +0 -810
- package/docs/goals/sync.md +0 -217
- package/docs/goals/worker-client.md +0 -1005
- package/docs/goals/worker-daemon.md +0 -1547
- package/docs/modules/overview.md +0 -319
- package/docs/modules/rust/embedding/module.md +0 -91
- package/docs/modules/rust/fs-database/functions/document.md +0 -48
- package/docs/modules/rust/fs-database/functions/integrity.md +0 -79
- package/docs/modules/rust/fs-database/functions/io.md +0 -87
- package/docs/modules/rust/fs-database/functions/module.md +0 -22
- package/docs/modules/rust/fs-database/functions/query-exec.md +0 -44
- package/docs/modules/rust/fs-database/functions/section.md +0 -76
- package/docs/modules/rust/fs-database/indexes/btree.md +0 -35
- package/docs/modules/rust/fs-database/indexes/corpus.md +0 -61
- package/docs/modules/rust/fs-database/indexes/inverted.md +0 -60
- package/docs/modules/rust/fs-database/indexes/module.md +0 -64
- package/docs/modules/rust/fs-database/indexes/positional.md +0 -39
- package/docs/modules/rust/fs-database/indexes/vectors.md +0 -110
- package/docs/modules/rust/fs-database/module.md +0 -150
- package/docs/modules/rust/fs-database/types/catalog.md +0 -53
- package/docs/modules/rust/fs-database/types/module.md +0 -12
- package/docs/modules/rust/fs-database/types/section.md +0 -127
- package/docs/modules/rust/kb-worker/functions/compose.md +0 -57
- package/docs/modules/rust/kb-worker/functions/module.md +0 -20
- package/docs/modules/rust/kb-worker/functions/retrieve.md +0 -194
- package/docs/modules/rust/kb-worker/functions/search.md +0 -173
- package/docs/modules/rust/kb-worker/functions/write.md +0 -172
- package/docs/modules/rust/kb-worker/module.md +0 -171
- package/docs/modules/rust/kb-worker/types/cache.md +0 -90
- package/docs/modules/rust/kb-worker/types/context.md +0 -26
- package/docs/modules/rust/kb-worker/types/module.md +0 -16
- package/docs/modules/rust/kb-worker/types/status.md +0 -29
- package/docs/modules/rust/kb-worker/types/token.md +0 -50
- package/docs/modules/rust/overview.md +0 -531
- package/docs/modules/rust/query-parser/functions/extract.md +0 -56
- package/docs/modules/rust/query-parser/functions/module.md +0 -18
- package/docs/modules/rust/query-parser/functions/parse.md +0 -33
- package/docs/modules/rust/query-parser/functions/pipeline.md +0 -9
- package/docs/modules/rust/query-parser/functions/rank.md +0 -85
- package/docs/modules/rust/query-parser/module.md +0 -131
- package/docs/modules/rust/query-parser/types/cache.md +0 -39
- package/docs/modules/rust/query-parser/types/keyphrase.md +0 -71
- package/docs/modules/rust/query-parser/types/module.md +0 -18
- package/docs/modules/rust/query-parser/types/token.md +0 -46
- package/docs/modules/rust/shared/content-composer/functions.md +0 -114
- package/docs/modules/rust/shared/content-composer/module.md +0 -147
- package/docs/modules/rust/shared/content-composer/types.md +0 -93
- package/docs/modules/rust/shared/content-parser/functions.md +0 -72
- package/docs/modules/rust/shared/content-parser/module.md +0 -99
- package/docs/modules/rust/shared/content-parser/types.md +0 -71
- package/docs/modules/rust/shared/corpus/module.md +0 -24
- package/docs/modules/rust/shared/corpus/types.md +0 -141
- package/docs/modules/rust/shared/document/module.md +0 -25
- package/docs/modules/rust/shared/document/types.md +0 -154
- package/docs/modules/rust/shared/encode/module.md +0 -22
- package/docs/modules/rust/shared/encode/traits.md +0 -304
- package/docs/modules/rust/shared/error/module.md +0 -28
- package/docs/modules/rust/shared/error/types.md +0 -302
- package/docs/modules/rust/shared/kbid/module.md +0 -24
- package/docs/modules/rust/shared/kbid/types.md +0 -147
- package/docs/modules/rust/shared/memory/functions.md +0 -209
- package/docs/modules/rust/shared/memory/module.md +0 -24
- package/docs/modules/rust/shared/module.md +0 -196
- package/docs/modules/rust/shared/pipeline/functions.md +0 -141
- package/docs/modules/rust/shared/pipeline/module.md +0 -62
- package/docs/modules/rust/shared/pipeline/types.md +0 -43
- package/docs/modules/rust/shared/query/module.md +0 -27
- package/docs/modules/rust/shared/query/types.md +0 -236
- package/docs/modules/rust/shared/section/module.md +0 -25
- package/docs/modules/rust/shared/section/types.md +0 -294
- package/docs/modules/rust/shared/term/module.md +0 -25
- package/docs/modules/rust/shared/term/types.md +0 -139
- package/docs/modules/typescript/cli.md +0 -131
- package/docs/modules/typescript/kbdb-worker.md +0 -150
- package/docs/modules/typescript/overview.md +0 -400
- package/docs/modules/typescript/shared/auto-capture/module.md +0 -73
- package/docs/modules/typescript/shared/cli/constants.md +0 -23
- package/docs/modules/typescript/shared/cli/functions.md +0 -809
- package/docs/modules/typescript/shared/cli/module.md +0 -138
- package/docs/modules/typescript/shared/cli/typings.md +0 -197
- package/docs/modules/typescript/shared/embedding/functions.md +0 -200
- package/docs/modules/typescript/shared/embedding/module.md +0 -77
- package/docs/modules/typescript/shared/embedding/typings.md +0 -124
- package/docs/modules/typescript/shared/hash/functions.md +0 -20
- package/docs/modules/typescript/shared/hash/module.md +0 -21
- package/docs/modules/typescript/shared/log/constants.md +0 -82
- package/docs/modules/typescript/shared/log/functions.md +0 -131
- package/docs/modules/typescript/shared/log/module.md +0 -153
- package/docs/modules/typescript/shared/log/typings.md +0 -82
- package/docs/modules/typescript/shared/mcp/classes.md +0 -179
- package/docs/modules/typescript/shared/mcp/functions.md +0 -386
- package/docs/modules/typescript/shared/mcp/module.md +0 -160
- package/docs/modules/typescript/shared/mcp/typings.md +0 -529
- package/docs/modules/typescript/shared/module.md +0 -110
- package/docs/modules/typescript/shared/platform/functions.md +0 -42
- package/docs/modules/typescript/shared/platform/module.md +0 -25
- package/docs/modules/typescript/shared/runtime/functions.md +0 -26
- package/docs/modules/typescript/shared/runtime/module.md +0 -27
- package/docs/modules/typescript/shared/runtime/typings.md +0 -13
- package/docs/modules/typescript/shared/version/module.md +0 -27
- package/docs/modules/typescript/shared/wasm-codec/functions.md +0 -391
- package/docs/modules/typescript/shared/wasm-codec/module.md +0 -98
- package/docs/modules/typescript/shared/worker-client/classes.md +0 -264
- package/docs/modules/typescript/shared/worker-client/functions.md +0 -175
- package/docs/modules/typescript/shared/worker-client/module.md +0 -92
- package/docs/modules/typescript/shared/worker-client/typings.md +0 -511
- package/docs/modules/typescript/shared/worker-daemon/classes.md +0 -239
- package/docs/modules/typescript/shared/worker-daemon/constants.md +0 -37
- package/docs/modules/typescript/shared/worker-daemon/functions.md +0 -234
- package/docs/modules/typescript/shared/worker-daemon/module.md +0 -118
- package/docs/modules/typescript/shared/worker-daemon/typings.md +0 -132
- package/docs/overview.md +0 -191
- package/docs/release-notes/0.1.0.md +0 -189
- package/docs/release-notes/0.1.1.md +0 -46
- package/docs/release-notes/0.1.2.md +0 -38
- package/docs/release-notes/0.1.3.md +0 -42
- package/docs/release-notes/0.2.0.md +0 -147
- package/docs/release-notes/0.3.0.md +0 -89
- package/docs/release-notes/0.3.1.md +0 -121
- package/docs/release-notes/0.3.2.md +0 -64
- package/docs/release-notes/0.4.0.md +0 -80
- package/docs/release-notes/0.4.1.md +0 -149
- package/docs/release-notes/0.4.3.md +0 -60
- package/docs/release-notes/README.md +0 -13
|
@@ -1,1005 +0,0 @@
|
|
|
1
|
-
# Worker Client
|
|
2
|
-
|
|
3
|
-
*A shared TypeScript module that discovers or spawns
|
|
4
|
-
the worker daemon and provides a typed API for all
|
|
5
|
-
knowledge base operations over IPC.*
|
|
6
|
-
|
|
7
|
-
## Goal
|
|
8
|
-
|
|
9
|
-
`src/cli.ts` needs to communicate with the worker
|
|
10
|
-
daemon (`src/worker.ts`) to perform database
|
|
11
|
-
operations. The worker client module is the single
|
|
12
|
-
shared entry point for that communication. It
|
|
13
|
-
handles:
|
|
14
|
-
|
|
15
|
-
- Resolving the database directory path.
|
|
16
|
-
- Computing the context ID.
|
|
17
|
-
- Discovering an existing daemon or spawning a new one.
|
|
18
|
-
- Connecting to the daemon's IPC socket.
|
|
19
|
-
- Exposing every daemon IPC method as a typed async
|
|
20
|
-
function.
|
|
21
|
-
- Managing connection lifecycle (connect, reconnect,
|
|
22
|
-
disconnect).
|
|
23
|
-
|
|
24
|
-
No other module in the TypeScript layer communicates
|
|
25
|
-
with the daemon directly. `src/cli.ts` imports the
|
|
26
|
-
worker client and calls its methods -- it never
|
|
27
|
-
constructs IPC messages, manages PID files, or
|
|
28
|
-
spawns processes itself.
|
|
29
|
-
|
|
30
|
-
```
|
|
31
|
-
src/cli.ts ──→ worker-client ──→ src/worker.ts
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Module Location
|
|
35
|
-
|
|
36
|
-
```
|
|
37
|
-
src/shared/worker-client/
|
|
38
|
-
├── mod.ts # public API: createWorkerClient
|
|
39
|
-
├── resolve.ts # database directory resolution
|
|
40
|
-
├── context.ts # context ID computation
|
|
41
|
-
├── discovery.ts # PID file lookup, process check
|
|
42
|
-
├── spawn.ts # daemon spawning (Node + Deno)
|
|
43
|
-
├── connection.ts # IPC socket connect / reconnect
|
|
44
|
-
└── types.ts # request/response type defs
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
The module is internal to the package. It is not
|
|
48
|
-
exported from the library's public API (`mod.ts` at
|
|
49
|
-
the project root). Only `src/cli.ts` imports it.
|
|
50
|
-
|
|
51
|
-
## When the Daemon Is Needed
|
|
52
|
-
|
|
53
|
-
The worker client is only created when the caller needs
|
|
54
|
-
to perform operations that involve the `.kbdb` database
|
|
55
|
-
directory:
|
|
56
|
-
|
|
57
|
-
- **Search and query** -- Looking up indexes, ranking
|
|
58
|
-
results, retrieving sections or documents.
|
|
59
|
-
- **Content management** -- `kbdb learn` (import,
|
|
60
|
-
index) and `kbdb unlearn` (remove, re-index).
|
|
61
|
-
- **Status and retrieval** -- Database metadata,
|
|
62
|
-
content retrieval by ID.
|
|
63
|
-
|
|
64
|
-
Operations that do not touch the database do not create
|
|
65
|
-
a worker client and do not require a daemon:
|
|
66
|
-
|
|
67
|
-
- `kbdb --help`, `kbdb --version`
|
|
68
|
-
- Invalid commands or argument validation errors
|
|
69
|
-
- Any subcommand that exits before reaching the
|
|
70
|
-
database
|
|
71
|
-
|
|
72
|
-
## Database Directory Resolution
|
|
73
|
-
|
|
74
|
-
The worker client resolves the database directory path
|
|
75
|
-
before connecting to the daemon:
|
|
76
|
-
|
|
77
|
-
1. If the caller provides a `targetDir` (e.g. from a
|
|
78
|
-
CLI `--db <path>` flag), use it.
|
|
79
|
-
2. Otherwise, use the current working directory.
|
|
80
|
-
|
|
81
|
-
The database directory is always
|
|
82
|
-
`${targetDir}/.kbdb`. The worker client checks that
|
|
83
|
-
this directory exists before proceeding. If `.kbdb`
|
|
84
|
-
does not exist, the factory throws -- the caller is
|
|
85
|
-
responsible for database initialization (e.g. the CLI
|
|
86
|
-
prompts the user to create it).
|
|
87
|
-
|
|
88
|
-
## Context ID
|
|
89
|
-
|
|
90
|
-
The worker client computes the context ID during
|
|
91
|
-
`createWorkerClient()` and includes it in every IPC
|
|
92
|
-
request. The context ID is a **SHA-256 hash of the
|
|
93
|
-
database directory's absolute path**, truncated to 16
|
|
94
|
-
lowercase hexadecimal characters.
|
|
95
|
-
|
|
96
|
-
```
|
|
97
|
-
SHA-256(absolutePath) → truncate to 16 hex chars → ctx
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
See [Worker Daemon -- Context ID](
|
|
101
|
-
worker-daemon.md#context-id) for the full
|
|
102
|
-
specification, including how the daemon validates the
|
|
103
|
-
context ID on every request.
|
|
104
|
-
|
|
105
|
-
## Daemon Discovery
|
|
106
|
-
|
|
107
|
-
When the worker client is created, it follows this
|
|
108
|
-
discovery sequence:
|
|
109
|
-
|
|
110
|
-
1. Compute the context ID from the database directory
|
|
111
|
-
path.
|
|
112
|
-
2. Look for `kb-worker-{ctx}.pid` in the OS temporary
|
|
113
|
-
directory (e.g., `/tmp` on Linux/macOS, `%TEMP%` on
|
|
114
|
-
Windows).
|
|
115
|
-
3. If the PID file exists, read the process ID and
|
|
116
|
-
check whether the process is alive (see
|
|
117
|
-
Platform-Specific PID Handling below).
|
|
118
|
-
4. If the process is alive, connect to the IPC socket
|
|
119
|
-
at `kb-worker-{ctx}.sock` (or the named pipe on
|
|
120
|
-
Windows).
|
|
121
|
-
5. If the PID file is missing or the process is dead
|
|
122
|
-
(stale PID file), remove the stale file and spawn
|
|
123
|
-
a new daemon (see Spawning below).
|
|
124
|
-
|
|
125
|
-
The PID file contains only the process ID number as
|
|
126
|
-
plain text (e.g., `12345`). No other data.
|
|
127
|
-
|
|
128
|
-
### Platform-Specific PID Handling
|
|
129
|
-
|
|
130
|
-
Both Linux/macOS and Windows use integer process IDs.
|
|
131
|
-
The PID file format is identical across platforms -- a
|
|
132
|
-
single integer as plain text. The differences are in
|
|
133
|
-
how the worker client resolves the temporary directory
|
|
134
|
-
and checks whether the process is alive.
|
|
135
|
-
|
|
136
|
-
**Temporary directory:**
|
|
137
|
-
|
|
138
|
-
| Platform | Resolution |
|
|
139
|
-
|---------------|---------------------------------------|
|
|
140
|
-
| Linux / macOS | `$TMPDIR` if set, otherwise `/tmp` |
|
|
141
|
-
| Windows | `%TEMP%` (typically `C:\Users\{user}\AppData\Local\Temp`) |
|
|
142
|
-
|
|
143
|
-
In Node.js, `os.tmpdir()` returns the correct path on
|
|
144
|
-
all platforms. In Deno, `Deno.env.get("TMPDIR")` (Unix)
|
|
145
|
-
or `Deno.env.get("TEMP")` (Windows) provides the same.
|
|
146
|
-
|
|
147
|
-
**Checking if a process is alive:**
|
|
148
|
-
|
|
149
|
-
The worker client must verify that the PID read from
|
|
150
|
-
the file refers to a running daemon, not a dead process
|
|
151
|
-
whose PID has been recycled by the OS.
|
|
152
|
-
|
|
153
|
-
**Linux / macOS** -- Use a zero signal via `kill()`:
|
|
154
|
-
|
|
155
|
-
```ts
|
|
156
|
-
// Node.js
|
|
157
|
-
try {
|
|
158
|
-
process.kill(pid, 0); // signal 0 = no signal sent
|
|
159
|
-
// process exists
|
|
160
|
-
} catch {
|
|
161
|
-
// process does not exist (ESRCH) or
|
|
162
|
-
// not permitted (EPERM -- process exists
|
|
163
|
-
// but is owned by another user)
|
|
164
|
-
}
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
`kill(pid, 0)` sends no actual signal. It returns
|
|
168
|
-
successfully if the process exists and the caller has
|
|
169
|
-
permission to signal it. It throws `ESRCH` if the
|
|
170
|
-
process does not exist. An `EPERM` error means the
|
|
171
|
-
process exists but belongs to another user -- this
|
|
172
|
-
should be treated as "alive" since the PID is in use.
|
|
173
|
-
|
|
174
|
-
**Windows** -- `process.kill(pid, 0)` is supported in
|
|
175
|
-
Node.js on Windows and behaves the same way: it checks
|
|
176
|
-
process existence without sending a signal. This is the
|
|
177
|
-
preferred cross-platform method.
|
|
178
|
-
|
|
179
|
-
For Deno on Windows, where `Deno.kill()` does not
|
|
180
|
-
support signal 0, use a subprocess check:
|
|
181
|
-
|
|
182
|
-
```ts
|
|
183
|
-
// Deno (Windows fallback)
|
|
184
|
-
const cmd = new Deno.Command('tasklist', {
|
|
185
|
-
args: ['/FI', `PID eq ${pid}`, '/NH', '/FO', 'CSV'],
|
|
186
|
-
stdout: 'piped',
|
|
187
|
-
stderr: 'null',
|
|
188
|
-
});
|
|
189
|
-
const { stdout } = await cmd.output();
|
|
190
|
-
const output = new TextDecoder().decode(stdout);
|
|
191
|
-
const alive = output.includes(`"${pid}"`);
|
|
192
|
-
```
|
|
193
|
-
|
|
194
|
-
`tasklist /FI "PID eq {pid}"` filters the process list
|
|
195
|
-
by PID. If the output contains the PID, the process is
|
|
196
|
-
alive. The `/NH` flag suppresses the header and `/FO
|
|
197
|
-
CSV` outputs in CSV format for reliable parsing.
|
|
198
|
-
|
|
199
|
-
**Cross-platform wrapper:**
|
|
200
|
-
|
|
201
|
-
The worker client implements a single
|
|
202
|
-
`isProcessAlive(pid)` function that selects the correct
|
|
203
|
-
method based on `process.platform` (Node.js) or
|
|
204
|
-
`Deno.build.os` (Deno):
|
|
205
|
-
|
|
206
|
-
| Runtime | Linux / macOS | Windows |
|
|
207
|
-
|-----------|------------------------|--------------------------|
|
|
208
|
-
| Node.js | `process.kill(pid, 0)` | `process.kill(pid, 0)` |
|
|
209
|
-
| Deno | `Deno.kill(pid, "SIGCONT")` | `tasklist` subprocess |
|
|
210
|
-
|
|
211
|
-
This function returns `true` if the process is alive,
|
|
212
|
-
`false` if it is not. The daemon discovery sequence
|
|
213
|
-
calls it after reading the PID file to decide whether
|
|
214
|
-
to connect or spawn.
|
|
215
|
-
|
|
216
|
-
## Spawning
|
|
217
|
-
|
|
218
|
-
The worker client spawns the daemon as a **detached
|
|
219
|
-
child process** so it outlives the parent. Unlike a
|
|
220
|
-
typical fire-and-forget detach, the worker client
|
|
221
|
-
**captures the daemon's stderr** during the spawn
|
|
222
|
-
window so that startup failures can be relayed to the
|
|
223
|
-
caller.
|
|
224
|
-
|
|
225
|
-
**Node.js:**
|
|
226
|
-
|
|
227
|
-
```js
|
|
228
|
-
const child = spawn(
|
|
229
|
-
'node',
|
|
230
|
-
['kb-worker.js', contextPath],
|
|
231
|
-
{
|
|
232
|
-
detached: true,
|
|
233
|
-
stdio: ['ignore', 'ignore', 'pipe'],
|
|
234
|
-
},
|
|
235
|
-
);
|
|
236
|
-
```
|
|
237
|
-
|
|
238
|
-
**Deno:**
|
|
239
|
-
|
|
240
|
-
```ts
|
|
241
|
-
const cmd = new Deno.Command('deno', {
|
|
242
|
-
args: [
|
|
243
|
-
'run', '--allow-all',
|
|
244
|
-
'src/worker.ts', contextPath,
|
|
245
|
-
],
|
|
246
|
-
stdin: 'null',
|
|
247
|
-
stdout: 'null',
|
|
248
|
-
stderr: 'piped',
|
|
249
|
-
});
|
|
250
|
-
const child = cmd.spawn();
|
|
251
|
-
```
|
|
252
|
-
|
|
253
|
-
Note: `stderr` is `'pipe'` / `'piped'`, not
|
|
254
|
-
`'ignore'`. The worker client reads from it during
|
|
255
|
-
the spawn window. Once the PID file appears
|
|
256
|
-
(indicating successful startup), the worker client
|
|
257
|
-
closes the stderr pipe and unrefs the child so the
|
|
258
|
-
daemon can outlive the parent.
|
|
259
|
-
|
|
260
|
-
The spawning sequence:
|
|
261
|
-
|
|
262
|
-
1. Worker client ensures `worker.toml` exists in the
|
|
263
|
-
context directory (writes defaults if missing).
|
|
264
|
-
2. Worker client spawns `src/worker.ts` (or its
|
|
265
|
-
compiled `.js` equivalent) with the context path as
|
|
266
|
-
the sole command-line argument. stderr is piped to
|
|
267
|
-
the worker client.
|
|
268
|
-
3. Worker client polls for the PID file to appear (with
|
|
269
|
-
a short timeout, default 10 000 ms) to confirm the
|
|
270
|
-
daemon started.
|
|
271
|
-
4. **On PID file found (success):** worker client
|
|
272
|
-
closes the stderr pipe, detaches the child, unrefs
|
|
273
|
-
it, and connects to the IPC socket. Because the
|
|
274
|
-
daemon writes the PID file only after the socket
|
|
275
|
-
is listening, the client can connect immediately.
|
|
276
|
-
5. **On timeout or child exit (failure):** worker
|
|
277
|
-
client reads all buffered stderr from the daemon,
|
|
278
|
-
includes it in the thrown error (see Startup
|
|
279
|
-
Failure Handling below).
|
|
280
|
-
|
|
281
|
-
There is no fallback to in-process WASM loading -- the
|
|
282
|
-
daemon is required for all database operations.
|
|
283
|
-
|
|
284
|
-
### Startup Failure Handling
|
|
285
|
-
|
|
286
|
-
When the daemon fails to start -- either because a
|
|
287
|
-
startup precondition failed (see
|
|
288
|
-
[Worker Daemon -- Startup Preconditions](
|
|
289
|
-
worker-daemon.md#startup-preconditions)) or the
|
|
290
|
-
process exited unexpectedly -- the worker client
|
|
291
|
-
detects the failure by one of:
|
|
292
|
-
|
|
293
|
-
- The child process exits before the PID file appears.
|
|
294
|
-
- The PID file poll times out.
|
|
295
|
-
|
|
296
|
-
In either case, the worker client:
|
|
297
|
-
|
|
298
|
-
1. Reads the daemon's buffered stderr output.
|
|
299
|
-
2. Throws a `WorkerClientError` that includes the
|
|
300
|
-
daemon's stderr message in the error's `message`
|
|
301
|
-
property.
|
|
302
|
-
|
|
303
|
-
```ts
|
|
304
|
-
interface WorkerClientError extends Error {
|
|
305
|
-
code: number;
|
|
306
|
-
/** Daemon stderr output, if available. */
|
|
307
|
-
daemonStderr?: string;
|
|
308
|
-
}
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
The error code is `-32003` (daemon unavailable).
|
|
312
|
-
|
|
313
|
-
The caller (CLI or MCP) is responsible for printing
|
|
314
|
-
the error. In practice, the CLI prints the error to
|
|
315
|
-
its own stderr and exits non-zero. This means the
|
|
316
|
-
daemon's error message flows through to the user's
|
|
317
|
-
terminal:
|
|
318
|
-
|
|
319
|
-
```
|
|
320
|
-
kbdb search "query"
|
|
321
|
-
-> worker client spawns daemon
|
|
322
|
-
-> daemon checks preconditions
|
|
323
|
-
-> daemon prints to its stderr:
|
|
324
|
-
"error: database version 3 is newer than kbdb
|
|
325
|
-
version 2 -- upgrade kbdb to open this database"
|
|
326
|
-
-> daemon exits with code 1
|
|
327
|
-
-> worker client reads daemon stderr
|
|
328
|
-
-> worker client throws WorkerClientError with
|
|
329
|
-
daemonStderr containing the message
|
|
330
|
-
-> src/cli.ts catches error, prints to stderr:
|
|
331
|
-
"error: daemon failed to start: database version
|
|
332
|
-
3 is newer than kbdb version 2 -- upgrade kbdb
|
|
333
|
-
to open this database"
|
|
334
|
-
-> src/cli.ts sets process.exitCode = 1
|
|
335
|
-
```
|
|
336
|
-
|
|
337
|
-
This relay ensures the user sees the daemon's specific
|
|
338
|
-
error reason even though the daemon runs as a separate
|
|
339
|
-
detached process. The daemon's stderr is the single
|
|
340
|
-
source of truth for startup failure diagnostics.
|
|
341
|
-
|
|
342
|
-
## Connection Retry
|
|
343
|
-
|
|
344
|
-
When the worker client calls `connect()`, transient
|
|
345
|
-
errors (`ENOENT` -- socket not yet created, or
|
|
346
|
-
`ECONNREFUSED` -- server not yet accepting) are
|
|
347
|
-
retried with exponential backoff:
|
|
348
|
-
|
|
349
|
-
- **Max retries**: 5
|
|
350
|
-
- **Initial delay**: 50 ms
|
|
351
|
-
- **Backoff**: doubles each attempt (50, 100, 200,
|
|
352
|
-
400, 800 ms = ~1.55 s worst case)
|
|
353
|
-
- **Non-transient errors** (`EACCES`, `EINVAL`, etc.)
|
|
354
|
-
fail immediately without retry.
|
|
355
|
-
|
|
356
|
-
This covers the narrow window where the PID file
|
|
357
|
-
exists but the socket has not yet finished binding
|
|
358
|
-
(e.g. under heavy load or slow I/O).
|
|
359
|
-
|
|
360
|
-
## Post-Connect Failure Handling
|
|
361
|
-
|
|
362
|
-
After a successful `connect()`, if the daemon crashes
|
|
363
|
-
or the IPC socket dies:
|
|
364
|
-
|
|
365
|
-
1. All pending RPC calls (in `this.pending`) are
|
|
366
|
-
rejected with a descriptive error.
|
|
367
|
-
2. The client marks itself as disconnected
|
|
368
|
-
(`this.socket = null`).
|
|
369
|
-
3. The socket is destroyed.
|
|
370
|
-
|
|
371
|
-
The `isConnected` getter exposes the connection
|
|
372
|
-
state so callers (and the cached client factory) can
|
|
373
|
-
check before reusing a client.
|
|
374
|
-
|
|
375
|
-
## Cached Client Factory
|
|
376
|
-
|
|
377
|
-
`getOrCreateClient(options?)` returns a cached
|
|
378
|
-
`WorkerClient` for the given context path, or creates
|
|
379
|
-
and caches a new one if the cached client is
|
|
380
|
-
disconnected or absent.
|
|
381
|
-
|
|
382
|
-
```ts
|
|
383
|
-
import { getOrCreateClient } from '@dikolab/kbdb';
|
|
384
|
-
|
|
385
|
-
const client = await getOrCreateClient({
|
|
386
|
-
dbPath: '/path/to/project',
|
|
387
|
-
});
|
|
388
|
-
```
|
|
389
|
-
|
|
390
|
-
Intended for library consumers and the MCP server.
|
|
391
|
-
CLI one-shots should use `createWorkerClient()`
|
|
392
|
-
directly (each is a separate process anyway).
|
|
393
|
-
|
|
394
|
-
## Socket Keepalive
|
|
395
|
-
|
|
396
|
-
Both the client and server enable `SO_KEEPALIVE`
|
|
397
|
-
with a 30-second interval. This ensures both ends
|
|
398
|
-
detect a dead peer within ~30 seconds, even if no
|
|
399
|
-
application-level traffic is flowing.
|
|
400
|
-
|
|
401
|
-
## `createWorkerClient(options?)`
|
|
402
|
-
|
|
403
|
-
Factory function. Returns a connected `WorkerClient`
|
|
404
|
-
instance.
|
|
405
|
-
|
|
406
|
-
```ts
|
|
407
|
-
interface WorkerClientOptions {
|
|
408
|
-
targetDir?: string;
|
|
409
|
-
}
|
|
410
|
-
|
|
411
|
-
function createWorkerClient(
|
|
412
|
-
options?: WorkerClientOptions,
|
|
413
|
-
): Promise<WorkerClient>;
|
|
414
|
-
```
|
|
415
|
-
|
|
416
|
-
**Resolution order for `targetDir`:**
|
|
417
|
-
|
|
418
|
-
1. If `options.targetDir` is provided, use it.
|
|
419
|
-
2. Otherwise, use the current working directory.
|
|
420
|
-
|
|
421
|
-
The factory then:
|
|
422
|
-
|
|
423
|
-
1. Resolves the absolute path: `${targetDir}/.kbdb`.
|
|
424
|
-
2. Computes the context ID (SHA-256 of the absolute
|
|
425
|
-
path, truncated to 16 lowercase hex characters).
|
|
426
|
-
3. Checks whether `.kbdb` exists at the resolved path.
|
|
427
|
-
If not, throws -- the caller is responsible for
|
|
428
|
-
prompting the user or initializing the database
|
|
429
|
-
before creating a client.
|
|
430
|
-
4. Looks for `kb-worker-{ctx}.pid` in the OS temp
|
|
431
|
-
directory and checks whether the daemon is alive
|
|
432
|
-
(see Daemon Discovery above).
|
|
433
|
-
5. If no daemon is running, spawns one (see Spawning
|
|
434
|
-
above).
|
|
435
|
-
6. Connects to the IPC socket
|
|
436
|
-
(`kb-worker-{ctx}.sock` on Unix,
|
|
437
|
-
`\\.\pipe\kb-worker-{ctx}` on Windows).
|
|
438
|
-
7. Returns the connected `WorkerClient`.
|
|
439
|
-
|
|
440
|
-
If the daemon fails to start or the IPC connection
|
|
441
|
-
cannot be established, the factory throws.
|
|
442
|
-
|
|
443
|
-
## `WorkerClient`
|
|
444
|
-
|
|
445
|
-
All methods are async. High-level methods (`search`,
|
|
446
|
-
`readSections`, `addSection`, `updateSection`,
|
|
447
|
-
`removeSection`, `groupSections`, `removeGrouping`,
|
|
448
|
-
`retrieveDocument`, `content`) are composed from
|
|
449
|
-
the low-level IPC primitives (`query` →
|
|
450
|
-
`get_result_status` → `get_result`) documented in
|
|
451
|
-
[Worker Daemon -- IPC API](worker-daemon.md). The
|
|
452
|
-
daemon routes these queries to `kb-worker.wasm`,
|
|
453
|
-
which orchestrates `fs-database.wasm` and
|
|
454
|
-
`query-parser.wasm`.
|
|
455
|
-
|
|
456
|
-
`content()` takes one or more kbid or docid
|
|
457
|
-
identifiers (mixed freely) and returns a composed
|
|
458
|
-
Markdown document. It delegates to the daemon's
|
|
459
|
-
`compose` IPC method, which triggers `kb-worker.wasm`
|
|
460
|
-
to resolve identifiers, fetch sections, extract
|
|
461
|
-
markers recursively, and call the `content-composer`
|
|
462
|
-
module in `kbdb-shared`. See
|
|
463
|
-
[Content Composer](content-composer.md) for the full
|
|
464
|
-
composition specification.
|
|
465
|
-
|
|
466
|
-
Maintenance methods (`integrityCheck`, `gc`,
|
|
467
|
-
`rebuildIndexes`) delegate to the corresponding
|
|
468
|
-
daemon IPC methods (`integrity_check`, `gc`,
|
|
469
|
-
`rebuild_indexes`). These return results directly
|
|
470
|
-
(not via the token pattern) because maintenance
|
|
471
|
-
operations are infrequent and their results are not
|
|
472
|
-
cached.
|
|
473
|
-
|
|
474
|
-
`dbStatus()` returns data from the initialization
|
|
475
|
-
response cached at connect time -- it does not
|
|
476
|
-
issue a separate IPC call.
|
|
477
|
-
|
|
478
|
-
```ts
|
|
479
|
-
interface WorkerClient {
|
|
480
|
-
readonly ctx: string;
|
|
481
|
-
readonly targetDir: string;
|
|
482
|
-
|
|
483
|
-
// Lifecycle
|
|
484
|
-
disconnect(): Promise<void>;
|
|
485
|
-
|
|
486
|
-
// Status
|
|
487
|
-
status(): Promise<DaemonStatus>;
|
|
488
|
-
idleAt(): Promise<number>;
|
|
489
|
-
config(key: ConfigKey): Promise<number>;
|
|
490
|
-
dbStatus(): Promise<DbStatus>;
|
|
491
|
-
|
|
492
|
-
// Search
|
|
493
|
-
search(params: SearchParams):
|
|
494
|
-
Promise<PagedSearchResult>;
|
|
495
|
-
|
|
496
|
-
// Context expansion
|
|
497
|
-
recall(params: {
|
|
498
|
-
kbid?: string;
|
|
499
|
-
kbids?: string[];
|
|
500
|
-
depth?: number;
|
|
501
|
-
}): Promise<RecallResult | RecallResult[]>;
|
|
502
|
-
|
|
503
|
-
// Sections
|
|
504
|
-
readSections(kbIds: string[]):
|
|
505
|
-
Promise<KbdbResult<Section[]>>;
|
|
506
|
-
addSection(params: AddSectionParams):
|
|
507
|
-
Promise<AddSectionResult>;
|
|
508
|
-
updateSection(params: UpdateSectionParams):
|
|
509
|
-
Promise<UpdateSectionResult>;
|
|
510
|
-
removeSection(kbId: string):
|
|
511
|
-
Promise<RemoveSectionResult>;
|
|
512
|
-
|
|
513
|
-
// Documents
|
|
514
|
-
groupSections(params: GroupSectionsParams):
|
|
515
|
-
Promise<GroupSectionsResult>;
|
|
516
|
-
removeGrouping(docid: string):
|
|
517
|
-
Promise<RemoveGroupingResult>;
|
|
518
|
-
retrieveDocument(docid: string):
|
|
519
|
-
Promise<KbdbResult<DocumentResult>>;
|
|
520
|
-
|
|
521
|
-
// Content composition
|
|
522
|
-
content(ids: string[]):
|
|
523
|
-
Promise<ContentResult>;
|
|
524
|
-
|
|
525
|
-
// Maintenance
|
|
526
|
-
integrityCheck(): Promise<IntegrityCheckResult>;
|
|
527
|
-
gc(): Promise<GcResult>;
|
|
528
|
-
rebuildIndexes(): Promise<RebuildResult>;
|
|
529
|
-
|
|
530
|
-
// Cache
|
|
531
|
-
invalidate(keys: string[]): Promise<number>;
|
|
532
|
-
invalidateAll(): Promise<number>;
|
|
533
|
-
|
|
534
|
-
// Low-level query (token-based)
|
|
535
|
-
query(kbId: string): Promise<string>;
|
|
536
|
-
getResultStatus(token: string):
|
|
537
|
-
Promise<ResultStatus>;
|
|
538
|
-
getResult<T extends ResultType>(
|
|
539
|
-
token: string,
|
|
540
|
-
type: T,
|
|
541
|
-
): Promise<ResultData<T> | null>;
|
|
542
|
-
}
|
|
543
|
-
```
|
|
544
|
-
|
|
545
|
-
## Type Definitions
|
|
546
|
-
|
|
547
|
-
**Status types:**
|
|
548
|
-
|
|
549
|
-
```ts
|
|
550
|
-
type DaemonStatus = 'busy' | 'idle';
|
|
551
|
-
|
|
552
|
-
type ConfigKey =
|
|
553
|
-
| 'timeout_ms'
|
|
554
|
-
| 'item_ttl_ms'
|
|
555
|
-
| 'item_limit'
|
|
556
|
-
| 'index_limit';
|
|
557
|
-
|
|
558
|
-
interface DbStatus {
|
|
559
|
-
target_dir: string;
|
|
560
|
-
document_count: number;
|
|
561
|
-
section_count: number;
|
|
562
|
-
index_size_bytes: number;
|
|
563
|
-
last_modified: string;
|
|
564
|
-
cache_stats: {
|
|
565
|
-
items_cached: number;
|
|
566
|
-
hit_rate: number;
|
|
567
|
-
};
|
|
568
|
-
}
|
|
569
|
-
```
|
|
570
|
-
|
|
571
|
-
**Search types:**
|
|
572
|
-
|
|
573
|
-
`search()` now returns a `PagedSearchResult` envelope.
|
|
574
|
-
Individual `SectionMatch` items include `heading`,
|
|
575
|
-
`type`, `docids`, `snippet`, and `matched_terms`
|
|
576
|
-
alongside `score`. The MCP layer strips `score`; the
|
|
577
|
-
CLI and IPC layers retain it.
|
|
578
|
-
|
|
579
|
-
```ts
|
|
580
|
-
interface SearchParams {
|
|
581
|
-
query: string;
|
|
582
|
-
mode: 'sections' | 'documents' | 'stats';
|
|
583
|
-
limit?: number;
|
|
584
|
-
offset?: number;
|
|
585
|
-
}
|
|
586
|
-
|
|
587
|
-
interface PagedSearchResult {
|
|
588
|
-
items: SectionMatch[];
|
|
589
|
-
total: number;
|
|
590
|
-
offset: number;
|
|
591
|
-
limit: number;
|
|
592
|
-
has_more: boolean;
|
|
593
|
-
}
|
|
594
|
-
|
|
595
|
-
type SearchResult =
|
|
596
|
-
| SectionSearchResult
|
|
597
|
-
| DocumentSearchResult
|
|
598
|
-
| StatsSearchResult;
|
|
599
|
-
|
|
600
|
-
interface SectionSearchResult {
|
|
601
|
-
type: 'section';
|
|
602
|
-
data: SectionMatch[];
|
|
603
|
-
references: (Section | DocumentResult)[];
|
|
604
|
-
total: number;
|
|
605
|
-
query_time_ms: number;
|
|
606
|
-
query: string;
|
|
607
|
-
}
|
|
608
|
-
|
|
609
|
-
interface DocumentSearchResult {
|
|
610
|
-
type: 'document';
|
|
611
|
-
data: DocumentMatch[];
|
|
612
|
-
references: (Section | DocumentResult)[];
|
|
613
|
-
total: number;
|
|
614
|
-
query_time_ms: number;
|
|
615
|
-
query: string;
|
|
616
|
-
}
|
|
617
|
-
|
|
618
|
-
interface StatsSearchResult {
|
|
619
|
-
total_matches: number;
|
|
620
|
-
query_time_ms: number;
|
|
621
|
-
top_terms: TermStat[];
|
|
622
|
-
query: string;
|
|
623
|
-
}
|
|
624
|
-
```
|
|
625
|
-
|
|
626
|
-
**Match types:**
|
|
627
|
-
|
|
628
|
-
```ts
|
|
629
|
-
interface SectionMatch {
|
|
630
|
-
kbid: string;
|
|
631
|
-
docids: string[];
|
|
632
|
-
type: string;
|
|
633
|
-
heading: string | null;
|
|
634
|
-
score: number;
|
|
635
|
-
snippet: string;
|
|
636
|
-
matched_terms: string[];
|
|
637
|
-
}
|
|
638
|
-
|
|
639
|
-
interface DocumentMatch {
|
|
640
|
-
docid: string;
|
|
641
|
-
title: string;
|
|
642
|
-
score: number;
|
|
643
|
-
matching_sections: Array<{
|
|
644
|
-
kbid: string;
|
|
645
|
-
type: string;
|
|
646
|
-
heading: string;
|
|
647
|
-
score: number;
|
|
648
|
-
snippet: string;
|
|
649
|
-
}>;
|
|
650
|
-
}
|
|
651
|
-
|
|
652
|
-
interface TermStat {
|
|
653
|
-
term: string;
|
|
654
|
-
frequency: number;
|
|
655
|
-
sections_matched: number;
|
|
656
|
-
}
|
|
657
|
-
```
|
|
658
|
-
|
|
659
|
-
**Recall types:**
|
|
660
|
-
|
|
661
|
-
`recall()` returns progressive context expansion for
|
|
662
|
-
one or more sections. Returns `RecallResult` for a
|
|
663
|
-
single kbid or `RecallResult[]` for a batch. The
|
|
664
|
-
`depth` parameter controls how much context is
|
|
665
|
-
included (0--3).
|
|
666
|
-
|
|
667
|
-
```ts
|
|
668
|
-
interface RecallResult {
|
|
669
|
-
kbid: string;
|
|
670
|
-
heading: string | null;
|
|
671
|
-
type: string;
|
|
672
|
-
content: string;
|
|
673
|
-
docids: string[];
|
|
674
|
-
// Present at depth >= 1:
|
|
675
|
-
documents?: RecallDocument[];
|
|
676
|
-
back_references?: RecallReference[];
|
|
677
|
-
// Present at depth >= 2:
|
|
678
|
-
siblings?: RecallResult[];
|
|
679
|
-
// Present at depth >= 3:
|
|
680
|
-
references?: RecallResult[];
|
|
681
|
-
}
|
|
682
|
-
|
|
683
|
-
interface RecallDocument {
|
|
684
|
-
docid: string;
|
|
685
|
-
title: string;
|
|
686
|
-
sections: Array<{
|
|
687
|
-
kbid: string;
|
|
688
|
-
heading: string | null;
|
|
689
|
-
type: string;
|
|
690
|
-
}>;
|
|
691
|
-
}
|
|
692
|
-
|
|
693
|
-
interface RecallReference {
|
|
694
|
-
kbid: string;
|
|
695
|
-
heading: string | null;
|
|
696
|
-
type: string;
|
|
697
|
-
marker_type: string;
|
|
698
|
-
context_snippet: string;
|
|
699
|
-
direction: 'back';
|
|
700
|
-
}
|
|
701
|
-
```
|
|
702
|
-
|
|
703
|
-
Maps to the daemon's `recall` IPC method, which
|
|
704
|
-
calls `worker_recall` in the WASM engine.
|
|
705
|
-
|
|
706
|
-
**Section types:**
|
|
707
|
-
|
|
708
|
-
```ts
|
|
709
|
-
interface Section {
|
|
710
|
-
kbid: string;
|
|
711
|
-
docids: string[];
|
|
712
|
-
type: string;
|
|
713
|
-
title?: string;
|
|
714
|
-
description?: string;
|
|
715
|
-
token_count: number;
|
|
716
|
-
heading_token_count: number;
|
|
717
|
-
body_token_count: number;
|
|
718
|
-
code_token_count: number;
|
|
719
|
-
content: string;
|
|
720
|
-
}
|
|
721
|
-
|
|
722
|
-
interface AddSectionParams {
|
|
723
|
-
content: string;
|
|
724
|
-
type: string;
|
|
725
|
-
title?: string;
|
|
726
|
-
description?: string;
|
|
727
|
-
docid?: string;
|
|
728
|
-
}
|
|
729
|
-
|
|
730
|
-
interface AddSectionResult {
|
|
731
|
-
kbid: string;
|
|
732
|
-
type: string;
|
|
733
|
-
title?: string;
|
|
734
|
-
description?: string;
|
|
735
|
-
token_count: number;
|
|
736
|
-
indexed_terms_count: number;
|
|
737
|
-
elapsed_ms: number;
|
|
738
|
-
attached: boolean;
|
|
739
|
-
}
|
|
740
|
-
|
|
741
|
-
interface UpdateSectionParams {
|
|
742
|
-
kbid: string;
|
|
743
|
-
content: string;
|
|
744
|
-
}
|
|
745
|
-
|
|
746
|
-
interface UpdateSectionResult {
|
|
747
|
-
old_kbid: string;
|
|
748
|
-
new_kbid: string;
|
|
749
|
-
reindexed_terms_count: number;
|
|
750
|
-
elapsed_ms: number;
|
|
751
|
-
}
|
|
752
|
-
|
|
753
|
-
interface RemoveSectionResult {
|
|
754
|
-
removed: boolean;
|
|
755
|
-
elapsed_ms: number;
|
|
756
|
-
}
|
|
757
|
-
```
|
|
758
|
-
|
|
759
|
-
**Document types:**
|
|
760
|
-
|
|
761
|
-
```ts
|
|
762
|
-
interface GroupSectionsParams {
|
|
763
|
-
docid: string;
|
|
764
|
-
title: string;
|
|
765
|
-
kbids: string[];
|
|
766
|
-
}
|
|
767
|
-
|
|
768
|
-
interface GroupSectionsResult {
|
|
769
|
-
docid: string;
|
|
770
|
-
section_count: number;
|
|
771
|
-
elapsed_ms: number;
|
|
772
|
-
}
|
|
773
|
-
|
|
774
|
-
interface RemoveGroupingResult {
|
|
775
|
-
removed: boolean;
|
|
776
|
-
sections_ungrouped: number;
|
|
777
|
-
elapsed_ms: number;
|
|
778
|
-
}
|
|
779
|
-
|
|
780
|
-
interface DocumentResult {
|
|
781
|
-
docid: string;
|
|
782
|
-
title: string;
|
|
783
|
-
created_at: string;
|
|
784
|
-
sections: Array<{
|
|
785
|
-
kbid: string;
|
|
786
|
-
docids: string[];
|
|
787
|
-
type: string;
|
|
788
|
-
title?: string;
|
|
789
|
-
description?: string;
|
|
790
|
-
content: string;
|
|
791
|
-
position: number;
|
|
792
|
-
}>;
|
|
793
|
-
}
|
|
794
|
-
```
|
|
795
|
-
|
|
796
|
-
**Unified result shape:**
|
|
797
|
-
|
|
798
|
-
```ts
|
|
799
|
-
interface KbdbResult<T> {
|
|
800
|
-
type: 'section' | 'document';
|
|
801
|
-
data: T;
|
|
802
|
-
references: (Section | DocumentResult)[];
|
|
803
|
-
total: number;
|
|
804
|
-
}
|
|
805
|
-
```
|
|
806
|
-
|
|
807
|
-
`KbdbResult<T>` wraps all retrieval and search
|
|
808
|
-
responses. The `references` array contains unique,
|
|
809
|
-
recursively resolved records for every marker found
|
|
810
|
-
in the returned content (see
|
|
811
|
-
[Document](document.md) for marker types and
|
|
812
|
-
[Content Parser](content-parser.md) for extraction).
|
|
813
|
-
`total` is the full count of matching entities --
|
|
814
|
-
`data.length` may be smaller when `limit` is applied.
|
|
815
|
-
|
|
816
|
-
`SearchResult` (`SectionSearchResult` and
|
|
817
|
-
`DocumentSearchResult`) follows the same shape with
|
|
818
|
-
additional search-specific fields (`query_time_ms`,
|
|
819
|
-
`query`). `StatsSearchResult` does not use the
|
|
820
|
-
unified shape since it returns aggregate statistics,
|
|
821
|
-
not entities.
|
|
822
|
-
|
|
823
|
-
**Content composition result:**
|
|
824
|
-
|
|
825
|
-
```ts
|
|
826
|
-
interface ContentResult {
|
|
827
|
-
type: 'text/markdown';
|
|
828
|
-
data: string;
|
|
829
|
-
total: number;
|
|
830
|
-
}
|
|
831
|
-
```
|
|
832
|
-
|
|
833
|
-
`ContentResult` wraps the output of the content
|
|
834
|
-
composition engine. `data` is the composed Markdown
|
|
835
|
-
string. `total` is the character count of `data`
|
|
836
|
-
(not byte count). See
|
|
837
|
-
[Content Composer](content-composer.md) for the full
|
|
838
|
-
composition specification.
|
|
839
|
-
|
|
840
|
-
**Maintenance types:**
|
|
841
|
-
|
|
842
|
-
```ts
|
|
843
|
-
interface IntegrityCheckResult {
|
|
844
|
-
ok: boolean;
|
|
845
|
-
errors: IntegrityError[];
|
|
846
|
-
sections_checked: number;
|
|
847
|
-
documents_checked: number;
|
|
848
|
-
references_checked: number;
|
|
849
|
-
elapsed_ms: number;
|
|
850
|
-
}
|
|
851
|
-
|
|
852
|
-
interface IntegrityError {
|
|
853
|
-
type:
|
|
854
|
-
| 'checksum_mismatch'
|
|
855
|
-
| 'orphan_section'
|
|
856
|
-
| 'orphan_document'
|
|
857
|
-
| 'broken_reference'
|
|
858
|
-
| 'type_mismatch'
|
|
859
|
-
| 'circular_reference';
|
|
860
|
-
entity: string;
|
|
861
|
-
detail: string;
|
|
862
|
-
}
|
|
863
|
-
|
|
864
|
-
interface GcResult {
|
|
865
|
-
removed_sections: number;
|
|
866
|
-
removed_bytes: number;
|
|
867
|
-
elapsed_ms: number;
|
|
868
|
-
}
|
|
869
|
-
|
|
870
|
-
interface RebuildResult {
|
|
871
|
-
sections_reindexed: number;
|
|
872
|
-
terms_indexed: number;
|
|
873
|
-
elapsed_ms: number;
|
|
874
|
-
}
|
|
875
|
-
```
|
|
876
|
-
|
|
877
|
-
`IntegrityCheckResult` reports all issues found in
|
|
878
|
-
the database. Error types match
|
|
879
|
-
[Database -- Integrity and Maintenance](database.md).
|
|
880
|
-
`GcResult` reports sections removed and bytes freed.
|
|
881
|
-
`RebuildResult` reports sections and terms
|
|
882
|
-
re-indexed.
|
|
883
|
-
|
|
884
|
-
**Low-level query types:**
|
|
885
|
-
|
|
886
|
-
```ts
|
|
887
|
-
type ResultType =
|
|
888
|
-
| 'kbid'
|
|
889
|
-
| 'docid'
|
|
890
|
-
| 'section'
|
|
891
|
-
| 'content'
|
|
892
|
-
| 'kbid[]'
|
|
893
|
-
| 'docid[]'
|
|
894
|
-
| 'section[]'
|
|
895
|
-
| 'content[]';
|
|
896
|
-
|
|
897
|
-
interface ResultStatus {
|
|
898
|
-
status: 'busy' | 'success' | 'fail';
|
|
899
|
-
reason: string;
|
|
900
|
-
}
|
|
901
|
-
```
|
|
902
|
-
|
|
903
|
-
## Connection Lifecycle
|
|
904
|
-
|
|
905
|
-
**Short-lived (CLI):**
|
|
906
|
-
|
|
907
|
-
```
|
|
908
|
-
src/cli.ts
|
|
909
|
-
│
|
|
910
|
-
├─ parse args, resolve targetDir
|
|
911
|
-
├─ createWorkerClient({ targetDir })
|
|
912
|
-
│ ├─ find or spawn daemon
|
|
913
|
-
│ └─ connect to IPC socket
|
|
914
|
-
├─ client.search(...) / client.addSection(...)
|
|
915
|
-
├─ print result to stdout
|
|
916
|
-
└─ client.disconnect()
|
|
917
|
-
```
|
|
918
|
-
|
|
919
|
-
The CLI creates the client, makes one or a few calls,
|
|
920
|
-
and disconnects. The daemon stays alive for subsequent
|
|
921
|
-
invocations.
|
|
922
|
-
|
|
923
|
-
**Long-lived (MCP):**
|
|
924
|
-
|
|
925
|
-
```
|
|
926
|
-
src/cli.ts
|
|
927
|
-
│
|
|
928
|
-
├─ createWorkerClient({ targetDir })
|
|
929
|
-
│ ├─ find or spawn daemon
|
|
930
|
-
│ └─ connect to IPC socket
|
|
931
|
-
├─ listen for MCP requests
|
|
932
|
-
│ ├─ MCP search → client.search(...)
|
|
933
|
-
│ ├─ MCP add → client.addSection(...)
|
|
934
|
-
│ ├─ MCP remove → client.removeSection(...)
|
|
935
|
-
│ └─ ...
|
|
936
|
-
└─ on shutdown: client.disconnect()
|
|
937
|
-
```
|
|
938
|
-
|
|
939
|
-
The MCP server holds the connection open. All incoming
|
|
940
|
-
MCP requests route through the same `WorkerClient`.
|
|
941
|
-
|
|
942
|
-
## Reconnection
|
|
943
|
-
|
|
944
|
-
If the IPC connection drops (daemon crashed, idle
|
|
945
|
-
timeout, OS killed the process), the client does not
|
|
946
|
-
throw immediately. On the next method call, it:
|
|
947
|
-
|
|
948
|
-
1. Detects the broken connection.
|
|
949
|
-
2. Runs the discovery sequence again (PID file check,
|
|
950
|
-
spawn if needed).
|
|
951
|
-
3. Reconnects to the new daemon's IPC socket.
|
|
952
|
-
4. Retries the method call once.
|
|
953
|
-
|
|
954
|
-
If reconnection fails, the method throws. The caller
|
|
955
|
-
handles the error.
|
|
956
|
-
|
|
957
|
-
This makes the MCP server resilient to daemon restarts
|
|
958
|
-
without manual reconnection logic. The CLI benefits
|
|
959
|
-
too -- if a daemon dies between the spawn check and
|
|
960
|
-
the first method call, the client recovers.
|
|
961
|
-
|
|
962
|
-
## Error Handling
|
|
963
|
-
|
|
964
|
-
All client methods throw on:
|
|
965
|
-
|
|
966
|
-
- **Connection failure** -- Cannot connect to the
|
|
967
|
-
daemon after discovery and spawn attempts.
|
|
968
|
-
- **Spawn failure** -- Daemon did not start within
|
|
969
|
-
the timeout.
|
|
970
|
-
- **IPC error** -- JSON-RPC 2.0 error response from
|
|
971
|
-
the daemon (context mismatch, invalid token, method
|
|
972
|
-
not found).
|
|
973
|
-
- **Reconnection failure** -- Broken connection and
|
|
974
|
-
the retry also failed.
|
|
975
|
-
|
|
976
|
-
Errors are thrown as typed error objects with a `code`
|
|
977
|
-
property matching the JSON-RPC 2.0 error codes defined
|
|
978
|
-
in [Worker Daemon -- Error Responses](
|
|
979
|
-
worker-daemon.md#error-responses):
|
|
980
|
-
|
|
981
|
-
```ts
|
|
982
|
-
interface WorkerClientError extends Error {
|
|
983
|
-
code: number;
|
|
984
|
-
}
|
|
985
|
-
```
|
|
986
|
-
|
|
987
|
-
## Design Constraints
|
|
988
|
-
|
|
989
|
-
- The worker client is the **sole IPC client** for the
|
|
990
|
-
daemon. No other TypeScript module sends messages to
|
|
991
|
-
`src/worker.ts`.
|
|
992
|
-
- The client does not cache data. Caching is the
|
|
993
|
-
daemon's responsibility (via `kb-worker.wasm`).
|
|
994
|
-
- The client does not perform database I/O. All
|
|
995
|
-
storage operations go through the daemon.
|
|
996
|
-
- The client does not perform text processing. That is
|
|
997
|
-
`query-parser.wasm`'s job inside the daemon.
|
|
998
|
-
- The module works on both Deno and Node.js. Platform
|
|
999
|
-
differences (IPC sockets, process spawning, PID
|
|
1000
|
-
checks) are abstracted inside internal files.
|
|
1001
|
-
- The client's public API mirrors the IPC wire format.
|
|
1002
|
-
Types match the JSON shapes defined in the Endpoints
|
|
1003
|
-
section of [Database](database.md) and the IPC API
|
|
1004
|
-
sections of [Worker Daemon](worker-daemon.md). No
|
|
1005
|
-
transformation layer.
|