@dikolab/kbdb 0.3.1 → 0.4.0

package/docs/details/search-and-ranking.md

@@ -1,5 +1,17 @@
 # How Search Works
 
+When you search, kbdb finds the sections that best
+match your query and ranks them by relevance. By
+default it uses keyword search -- matching terms with
+stemming, synonym expansion, and BM25F ranking. Use
+`--algo hybrid` to combine keyword matching with
+AI-powered similarity for broader recall. The rest
+of this page explains how ranking
+works under the hood -- you don't need to read it
+to use kbdb.
+
+---
+
 kbdb uses field-weighted BM25F ranking to score search
 results by relevance. Sections are indexed across
 three fields -- heading, body, and code -- each with a
@@ -513,3 +525,47 @@ Neither standalone module calls the other directly.
 | `sections`  | Individual matching sections with scores, snippets, and matched terms |
 | `documents` | Documents containing matches, with aggregated scores and per-section details |
 | `stats`     | Match counts and top terms without content bodies |
+
+## Search Algorithms
+
+kbdb supports three search algorithms, selected
+with `--algo` (CLI) or the `algo` parameter (MCP).
+
+### Lexical
+
+Keyword matching with stemming and synonym
+expansion, scored by BM25F as described above.
+Fast, works offline, and gives consistent results.
+
+### Vector
+
+Embedding-based similarity search. Each section's
+content is converted into a numeric vector (an
+"embedding") by an AI model. Search finds sections
+whose vectors are closest to the query's vector
+using cosine similarity.
+
+Vector search catches matches that keyword search
+misses -- for example, searching "database
+rationale" finds a section about "we chose Postgres
+because of JSONB" even though none of the words
+overlap.
+
+Requires an embedding provider to be configured.
+Without one, vector mode returns an error.
+
+### Hybrid
+
+Runs both lexical and vector searches in parallel,
+then merges the two ranked lists using Reciprocal
+Rank Fusion (RRF). This combines the precision of
+keyword matching with the associative recall of
+embeddings. Use `--algo hybrid` to enable.
+
+RRF works by converting each result's rank position
+into a score (`1 / (k + rank)` with k=60), then
+summing scores across both lists. Results that rank
+well in both lists get the highest combined scores.
+
+Hybrid mode requires an embedding provider. Without
+one, it returns an error.

package/docs/goals/cli.md

@@ -61,6 +61,8 @@ kbdb gc [options]
 kbdb rebuild [options]
 kbdb status [options]
 kbdb export [options] [<path>]
+kbdb worker stop [options]
+kbdb worker restart [options]
 kbdb db init [options]
 kbdb db migrate [options]
 kbdb skill learn [options] --name <n> --body <file|->
@@ -104,10 +106,9 @@ with the AI agent host over stdio using JSON-RPC 2.0,
 and delegates all operations to the worker daemon via
 the same worker client module.
 
-MCP mode requires `--db <path>` or the `KBDB_DIR`
-environment variable. It does not fall back to the
-current working directory because the server's cwd is
-determined by the host application, not the user.
+MCP mode accepts `--db <path>` or the `KBDB_DB_DIR`
+environment variable. When neither is set, it falls
+back to the current working directory.
 
 See [MCP](mcp.md) for the full MCP mode
 specification.
@@ -116,11 +117,14 @@ specification.
 
 These options apply to all commands.
 
-| Option        | Short | Description              |
-|---------------|-------|--------------------------|
-| `--help`      | `-h`  | Print help text and exit |
-| `--version`   | `-v`  | Print version and exit   |
-| `--db <path>` |       | Database directory path  |
+| Option             | Short | Description              |
+|--------------------|-------|--------------------------|
+| `--help`           | `-h`  | Print help text and exit |
+| `--version`        | `-v`  | Print version and exit   |
+| `--db <path>`      |       | Database directory path (optional) |
+| `--format <fmt>`   | `-f`  | Output format (json default; text when stdout is a TTY) |
+| `--timeout-ms <ms>`|       | Request timeout in ms    |
+| `--retry <n>`      |       | Max retries on timeout   |
 
 ### `--help`, `-h`
 
@@ -156,12 +160,13 @@ kbdb search "query" --db /path/to/project
 ```
 
 When omitted:
-- **CLI mode**: defaults to the current working
-  directory (`process.cwd()`).
-- **MCP mode**: falls back to the `KBDB_DIR`
-  environment variable. If neither `--db` nor
-  `KBDB_DIR` is set, the MCP server prints an error
-  and exits non-zero.
+- **CLI mode**: falls back to the `KBDB_DB_DIR`
+  environment variable when `--db` is omitted,
+  then to the current working directory
+  (`process.cwd()`).
+- **MCP mode**: falls back to the `KBDB_DB_DIR`
+  environment variable, then to the current
+  working directory.
 
 ## Commands
 
@@ -300,7 +305,10 @@ Remove knowledge base sections by their kb-id.
 
 **Arguments:**
 - `<kbid>...` -- one or more 26-character kb-id
-  strings. Each is processed independently.
+  strings (or unique prefixes -- kbdb resolves
+  short prefixes to full identifiers, similar to
+  git short hashes). Each is processed
+  independently.
 
 **Behaviour:**
 - The engine removes the referenced content and
@@ -360,6 +368,7 @@ Search the knowledge base by keywords or phrases.
 | `--offset <n>`   | `-o`  | `0`        | Result offset      |
 | `--format <fmt>` | `-f`  | `json`     | Output format      |
 | `--content`      | `-c`  |            | Hydrate previews   |
+| `--algo <algo>`  |       | `lexical`  | Search algorithm   |
 | `--db <path>`    |       | cwd        | Database directory |
 
 #### `--mode`, `-m`
@@ -406,8 +415,8 @@ Controls the output format printed to stdout.
 
 | Format | Description                             |
 |--------|-----------------------------------------|
-| `json` | Pretty-printed JSON array (default).    |
-| `text` | Numbered human-readable list with scores and matched terms. |
+| `json` | Pretty-printed JSON array (default when stdout is piped). |
+| `text` | Numbered human-readable list with scores and matched terms (default when stdout is a TTY). |
 | `mcp`  | JSON envelope for MCP protocol consumers. |
 
 ```
@@ -553,7 +562,10 @@ more knowledge base identifiers and output it to
 stdout.
 
 **Arguments:**
-- `<id>...` -- one or more kbid or docid identifiers.
+- `<id>...` -- one or more kbid or docid identifiers
+  (or unique prefixes -- kbdb resolves short
+  prefixes to full identifiers, similar to git
+  short hashes).
 
 **Output:**
 - **stdout**: JSON envelope containing the composed
@@ -600,7 +612,9 @@ expansion depths.
 
 **Arguments:**
 - `<kbid>...` -- one or more 26-character kb-id
-  strings.
+  strings (or unique prefixes -- kbdb resolves
+  short prefixes to full identifiers, similar to
+  git short hashes).
 
 **Options:**
 
@@ -722,9 +736,9 @@ Start a long-lived MCP server on stdio.
 **Behaviour:**
 - Communicates with the AI agent host over stdio
   using JSON-RPC 2.0.
-- Requires `--db <path>` or the `KBDB_DIR` environment
-  variable. Does NOT fall back to the current working
-  directory.
+- Accepts `--db <path>` or the `KBDB_DB_DIR`
+  environment variable. Falls back to the current
+  working directory when neither is set.
 - Runs a version gate before starting: exits with an
   error if the database needs migration or is too new.
 - The MCP server does not prompt, does not print
@@ -734,8 +748,8 @@ Start a long-lived MCP server on stdio.
   a shutdown signal.
 
 **Environment variables:**
-- `KBDB_DIR` -- fallback database directory path when
-  `--db` is not specified.
+- `KBDB_DB_DIR` -- fallback database directory path
+  when `--db` is not specified.
 
 **Exit code:** `0` on clean shutdown, `1` on error.
 
@@ -943,6 +957,43 @@ src/cli.ts  parseArgs()
   -> stdout
 ```
 
+### `kbdb worker` -- daemon lifecycle
+
+The `worker` command group manages the background
+worker daemon process.
+
+#### `worker stop` -- stop the daemon
+
+```
+kbdb worker stop [options]
+```
+
+Stop the running worker daemon for the current
+database. If no daemon is running, prints a message
+and exits successfully.
+
+**Exit code:** `0` on success, `1` on error.
+
+**Delegation chain:**
+```
+src/cli.ts  parseArgs()
+  -> dispatchCommand(client, options)
+       -> runWorkerStop(client, options)
+            -> sends shutdown signal via IPC
+```
+
+#### `worker restart` -- restart the daemon
+
+```
+kbdb worker restart [options]
+```
+
+Stop and restart the worker daemon. Equivalent to
+`worker stop` followed by the next command
+auto-spawning a new daemon.
+
+**Exit code:** `0` on success, `1` on error.
+
 ### `kbdb db` -- database management
 
 The `db` command group manages the `.kbdb` database
@@ -1721,18 +1772,21 @@ processes the raw argument vector
    deprecation warning to stderr, and continues
    parsing.
 4. When the first positional token is `db`, `skill`,
-   or `agent`, the parser consumes the next positional
+   `agent`, or `worker`, the parser consumes the next positional
    token as the subcommand and combines them into a
    hyphenated command string: `db init` →
    `'db-init'`, `skill learn` → `'skill-learn'`,
-   `agent create` → `'agent-create'`, etc.
+   `agent create` → `'agent-create'`,
+   `worker stop` → `'worker-stop'`, etc.
 5. All other flags (`--db`, `--format`, `--mode`,
    `--limit`, `--offset`, `--title`, `--description`,
    `--docid`, `--name`, `--persona`, `--body`,
-   `--skills`, `--depth` / `-d`, `--tags`) consume
-   the next token as their value. `--depth` is parsed
-   as a number and clamped to the range 0--3. `--tags`
-   splits its value on commas into a `string[]`.
+   `--skills`, `--depth` / `-d`, `--tags`,
+   `--timeout-ms`, `--retry`) consume the next token
+   as their value. `--depth` is parsed as a number
+   and clamped to the range 0--3. `--tags` splits its
+   value on commas into a `string[]`. `--timeout-ms`
+   and `--retry` are parsed as numbers.
 6. `--content` / `-c` and `--replace` are boolean
    flags; they set their respective option to `true`
    without consuming the next token.
@@ -1802,6 +1856,10 @@ interface CliOptions {
    tags?: string[];
    /** Enable supersession by source key (learn). */
    replace?: boolean;
+   /** Request timeout in ms. */
+   timeoutMs?: number;
+   /** Max retry count on timeout. */
+   retry?: number;
    /** Positional arguments following the sub-command. */
    args: string[];
 }
@@ -1883,6 +1941,10 @@ Agents:
    agent get           Retrieve an agent by docid
    agent delete        Remove an agent
 
+Worker:
+   worker stop         Stop the background daemon
+   worker restart      Restart the background daemon
+
 Server:
    mcp                 Start MCP server mode (stdio)
 
@@ -2114,6 +2176,8 @@ Routing table:
 | `agent-create`   | `runAgentCreate`   | yes             |
 | `agent-list`     | `runAgentList`     | yes             |
 | `agent-get`      | `runAgentGet`      | yes             |
+| `worker-stop`    | `runWorkerStop`    | yes             |
+| `worker-restart` | `runWorkerRestart` | yes             |
 | `agent-delete`   | `runAgentDelete`   | yes             |
 
 Unknown commands return exit code 1 with an "unknown
@@ -2504,9 +2568,21 @@ kbdb search "machine learning" --limit 5
 
 ## Environment Variables
 
-| Variable   | Used by  | Description                |
-|------------|----------|----------------------------|
-| `KBDB_DIR` | MCP mode | Fallback database directory when `--db` is not specified. Not used in CLI mode. |
+| Variable              | Used by   | Description             |
+|-----------------------|-----------|-------------------------|
+| `KBDB_DB_DIR`         | CLI + MCP | Fallback database directory when `--db` is not specified |
+| `KBDB_DAEMON_TIMEOUT` | CLI + MCP | Request timeout fallback (ms) |
+| `KBDB_RETRY`          | CLI + MCP | Retry count fallback    |
+
+### Resolution Priority
+
+When multiple sources set the same option, the
+most specific source wins:
+
+- **timeout**: `--timeout-ms` >
+  `KBDB_DAEMON_TIMEOUT` > `30000`
+- **retry**: `--retry` > `KBDB_RETRY` > `5`
+- **db path**: `--db` > `KBDB_DB_DIR` > cwd
 
 ## Exit Codes
 

package/docs/goals/mcp.md

@@ -27,25 +27,28 @@ identically.
 ## Invocation
 
 ```
-kbdb mcp [--db <path>]
+kbdb mcp [--db <path>] [--timeout-ms <ms>]
+   [--retry <n>]
 ```
 
-The `--db` flag specifies the absolute path to the
-`.kbdb` database directory. If omitted, the server
-reads the `KBDB_DIR` environment variable. One of the
-two must be provided -- the MCP server does not fall
-back to the current working directory.
+All flags are optional. `--db` specifies the path
+to the `.kbdb` database directory. If omitted, the
+server reads the `KBDB_DB_DIR` environment variable,
+then falls back to the current working directory.
 
 Resolution order:
 
 1. `--db <path>` command-line flag.
-2. `KBDB_DIR` environment variable.
-3. Error: no database directory specified.
+2. `KBDB_DB_DIR` environment variable.
+3. Current working directory.
 
-The MCP server does not fall back to the current
-working directory because the server's working
-directory is determined by the host application, not
-the user.
+### Environment Variables
+
+| Variable              | Description             |
+|-----------------------|-------------------------|
+| `KBDB_DB_DIR`         | Fallback database directory when `--db` is not specified |
+| `KBDB_DAEMON_TIMEOUT` | Request timeout fallback (ms); default 30 000 |
+| `KBDB_RETRY`          | Retry count fallback; default 5 |
 
 See [CLI -- Run Modes](cli.md#run-modes) for how
 the `kbdb` executable dispatches to MCP mode.
@@ -127,7 +130,7 @@ Add to `.claude/settings.json` or project settings:
 
 ### Environment Variable
 
-The `--db` flag can be replaced by `KBDB_DIR`:
+The `--db` flag can be replaced by `KBDB_DB_DIR`:
 
 ```json
 {
@@ -136,7 +139,7 @@ The `--db` flag can be replaced by `KBDB_DIR`:
          "command": "npx",
          "args": ["@dikolab/kbdb", "mcp"],
          "env": {
-            "KBDB_DIR": "/path/to/.kbdb"
+            "KBDB_DB_DIR": "/path/to/.kbdb"
          }
       }
    }
@@ -1125,7 +1128,7 @@ interface McpToolAnnotations {
 }
 ```
 
-Annotations for all 19 kbdb tools:
+Annotations for all 20 kbdb tools:
 
 | Tool           | title             | readOnly | destructive | idempotent | openWorld |
 |----------------|-------------------|----------|-------------|------------|-----------|
@@ -1148,6 +1151,7 @@ Annotations for all 19 kbdb tools:
 | `agent-list`   | List Agents       | true     | false       | true       | false     |
 | `agent-get`    | Get Agent         | true     | false       | true       | false     |
 | `agent-delete` | Delete Agent      | false    | true        | true       | false     |
+| `auto-capture-review` | Review Captures | false | false    | false      | false     |
 
 ## Completions
 
@@ -1361,7 +1365,7 @@ The difference is the interface:
 | Interface | Command-line arguments | MCP protocol (stdio)   |
 | Consumer  | Human / shell script   | AI agent               |
 | Output    | stdout (JSON/text/MCP) | MCP tool result format |
-| DB path   | CWD or `--db` flag     | `--db` flag or `KBDB_DIR` env |
+| DB path   | CWD or `--db` flag     | `--db` flag or `KBDB_DB_DIR` env |
 
 See [CLI](cli.md) for CLI mode details,
 [Worker Client](worker-client.md) for the shared
@@ -1513,9 +1517,8 @@ returns `-32601` if they are called.
 - The server works with both Deno and Node.js
   runtimes, consistent with the project's
   dual-runtime requirement.
-- The database path must be explicitly configured
-  via `--db` or `KBDB_DIR`. No working directory
-  fallback.
+- The database path is resolved from `--db`,
+  `KBDB_DB_DIR`, or the current working directory.
 - The MCP server does not perform keyword extraction,
   indexing, ranking, or database I/O itself. All
   processing is delegated to the worker daemon via

package/docs/goals/query-parser.md

@@ -193,6 +193,14 @@ hardcoded. Future versions may allow user-defined
 synonym maps via a `synonyms.toml` configuration file
 in the `.kbdb` directory.
 
+**Implementation status:** Synonym expansion is
+implemented in the Rust pipeline
+(`query-parser/src/pipeline/synonyms.rs`). The
+built-in map covers common technical synonyms.
+The `query-guidance` default skill complements this
+by instructing AI agents to rephrase queries with
+alternative terms before searching.
+
 ## Query Parsing
 
 Search queries are parsed into a structured query plan

package/docs/goals/worker-client.md

@@ -270,7 +270,9 @@ The spawning sequence:
    daemon started.
 4. **On PID file found (success):** worker client
    closes the stderr pipe, detaches the child, unrefs
-   it, and connects to the IPC socket.
+   it, and connects to the IPC socket. Because the
+   daemon writes the PID file only after the socket
+   is listening, the client can connect immediately.
 5. **On timeout or child exit (failure):** worker
    client reads all buffered stderr from the daemon,
    includes it in the thrown error (see Startup
@@ -337,6 +339,65 @@ error reason even though the daemon runs as a separate
 detached process. The daemon's stderr is the single
 source of truth for startup failure diagnostics.
 
+## Connection Retry
+
+When the worker client calls `connect()`, transient
+errors (`ENOENT` -- socket not yet created, or
+`ECONNREFUSED` -- server not yet accepting) are
+retried with exponential backoff:
+
+- **Max retries**: 5
+- **Initial delay**: 50 ms
+- **Backoff**: doubles each attempt (50, 100, 200,
+  400, 800 ms = ~1.55 s worst case)
+- **Non-transient errors** (`EACCES`, `EINVAL`, etc.)
+  fail immediately without retry.
+
+This covers the narrow window where the PID file
+exists but the socket has not yet finished binding
+(e.g. under heavy load or slow I/O).
+
+## Post-Connect Failure Handling
+
+After a successful `connect()`, if the daemon crashes
+or the IPC socket dies:
+
+1. All pending RPC calls (in `this.pending`) are
+   rejected with a descriptive error.
+2. The client marks itself as disconnected
+   (`this.socket = null`).
+3. The socket is destroyed.
+
+The `isConnected` getter exposes the connection
+state so callers (and the cached client factory) can
+check before reusing a client.
+
+## Cached Client Factory
+
+`getOrCreateClient(options?)` returns a cached
+`WorkerClient` for the given context path, or creates
+and caches a new one if the cached client is
+disconnected or absent.
+
+```ts
+import { getOrCreateClient } from '@dikolab/kbdb';
+
+const client = await getOrCreateClient({
+   dbPath: '/path/to/project',
+});
+```
+
+Intended for library consumers and the MCP server.
+CLI one-shots should use `createWorkerClient()`
+directly (each is a separate process anyway).
+
+## Socket Keepalive
+
+Both the client and server enable `SO_KEEPALIVE`
+with a 30-second interval. This ensures both ends
+detect a dead peer within ~30 seconds, even if no
+application-level traffic is flowing.
+
 ## `createWorkerClient(options?)`
 
 Factory function. Returns a connected `WorkerClient`

package/docs/goals/worker-daemon.md

@@ -163,18 +163,21 @@ daemon process performs the following startup sequence:
    to startup. The health status (`integrity_ok`,
    `integrity_errors`) is included in subsequent
    `status` responses.
-7. Writes its PID to `kbdb-{ctx}.pid` in the OS
-   temporary directory.
+7. Removes the stale IPC socket if present.
 8. Creates the IPC socket (`kbdb-{ctx}.sock` on Unix,
-   named pipe on Windows).
-9. Begins listening for connections.
+   named pipe on Windows) and begins listening
+   (async, with backlog 256).
+9. Writes its PID to `kbdb-{ctx}.pid` in the OS
+   temporary directory.
 
-The daemon only writes its PID file and creates the
-IPC socket after all preconditions pass and the WASM
-module is loaded. If the daemon exits before step 7,
-the worker client's PID-file poll times out, and the
-worker client reads the crash log (or the daemon's
-stderr) to report the failure reason.
+The daemon writes its PID file only after the IPC
+socket is bound and listening. This guarantees the
+worker client's PID-file poll succeeds only when
+the socket is ready to accept connections. If the
+daemon exits before step 9, the worker client's
+PID-file poll times out, and the worker client
+reads the crash log (or the daemon's stderr) to
+report the failure reason.
 
 ### Startup Preconditions
 
@@ -273,6 +276,14 @@ a client connects, disconnects, or sends a request.
 The daemon stays alive as long as at least one client
 is connected, regardless of the idle timer.
 
+`SO_KEEPALIVE` (30-second interval) provides a
+secondary mechanism for detecting dead connections
+beyond the counter-based idle timer. If a client
+process crashes without closing the socket, the OS
+will detect the dead peer within ~30 seconds and
+close the connection, allowing the daemon to
+decrement its counter and eventually idle-shutdown.
+
 ## Graceful Shutdown
 
 On shutdown (idle timeout or OS signal SIGTERM/SIGINT),
@@ -447,6 +458,26 @@ Configuration is set once when the daemon spawns and
 does not change until the daemon shuts down and a new
 one is spawned.
 
+### Client vs Daemon Timeouts
+
+The `KBDB_DAEMON_TIMEOUT` environment variable (or
+`--timeout-ms` CLI flag) controls the **client-side
+request timeout** -- how long a CLI or MCP client
+waits for a response from the daemon before giving
+up and optionally retrying.
+
+The `timeout_ms` setting in `worker.toml` controls
+the **daemon-side idle timeout** -- how long the
+daemon stays alive after all client connections have
+closed.
+
+These are independent. A long-running query may
+exceed the client request timeout while the daemon
+continues processing. Conversely, a short idle
+timeout shuts the daemon down quickly between
+invocations but has no effect on in-flight
+requests.
+
 ## IPC Transport
 
 The daemon listens on a local IPC channel:
@@ -471,6 +502,17 @@ No third-party IPC library is required. The daemon
 creates the IPC socket at startup and removes it on
 shutdown.
 
+### Socket Options
+
+- **Backlog**: `256`. Handles connection bursts from
+  concurrent CLI invocations and benchmarks without
+  dropping connections.
+- **SO_KEEPALIVE**: Enabled on every accepted
+  connection with a 30-second interval. Detects dead
+  clients without relying solely on the idle timer.
+  Both the server and client enable keepalive (see
+  [Worker Client](worker-client.md)).
+
 ## IPC Protocol
 
 Communication uses **JSON-RPC 2.0** over the IPC

package/docs/modules/typescript/cli.md

@@ -73,7 +73,7 @@ src/cli.ts
 src/cli.ts
   │
   ├─ argv[0] === 'mcp'
-  ├─ resolve --db flag or KBDB_DIR env
+  ├─ resolve --db flag or KBDB_DB_DIR env or cwd
   ├─ client = await createWorkerClient({ targetDir })
   ├─ server = createMcpServer(client)
   ├─ server.listen()   ← stdio, blocks until closed