@dikolab/kbdb 0.3.2 → 0.4.1

package/docs/details/README.md

@@ -4,6 +4,10 @@ In-depth guides and reference for @dikolab/kbdb.
 
 ## Guides
 
+- [Knowledge Base Guide](knowledge-base.md) --
+  Importing, searching, recall, sync & backup
+- [Agent Tooling Guide](agent-tooling.md) -- MCP,
+  skills, agents, library API, worker daemon
 - [CLI Reference](cli.md) -- Full command reference
   with examples
 - [MCP Server Setup](mcp-server.md) -- Configure

package/docs/details/agent-tooling.md

@@ -0,0 +1,132 @@
+# Agent Tooling Guide
+
+How to integrate kbdb with AI agents using the MCP
+server, skills, agent profiles, and the library API.
+
+## Overview
+
+kbdb runs as an
+[MCP](https://modelcontextprotocol.io/) server with
+20 tools, giving AI agents a searchable second brain
+that persists between sessions. Agents can search
+your knowledge base, recall context progressively,
+store reusable skills, and build agent profiles.
+
+## MCP Server Setup
+
+Add kbdb to your AI tool's MCP config. The setup is
+the same for Claude Code, VS Code, Cursor, and
+Claude Desktop -- just put the config in the right
+file for your tool.
+
+```json
+{
+   "mcpServers": {
+      "kbdb": {
+         "command": "npx",
+         "args": [
+            "-y", "@dikolab/kbdb",
+            "mcp", "--db", "/path/to/.kbdb"
+         ]
+      }
+   }
+}
+```
+
+**Deno users:** replace `npx` / `-y` /
+`@dikolab/kbdb` with `deno` / `run` / `-A` /
+`jsr:@dikolab/kbdb/cli`.
+
+Environment variables:
+
+| Variable | Description |
+|----------|-------------|
+| `KBDB_DB_DIR` | Database path (replaces `--db`) |
+| `KBDB_LOG_LEVEL` | `debug`, `info`, `warning`, `error` |
+| `KBDB_NON_INTERACTIVE` | Suppress prompts (`1` / `true`) |
+| `KBDB_DAEMON_TIMEOUT` | Request timeout in ms |
+| `KBDB_RETRY` | Max retries on timeout |
+
+See the [MCP Server Guide](mcp-server.md) for
+environment variable config, multiple knowledge
+bases, and the full 20-tool reference.
+
+## Skills
+
+Skills are reusable prompt templates with
+fill-in-the-blank arguments.
+
+```sh
+kbdb skill learn --name "code-review" \
+   --body "Review this code for: {{focus}}" \
+   --db ./my-kb
+```
+
+Skills are stored in the knowledge base and
+accessible to agents via MCP tools.
+
+## Agent Profiles
+
+Agent profiles combine a persona with a set of
+skills.
+
+```sh
+kbdb agent create --name "reviewer" \
+   --persona "You are a senior code reviewer." \
+   --skills "code-review,style-check" \
+   --db ./my-kb
+```
+
+## Auto-Capture
+
+The MCP server can proactively suggest knowledge
+to store from conversations. When enabled,
+auto-capture monitors tool call patterns and
+triggers capture suggestions based on configurable
+thresholds.
+
+## Library API
+
+Use kbdb programmatically in Node.js or Deno:
+
+```typescript
+import { createWorkerClient } from '@dikolab/kbdb';
+
+const client = await createWorkerClient({
+   contextPath: '/path/to/.kbdb',
+});
+
+const results = await client.search({
+   query: 'authentication',
+   limit: 10,
+});
+
+console.log(results.items);
+client.disconnect();
+```
+
+See the
+[Library API Reference](library-api.md)
+for the full API.
+
+## Worker Daemon
+
+kbdb runs a background worker daemon that handles
+indexing and search. The daemon auto-spawns on the
+first CLI or MCP command and stops when idle.
+
+```sh
+kbdb worker stop --db ./my-kb      # stop daemon
+kbdb worker restart --db ./my-kb   # restart daemon
+```
+
+The daemon writes a PID file and Unix socket under
+the system temp directory. After upgrading kbdb,
+restart the daemon to pick up the new version.
+
+## Deep Dives
+
+- [MCP Server Guide](mcp-server.md) -- full tool
+  list, environment config, multi-KB setup
+- [Library API Reference](library-api.md) --
+  programmatic access for custom integrations

package/docs/details/cli.md

@@ -12,7 +12,7 @@ the command line.
 npm install -g @dikolab/kbdb
 
 # Deno
-deno install
+deno install -Agf jsr:@dikolab/kbdb/cli
 ```
 
 ## Synopsis
@@ -21,6 +21,10 @@ deno install
 kbdb [options]
 kbdb <command> [options] ...[parameters]
 kbdb <command> <subcommand> [options] ...[parameters]
+kbdb --version
+kbdb -v <command>          (verbose: warnings)
+kbdb -vv <command>         (verbose: info)
+kbdb -vvv <command>        (verbose: debug)
 ```
 
 ## Command Groups
@@ -48,27 +52,59 @@ kbdb <command> <subcommand> [options] ...[parameters]
 | Agents | `agent get` | Retrieve an agent by docid |
 | Agents | `agent delete` | Remove an agent |
 | Server | `mcp` | Start MCP server mode (stdio) |
+| Worker | `worker stop` | Stop the background worker daemon |
+| Worker | `worker restart` | Restart the background worker daemon |
+
+### Reading Data
+
+Three commands retrieve stored content at different
+granularities:
+
+| Command | Purpose |
+|---------|---------|
+| `search` | Find sections by query; returns ranked snippets |
+| `recall` | Expand full context around known kbids (post-search deep-dive) |
+| `content` | Compose rendered Markdown from kbid or docid identifiers |
+
+**When to use which:** Start with `search` to find
+relevant sections by keyword. Use `recall` to expand
+a search result into its full context -- parent
+documents, sibling sections, and references. Use
+`content` when you already know the exact kbid or
+docid and want rendered Markdown output.
 
 ## Global Options
 
 | Option | Short | Default | Description |
 |--------|-------|---------|-------------|
 | `--help` | `-h` | | Print help text and exit |
-| `--version` | `-v` | | Print version and exit |
-| `--db <path>` | | cwd | Database directory path |
-| `--format <fmt>` | `-f` | `json` | Output format (`json`, `text`, `mcp`) |
+| `--version` | | | Print version and exit |
+| `--verbose` | `-v` | | Set log level to WARNING (errors + warnings) |
+| `--verbose-info` | `-vv` | | Set log level to INFO |
+| `--verbose-debug` | `-vvv` | | Set log level to DEBUG (all messages) |
+| `--db <path>` | | cwd | Database directory path (optional) |
+| `--format <fmt>` | `-f` | `json` (text when interactive) | Output format (`json`, `text`, `mcp`) |
+| `--timeout-ms <ms>` | | `30000` | Per-request daemon timeout in milliseconds |
+| `--retry <n>` | | `1` | Max retries with daemon respawn on timeout |
+| `--non-interactive` | | | Suppress prompts; use defaults |
 
 ### `--db <path>`
 
 Set the path to the directory containing the
 `.kbdb` database folder. The actual database is
-always at `${path}/.kbdb`.
+always at `${path}/.kbdb`. This flag is optional.
 
-- **CLI mode**: defaults to the current working
-  directory.
-- **MCP mode**: falls back to the `KBDB_DIR`
+Resolution order:
+
+1. `--db` flag (highest priority)
+2. `KBDB_DB_DIR` environment variable
+3. Current working directory (CLI mode only)
+
+- **CLI mode**: resolves using the order above.
+- **MCP mode**: falls back to the `KBDB_DB_DIR`
   environment variable. If neither `--db` nor
-  `KBDB_DIR` is set, the server exits with an error.
+  `KBDB_DB_DIR` is set, the server exits with an
+  error.
 
 ## Search Options
 
@@ -87,10 +123,24 @@ These options apply to the `search` command.
 
 | Algorithm | Description |
 |-----------|-------------|
-| `lexical` | Keyword matching with stemming and synonyms (default) |
-| `vector` | AI-powered similarity search using embeddings (requires a configured embedding provider) |
+| `lexical` | Keyword matching with stemming and synonyms |
+| `vector` | Similarity search using embeddings |
 | `hybrid` | Combines keyword and similarity results for best coverage |
 
+### Configuring Hybrid Search
+
+Hybrid and vector search use a built-in TF-IDF
+embedding provider by default -- no configuration
+needed. For higher quality embeddings, configure a
+provider via `worker.toml`:
+
+| Provider | Description |
+|----------|-------------|
+| *(none)* | Built-in TF-IDF (256 dimensions, works offline) |
+| `none` | Explicitly disabled; hybrid and vector return an error |
+| `onnx` | Local ONNX model (no network required) |
+| `api` | Remote embedding API (OpenAI-compatible endpoint) |
+
 ### Search Modes
 
 | Mode | Description |
@@ -167,6 +217,9 @@ and ranking data.
 ]
 ```
 
+> kbid arguments accept unique prefixes -- kbdb
+> resolves them to full 26-character identifiers.
+
 ### `search` -- Query the Knowledge Base
 
 ```
@@ -220,6 +273,10 @@ Retrieve full content and progressive context for
 one or more sections. Use after `search` to read
 relevant results.
 
+> Identifiers can be shortened to a unique prefix
+> (like git short hashes). kbdb resolves the prefix
+> to the full 26-character kbid.
+
 **Options:**
 
 | Option | Default | Description |
@@ -260,6 +317,10 @@ kbdb content [options] <id>...
 Compose a rendered Markdown document from one or
 more kbid or docid identifiers.
 
+> Identifiers can be shortened to a unique prefix
+> (like git short hashes). kbdb resolves the prefix
+> to the full 26-character kbid.
+
 **Output:**
 
 ```json
@@ -561,7 +622,7 @@ kbdb mcp [options]
 ```
 
 Start a long-lived MCP server on stdio. Requires
-`--db <path>` or the `KBDB_DIR` environment
+`--db <path>` or the `KBDB_DB_DIR` environment
 variable. See the
 [MCP Server Guide](mcp-server.md) for setup and
 tool reference.
@@ -660,9 +721,13 @@ kbdb agent delete agent-code-reviewer
 
 ## Environment Variables
 
-| Variable | Used By | Description |
-|----------|---------|-------------|
-| `KBDB_DIR` | MCP mode | Fallback database directory path when `--db` is not specified. Not used in CLI mode (CLI defaults to cwd). |
+| Variable | Used By | Default | Description |
+|----------|---------|---------|-------------|
+| `KBDB_DB_DIR` | CLI + MCP | cwd (CLI) / none (MCP) | Fallback database directory path when `--db` is not specified. |
+| `KBDB_DAEMON_TIMEOUT` | CLI + MCP | `30000` | Per-request daemon timeout in milliseconds. Overridden by `--timeout-ms`. |
+| `KBDB_RETRY` | CLI + MCP | `1` | Max retries with daemon respawn on timeout. Overridden by `--retry`. |
+| `KBDB_LOG_LEVEL` | CLI + MCP + Daemon | `error` | Log verbosity: `debug`, `info`, `warning`, `error`. Overridden by `-v`/`-vv`/`-vvv`. |
+| `KBDB_NON_INTERACTIVE` | CLI + MCP + Daemon | off | Suppress interactive prompts. Accepts `1` or `true`. Overridden by `--non-interactive`. |
 
 ## Exit Codes
 
@@ -681,6 +746,12 @@ Before any data command (`learn`, `unlearn`,
    prompts to create one.
 2. **No `.kbdb` found + non-interactive** -- prints
    error and exits.
+
+The `--non-interactive` flag (or
+`KBDB_NON_INTERACTIVE=1`) forces the non-interactive
+path regardless of terminal type. This is useful in
+scripts, CI pipelines, and automated testing.
+
 3. **`.kbdb` found** -- runs a version gate before
    proceeding. If migration is needed, prints an
    error directing the user to run `kbdb db migrate`.
@@ -696,9 +767,11 @@ database creation.
   modules.
 - **stdout** contains only command output. No
   banners, progress bars, or decorative text.
-- **stderr** contains only errors (and deprecation
-  warnings for the legacy `--init`/`--migrate`
-  flags).
+- **stderr** contains log messages. Default level is
+  ERROR (errors only). Use `-v`/`-vv`/`-vvv` or
+  `KBDB_LOG_LEVEL` to increase verbosity. Log lines
+  are prefixed with per-level icons (✖⚠ℹ▸) and ANSI
+  colors.
 - Works on both Deno and Node.js (>= 20).
 - The `--init` and `--migrate` flags are deprecated.
   Use `kbdb db init` and `kbdb db migrate` instead.

package/docs/details/knowledge-base.md

@@ -0,0 +1,180 @@
+# Knowledge Base Guide
+
+How to build, search, and maintain a knowledge base
+with kbdb.
+
+## Overview
+
+kbdb stores your documents in a plain `.kbdb` folder on
+disk and makes them searchable. No external server
+required -- just a directory you can back up, version
+with git, or move between machines.
+
+## Getting Started
+
+**Create a knowledge base:**
+
+```sh
+kbdb db init --db ./my-project
+```
+
+**Feed it your docs:**
+
+```sh
+kbdb learn ./docs --db ./my-project
+```
+
+kbdb reads Markdown (`.md`) and plain text (`.txt`)
+files.
+It breaks them into sections, extracts keywords, and
+builds a search index.
+
+**Search:**
+
+```sh
+kbdb search "how does auth work" --db ./my-project
+```
+
+Results are ranked by relevance. Headings carry 2x
+weight, so well-titled sections surface first.
+
+## Importing Content
+
+### File import
+
+```sh
+kbdb learn ./docs --db ./my-project
+```
+
+Recursively imports all supported files. Each file is
+split into sections by heading.
+
+### Stdin import
+
+```sh
+echo "My note about auth." \
+   | kbdb learn - --title "Auth notes" --db ./my-project
+```
+
+### Tags and source tracking
+
+```sh
+kbdb learn ./docs --tags design,v2 --db ./my-project
+```
+
+Tags scope content for filtering. Use `--replace` to
+update existing sections from the same source instead
+of creating duplicates.
+
+## Search Modes
+
+| Flag | Mode | Description |
+|------|------|-------------|
+| _(default)_ | lexical | Keyword matching with stemming |
+| `--algo hybrid` | hybrid | Keyword + AI similarity |
+| `--algo vector` | vector | AI similarity only |
+
+Hybrid and vector modes work out of the box using a
+built-in TF-IDF embedding provider. For higher
+quality, configure an ONNX or API provider in
+`worker.toml`. See the [CLI Reference](cli.md) for
+provider configuration.
+
+When a query returns no exact matches, kbdb
+automatically loosens the search (OR instead of AND)
+and flags results with `relaxed: true`.
+
+## Reading Data
+
+Three commands retrieve stored content at different
+granularities:
+
+| Command | Purpose |
+|---------|---------|
+| `search` | Find sections by query; ranked snippets |
+| `recall` | Expand context around known kbids |
+| `content` | Compose Markdown from kbid/docid lists |
+
+**Recall depths:**
+
+| Depth | Returns |
+|-------|---------|
+| 0 | Section content only |
+| 1 | + parent documents, back-references |
+| 2 | + siblings, forward references |
+| 3 | + full text of referenced sections |
+
+**kbid prefixes:** You don't need to paste full 26-char
+identifiers. Short prefixes work (like git short
+hashes) as long as they're unambiguous.
+
+## Maintenance
+
+```sh
+kbdb check --db ./my-project    # integrity check
+kbdb gc --db ./my-project       # remove stale data
+kbdb rebuild --db ./my-project  # rebuild all indexes
+kbdb export --db ./my-project   # portable snapshot
+```
+
+## Sync & Backup
+
+The `.kbdb` directory is self-contained and portable.
+
+**Git-based sync:**
+
+```sh
+cd .kbdb
+git init && git add -A && git commit -m "snapshot"
+git remote add origin <url> && git push
+```
+
+On another machine:
+
+```sh
+git clone <url> .kbdb
+kbdb rebuild
+```
+
+The `.gitignore` inside `.kbdb` already excludes
+binary indexes, so git only tracks raw section and
+document files. After cloning, `rebuild` reconstructs
+the indexes.
+
+**Export/import:**
+
+```sh
+kbdb export --db ./project
+# copy the export directory to another machine
+kbdb db init --db ./project
+# re-import sections from the export
+```
+
+## Daemon Timeout and Retry
+
+All commands that talk to the worker daemon support
+two global flags:
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--timeout-ms <ms>` | `30000` | Per-request daemon timeout |
+| `--retry <n>` | `1` | Retries with daemon respawn on timeout |
+| `--non-interactive` | off | Suppress prompts; use defaults |
+
+These can also be set via environment variables.
+The flags take priority over env vars.
+
+| Variable | Description |
+|----------|-------------|
+| `KBDB_DAEMON_TIMEOUT` | Per-request daemon timeout (ms) |
+| `KBDB_RETRY` | Max retry count |
+| `KBDB_LOG_LEVEL` | Log verbosity: `debug`, `info`, `warning`, `error` |
+| `KBDB_NON_INTERACTIVE` | Suppress prompts (`1` or `true`) |
+
+## Deep Dives
+
+- [CLI Reference](cli.md) -- full command list
+- [Search and Ranking](search-and-ranking.md) --
+  scoring, stemming, hybrid search internals
+- [Storage Architecture](storage.md) -- file formats,
+  directory layout

package/docs/details/library-api.md

@@ -53,12 +53,37 @@ function createWorkerClient(
 
 | Name | Type | Description |
 |------|------|-------------|
-| `options.dbPath` | `string?` | Directory containing `.kbdb`. Defaults to `cwd()`. |
-| `options.contextPath` | `string?` | Absolute path to `.kbdb` directory. Skips discovery. |
+| `options.dbPath` | `string?` | Parent directory containing `.kbdb`. The client discovers `.kbdb` inside it. Defaults to `cwd()`. |
+| `options.contextPath` | `string?` | Direct absolute path to the `.kbdb` directory itself. Skips discovery. |
 | `options.autoSpawn` | `boolean?` | Auto-spawn daemon if not running. Default `true`. |
 | `options.connectTimeoutMs` | `number?` | Max ms to wait for daemon. Default `10000`. |
+| `options.requestTimeoutMs` | `number?` | Per-request timeout in milliseconds. Default `30000`. |
 | `options.workerScript` | `string?` | Path to worker daemon script. |
 
+#### `getOrCreateClient(options?)`
+
+Returns a cached `WorkerClient`, creating one on
+first call. Subsequent calls with the same
+`contextPath` return the existing connected client.
+Use this in long-lived processes to avoid spawning
+multiple daemon connections.
+
+```typescript
+function getOrCreateClient(
+   options?: WorkerClientOptions,
+): Promise<WorkerClient>;
+```
+
+#### `clearClientCache()`
+
+Disconnects and removes all cached clients created
+by `getOrCreateClient`. Call during shutdown to
+clean up connections.
+
+```typescript
+function clearClientCache(): void;
+```
+
 ---
 
 ### Search
@@ -79,6 +104,9 @@ search(
 | `params.query` | `string` | Search query string. |
 | `params.mode` | `'sections' \| 'documents' \| 'stats'?` | Entity type to return. Default `'sections'`. |
 | `params.limit` | `number?` | Maximum results to return. |
+| `params.offset` | `number?` | Zero-based result offset for pagination. |
+| `params.algo` | `'lexical' \| 'vector' \| 'hybrid'?` | Search algorithm. Default `'lexical'`. |
+| `params.relaxed` | `boolean?` | Use relaxed (OR) matching when strict returns no results. |
 
 ---
 
@@ -348,6 +376,15 @@ Destroys the IPC socket and closes the connection.
 disconnect(): void
 ```
 
+#### `isConnected`
+
+Read-only getter that returns `true` when the IPC
+socket is connected and healthy.
+
+```typescript
+get isConnected(): boolean;
+```
+
 ---
 
 ## Types
@@ -360,6 +397,7 @@ interface WorkerClientOptions {
    readonly contextPath?: string;
    readonly autoSpawn?: boolean;
    readonly connectTimeoutMs?: number;
+   readonly requestTimeoutMs?: number;
    readonly workerScript?: string;
 }
 ```
@@ -397,6 +435,8 @@ interface SearchResult {
    readonly heading: string | null;
    readonly type: string;
    readonly docids: readonly string[];
+   readonly score: number;
+   readonly confidence: number;
    readonly snippet: string;
    readonly matched_terms: readonly string[];
    readonly matched_fields: readonly string[];