@dikolab/kbdb 0.3.1 → 0.4.0

package/docs/details/README.md

@@ -4,6 +4,10 @@ In-depth guides and reference for @dikolab/kbdb.
 
 ## Guides
 
+- [Knowledge Base Guide](knowledge-base.md) --
+  Importing, searching, recall, sync & backup
+- [Agent Tooling Guide](agent-tooling.md) -- MCP,
+  skills, agents, library API, worker daemon
 - [CLI Reference](cli.md) -- Full command reference
   with examples
 - [MCP Server Setup](mcp-server.md) -- Configure

package/docs/details/agent-tooling.md

@@ -0,0 +1,122 @@
+# Agent Tooling Guide
+
+How to integrate kbdb with AI agents using the MCP
+server, skills, agent profiles, and the library API.
+
+## Overview
+
+kbdb runs as an
+[MCP](https://modelcontextprotocol.io/) server with
+20 tools, giving AI agents a searchable second brain
+that persists between sessions. Agents can search
+your knowledge base, recall context progressively,
+store reusable skills, and build agent profiles.
+
+## MCP Server Setup
+
+Add kbdb to your AI tool's MCP config. The setup is
+the same for Claude Code, VS Code, Cursor, and
+Claude Desktop -- just put the config in the right
+file for your tool.
+
+```json
+{
+   "mcpServers": {
+      "kbdb": {
+         "command": "npx",
+         "args": [
+            "-y", "@dikolab/kbdb",
+            "mcp", "--db", "/path/to/.kbdb"
+         ]
+      }
+   }
+}
+```
+
+**Deno users:** replace `npx` / `-y` /
+`@dikolab/kbdb` with `deno` / `run` / `-A` /
+`jsr:@dikolab/kbdb/cli`.
+
+See the [MCP Server Guide](mcp-server.md) for
+environment variable config, multiple knowledge
+bases, and the full 20-tool reference.
+
+## Skills
+
+Skills are reusable prompt templates with
+fill-in-the-blank arguments.
+
+```sh
+kbdb skill learn --name "code-review" \
+   --body "Review this code for: {{focus}}" \
+   --db ./my-kb
+```
+
+Skills are stored in the knowledge base and
+accessible to agents via MCP tools.
+
+## Agent Profiles
+
+Agent profiles combine a persona with a set of
+skills.
+
+```sh
+kbdb agent create --name "reviewer" \
+   --persona "You are a senior code reviewer." \
+   --skills "code-review,style-check" \
+   --db ./my-kb
+```
+
+## Auto-Capture
+
+The MCP server can proactively suggest knowledge
+to store from conversations. When enabled,
+auto-capture monitors tool call patterns and
+triggers capture suggestions based on configurable
+thresholds.
+
+## Library API
+
+Use kbdb programmatically in Node.js or Deno:
+
+```typescript
+import { createWorkerClient } from '@dikolab/kbdb';
+
+const client = await createWorkerClient({
+   contextPath: '/path/to/.kbdb',
+});
+
+const results = await client.search({
+   query: 'authentication',
+   limit: 10,
+});
+
+console.log(results.items);
+client.disconnect();
+```
+
+See the
+[Library API Reference](library-api.md)
+for the full API.
+
+## Worker Daemon
+
+kbdb runs a background worker daemon that handles
+indexing and search. The daemon auto-spawns on the
+first CLI or MCP command and stops when idle.
+
+```sh
+kbdb worker stop --db ./my-kb      # stop daemon
+kbdb worker restart --db ./my-kb   # restart daemon
+```
+
+The daemon writes a PID file and Unix socket under
+the system temp directory. After upgrading kbdb,
+restart the daemon to pick up the new version.
+
+## Deep Dives
+
+- [MCP Server Guide](mcp-server.md) -- full tool
+  list, environment config, multi-KB setup
+- [Library API Reference](library-api.md) --
+  programmatic access for custom integrations

package/docs/details/cli.md

@@ -12,7 +12,7 @@ the command line.
 npm install -g @dikolab/kbdb
 
 # Deno
-deno install
+deno install -Agf jsr:@dikolab/kbdb/cli
 ```
 
 ## Synopsis
@@ -48,6 +48,26 @@ kbdb <command> <subcommand> [options] ...[parameters]
 | Agents | `agent get` | Retrieve an agent by docid |
 | Agents | `agent delete` | Remove an agent |
 | Server | `mcp` | Start MCP server mode (stdio) |
+| Worker | `worker stop` | Stop the background worker daemon |
+| Worker | `worker restart` | Restart the background worker daemon |
+
+### Reading Data
+
+Three commands retrieve stored content at different
+granularities:
+
+| Command | Purpose |
+|---------|---------|
+| `search` | Find sections by query; returns ranked snippets |
+| `recall` | Expand full context around known kbids (post-search deep-dive) |
+| `content` | Compose rendered Markdown from kbid or docid identifiers |
+
+**When to use which:** Start with `search` to find
+relevant sections by keyword. Use `recall` to expand
+a search result into its full context -- parent
+documents, sibling sections, and references. Use
+`content` when you already know the exact kbid or
+docid and want rendered Markdown output.
 
 ## Global Options
 
@@ -55,20 +75,28 @@ kbdb <command> <subcommand> [options] ...[parameters]
 |--------|-------|---------|-------------|
 | `--help` | `-h` | | Print help text and exit |
 | `--version` | `-v` | | Print version and exit |
-| `--db <path>` | | cwd | Database directory path |
-| `--format <fmt>` | `-f` | `json` | Output format (`json`, `text`, `mcp`) |
+| `--db <path>` | | cwd | Database directory path (optional) |
+| `--format <fmt>` | `-f` | `json` (text when interactive) | Output format (`json`, `text`, `mcp`) |
+| `--timeout-ms <ms>` | | `30000` | Per-request daemon timeout in milliseconds |
+| `--retry <n>` | | `1` | Max retries with daemon respawn on timeout |
 
 ### `--db <path>`
 
 Set the path to the directory containing the
 `.kbdb` database folder. The actual database is
-always at `${path}/.kbdb`.
+always at `${path}/.kbdb`. This flag is optional.
 
-- **CLI mode**: defaults to the current working
-  directory.
-- **MCP mode**: falls back to the `KBDB_DIR`
+Resolution order:
+
+1. `--db` flag (highest priority)
+2. `KBDB_DB_DIR` environment variable
+3. Current working directory (CLI mode only)
+
+- **CLI mode**: resolves using the order above.
+- **MCP mode**: falls back to the `KBDB_DB_DIR`
   environment variable. If neither `--db` nor
-  `KBDB_DIR` is set, the server exits with an error.
+  `KBDB_DB_DIR` is set, the server exits with an
+  error.
 
 ## Search Options
 
@@ -80,8 +108,33 @@ These options apply to the `search` command.
 | `--limit <n>` | `-l` | `20` | positive integer | Maximum results returned |
 | `--offset <n>` | | `0` | non-negative integer | Zero-based result offset for pagination |
 | `--format <fmt>` | `-f` | `json` | `json`, `text`, `mcp` | Output format |
+| `--algo <algo>` | | `lexical` | `lexical`, `vector`, `hybrid` | Search algorithm |
 | `--content` | `-c` | off | boolean flag | Hydrate results with section content |
 
+### Search Algorithms
+
+| Algorithm | Description |
+|-----------|-------------|
+| `lexical` | Keyword matching with stemming and synonyms |
+| `vector` | AI-powered similarity search using embeddings (requires a configured embedding provider) |
+| `hybrid` | Combines keyword and similarity results for best coverage |
+
+### Configuring Hybrid Search
+
+Hybrid and vector search require an embedding provider.
+kbdb supports pluggable providers configured via the
+`EmbeddingConfig` interface:
+
+| Provider | Description |
+|----------|-------------|
+| `none` | No embeddings; hybrid and vector return an error |
+| `onnx` | Local ONNX model (no network required) |
+| `api` | Remote embedding API (OpenAI-compatible endpoint) |
+
+When no provider is configured, `--algo hybrid` and
+`--algo vector` return an error prompting you to
+configure an embedding provider.
+
 ### Search Modes
 
 | Mode | Description |
@@ -158,6 +211,9 @@ and ranking data.
 ]
 ```
 
+> kbid arguments accept unique prefixes -- kbdb
+> resolves them to full 26-character identifiers.
+
 ### `search` -- Query the Knowledge Base
 
 ```
@@ -211,6 +267,10 @@ Retrieve full content and progressive context for
 one or more sections. Use after `search` to read
 relevant results.
 
+> Identifiers can be shortened to a unique prefix
+> (like git short hashes). kbdb resolves the prefix
+> to the full 26-character kbid.
+
 **Options:**
 
 | Option | Default | Description |
@@ -251,6 +311,10 @@ kbdb content [options] <id>...
 Compose a rendered Markdown document from one or
 more kbid or docid identifiers.
 
+> Identifiers can be shortened to a unique prefix
+> (like git short hashes). kbdb resolves the prefix
+> to the full 26-character kbid.
+
 **Output:**
 
 ```json
@@ -552,7 +616,7 @@ kbdb mcp [options]
 ```
 
 Start a long-lived MCP server on stdio. Requires
-`--db <path>` or the `KBDB_DIR` environment
+`--db <path>` or the `KBDB_DB_DIR` environment
 variable. See the
 [MCP Server Guide](mcp-server.md) for setup and
 tool reference.
@@ -651,9 +715,11 @@ kbdb agent delete agent-code-reviewer
 
 ## Environment Variables
 
-| Variable | Used By | Description |
-|----------|---------|-------------|
-| `KBDB_DIR` | MCP mode | Fallback database directory path when `--db` is not specified. Not used in CLI mode (CLI defaults to cwd). |
+| Variable | Used By | Default | Description |
+|----------|---------|---------|-------------|
+| `KBDB_DB_DIR` | CLI + MCP | cwd (CLI) / none (MCP) | Fallback database directory path when `--db` is not specified. |
+| `KBDB_DAEMON_TIMEOUT` | CLI + MCP | `30000` | Per-request daemon timeout in milliseconds. Overridden by `--timeout-ms`. |
+| `KBDB_RETRY` | CLI + MCP | `1` | Max retries with daemon respawn on timeout. Overridden by `--retry`. |
 
 ## Exit Codes
 

package/docs/details/knowledge-base.md

@@ -0,0 +1,173 @@
+# Knowledge Base Guide
+
+How to build, search, and maintain a knowledge base
+with kbdb.
+
+## Overview
+
+kbdb stores your documents in a plain `.kbdb` folder on
+disk and makes them searchable. No external server
+required -- just a directory you can back up, version
+with git, or move between machines.
+
+## Getting Started
+
+**Create a knowledge base:**
+
+```sh
+kbdb db init --db ./my-project
+```
+
+**Feed it your docs:**
+
+```sh
+kbdb learn ./docs --db ./my-project
+```
+
+kbdb reads Markdown (`.md`) and plain text (`.txt`)
+files.
+It breaks them into sections, extracts keywords, and
+builds a search index.
+
+**Search:**
+
+```sh
+kbdb search "how does auth work" --db ./my-project
+```
+
+Results are ranked by relevance. Headings carry 2x
+weight, so well-titled sections surface first.
+
+## Importing Content
+
+### File import
+
+```sh
+kbdb learn ./docs --db ./my-project
+```
+
+Recursively imports all supported files. Each file is
+split into sections by heading.
+
+### Stdin import
+
+```sh
+echo "My note about auth." \
+   | kbdb learn - --title "Auth notes" --db ./my-project
+```
+
+### Tags and source tracking
+
+```sh
+kbdb learn ./docs --tags design,v2 --db ./my-project
+```
+
+Tags scope content for filtering. Use `--replace` to
+update existing sections from the same source instead
+of creating duplicates.
+
+## Search Modes
+
+| Flag | Mode | Description |
+|------|------|-------------|
+| _(default)_ | lexical | Keyword matching with stemming |
+| `--algo hybrid` | hybrid | Keyword + AI similarity |
+| `--algo vector` | vector | AI similarity only |
+
+Hybrid and vector modes need an embedding provider.
+Without one, they return an error prompting you to
+configure an embedding provider.
+See the [CLI Reference](cli.md) for provider
+configuration.
+
+When a query returns no exact matches, kbdb
+automatically loosens the search (OR instead of AND)
+and flags results with `relaxed: true`.
+
+## Reading Data
+
+Three commands retrieve stored content at different
+granularities:
+
+| Command | Purpose |
+|---------|---------|
+| `search` | Find sections by query; ranked snippets |
+| `recall` | Expand context around known kbids |
+| `content` | Compose Markdown from kbid/docid lists |
+
+**Recall depths:**
+
+| Depth | Returns |
+|-------|---------|
+| 0 | Section content only |
+| 1 | + parent documents, back-references |
+| 2 | + siblings, forward references |
+| 3 | + full text of referenced sections |
+
+**kbid prefixes:** You don't need to paste full 26-char
+identifiers. Short prefixes work (like git short
+hashes) as long as they're unambiguous.
+
+## Maintenance
+
+```sh
+kbdb check --db ./my-project    # integrity check
+kbdb gc --db ./my-project       # remove stale data
+kbdb rebuild --db ./my-project  # rebuild all indexes
+kbdb export --db ./my-project   # portable snapshot
+```
+
+## Sync & Backup
+
+The `.kbdb` directory is self-contained and portable.
+
+**Git-based sync:**
+
+```sh
+cd .kbdb
+git init && git add -A && git commit -m "snapshot"
+git remote add origin <url> && git push
+```
+
+On another machine:
+
+```sh
+git clone <url> .kbdb
+kbdb rebuild
+```
+
+The `.gitignore` inside `.kbdb` already excludes
+binary indexes, so git only tracks raw section and
+document files. After cloning, `rebuild` reconstructs
+the indexes.
+
+**Export/import:**
+
+```sh
+kbdb export --db ./project
+# copy the export directory to another machine
+kbdb db init --db ./project
+# re-import sections from the export
+```
+
+## Daemon Timeout and Retry
+
+All commands that talk to the worker daemon support
+two global flags:
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--timeout-ms <ms>` | `30000` | Per-request daemon timeout |
+| `--retry <n>` | `1` | Retries with daemon respawn on timeout |
+
+These can also be set via the `KBDB_DAEMON_TIMEOUT`
+and `KBDB_RETRY` environment variables. The flags
+take priority over env vars.
+
+## Deep Dives
+
+- [CLI Reference](cli.md) -- full command list
+- [Search and Ranking](search-and-ranking.md) --
+  scoring, stemming, hybrid search internals
+- [Storage Architecture](storage.md) -- file formats,
+  directory layout

package/docs/details/library-api.md

@@ -53,12 +53,37 @@ function createWorkerClient(
 
 | Name | Type | Description |
 |------|------|-------------|
-| `options.dbPath` | `string?` | Directory containing `.kbdb`. Defaults to `cwd()`. |
-| `options.contextPath` | `string?` | Absolute path to `.kbdb` directory. Skips discovery. |
+| `options.dbPath` | `string?` | Parent directory containing `.kbdb`. The client discovers `.kbdb` inside it. Defaults to `cwd()`. |
+| `options.contextPath` | `string?` | Direct absolute path to the `.kbdb` directory itself. Skips discovery. |
 | `options.autoSpawn` | `boolean?` | Auto-spawn daemon if not running. Default `true`. |
 | `options.connectTimeoutMs` | `number?` | Max ms to wait for daemon. Default `10000`. |
+| `options.requestTimeoutMs` | `number?` | Per-request timeout in milliseconds. Default `30000`. |
 | `options.workerScript` | `string?` | Path to worker daemon script. |
 
+#### `getOrCreateClient(options?)`
+
+Returns a cached `WorkerClient`, creating one on
+first call. Subsequent calls with the same
+`contextPath` return the existing connected client.
+Use this in long-lived processes to avoid spawning
+multiple daemon connections.
+
+```typescript
+function getOrCreateClient(
+   options?: WorkerClientOptions,
+): Promise<WorkerClient>;
+```
+
+#### `clearClientCache()`
+
+Disconnects and removes all cached clients created
+by `getOrCreateClient`. Call during shutdown to
+clean up connections.
+
+```typescript
+function clearClientCache(): void;
+```
+
 ---
 
 ### Search
@@ -79,6 +104,9 @@ search(
 | `params.query` | `string` | Search query string. |
 | `params.mode` | `'sections' \| 'documents' \| 'stats'?` | Entity type to return. Default `'sections'`. |
 | `params.limit` | `number?` | Maximum results to return. |
+| `params.offset` | `number?` | Zero-based result offset for pagination. |
+| `params.algo` | `'lexical' \| 'vector' \| 'hybrid'?` | Search algorithm. Default `'lexical'`. |
+| `params.relaxed` | `boolean?` | Use relaxed (OR) matching when strict returns no results. |
 
 ---
 
@@ -348,6 +376,15 @@ Destroys the IPC socket and closes the connection.
 disconnect(): void
 ```
 
+#### `isConnected`
+
+Read-only getter that returns `true` when the IPC
+socket is connected and healthy.
+
+```typescript
+get isConnected(): boolean;
+```
+
 ---
 
 ## Types
@@ -360,6 +397,7 @@ interface WorkerClientOptions {
    readonly contextPath?: string;
    readonly autoSpawn?: boolean;
    readonly connectTimeoutMs?: number;
+   readonly requestTimeoutMs?: number;
    readonly workerScript?: string;
 }
 ```
@@ -397,6 +435,8 @@ interface SearchResult {
    readonly heading: string | null;
    readonly type: string;
    readonly docids: readonly string[];
+   readonly score: number;
+   readonly confidence: number;
    readonly snippet: string;
    readonly matched_terms: readonly string[];
    readonly matched_fields: readonly string[];

package/docs/details/mcp-server.md

@@ -8,7 +8,7 @@ agents over stdio using JSON-RPC 2.0.
 
 Configure kbdb as an MCP server for your AI agent
 platform. The server requires a database path via
-`--db` or the `KBDB_DIR` environment variable.
+`--db` or the `KBDB_DB_DIR` environment variable.
 
 ### Claude Code (CLI)
 
@@ -131,9 +131,9 @@ deno run -A jsr:@dikolab/kbdb/cli mcp \
    --db /path/to/.kbdb
 ```
 
-### Using `KBDB_DIR` Environment Variable
+### Using `KBDB_DB_DIR` Environment Variable
 
-The `--db` flag can be replaced by the `KBDB_DIR`
+The `--db` flag can be replaced by the `KBDB_DB_DIR`
 environment variable in any configuration:
 
 ```json
@@ -143,20 +143,51 @@ environment variable in any configuration:
          "command": "npx",
          "args": ["-y", "@dikolab/kbdb", "mcp"],
          "env": {
-            "KBDB_DIR": "/path/to/.kbdb"
+            "KBDB_DB_DIR": "/path/to/.kbdb"
          }
       }
    }
 }
 ```
 
-Resolution order: `--db` flag, then `KBDB_DIR`
+Resolution order: `--db` flag, then `KBDB_DB_DIR`
 env var. If neither is set, the server exits with
 an error. The server does not fall back to the
 current working directory because the server's cwd
 is determined by the host application, not the
 user.
 
+### `KBDB_DAEMON_TIMEOUT`
+
+Per-request daemon timeout in milliseconds.
+Defaults to `30000` (30 seconds). Overridden by
+the `--timeout-ms` flag.
+
+### `KBDB_RETRY`
+
+Maximum number of retries with daemon respawn when
+a request times out. Defaults to `1`. Overridden
+by the `--retry` flag.
+
+These variables can be set in the `env` block of
+your MCP configuration:
+
+```json
+{
+   "mcpServers": {
+      "kbdb": {
+         "command": "npx",
+         "args": ["-y", "@dikolab/kbdb", "mcp"],
+         "env": {
+            "KBDB_DB_DIR": "/path/to/.kbdb",
+            "KBDB_DAEMON_TIMEOUT": "60000",
+            "KBDB_RETRY": "2"
+         }
+      }
+   }
+}
+```
+
 ### Multiple Knowledge Bases
 
 Register multiple server instances pointing to
@@ -196,7 +227,7 @@ daemon via JSON-RPC 2.0 over IPC.
 
 | Tool | Parameters | Description | Read-only | Destructive |
 |------|-----------|-------------|-----------|-------------|
-| `search` | `query` (string, required), `mode` (string), `limit` (number), `offset` (number), `content` (boolean) | Query the knowledge base by keywords. Returns ranked results with heading, snippet, and matched terms. Use multiple short keyword queries for best coverage. | yes | no |
+| `search` | `query` (string, required), `mode` (string), `algo` (string: lexical, vector, hybrid), `limit` (number), `offset` (number), `content` (boolean) | Query the knowledge base by keywords. Returns ranked results with heading, snippet, and matched terms. Use `algo` to select search algorithm (default: lexical). | yes | no |
 | `learn` | `content` (string, required), `type` (string), `title` (string), `description` (string), `docid` (string), `tags` (string[]), `replace` (boolean) | Import content into the knowledge base. Use for durable facts, decisions, and corrections. Use `replace` when updating existing knowledge. Attach `tags` for scoping. | no | no |
 | `unlearn` | `kbid` (string, required) | Remove a section by kb-id. Cascades deletion to index entries and ranking data. | no | yes |
 | `retrieve` | `kbid` (string), `docid` (string) | Fetch a specific section or document by identifier. At least one of `kbid` or `docid` required. | yes | no |
@@ -213,7 +244,7 @@ daemon via JSON-RPC 2.0 over IPC.
 
 | Tool | Parameters | Description | Read-only | Destructive |
 |------|-----------|-------------|-----------|-------------|
-| `skill-learn` | `name` (string, required), `description` (string, required), `body` (string, required), `arguments` (array) | Create a new skill definition. Body supports `{{argName}}` template placeholders. | no | no |
+| `skill-learn` | `name` (string, required), `description` (string), `body` (string, required) | Create or update a skill definition. Body supports `{{argName}}` template placeholders. | no | no |
 | `skill-list` | (none) | List all skill definitions. Returns array of summaries with kbid, name, description, argumentCount. | yes | no |
 | `skill-get` | `name` (string), `kbid` (string) | Retrieve a single skill by name or kb-id. At least one required; `kbid` takes precedence. | yes | no |
 | `skill-delete` | `kbid` (string, required) | Remove a skill by kb-id. Agent documents referencing this skill will have dangling references. | no | yes |
@@ -222,7 +253,7 @@ daemon via JSON-RPC 2.0 over IPC.
 
 | Tool | Parameters | Description | Read-only | Destructive |
 |------|-----------|-------------|-----------|-------------|
-| `agent-create` | `name` (string, required), `description` (string, required), `persona` (string, required), `skills` (string[], required) | Create an agent document with a persona and ordered skill references. | no | no |
+| `agent-create` | `name` (string, required), `description` (string), `persona` (string, required), `skills` (string[]) | Create an agent with a persona and optional skill references. | no | no |
 | `agent-list` | (none) | List all agent documents. Returns array of summaries with docid, name, description, skillCount. | yes | no |
 | `agent-get` | `docid` (string, required) | Retrieve an agent with resolved skill definitions. Deleted skills appear with `name: null`. | yes | no |
 | `agent-delete` | `docid` (string, required) | Remove an agent document. Referenced skill sections are preserved. | no | yes |