@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/goals/architecture.svg

@@ -1,198 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg"
-     width="780" height="770" viewBox="0 0 780 770"
-     font-family="system-ui, -apple-system, sans-serif">
-
-  <defs>
-    <marker id="arrowhead" viewBox="0 0 10 7"
-            refX="10" refY="3.5"
-            markerWidth="8" markerHeight="6" orient="auto">
-      <path d="M0,0 L10,3.5 L0,7 z" fill="#6b7280"/>
-    </marker>
-    <marker id="arrowhead-red" viewBox="0 0 10 7"
-            refX="10" refY="3.5"
-            markerWidth="8" markerHeight="6" orient="auto">
-      <path d="M0,0 L10,3.5 L0,7 z" fill="#dc2626"/>
-    </marker>
-  </defs>
-
-  <!-- Background -->
-  <rect width="780" height="770" fill="#ffffff"/>
-
-  <!-- =========================================== -->
-  <!-- Process boundary: kbdb executable            -->
-  <!-- =========================================== -->
-  <rect x="35" y="90" width="710" height="200" rx="12"
-        fill="none" stroke="#d1d5db" stroke-width="1.5"
-        stroke-dasharray="8,4"/>
-  <text x="50" y="105" font-size="11" fill="#9ca3af"
-        font-weight="500">kbdb executable</text>
-
-  <!-- =========================================== -->
-  <!-- Process boundary: worker daemon              -->
-  <!-- =========================================== -->
-  <rect x="35" y="355" width="710" height="285" rx="12"
-        fill="none" stroke="#d1d5db" stroke-width="1.5"
-        stroke-dasharray="8,4"/>
-  <text x="50" y="370" font-size="11" fill="#9ca3af"
-        font-weight="500">worker daemon (detached process)</text>
-
-  <!-- =========================================== -->
-  <!-- External: User / Script                      -->
-  <!-- =========================================== -->
-  <rect x="85" y="20" width="200" height="44" rx="8"
-        fill="#f8f9fa" stroke="#9ca3af" stroke-width="1.5"
-        stroke-dasharray="6,3"/>
-  <text x="185" y="47" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#495057">User / Script</text>
-
-  <!-- =========================================== -->
-  <!-- External: AI Agent Host                      -->
-  <!-- =========================================== -->
-  <rect x="495" y="20" width="200" height="44" rx="8"
-        fill="#f8f9fa" stroke="#9ca3af" stroke-width="1.5"
-        stroke-dasharray="6,3"/>
-  <text x="595" y="47" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#495057">AI Agent Host</text>
-
-  <!-- =========================================== -->
-  <!-- CLI Mode                                     -->
-  <!-- =========================================== -->
-  <rect x="85" y="110" width="200" height="52" rx="8"
-        fill="#dbeafe" stroke="#3b82f6" stroke-width="1.5"/>
-  <text x="185" y="134" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#1e40af">CLI Mode</text>
-  <text x="185" y="150" text-anchor="middle"
-        font-size="11" fill="#6b7280">cli.ts</text>
-
-  <!-- =========================================== -->
-  <!-- MCP Mode                                     -->
-  <!-- =========================================== -->
-  <rect x="495" y="110" width="200" height="52" rx="8"
-        fill="#dbeafe" stroke="#3b82f6" stroke-width="1.5"/>
-  <text x="595" y="134" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#1e40af">MCP Mode</text>
-  <text x="595" y="150" text-anchor="middle"
-        font-size="11" fill="#6b7280">mcp.ts (stdio)</text>
-
-  <!-- =========================================== -->
-  <!-- Worker Client                                -->
-  <!-- =========================================== -->
-  <rect x="240" y="220" width="300" height="52" rx="8"
-        fill="#d1fae5" stroke="#10b981" stroke-width="1.5"/>
-  <text x="390" y="244" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#065f46">Worker Client</text>
-  <text x="390" y="260" text-anchor="middle"
-        font-size="11" fill="#6b7280">src/shared/worker-client/</text>
-
-  <!-- =========================================== -->
-  <!-- Worker Daemon                                -->
-  <!-- =========================================== -->
-  <rect x="240" y="375" width="300" height="52" rx="8"
-        fill="#fed7aa" stroke="#f97316" stroke-width="1.5"/>
-  <text x="390" y="399" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#9a3412">Worker Daemon</text>
-  <text x="390" y="415" text-anchor="middle"
-        font-size="11" fill="#6b7280">kbdb-worker.ts</text>
-
-  <!-- =========================================== -->
-  <!-- kb-worker.wasm                               -->
-  <!-- =========================================== -->
-  <rect x="265" y="475" width="250" height="46" rx="8"
-        fill="#ede9fe" stroke="#8b5cf6" stroke-width="1.5"/>
-  <text x="390" y="503" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#5b21b6">kb-worker.wasm</text>
-
-  <!-- =========================================== -->
-  <!-- query-parser.wasm                            -->
-  <!-- =========================================== -->
-  <rect x="55" y="575" width="220" height="46" rx="8"
-        fill="#ede9fe" stroke="#8b5cf6" stroke-width="1.5"/>
-  <text x="165" y="603" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#5b21b6">query-parser.wasm</text>
-
-  <!-- =========================================== -->
-  <!-- fs-database.wasm                             -->
-  <!-- =========================================== -->
-  <rect x="505" y="575" width="220" height="46" rx="8"
-        fill="#ede9fe" stroke="#8b5cf6" stroke-width="1.5"/>
-  <text x="615" y="603" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#5b21b6">fs-database.wasm</text>
-
-  <!-- =========================================== -->
-  <!-- .kbdb/ directory                             -->
-  <!-- =========================================== -->
-  <rect x="505" y="700" width="220" height="50" rx="8"
-        fill="#f3f4f6" stroke="#6b7280" stroke-width="1.5"/>
-  <text x="615" y="722" text-anchor="middle"
-        font-size="13" font-weight="600" fill="#374151">.kbdb/</text>
-  <text x="615" y="738" text-anchor="middle"
-        font-size="11" fill="#6b7280">directory on disk</text>
-
-  <!-- =========================================== -->
-  <!-- Arrows                                       -->
-  <!-- =========================================== -->
-
-  <!-- User → CLI -->
-  <line x1="185" y1="64" x2="185" y2="108"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-  <text x="195" y="91" font-size="10" fill="#6b7280"
-        font-style="italic">args / stdout</text>
-
-  <!-- AI Agent → MCP -->
-  <line x1="595" y1="64" x2="595" y2="108"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-  <text x="605" y="84" font-size="10" fill="#6b7280"
-        font-style="italic">stdio</text>
-  <text x="605" y="96" font-size="10" fill="#6b7280"
-        font-style="italic">(JSON-RPC 2.0)</text>
-
-  <!-- CLI → Worker Client -->
-  <line x1="215" y1="162" x2="320" y2="218"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-
-  <!-- MCP → Worker Client -->
-  <line x1="565" y1="162" x2="460" y2="218"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-
-  <!-- Worker Client → Worker Daemon (IPC boundary) -->
-  <line x1="390" y1="272" x2="390" y2="373"
-        stroke="#dc2626" stroke-width="2"
-        marker-end="url(#arrowhead-red)"/>
-  <text x="400" y="318" font-size="11" fill="#dc2626"
-        font-weight="600">IPC (JSON-RPC 2.0)</text>
-  <text x="400" y="333" font-size="10" fill="#dc2626"
-        font-style="italic">Unix socket / named pipe</text>
-
-  <!-- Worker Daemon → kb-worker.wasm -->
-  <line x1="390" y1="427" x2="390" y2="473"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-  <text x="400" y="455" font-size="10" fill="#6b7280"
-        font-style="italic">loads</text>
-
-  <!-- kb-worker.wasm → query-parser.wasm -->
-  <line x1="315" y1="521" x2="210" y2="573"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-  <text x="232" y="543" font-size="10" fill="#6b7280"
-        font-style="italic">imports</text>
-
-  <!-- kb-worker.wasm → fs-database.wasm -->
-  <line x1="465" y1="521" x2="570" y2="573"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-  <text x="508" y="543" font-size="10" fill="#6b7280"
-        font-style="italic">imports</text>
-
-  <!-- fs-database.wasm → .kbdb/ -->
-  <line x1="615" y1="621" x2="615" y2="698"
-        stroke="#6b7280" stroke-width="1.5"
-        marker-end="url(#arrowhead)"/>
-  <text x="625" y="665" font-size="10" fill="#6b7280"
-        font-style="italic">host file I/O</text>
-
-</svg>

package/docs/goals/auto-capture.md

@@ -1,378 +0,0 @@
-# Auto-Capture -- Proactive Knowledge Capture Hooks
-
-*Server-side enforcement of knowledge capture that
-closes the gap between agent intent and reliable
-storage -- without requiring the agent to remember
-to act.*
-
-## Goal
-
-The `memory-protocol` skill (see [Skills](skills.md))
-teaches an agent what to store, but enforcement
-depends entirely on the agent choosing to act.
-Agents are unreliable at remembering to call `learn`
-after every significant exchange. Auto-capture closes
-this gap by having the MCP server proactively issue
-a sampling request to the host LLM, asking it to
-review recent context and identify storable knowledge.
-
-This is gap #2 from the
-[Opus 4.8 second brain review](../issues/2026-06-23-opus-48-second-brain-review.md):
-
-> A memory-protocol *skill* is intent, not
-> enforcement -- agents are unreliable at remembering
-> to write. Real second brains lean on auto-capture
-> hooks.
-
-Auto-capture is opt-in and default-disabled. It never
-writes to the database without the host LLM's
-explicit response identifying items to store.
-
-## MCP Sampling Approach
-
-Auto-capture uses the MCP `sampling/createMessage`
-capability. When a trigger fires, the kbdb MCP server
-sends a `sampling/createMessage` request to the host
-asking it to identify storable knowledge from the
-current conversation context.
-
-If the host's `initialize` response does not include
-a `sampling` capability entry, auto-capture is
-silently disabled for that session regardless of
-configuration.
-
-### Sampling Prompt Template
-
-```
-Review the recent conversation context. Identify any
-durable facts, decisions, corrections, architecture
-choices, or configuration details that should be
-stored in the knowledge base for future sessions.
-
-For each item, provide:
-- title: short heading
-- content: the fact or decision
-- tags: relevant topic tags
-
-Respond with JSON: {"items": [...]} or {"items": []}
-if nothing is worth storing. Apply the same criteria
-as the memory-protocol skill: store durable facts,
-not transient context.
-```
-
-The server sets `maxTokens` to a fixed ceiling (e.g.
-2048) to bound cost. The sampling request carries no
-`systemPrompt` -- context comes from the host's own
-conversation window.
-
-### Sampling Response Schema
-
-The host LLM must respond with a JSON object at the
-top level. The server parses the text content of the
-sampling response and extracts:
-
-```json
-{
-   "items": [
-      {
-         "title": "Postgres chosen for JSONB support",
-         "content": "We chose Postgres because JSONB
-            support simplifies document storage.",
-         "tags": ["architecture", "database"]
-      }
-   ]
-}
-```
-
-When `items` is empty or the response cannot be parsed
-as valid JSON, the server takes no action and logs a
-`debug`-level notification.
-
-Each item in `items` is stored via `addSection` with
-`sectionType: 'text/markdown'` and the
-`auto-capture` tag appended to `tags`.
-
-## Trigger Points
-
-Three independent trigger mechanisms fire a capture
-review. All three can be active simultaneously.
-
-### 1. Tool-Call Count Trigger
-
-The MCP server maintains a per-session counter of
-tool invocations. When the count reaches a multiple
-of `trigger_interval`, a capture review fires.
-
-- The counter increments on every `tools/call`
-  dispatch, including read-only tools.
-- The counter does not increment during the capture
-  review itself (no recursion).
-- The review fires after the triggering tool call
-  returns its result to the agent, not before.
-
-Default: every 10 tool calls.
-
-### 2. Explicit Tool Trigger
-
-The `auto-capture-review` tool is always registered,
-regardless of whether auto-capture is enabled in
-`worker.toml`. Agents or users can call it at any
-time to request an immediate capture review.
-
-When `dry_run: true` is passed, the review runs but
-no sections are written. See [MCP Surface](#mcp-surface).
-
-### 3. Session-End Trigger
-
-When the MCP connection closes gracefully (stdin EOF
-or explicit disconnect), one final capture review
-runs before the server exits.
-
-The session-end trigger fires only when
-`auto_capture.enabled` is `true`. It does not fire
-when the connection is lost abruptly (SIGKILL, socket
-drop).
-
-## Configuration
-
-Auto-capture is configured in `worker.toml` under the
-`[auto_capture]` table:
-
-```toml
-[auto_capture]
-enabled = false          # opt-in, default off
-trigger_interval = 10    # tool calls between reviews
-max_captures_per_session = 20  # rate limit
-dry_run = false          # log without writing
-```
-
-| Field | Type | Default | Description |
-|-------|------|---------|-------------|
-| `enabled` | bool | `false` | Enable auto-capture for all MCP sessions |
-| `trigger_interval` | integer | `10` | Tool calls between automatic reviews |
-| `max_captures_per_session` | integer | `20` | Maximum sections stored per session |
-| `dry_run` | bool | `false` | Log what would be stored; write nothing |
-
-When `enabled` is `false`, only the `auto-capture-review`
-tool remains active. The tool-call count and
-session-end triggers are suppressed.
-
-## Guard Rails
-
-### Rate Limiting
-
-The server tracks the number of sections stored
-during the current MCP session. When
-`max_captures_per_session` is reached, further
-automatic reviews are skipped. The
-`auto-capture-review` tool still runs but returns a
-rate-limit message instead of executing the review.
-
-The counter resets when the MCP session ends. It is
-not persisted across sessions.
-
-### Dry-Run Mode
-
-When `dry_run` is `true` (either from `worker.toml`
-or the `auto-capture-review` tool parameter), the
-server executes the full review -- sends the sampling
-request, parses the response, and evaluates items --
-but does not call `addSection`. Instead, it emits a
-`notifications/message` at `info` level describing
-what would have been stored.
-
-Dry-run mode is useful for tuning `trigger_interval`
-and verifying that the sampling prompt produces
-reasonable output before enabling writes.
-
-### Deduplication
-
-Before storing each item from the sampling response,
-the server searches the database for existing sections
-with similar content. If any existing section has a
-Jaccard similarity >= 0.85 to the candidate item, the
-item is skipped and logged at `debug` level.
-
-This uses the same near-duplicate detection that the
-`learn` tool uses, reflected in the `near_duplicates`
-field of `AddSectionResult`.
-
-### No Recursion
-
-The tool-call counter does not increment during an
-active capture review. The `addSection` calls made
-by auto-capture do not trigger a nested review.
-
-### Agent Override
-
-Auto-capture never overrides an agent's explicit
-decision not to store something. If an agent
-calls `unlearn` to remove a section, auto-capture
-will not re-add it in the same session (the content's
-kbid is added to a per-session exclusion set after
-removal).
-
-## Interaction with the memory-protocol Skill
-
-| Aspect | memory-protocol | Auto-capture |
-|--------|-----------------|--------------|
-| Audience | Agent (reads skill text) | MCP server (server-side) |
-| Mechanism | Teaches filtering criteria | Sends sampling request |
-| Enforcement | Agent must choose to act | Server acts automatically |
-| Criteria | Defined in skill body | Sampling prompt mirrors skill |
-| Override | Agent decides | Agent's `unlearn` is respected |
-
-Both mechanisms apply the same filtering criteria --
-store durable facts, skip transient context. They are
-complementary: `memory-protocol` is agent-facing
-intent; auto-capture is server-side enforcement.
-
-## CLI Surface
-
-The `--auto-capture` flag on `kbdb mcp` enables
-auto-capture for the current session, overriding the
-`enabled` value in `worker.toml`:
-
-```
-kbdb mcp --db <path> [--auto-capture]
-```
-
-The flag does not accept a value -- its presence sets
-`enabled: true`. It does not override other fields
-(`trigger_interval`, `max_captures_per_session`,
-`dry_run`), which continue to come from `worker.toml`
-or their defaults.
-
-## MCP Surface
-
-### auto-capture-review Tool
-
-Triggers an immediate capture review regardless of
-whether auto-capture is enabled in configuration.
-
-**Parameters:**
-
-| Name | Type | Required | Description |
-|------|------|----------|-------------|
-| `dry_run` | boolean | no | When `true`, evaluate but do not write (default: `false`) |
-
-**Returns:**
-
-```json
-{
-   "captured": 2,
-   "items": [
-      {
-         "title": "Postgres chosen for JSONB support",
-         "kbid": "a1b2c3d4e5f6g7h8j9k0l1m2n3"
-      }
-   ],
-   "skipped": 1,
-   "dry_run": false
-}
-```
-
-| Field | Description |
-|-------|-------------|
-| `captured` | Number of sections written (0 when `dry_run: true`) |
-| `items` | Array of `{ title, kbid }` for each section stored |
-| `skipped` | Items returned by LLM but not stored (duplicates, parse errors, rate limit) |
-| `dry_run` | Reflects the effective dry-run mode used |
-
-When sampling is not supported by the host, the tool
-returns `isError: true` with message:
-`"Host does not support sampling; auto-capture-review
-unavailable"`.
-
-**Tool annotation:**
-
-| Attribute | Value |
-|-----------|-------|
-| `readOnlyHint` | `false` |
-| `destructiveHint` | `false` |
-| `idempotentHint` | `false` |
-| `openWorldHint` | `false` |
-
-### Capability Advertisement
-
-When `sampling` is present in the host's `initialize`
-capabilities, the server advertises the `sampling`
-capability in its own `initialize` response. When
-`sampling` is absent, the capability is omitted from
-the response and auto-capture is silently disabled.
-
-## Response Flow
-
-```
-Agent uses kbdb tools
-        |
-        v
-Tool call count reaches N
-        |
-        v
-Server sends sampling/createMessage
-        |
-        v
-Host LLM reviews conversation context
-        |
-        v
-Returns {"items": [...]} or {"items": []}
-        |
-        v
-Server deduplicates against existing sections
-        |
-        v
-Server stores each new item via addSection
-(with "auto-capture" tag appended)
-        |
-        v
-notifications/message logged (info level)
-```
-
-When the host does not support sampling, or when
-`enabled` is `false` and the trigger is automatic
-(not the explicit tool), the flow terminates after
-the trigger fires with a `debug`-level log and no
-sampling request is sent.
-
-## Tagging
-
-Every section stored by auto-capture receives the
-`auto-capture` tag appended to any tags the sampling
-response supplied. This makes auto-captured sections
-filterable and identifiable in search results and
-integrity checks.
-
-Example stored section tags when the sampling
-response supplies `["architecture", "database"]`:
-
-```
-["architecture", "database", "auto-capture"]
-```
-
-## Design Constraints
-
-- **Opt-in by default.** `enabled` is `false` in
-  `worker.toml` until explicitly set. No sections are
-  written without explicit configuration or the
-  `--auto-capture` flag.
-- **Sampling required.** Auto-capture cannot function
-  without MCP sampling support. It is silently
-  disabled when the host omits the `sampling`
-  capability, with no error and no user-visible
-  change to other tools.
-- **No recursion.** Auto-capture calls do not
-  increment the tool-call counter and do not trigger
-  nested reviews.
-- **Agent override respected.** Sections removed by
-  an explicit `unlearn` call are not re-added by
-  auto-capture within the same session.
-- **Rate-limited.** `max_captures_per_session`
-  prevents runaway storage from a misconfigured
-  trigger interval or a verbose LLM response.
-- **Traceability.** Auto-captured sections always
-  carry the `auto-capture` tag so they can be
-  distinguished from manually learned content.
-- **Dry-run available.** `dry_run` mode allows
-  operators to verify sampling output and tune
-  configuration before enabling writes.