@dikolab/kbdb 0.3.0 → 0.3.2

package/README.md

@@ -6,56 +6,47 @@ required.
 [GitLab](https://gitlab.com/tech-slaves/knowledge-base-db)
 | [NPM](https://www.npmjs.com/package/@dikolab/kbdb)
 | [JSR](https://jsr.io/@dikolab/kbdb)
-| [License: ISC](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/LICENSE)
+| [License: ISC](LICENSE)
 
 ---
 
 ## What is kbdb?
 
-Storing and searching documentation shouldn't
-require setting up a database server, tuning
-indexes, or managing infrastructure.
-
-**kbdb** is a knowledge base that stores your docs
-in a plain folder on disk. No server to install.
-No cloud account to create. It handles indexing
-and fast search out of the box -- just point it
-at your files and start searching.
-
-It runs anywhere Node.js or Deno runs. Import your
-Markdown files, and kbdb indexes them automatically.
+**kbdb** stores your docs in a plain folder on
+disk and makes them searchable. No server to
+install. No cloud account. Import your Markdown
+files, and kbdb indexes them automatically.
 Search returns ranked results instantly.
 
-kbdb uses **lexical search** (keyword matching with
-stemming and synonym expansion), not semantic
-embeddings. This is a deliberate tradeoff: fast,
-offline, deterministic, and zero embedding cost.
-When AI agents are the primary consumers, lexical
-search works well -- agents can issue multiple
-keyword queries, use boolean/phrase syntax, and
-expand vocabulary themselves. When an exact-match
-query returns nothing, kbdb automatically retries
-with relaxed OR matching so you always get the
-best available results. Write clear headings
-(they carry 2x ranking weight) and use specific
-terms for best recall.
-
-kbdb tracks **knowledge freshness** automatically.
-Re-learn a file and supersession replaces the old
-version instead of duplicating it. Near-duplicate
-detection warns you before content drifts.
-Integrity checks catch contradictions between
-versions of the same source. Confidence scores
-let agents tell strong matches from weak tail
-results.
-
-As a bonus, kbdb also works as an MCP server --
-giving AI agents like Claude a searchable second
-brain. Store skills, agent profiles, and
-documentation that your AI can query across
-sessions. Default skills (`query-guidance` and
-`memory-protocol`) teach agents how to search
-effectively and what to store.
+It runs anywhere Node.js or Deno runs, and also
+works as an [MCP](https://modelcontextprotocol.io/)
+server -- giving AI agents like Claude a searchable
+second brain that persists between sessions.
+
+**How search works:** kbdb searches by keywords --
+it matches your query against indexed terms, expands
+synonyms, and ranks results by relevance. This is
+the default **lexical** mode: fast, works offline,
+and gives consistent results every time.
+
+When an exact query finds nothing, kbdb automatically
+loosens the match so you still get the best available
+results. Headings count for more in ranking, so
+writing clear headings helps.
+
+Want smarter matching? kbdb supports **hybrid search**
+(`--algo hybrid`) that blends keyword matching with
+AI-powered similarity. This catches results that use
+different words for the same concept. Hybrid mode
+needs an embedding provider -- without one, kbdb
+sticks to keyword search.
+
+**Knowledge stays fresh:** Re-learn a file and kbdb
+replaces the old version automatically.
+Near-duplicate detection warns you before content
+drifts. Integrity checks catch contradictions.
+Confidence scores help agents tell strong matches
+from weak ones.
 
 ---
 
@@ -106,7 +97,7 @@ kbdb learn ./docs --db ./my-kb
 Point it at a folder of Markdown files. kbdb reads
 them, breaks them into sections, and builds a
 search index. Add `--tags design,v2` to tag
-sections for scoping, or `--replace` to supersede
+sections for scoping, or `--replace` to update
 existing sections from the same source.
 
 **3. Search**
@@ -115,11 +106,16 @@ existing sections from the same source.
 kbdb search "how does auth work" --db ./my-kb
 ```
 
-Results are ranked by relevance with confidence
-scores and snippets showing matched terms in
-context. Each result includes `matched_fields`
-(heading, body, code) and `created_at` timestamps.
-Use `--offset` to page through large result sets.
+Results are ranked by relevance with snippets
+showing where your terms matched. Use `--offset`
+to page through large result sets.
+
+To try hybrid search (keyword + AI similarity):
+
+```sh
+kbdb search "how does auth work" --algo hybrid \
+   --db ./my-kb
+```
 
 **4. Recall context**
 
@@ -140,25 +136,28 @@ referenced sections.
 
 - **Import** Markdown files and other documents,
   with tags and source tracking
-- **Supersede** outdated sections automatically
-  when re-learning from the same source
-- **Search** with ranked results, confidence
-  scores, snippets, and paginated output
-- **Relaxed fallback** -- automatic OR retry when
-  exact AND queries return nothing
+- **Smart updates** -- re-learning a file replaces
+  the old version instead of duplicating it
+- **Search** with three modes: keyword (default),
+  AI similarity, or hybrid (both combined)
+- **Auto-fallback** -- if your exact query finds
+  nothing, kbdb loosens the match automatically
 - **Recall** sections with progressive context --
   from a quick summary to full related content
-- **Export** the database as a portable snapshot
+- **Export and sync** -- snapshot your knowledge
+  base for backup or sharing across machines
 - **Skills** -- store reusable prompt templates
-  with placeholder arguments
+  with fill-in-the-blank arguments
 - **Agents** -- create AI agent profiles that
   combine a persona with skills
-- **Manage** documents, sections, and metadata
-- **Verify** database integrity, detect
-  contradictions, and clean up superseded data
+- **Auto-capture** -- the MCP server can
+  proactively suggest knowledge to store from
+  your conversations
+- **Verify** database integrity and clean up
+  stale data
 - **Rebuild** indexes if anything goes wrong
 
-See the [CLI Reference](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/details/cli.md)
+See the [CLI Reference](docs/details/cli.md)
 for the full command list and examples.
 
 ---
@@ -167,56 +166,22 @@ for the full command list and examples.
 
 kbdb runs as an
 [MCP](https://modelcontextprotocol.io/) server
-with 19 tools, so AI agents can search your docs,
+with 20 tools, so AI agents can search your docs,
 recall context progressively, export snapshots,
 store reusable skills, and build agent profiles --
 a second brain that persists between sessions.
 
-### Claude Code (CLI)
+### Setup
 
-Create `.mcp.json` in your project root:
+Add kbdb to your AI tool's MCP config. The setup
+is the same for Claude Code, VS Code, Cursor, and
+Claude Desktop -- just put the config in the right
+file for your tool.
 
-```json
-{
-   "mcpServers": {
-      "kbdb": {
-         "command": "npx",
-         "args": [
-            "-y",
-            "@dikolab/kbdb",
-            "mcp",
-            "--db",
-            "/path/to/.kbdb"
-         ]
-      }
-   }
-}
-```
-
-### Claude for VS Code
-
-Create `.vscode/mcp.json` in your project:
-
-```json
-{
-   "servers": {
-      "kbdb": {
-         "command": "npx",
-         "args": [
-            "-y",
-            "@dikolab/kbdb",
-            "mcp",
-            "--db",
-            "${workspaceFolder}/.kbdb"
-         ]
-      }
-   }
-}
-```
-
-### Claude Desktop
-
-Add to your `claude_desktop_config.json`:
+**Claude Code** -- `.mcp.json` in your project root.
+**VS Code / Cursor** -- `.vscode/mcp.json` in your
+project. **Claude Desktop** --
+`claude_desktop_config.json`.
 
 ```json
 {
@@ -235,13 +200,18 @@ Add to your `claude_desktop_config.json`:
 }
 ```
 
-### Other AI Agents
+For VS Code / Cursor, wrap in `{"servers": ...}`
+instead of `{"mcpServers": ...}`. Use
+`"${workspaceFolder}/.kbdb"` for the db path.
 
-Any MCP-compatible agent can connect. See the
-[MCP Server Guide](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/details/mcp-server.md) for
-setup instructions, Deno variants, environment
-variable configuration, and the full list of
-available tools.
+**Deno users:** replace `npx` / `-y` /
+`@dikolab/kbdb` with `deno` / `run` / `-A` /
+`jsr:@dikolab/kbdb/cli`.
+
+See the
+[MCP Server Guide](docs/details/mcp-server.md)
+for Deno examples, environment variable config,
+multiple knowledge bases, and the full tool list.
 
 ---
 
@@ -265,13 +235,12 @@ const results = await client.search({
    offset: 0,
 });
 
-// results = { items, total, offset, limit, has_more, relaxed }
 console.log(results.items);
 client.disconnect();
 ```
 
 See the
-[Library API Reference](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/details/library-api.md)
+[Library API Reference](docs/details/library-api.md)
 for the full API.
 
 ### Development Setup
@@ -292,7 +261,7 @@ A Docker setup is included with all build tools
 HOST_UMASK=$(umask) docker compose run --rm dev bash
 ```
 
-See the [Makefile](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/Makefile) for all available
+See the [Makefile](Makefile) for all available
 build targets.
 
 ### Contributing
@@ -307,17 +276,17 @@ build targets.
 
 ## Documentation
 
-- [Detailed Guides](https://gitlab.com/tech-slaves/knowledge-base-db/-/tree/main/docs/details) -- CLI
-  reference, MCP setup, search internals, storage
-  format, library API
-- [Release Notes](https://gitlab.com/tech-slaves/knowledge-base-db/-/tree/main/docs/release-notes)
-- [Architecture Overview](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/docs/overview.md)
+- [Detailed Guides](docs/details/) -- CLI reference,
+  MCP setup, search internals, storage format,
+  library API
+- [Release Notes](docs/release-notes/)
+- [Architecture Overview](docs/overview.md)
 
 ---
 
 ## License
 
-ISC License -- see [LICENSE](https://gitlab.com/tech-slaves/knowledge-base-db/-/blob/main/LICENSE) for details.
+ISC License -- see [LICENSE](LICENSE) for details.
 
 ---
 

package/dist/bench/constants/corpus-defaults.constant.d.ts

@@ -0,0 +1,3 @@
+import type { CorpusConfig } from '../typings/corpus-config.interface.ts';
+/** Default corpus generation settings. */
+export declare const CORPUS_DEFAULTS: CorpusConfig;

package/dist/bench/functions/format-report.function.d.ts

@@ -0,0 +1,11 @@
+import type { BenchSuiteResult } from '../typings/bench-suite-result.interface.ts';
+/**
+ * Formats benchmark results as a Markdown table.
+ *
+ * Renders a header row, a separator, and one data row per
+ * result in the suite.
+ *
+ * @param suite - completed benchmark suite to render
+ * @returns multi-line Markdown table string
+ */
+export declare function formatReport(suite: BenchSuiteResult): string;

package/dist/bench/functions/generate-corpus.function.d.ts

@@ -0,0 +1,12 @@
+import type { CorpusConfig } from '../typings/corpus-config.interface.ts';
+import type { CorpusSection } from '../typings/corpus-section.interface.ts';
+/**
+ * Generates a synthetic corpus of sections.
+ *
+ * Uses Zipfian word frequency distribution for realistic
+ * token patterns. Output is deterministic for the same seed.
+ *
+ * @param config - corpus generation parameters
+ * @returns array of generated corpus sections
+ */
+export declare function generateCorpus(config: CorpusConfig): CorpusSection[];

package/dist/bench/functions/measure-operation.function.d.ts

@@ -0,0 +1,14 @@
+import type { BenchResult } from '../typings/bench-result.interface.ts';
+/**
+ * Measures the duration of an async operation.
+ *
+ * Runs one warmup call then measures N iterations, returning
+ * the median duration. Uses `performance.now()` for timing.
+ *
+ * @param name - label for the operation
+ * @param corpusSize - corpus size associated with this run
+ * @param operation - the function to benchmark
+ * @param iterations - number of measured iterations (default 3)
+ * @returns benchmark result with median timing
+ */
+export declare function measureOperation(name: string, corpusSize: number, operation: () => Promise<void> | void, iterations?: number): Promise<BenchResult>;

package/dist/bench/functions/seeded-rng.function.d.ts

@@ -0,0 +1,11 @@
+/**
+ * Creates a deterministic pseudo-random number generator
+ * using a linear congruential algorithm.
+ *
+ * Each call to the returned function advances the internal
+ * state and returns the next value in [0, 1).
+ *
+ * @param seed - initial state for the LCG sequence
+ * @returns a stateful RNG function producing values in [0, 1)
+ */
+export declare function seededRng(seed: number): () => number;

package/dist/bench/functions/zipf-word.function.d.ts

@@ -0,0 +1,13 @@
+/**
+ * Selects a word index using Zipfian distribution.
+ *
+ * Higher-ranked words (lower indices) appear more frequently,
+ * modeling natural language word frequencies. Uses inverse CDF
+ * approximation: `floor(vocabSize * (1 - rng()^(1/s)))` where
+ * s ≈ 1.07 maps uniform random values to rank-biased indices.
+ *
+ * @param rng - stateful random number generator in [0, 1)
+ * @param vocabSize - total vocabulary size (exclusive upper bound)
+ * @returns integer index in [0, vocabSize)
+ */
+export declare function zipfWord(rng: () => number, vocabSize: number): number;

package/dist/bench/index.d.ts

@@ -0,0 +1,10 @@
+export type { CorpusConfig } from './typings/corpus-config.interface.ts';
+export type { CorpusSection } from './typings/corpus-section.interface.ts';
+export type { BenchResult } from './typings/bench-result.interface.ts';
+export type { BenchSuiteResult } from './typings/bench-suite-result.interface.ts';
+export { CORPUS_DEFAULTS } from './constants/corpus-defaults.constant.ts';
+export { seededRng } from './functions/seeded-rng.function.ts';
+export { zipfWord } from './functions/zipf-word.function.ts';
+export { generateCorpus } from './functions/generate-corpus.function.ts';
+export { measureOperation } from './functions/measure-operation.function.ts';
+export { formatReport } from './functions/format-report.function.ts';

package/dist/bench/typings/bench-result.interface.d.ts

@@ -0,0 +1,11 @@
+/** Result of a single benchmark run. */
+export interface BenchResult {
+    /** Operation name. */
+    readonly operation: string;
+    /** Corpus size. */
+    readonly corpusSize: number;
+    /** Duration in milliseconds. */
+    readonly durationMs: number;
+    /** Operations per second. */
+    readonly opsPerSec: number;
+}

package/dist/bench/typings/bench-suite-result.interface.d.ts

@@ -0,0 +1,10 @@
+import type { BenchResult } from './bench-result.interface.ts';
+/** Results from a complete benchmark suite. */
+export interface BenchSuiteResult {
+    /** Individual benchmark results. */
+    readonly results: BenchResult[];
+    /** Timestamp of the run. */
+    readonly timestamp: string;
+    /** System info. */
+    readonly system: string;
+}

package/dist/bench/typings/corpus-config.interface.d.ts

@@ -0,0 +1,11 @@
+/** Configuration for synthetic corpus generation. */
+export interface CorpusConfig {
+    /** Number of sections to generate. */
+    readonly count: number;
+    /** Seed for deterministic RNG. */
+    readonly seed: number;
+    /** Average tokens per section. */
+    readonly avgTokens: number;
+    /** Vocabulary size. */
+    readonly vocabSize: number;
+}

package/dist/bench/typings/corpus-section.interface.d.ts

@@ -0,0 +1,9 @@
+/** A generated section for benchmarking. */
+export interface CorpusSection {
+    /** Section heading. */
+    readonly heading: string;
+    /** Section body text. */
+    readonly body: string;
+    /** Source key for dedup. */
+    readonly sourceKey: string;
+}

package/dist/cli.cjs

@@ -125,6 +125,9 @@ function parseArgs(argv) {
           parseInt(argv[++i] ?? "", 10) || 0
         );
         break;
+      case "--algo":
+        options.algo = argv[++i];
+        break;
       case "--title":
         options.title = argv[++i];
         break;
@@ -254,7 +257,8 @@ async function runSearch(client, options) {
     query,
     mode: options.mode,
     limit: options.limit,
-    offset: options.offset
+    offset: options.offset,
+    algo: options.algo
   });
   return {
     items: results.items.map((r) => ({
@@ -2447,6 +2451,11 @@ function createToolDefinitions() {
           offset: {
             type: "number",
             description: "Number of results to skip for pagination (default: 0)"
+          },
+          algo: {
+            type: "string",
+            enum: ["lexical", "vector", "hybrid"],
+            description: "Search algorithm (default: lexical)"
           }
         },
         required: ["query"]
@@ -2860,6 +2869,27 @@ function createToolDefinitions() {
         idempotentHint: true,
         openWorldHint: false
       }
+    },
+    {
+      name: "auto-capture-review",
+      description: "Review and approve pending auto-captured knowledge before storage.",
+      inputSchema: {
+        type: "object",
+        properties: {
+          action: {
+            type: "string",
+            enum: ["list", "approve", "reject"],
+            description: "Action to perform on pending captures"
+          }
+        }
+      },
+      annotations: {
+        title: "Auto-Capture Review",
+        readOnlyHint: false,
+        destructiveHint: false,
+        idempotentHint: true,
+        openWorldHint: false
+      }
     }
   ];
 }
@@ -2933,7 +2963,7 @@ async function handleAgentCreate(client, args) {
     description
   });
   const allKbids = [metaResult.kbid, ...skills];
-  const docid = await client.groupSections(name, allKbids);
+  const docid = await client.groupSections(name, allKbids, "agent");
   return {
     content: [
       {
@@ -3146,6 +3176,8 @@ async function handleToolCall(client, name, args) {
         return await handleAgentGet(client, args);
       case "agent-delete":
         return await handleAgentDelete(client, args);
+      case "auto-capture-review":
+        return handleAutoCaptureReview();
       default:
         return {
           content: [
@@ -3174,7 +3206,8 @@ async function handleSearch2(client, args) {
     query: asString2(args["query"], ""),
     mode: args["mode"],
     limit: args["limit"],
-    offset: args["offset"]
+    offset: args["offset"],
+    algo: args["algo"]
   });
   const stripped = {
     ...results,
@@ -3230,7 +3263,8 @@ async function handleLearn2(client, args) {
     warnings: result.warnings ?? [],
     near_duplicates: (result.near_duplicates ?? []).map((d) => ({
       kbid: d.kbid,
-      similarity: d.similarity
+      similarity: d.similarity,
+      ...d.method !== void 0 ? { method: d.method } : {}
     }))
   };
   return {
@@ -3478,6 +3512,19 @@ async function handleSkillDelete(client, args) {
     ]
   };
 }
+function handleAutoCaptureReview() {
+  return {
+    content: [
+      {
+        type: "text",
+        text: JSON.stringify({
+          pending: [],
+          message: "No pending auto-captured knowledge entries."
+        })
+      }
+    ]
+  };
+}
 async function handleExport2(client, args) {
   const path = typeof args["path"] === "string" ? args["path"] : void 0;
   const result = await client.export(
@@ -3588,8 +3635,12 @@ function buildSkillsText(skillSections) {
 ${lines.join("\n")}`;
 }
 async function handleAgentPrompt(name, client) {
-  const docid = name.slice(6);
-  const doc = await client.retrieveDocument(docid);
+  const agentName = name.slice(6);
+  const agents = await client.listDocumentsByType("agent");
+  const doc = agents.find((a) => a.title === agentName);
+  if (!doc) {
+    throw new Error("agent not found: " + agentName);
+  }
   const sections = doc.sections;
   const metaSection = sections.at(0);
   const skillRefs = sections.slice(1);
@@ -3747,13 +3798,16 @@ var McpServer = class {
       input: process.stdin,
       terminal: false
     });
+    let pending = Promise.resolve();
     await new Promise((resolve4) => {
       rl.on("line", (line) => {
-        void this.dispatchLine(line);
+        pending = pending.then(() => this.dispatchLine(line));
       });
       rl.on("close", () => {
-        this.client.disconnect();
-        resolve4();
+        void pending.then(() => {
+          this.client.disconnect();
+          resolve4();
+        });
       });
     });
   }