@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/modules/rust/shared/term/module.md

@@ -1,25 +0,0 @@
-# term -- Normalized Search Term
-
-*Newtype wrapper for normalized, stemmed search terms
-used as keys in inverted indexes and query plans.*
-
-## Source
-
-`engine/crates/shared/src/term/`
-
-## Responsibility
-
-Defines the `Term` newtype with ordering support for
-B+ tree keys. Terms are the atomic unit of the
-inverted index -- every index lookup, ranking
-computation, and query plan operates on `Term`
-values.
-
-See [Query Parser -- Tokenization Pipeline](
-../../../../goals/query-parser.md) for the pipeline
-that produces terms.
-
-## Exports
-
-- **[types.md](types.md)** -- `Term` newtype,
-  `from_stemmed`, `TermError`.

package/docs/modules/rust/shared/term/types.md

@@ -1,139 +0,0 @@
-# types::term -- Normalized Search Term
-
-*Newtype wrapper around a normalized, stemmed term
-used as the key in inverted indexes and query plans.*
-
-## File
-
-`engine/crates/shared/src/term/term.rs`
-
-## Overview
-
-A `Term` represents a single searchable token after
-the full tokenization pipeline has run: Unicode
-normalization, case folding, stemming, and stop word
-removal. Terms are the atomic unit of the inverted
-index -- every index lookup, every ranking
-computation, and every query plan operates on `Term`
-values.
-
-See [Query Parser -- Tokenization Pipeline](
-../../../../../goals/query-parser.md) for the pipeline
-that produces terms.
-
-## Type Definition
-
-```rust
-#[derive(Clone, Debug, PartialEq, Eq,
-         Hash, Ord, PartialOrd)]
-pub struct Term(String);
-```
-
-The inner `String` is private. Construction goes
-through `Term::new`, which validates the invariants.
-
-## Invariants
-
-A valid `Term`:
-
-- Is non-empty.
-- Contains only lowercase characters (post-case-fold).
-- Contains no whitespace.
-- Is not a stop word (stop words are filtered before
-  `Term` construction).
-- May be a stemmed form (e.g. `"comput"` from
-  `"computing"`).
-
-These invariants are enforced at construction time
-by the tokenization pipeline. Code that receives a
-`Term` can rely on them without re-checking.
-
-## Construction
-
-```rust
-impl Term {
-   pub fn new(s: &str) -> Result<Self, TermError>;
-
-   pub fn from_stemmed(s: String) -> Self;
-}
-```
-
-### `Term::new`
-
-Validates the input string against the term
-invariants. Returns `TermError::Empty` for empty
-strings, `TermError::ContainsWhitespace` if
-whitespace is found.
-
-This is the safe constructor for terms created
-outside the pipeline (e.g. from deserialized index
-data).
-
-### `Term::from_stemmed`
-
-Unchecked constructor for terms produced by the
-stemming stage. The pipeline guarantees the
-invariants, so validation is skipped. This is a
-performance path -- only the tokenization pipeline
-should call it.
-
-## Methods
-
-```rust
-impl Term {
-   pub fn as_str(&self) -> &str;
-
-   pub fn len(&self) -> usize;
-}
-```
-
-### `as_str`
-
-Returns the inner string slice. Used for index
-lookups, display, and serialization.
-
-### `len`
-
-Returns the byte length. Used by TF-IDF weighting
-and B+ tree key sizing.
-
-## Trait Implementations
-
-| Trait        | Purpose                          |
-|--------------|----------------------------------|
-| `Clone`      | Terms are stored in multiple structures (index, cache, results) |
-| `Debug`      | Diagnostic output                |
-| `PartialEq` / `Eq` | Term matching in index lookups |
-| `Hash`       | HashMap keys (inverted index, query cache) |
-| `Ord` / `PartialOrd` | B+ tree key ordering in `fs-database` |
-| `Display`    | Formats as the raw term string   |
-| `Encode`     | Binary serialization for shared memory (see [encode](../encode/traits.md)) |
-
-`Ord` **is** derived for `Term` (unlike `KbId`)
-because the B+ tree index requires ordered keys for
-range queries and efficient lookups.
-
-## Error Type
-
-```rust
-#[derive(Debug)]
-pub enum TermError {
-   Empty,
-   ContainsWhitespace,
-}
-```
-
-## Testing
-
-Unit tests:
-
-- `new` accepts lowercase, non-empty, no-whitespace
-  strings.
-- `new` rejects empty strings and strings with
-  spaces, tabs, or newlines.
-- `from_stemmed` produces a valid term without
-  validation overhead.
-- Ordering: terms sort lexicographically
-  (`"apple" < "banana"`).
-- `Encode` round-trip: encode then decode produces
-  an equal `Term`.

package/docs/modules/typescript/cli.md

@@ -1,131 +0,0 @@
-# src/cli.ts -- CLI and MCP Executable
-
-*Thin entry point that dispatches to CLI mode or MCP
-server mode based on the first argument.*
-
-## File
-
-`src/cli.ts`
-
-## Built To
-
-| Format | Output           | Notes          |
-|--------|------------------|----------------|
-| ESM    | `dist/cli.mjs`   | shebang, npm bin |
-| CJS    | `dist/cli.cjs`   | shebang        |
-| `.d.ts`| `dist/cli.d.ts`  |                |
-
-The npm `bin` entry `kbdb` points to `dist/cli.mjs`.
-
-## Run Modes
-
-```
-kbdb mcp [--db <path>]             → MCP server mode
-kbdb <command> [options]            → CLI mode
-kbdb --non-interactive <command>    → non-interactive
-```
-
-The executable inspects its first argument. If `mcp`,
-it starts as a long-lived MCP server. Otherwise, it
-runs a single CLI command and exits.
-
-## Execution Flow
-
-### CLI Mode
-
-```
-src/cli.ts
-  │
-  ├─ import detectRuntime from shared/runtime/
-  ├─ import getArgs from shared/runtime/
-  ├─ import parseArgs from shared/cli/
-  ├─ import createWorkerClient from
-  │    shared/worker-client/
-  │
-  ├─ detect runtime (Deno / Node.js)
-  ├─ argv = getArgs()
-  ├─ options = parseArgs(argv)
-  │
-  ├─ if options.init:
-  │    └─ ensureDb(options.dbPath)
-  │
-  ├─ if options.command ≠ 'mcp':
-  │    ├─ client = await createWorkerClient(options)
-  │    ├─ if options.migrate:
-  │    │    └─ run database migration
-  │    ├─ match command:
-  │    │    ├─ learn    → runLearn(client, options)
-  │    │    ├─ unlearn  → runUnlearn(client, options)
-  │    │    ├─ search   → runSearch(client, options)
-  │    │    ├─ content  → runContent(client, options)
-  │    │    ├─ check    → runCheck(client, options)
-  │    │    ├─ gc       → runGc(client, options)
-  │    │    └─ rebuild  → runRebuild(client, options)
-  │    ├─ formatOutput(result, options.format)
-  │    ├─ print to stdout
-  │    └─ client.disconnect()
-  │
-  └─ exit
-```
-
-### MCP Mode
-
-```
-src/cli.ts
-  │
-  ├─ argv[0] === 'mcp'
-  ├─ resolve --db flag or KBDB_DB_DIR env or cwd
-  ├─ resolve --non-interactive or KBDB_NON_INTERACTIVE
-  ├─ client = await createWorkerClient({ targetDir })
-  ├─ server = createMcpServer(client)
-  ├─ server.listen()   ← stdio, blocks until closed
-  └─ client.disconnect()
-```
-
-## Imports
-
-```ts
-import {
-   detectRuntime,
-} from './src/shared/runtime/functions/detect-runtime.function.ts';
-import {
-   getArgs,
-} from './src/shared/runtime/functions/get-args.function.ts';
-import {
-   parseArgs,
-} from './src/shared/cli/functions/parse-args.function.ts';
-import {
-   createWorkerClient,
-} from './src/shared/worker-client/functions/create-worker-client.function.ts';
-import {
-   createMcpServer,
-} from './src/shared/mcp/functions/create-mcp-server.function.ts';
-```
-
-## No Business Logic
-
-This file contains **only** runtime detection, arg
-parsing dispatch, and mode selection. All logic lives
-in the shared sub-modules:
-
-| Concern           | Module                    |
-|-------------------|---------------------------|
-| Arg parsing       | `shared/cli/`             |
-| CLI commands      | `shared/cli/`             |
-| Output formatting | `shared/cli/`             |
-| MCP server        | `shared/mcp/`             |
-| IPC to daemon     | `shared/worker-client/`   |
-| Runtime detection | `shared/runtime/`         |
-
-## Dual Runtime
-
-Works on both Deno and Node.js. Runtime-specific
-code is handled by `shared/runtime/` and
-`shared/platform/`.
-
-- **Deno**: `deno run --allow-all src/cli.ts learn ./docs`
-- **Node**: `node dist/cli.mjs learn ./docs`
-- **npm bin**: `kbdb learn ./docs`
-
-See [CLI](../../goals/cli.md) for the full behavioral
-specification.

package/docs/modules/typescript/kbdb-worker.md

@@ -1,150 +0,0 @@
-# src/worker.ts -- Worker Daemon Executable
-
-*Thin entry point for the detached daemon process that
-loads WASM modules and serves IPC requests.*
-
-## File
-
-`src/worker.ts`
-
-## Built To
-
-| Format | Output                   | Notes   |
-|--------|--------------------------|---------|
-| ESM    | `dist/worker.mjs`        | shebang |
-| CJS    | `dist/worker.cjs`        | shebang |
-| `.d.ts`| `dist/worker.d.ts`       |         |
-
-Not registered as an npm `bin` -- this executable is
-not invoked directly by users. It is spawned as a
-detached child process by `shared/worker-client/`.
-
-## How It Is Spawned
-
-The worker-client module spawns this executable when
-no running daemon is found for the target database:
-
-**Node.js:**
-
-```ts
-spawn(
-   'node',
-   ['kbdb-worker.js', contextPath, '--log-level', logLevel],
-   { detached: true, stdio: 'ignore' },
-).unref();
-```
-
-**Deno:**
-
-```ts
-new Deno.Command('deno', {
-   args: [
-      'run', '--allow-all',
-      'src/worker.ts', contextPath,
-      '--log-level', logLevel,
-   ],
-   stdin: 'null',
-   stdout: 'null',
-   stderr: 'piped',
-}).spawn().unref();
-```
-
-The first command-line argument is the **context path**
--- the absolute path to the `.kbdb` database directory.
-The optional `--log-level <level>` argument controls
-daemon log verbosity. It is forwarded from the
-`KBDB_LOG_LEVEL` environment variable by the spawner.
-
-The `KBDB_NON_INTERACTIVE` environment variable is
-inherited by the daemon through normal process
-environment inheritance (no explicit forwarding
-needed).
-
-## Execution Flow
-
-```
-src/worker.ts
-  │
-  ├─ import startDaemon from shared/worker-daemon/
-  │
-  ├─ read contextPath from argv[0]
-  ├─ validate contextPath exists
-  ├─ parse --log-level from argv (if present)
-  ├─ configure log level
-  │    (CLI arg → KBDB_LOG_LEVEL env → ERROR default)
-  │
-  └─ await startDaemon(contextPath)
-       │
-       ├─ readConfig(contextPath)
-       │    └─ parse .kbdb/worker.toml
-       │
-       ├─ loadWasmModules(config)
-       │    ├─ allocate shared WebAssembly.Memory
-       │    ├─ compile query-parser.wasm (standalone)
-       │    ├─ compile fs-database.wasm (standalone)
-       │    ├─ compile default-embedding.wasm
-       │    │    (standalone, no host imports)
-       │    └─ compile kb-worker.wasm
-       │         (imports query-parser + fs-database)
-       │
-       ├─ write PID file
-       │    └─ kb-worker-{ctx}.pid in tmpdir
-       │
-       ├─ createIpcServer(ctx, handler)
-       │    └─ Unix socket or named pipe
-       │
-       ├─ listen for connections
-       │    └─ JSON-RPC 2.0, newline-delimited
-       │
-       └─ on idle timeout or signal:
-            ├─ stop accepting connections
-            ├─ wait for in-flight requests
-            ├─ purge cache
-            ├─ remove PID file
-            ├─ remove IPC socket
-            └─ exit
-```
-
-## Imports
-
-```ts
-import {
-   startDaemon,
-} from './src/shared/worker-daemon/functions/start-daemon.function.ts';
-```
-
-## No Business Logic
-
-This file contains **only** argument reading and a
-single function call. All daemon logic lives in
-`shared/worker-daemon/`:
-
-| Concern            | Module                      |
-|--------------------|-----------------------------|
-| WASM loading       | `shared/worker-daemon/`     |
-| IPC server         | `shared/worker-daemon/`     |
-| Config reading     | `shared/worker-daemon/`     |
-| Cache management   | WASM (`kb-worker.wasm`)     |
-| Query orchestration| WASM (`kb-worker.wasm`)     |
-| Index operations   | WASM (`fs-database.wasm`)   |
-| Tokenization       | WASM (`query-parser.wasm`)  |
-| TF-IDF embedding   | WASM (`default-embedding.wasm`) |
-| Platform helpers   | `shared/platform/`          |
-
-## Lifecycle
-
-1. Spawned by worker-client as detached child process
-2. Loads and compiles 4 WASM modules (once)
-3. Serves IPC requests from any connected client
-4. Shuts down after idle timeout (`timeout_ms`) or
-   OS signal (SIGTERM, SIGINT)
-5. Cleans up PID file and IPC socket on exit
-
-Multiple daemons can run simultaneously on the same
-machine, each serving a different database directory.
-Each daemon is identified by its **context ID**
-(SHA-256 of the database path, truncated to 16 hex
-chars).
-
-See [Worker Daemon](../../goals/worker-daemon.md) for
-the full behavioral specification.