@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/modules/typescript/shared/cli/module.md

@@ -1,138 +0,0 @@
-# cli -- CLI Command Module
-
-*Parses command-line arguments, delegates operations
-to the worker daemon via worker-client, and prints
-results to stdout.*
-
-## Source
-
-`src/shared/cli/`
-
-## Responsibility
-
-Thin pass-through between the user's shell and the
-worker daemon. Parses arguments, validates input,
-creates a `WorkerClient`, calls the appropriate client
-method, formats the result, and prints to stdout.
-Errors go to stderr.
-
-The CLI does **not** perform keyword extraction,
-indexing, ranking, or database I/O. All processing is
-delegated to the worker daemon.
-
-See [CLI](../../../../goals/cli.md) for the full
-behavioral specification.
-
-## Directory Layout
-
-```
-src/shared/cli/
-├── index.ts
-├── constants/
-│   ├── defaults.constant.ts
-│   └── help-text.constant.ts
-├── functions/
-│   ├── dispatch-command.function.ts
-│   ├── ensure-db.function.ts
-│   ├── error-to-message.function.ts
-│   ├── find-db.function.ts
-│   ├── format-output.function.ts
-│   ├── is-stdin-tty.function.ts
-│   ├── parse-args.function.ts
-│   ├── prompt-user.function.ts
-│   ├── resolve-kbid-prefix.function.ts
-│   ├── run-agent-create.function.ts
-│   ├── run-agent-delete.function.ts
-│   ├── run-agent-get.function.ts
-│   ├── run-agent-list.function.ts
-│   ├── run-check.function.ts
-│   ├── run-cli.function.ts
-│   ├── run-content.function.ts
-│   ├── run-db-init.function.ts
-│   ├── run-db-migrate.function.ts
-│   ├── run-export.function.ts
-│   ├── run-gc.function.ts
-│   ├── run-init.function.ts
-│   ├── run-learn.function.ts
-│   ├── run-migrate.function.ts
-│   ├── run-recall.function.ts
-│   ├── run-rebuild.function.ts
-│   ├── run-search.function.ts
-│   ├── run-skill-delete.function.ts
-│   ├── run-skill-get.function.ts
-│   ├── run-skill-learn.function.ts
-│   ├── run-skill-list.function.ts
-│   ├── run-status.function.ts
-│   ├── run-unlearn.function.ts
-│   ├── run-worker-restart.function.ts
-│   └── run-worker-stop.function.ts
-├── typings/
-│   ├── cli-options.interface.ts
-│   ├── command-result.model.ts
-│   ├── learn-result.model.ts
-│   └── search-display.model.ts
-└── *.spec.ts
-```
-
-## Exports
-
-- **[functions.md](functions.md)** -- `runCli`,
-  `dispatchCommand`, `parseArgs`, `findDb`,
-  `resolveKbidPrefix`, `runInit`, `runMigrate`,
-  `runLearn`, `runUnlearn`, `runSearch`, `runRecall`,
-  `runContent`, `runExport`, `runStatus`, `runCheck`,
-  `runGc`, `runRebuild`, `ensureDb`, `formatOutput`,
-  `errorToMessage`, `isStdinTty`, `promptUser`,
-  `runDbInit`, `runDbMigrate`, `runSkillLearn`,
-  `runSkillList`, `runSkillGet`, `runSkillDelete`,
-  `runAgentCreate`, `runAgentList`, `runAgentGet`,
-  `runAgentDelete`, `runWorkerStop`,
-  `runWorkerRestart`.
-- **[typings.md](typings.md)** -- `CliOptions`,
-  `CommandResult`, `LearnResult`, `SearchDisplay`,
-  `SearchMode`, `OutputFormat`.
-- **[constants.md](constants.md)** -- `DEFAULT_LIMIT`,
-  `DEFAULT_FORMAT`.
-
-## Execution Flow
-
-```
-cli.ts (entry point)
-  │
-  ├─ runCli(argv, deps)
-  │    ├─ parseArgs(argv) → CliOptions
-  │    ├─ command === 'help'       → print HELP_TEXT
-  │    ├─ command === 'version'    → print VERSION
-  │    ├─ command === 'init'       → runInit(options)
-  │    ├─ command === 'mcp'        → startMcpServer
-  │    ├─ command === 'migrate'    → runMigrate(client, options)
-  │    ├─ command === 'db-init'    → runDbInit(options)
-  │    ├─ command === 'db-migrate' → runDbMigrate(client, options)
-  │    ├─ command === 'worker-stop'    → runWorkerStop(dbDir)
-  │    ├─ command === 'worker-restart' → runWorkerRestart(dbDir)
-  │    └─ data command:
-  │         ├─ findDb(options.dbPath) → dbDir
-  │         ├─ if not found: isStdinTty() / promptUser()
-  │         ├─ createWorkerClient({ contextPath: dbDir })
-  │         ├─ version gate (checkVersion)
-  │         └─ dispatchCommand(client, options)
-  │              ├─ learn           → runLearn(client, options)
-  │              ├─ unlearn         → runUnlearn(client, options)
-  │              ├─ search          → runSearch(client, options)
-  │              ├─ recall          → runRecall(client, options)
-  │              ├─ content         → runContent(client, options)
-  │              ├─ export          → runExport(client, options)
-  │              ├─ status          → runStatus(client, options)
-  │              ├─ check           → runCheck(client, options)
-  │              ├─ gc              → runGc(client, options)
-  │              ├─ rebuild         → runRebuild(client, options)
-  │              ├─ skill-learn     → runSkillLearn(client, options)
-  │              ├─ skill-list      → runSkillList(client, options)
-  │              ├─ skill-get       → runSkillGet(client, options)
-  │              ├─ skill-delete    → runSkillDelete(client, options)
-  │              ├─ agent-create    → runAgentCreate(client, options)
-  │              ├─ agent-list      → runAgentList(client, options)
-  │              ├─ agent-get       → runAgentGet(client, options)
-  │              └─ agent-delete    → runAgentDelete(client, options)
-  └─ returns CommandResult { output?, error?, exitCode }
-```

package/docs/modules/typescript/shared/cli/typings.md

@@ -1,197 +0,0 @@
-# cli/typings -- CLI Type Definitions
-
-*Interfaces, type aliases, and data models for CLI
-argument parsing, command results, and display output.*
-
-## Source
-
-`src/shared/cli/typings/`
-
-## Typings
-
-### `CliOptions`
-
-**File:** `cli-options.interface.ts`
-
-Parsed representation of all CLI arguments for a
-single invocation.
-
-```ts
-interface CliOptions {
-   command: string;
-   args: string[];
-   dbPath?: string;
-   format: OutputFormat;
-   mode: SearchMode;
-   limit: number;
-   title?: string;
-   description?: string;
-   docid?: string;
-   content?: boolean;
-   name?: string;
-   persona?: string;
-   body?: string;
-   skills?: string[];
-   timeoutMs?: number;
-   retry?: number;
-   logLevel?: LogLevel;
-}
-```
-
-| Property      | Description                           |
-|---------------|---------------------------------------|
-| `command`     | CLI sub-command. Simple commands:     |
-|               | `'search'`, `'learn'`, `'unlearn'`,   |
-|               | `'content'`, `'check'`, `'gc'`,       |
-|               | `'rebuild'`, `'mcp'`, `'status'`.     |
-|               | Compound commands: `'db-init'`,       |
-|               | `'db-migrate'`, `'skill-learn'`,      |
-|               | `'skill-list'`, `'skill-get'`,        |
-|               | `'skill-delete'`, `'agent-create'`,   |
-|               | `'agent-list'`, `'agent-get'`,        |
-|               | `'agent-delete'`.                     |
-| `args`        | Positional arguments                  |
-| `dbPath`      | Database directory (`--db`)           |
-| `format`      | Output format flag                    |
-| `mode`        | Search result mode (`--mode`)         |
-| `limit`       | Max results (`--limit`)               |
-| `title`       | Section title (`--title`)             |
-| `description` | Section description                   |
-| `docid`       | Document ID for learn command         |
-| `content`     | Hydrate search results with           |
-|               | section content when `true`           |
-| `name`        | Skill or agent name (`--name`)        |
-| `persona`     | Agent persona file path (`--persona`) |
-| `body`        | Skill body file path (`--body`)       |
-| `skills`      | Ordered skill kbid list (`--skills`)  |
-| `timeoutMs`   | Request timeout in ms                 |
-| `retry`       | Max retry count on timeout            |
-| `logLevel`    | Log verbosity level from `-v`/`-vv`/`-vvv` |
-| `nonInteractive` | Suppress prompts (`--non-interactive`) |
-
-### `OutputFormat`
-
-**File:** `cli-options.interface.ts`
-
-Type alias for supported output formats.
-
-```ts
-type OutputFormat = 'json' | 'text' | 'mcp';
-```
-
-| Value  | Description                     |
-|--------|---------------------------------|
-| `json` | Machine-readable JSON array     |
-| `text` | Human-readable numbered list    |
-| `mcp`  | JSON envelope for MCP consumers |
-
-### `SearchMode`
-
-**File:** `cli-options.interface.ts`
-
-Controls which entity type is returned by a search
-query.
-
-```ts
-type SearchMode = 'sections' | 'documents' | 'stats';
-```
-
-### `LogLevel`
-
-**File:** `cli-options.interface.ts`
-
-Log verbosity level. Set via `-v`/`-vv`/`-vvv` flags
-or the `KBDB_LOG_LEVEL` environment variable.
-
-```ts
-type LogLevel = 'debug' | 'info' | 'warning' | 'error';
-```
-
-| Value     | Description                          |
-|-----------|--------------------------------------|
-| `error`   | Errors only (default)                |
-| `warning` | Errors and warnings (`-v`)           |
-| `info`    | Errors, warnings, and info (`-vv`)   |
-| `debug`   | All messages including debug (`-vvv`)|
-
-### `CommandResult`
-
-**File:** `command-result.model.ts`
-
-Result of dispatching a CLI command. Encapsulates
-output, error text, and exit code so the caller can
-decide how to present them without touching process
-globals.
-
-```ts
-interface CommandResult {
-   output?: string;
-   error?: string;
-   exitCode: number;
-}
-```
-
-| Property   | Description                          |
-|------------|--------------------------------------|
-| `output`   | Text to write to stdout (if any)     |
-| `error`    | Error message for stderr (if any)    |
-| `exitCode` | Process exit code (`0` = success)    |
-
-### `LearnResult`
-
-**File:** `learn-result.model.ts`
-
-Result record produced for each file processed by the
-learn command.
-
-```ts
-interface LearnResult {
-   kbid: string;
-   path: string;
-   size: number;
-   skipped: boolean;
-}
-```
-
-| Property  | Description                           |
-|-----------|---------------------------------------|
-| `kbid`    | Knowledge-base identifier assigned    |
-| `path`    | Absolute path to the source file      |
-| `size`    | Size of file content in bytes         |
-| `skipped` | `true` when file was skipped          |
-
-### `SearchDisplay`
-
-**File:** `search-display.model.ts`
-
-A single search result formatted for display to the
-user.
-
-```ts
-interface SearchDisplay {
-   kbid: string;
-   heading: string | null;
-   type: string;
-   docids: readonly string[];
-   score: number;
-   snippet: string;
-   matched_terms: readonly string[];
-   preview: string;
-}
-```
-
-| Property        | Description                          |
-|-----------------|--------------------------------------|
-| `kbid`          | Knowledge-base identifier            |
-| `heading`       | Section heading, or `null`           |
-| `type`          | Semantic section type label          |
-| `docids`        | Parent document IDs for the section  |
-| `score`         | Relevance score from ranking engine  |
-| `snippet`       | Short excerpt highlighting matched   |
-|                 | terms                                |
-| `matched_terms` | Terms from the query that matched    |
-|                 | this section                         |
-| `preview`       | Formatted preview string for         |
-|                 | terminal output                      |
-
-No spec -- typings are compile-time only.

package/docs/modules/typescript/shared/embedding/functions.md

@@ -1,200 +0,0 @@
-# embedding/functions -- Embedding Functions
-
-**Source:** `src/shared/embedding/functions/`
-
-## createEmbeddingProvider
-
-```ts
-createEmbeddingProvider(
-   config: EmbeddingConfig,
-): EmbeddingProvider
-```
-
-Factory that returns the appropriate `EmbeddingProvider`
-implementation based on the given config. Returns an
-`OnnxEmbeddingProvider` for `provider: 'onnx'`, an
-`ApiEmbeddingProvider` for `provider: 'api'`, or a
-`TfidfEmbeddingProvider` when config is undefined, or a
-`NullEmbeddingProvider` when `provider: 'none'`. The
-TF-IDF provider is the default when no embedding config
-is present, enabling hybrid search without external
-configuration.
-
-## hashToken
-
-```ts
-hashToken(token: string): number
-```
-
-Computes an FNV-1a hash of a string token, returning
-a non-negative 32-bit unsigned integer. Used by
-`TfidfEmbeddingProvider` to map tokens to fixed-size
-vector buckets (feature hashing).
-
-## tokenizeText
-
-```ts
-tokenizeText(text: string): string[]
-```
-
-Splits text into lowercase tokens by non-word
-characters, filtering out tokens shorter than 3
-characters. A simple JS tokenizer used by
-`TfidfEmbeddingProvider` for both document and query
-embedding.
-
-## cosineSimilarity
-
-```ts
-cosineSimilarity(
-   a: Float32Array,
-   b: Float32Array,
-): number
-```
-
-Computes the cosine similarity between two vectors. Returns a
-value in [-1, 1], where 1 means identical direction. Used by
-semantic near-duplicate detection (threshold >= 0.92) and by
-vector search ranking.
-
-Both vectors must have the same length (same embedding
-dimensions). Throws if lengths differ.
-
-## buildVectorIndex
-
-```ts
-buildVectorIndex(
-   provider: EmbeddingProvider,
-   sections: SectionRecord[],
-   outPath: string,
-): Promise<VectorIndexMeta>
-```
-
-Builds the vector index file (`vectors.idx`) from all
-sections. Embeds each section's content via `batchEmbed`,
-writes fixed-width records (kbid + vector) to `outPath`, and
-writes `vectors-meta.toml` alongside it with model name,
-dimensions, and record count.
-
-Called by `rebuild` when an embedding provider is configured.
-Returns metadata about the built index.
-
-## searchVectors
-
-```ts
-searchVectors(
-   provider: EmbeddingProvider,
-   query: string,
-   indexPath: string,
-   limit: number,
-): Promise<VectorSearchResult[]>
-```
-
-Performs vector similarity search. Embeds the query, reads
-`vectors.idx`, computes cosine similarity against all stored
-vectors, and returns the top `limit` results sorted by
-similarity descending.
-
-Each result includes the kbid and similarity score. The
-results are passed to the RRF fusion step alongside lexical
-search results.
-
-Note: This is a brute-force linear scan. For corpus sizes
-under 100k sections, this is fast enough (< 50ms). A future
-version may add approximate nearest neighbor (ANN) indexing
-if scale demands it.
-
-## appendVectorEntry
-
-```ts
-appendVectorEntry(
-   provider: EmbeddingProvider,
-   kbid: string,
-   content: string,
-   indexPath: string,
-): Promise<void>
-```
-
-Appends a single vector entry to an existing vector index.
-Used during incremental `learn` to add new sections without
-a full rebuild.
-
-## readVectorIndexMeta
-
-```ts
-readVectorIndexMeta(
-   indexPath: string,
-): Promise<VectorIndexMeta | null>
-```
-
-Reads the `vectors-meta.toml` file for an existing vector
-index. Returns `null` if the index does not exist.
-
-## rrfFuse
-
-```ts
-rrfFuse(
-   lexical: { kbid: string; rank: number }[],
-   vector: { kbid: string; score: number }[],
-   k?: number,
-): { kbid: string; score: number }[]
-```
-
-Merges lexical and vector result lists using Reciprocal
-Rank Fusion. The `k` parameter controls rank smoothing
-(default: 60).
-
-## jaccardCoefficient
-
-```ts
-jaccardCoefficient(
-   a: Set<string>,
-   b: Set<string>,
-): number
-```
-
-Computes the Jaccard similarity coefficient between two
-term sets. Returns a value in [0, 1]. Used as the first
-pass in near-duplicate detection.
-
-## detectNearDuplicates
-
-```ts
-detectNearDuplicates(
-   sections: SectionRecord[],
-   provider?: EmbeddingProvider,
-): Promise<NearDuplicateResult[]>
-```
-
-Two-pass near-duplicate detection: Jaccard coefficient on
-term sets (always), then cosine similarity on embeddings
-(when a provider is available). Returns pairs of sections
-that exceed the similarity thresholds.
-
-## detectStaleVectors
-
-```ts
-detectStaleVectors(
-   indexPath: string,
-   currentKbids: Set<string>,
-): Promise<StaleVectorsReport>
-```
-
-Compares the vector index against the current section set
-and reports entries whose kbid no longer exists. Used by
-`check` to flag stale vector data.
-
-## rebuildVectors
-
-```ts
-rebuildVectors(
-   provider: EmbeddingProvider,
-   sections: SectionRecord[],
-   outPath: string,
-): Promise<VectorIndexMeta>
-```
-
-Rebuilds the entire vector index from scratch. Embeds all
-sections and writes a fresh `vectors.idx` and
-`vectors-meta.toml`. Called by `rebuild` when an embedding
-provider is configured.

package/docs/modules/typescript/shared/embedding/module.md

@@ -1,77 +0,0 @@
-# embedding -- Pluggable Embedding Computation
-
-**Source:** `src/shared/embedding/`
-
-## Responsibility
-
-Pluggable embedding computation for hybrid lexical+vector search.
-Provides the `EmbeddingProvider` interface and implementations for
-different backends. Used by the worker daemon to embed sections at
-learn time and to compute query embeddings at search time.
-
-## Directory Layout
-
-```
-src/shared/embedding/
-   index.ts
-   functions/
-      embed.function.ts
-      batch-embed.function.ts
-      cosine-similarity.function.ts
-      build-vector-index.function.ts
-      search-vectors.function.ts
-   providers/
-      null-embedding.provider.ts
-      onnx-embedding.provider.ts
-      api-embedding.provider.ts
-   typings/
-      embedding-provider.typing.ts
-      embedding-config.typing.ts
-      vector-search-result.typing.ts
-      vector-index-meta.typing.ts
-```
-
-## Exports
-
-- [Functions](functions.md) -- `embed`, `batchEmbed`,
-  `cosineSimilarity`, `buildVectorIndex`, `searchVectors`
-- [Typings](typings.md) -- `EmbeddingProvider`, `EmbeddingConfig`,
-  `VectorSearchResult`, `VectorIndexMeta`
-
-## Providers
-
-Four implementations of the `EmbeddingProvider` interface:
-
-- **TfidfEmbeddingProvider** -- Built-in default provider backed by
-  the `kbdb-default-embedding` WASM crate. Delegates `embed()`
-  calls to WASM exports that compute feature-hashed sublinear TF
-  vectors (256 dimensions, L2-normalized) using kbdb-shared's
-  tokenization pipeline. Zero external dependencies, zero
-  configuration. Enables hybrid search out of the box.
-- **NullEmbeddingProvider** -- No-op provider (dimensions = 0).
-  Available for explicit use when embeddings should be fully
-  disabled.
-- **OnnxEmbeddingProvider** -- Local inference using ONNX Runtime.
-  Loads a model from disk (e.g., `all-MiniLM-L6-v2`). Zero network
-  calls, fully offline.
-- **ApiEmbeddingProvider** -- Remote inference via HTTP API (OpenAI,
-  Anthropic, or compatible endpoints). Configured via `worker.toml`
-  with endpoint URL and API key.
-
-## Dependency Direction
-
-```
-shared/embedding/
-  ├── shared/hash/        (content hashing for cache keys)
-  └── shared/platform/    (file paths for local models)
-
-shared/worker-daemon/
-  └── shared/embedding/   (init provider at startup)
-
-shared/mcp/
-  └── (indirect via worker-client)
-```
-
-The embedding module is a utility module -- it depends only on other
-utility modules. The worker daemon initializes the provider at startup
-based on `worker.toml` config.