@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/details/knowledge-base.md

@@ -1,180 +0,0 @@
-# Knowledge Base Guide
-
-How to build, search, and maintain a knowledge base
-with kbdb.
-
-## Overview
-
-kbdb stores your documents in a plain `.kbdb` folder on
-disk and makes them searchable. No external server
-required -- just a directory you can back up, version
-with git, or move between machines.
-
-## Getting Started
-
-**Create a knowledge base:**
-
-```sh
-kbdb db init --db ./my-project
-```
-
-**Feed it your docs:**
-
-```sh
-kbdb learn ./docs --db ./my-project
-```
-
-kbdb reads Markdown (`.md`) and plain text (`.txt`)
-files.
-It breaks them into sections, extracts keywords, and
-builds a search index.
-
-**Search:**
-
-```sh
-kbdb search "how does auth work" --db ./my-project
-```
-
-Results are ranked by relevance. Headings carry 2x
-weight, so well-titled sections surface first.
-
-## Importing Content
-
-### File import
-
-```sh
-kbdb learn ./docs --db ./my-project
-```
-
-Recursively imports all supported files. Each file is
-split into sections by heading.
-
-### Stdin import
-
-```sh
-echo "My note about auth." \
-   | kbdb learn - --title "Auth notes" --db ./my-project
-```
-
-### Tags and source tracking
-
-```sh
-kbdb learn ./docs --tags design,v2 --db ./my-project
-```
-
-Tags scope content for filtering. Use `--replace` to
-update existing sections from the same source instead
-of creating duplicates.
-
-## Search Modes
-
-| Flag | Mode | Description |
-|------|------|-------------|
-| _(default)_ | lexical | Keyword matching with stemming |
-| `--algo hybrid` | hybrid | Keyword + AI similarity |
-| `--algo vector` | vector | AI similarity only |
-
-Hybrid and vector modes work out of the box using a
-built-in TF-IDF embedding provider. For higher
-quality, configure an ONNX or API provider in
-`worker.toml`. See the [CLI Reference](cli.md) for
-provider configuration.
-
-When a query returns no exact matches, kbdb
-automatically loosens the search (OR instead of AND)
-and flags results with `relaxed: true`.
-
-## Reading Data
-
-Three commands retrieve stored content at different
-granularities:
-
-| Command | Purpose |
-|---------|---------|
-| `search` | Find sections by query; ranked snippets |
-| `recall` | Expand context around known kbids |
-| `content` | Compose Markdown from kbid/docid lists |
-
-**Recall depths:**
-
-| Depth | Returns |
-|-------|---------|
-| 0 | Section content only |
-| 1 | + parent documents, back-references |
-| 2 | + siblings, forward references |
-| 3 | + full text of referenced sections |
-
-**kbid prefixes:** You don't need to paste full 26-char
-identifiers. Short prefixes work (like git short
-hashes) as long as they're unambiguous.
-
-## Maintenance
-
-```sh
-kbdb check --db ./my-project    # integrity check
-kbdb gc --db ./my-project       # remove stale data
-kbdb rebuild --db ./my-project  # rebuild all indexes
-kbdb export --db ./my-project   # portable snapshot
-```
-
-## Sync & Backup
-
-The `.kbdb` directory is self-contained and portable.
-
-**Git-based sync:**
-
-```sh
-cd .kbdb
-git init && git add -A && git commit -m "snapshot"
-git remote add origin <url> && git push
-```
-
-On another machine:
-
-```sh
-git clone <url> .kbdb
-kbdb rebuild
-```
-
-The `.gitignore` inside `.kbdb` already excludes
-binary indexes, so git only tracks raw section and
-document files. After cloning, `rebuild` reconstructs
-the indexes.
-
-**Export/import:**
-
-```sh
-kbdb export --db ./project
-# copy the export directory to another machine
-kbdb db init --db ./project
-# re-import sections from the export
-```
-
-## Daemon Timeout and Retry
-
-All commands that talk to the worker daemon support
-two global flags:
-
-| Flag | Default | Description |
-|------|---------|-------------|
-| `--timeout-ms <ms>` | `30000` | Per-request daemon timeout |
-| `--retry <n>` | `1` | Retries with daemon respawn on timeout |
-| `--non-interactive` | off | Suppress prompts; use defaults |
-
-These can also be set via environment variables.
-The flags take priority over env vars.
-
-| Variable | Description |
-|----------|-------------|
-| `KBDB_DAEMON_TIMEOUT` | Per-request daemon timeout (ms) |
-| `KBDB_RETRY` | Max retry count |
-| `KBDB_LOG_LEVEL` | Log verbosity: `debug`, `info`, `warning`, `error` |
-| `KBDB_NON_INTERACTIVE` | Suppress prompts (`1` or `true`) |
-
-## Deep Dives
-
-- [CLI Reference](cli.md) -- full command list
-- [Search and Ranking](search-and-ranking.md) --
-  scoring, stemming, hybrid search internals
-- [Storage Architecture](storage.md) -- file formats,
-  directory layout

package/docs/details/library-api.md

@@ -1,495 +0,0 @@
-# Library API Reference
-
-## Overview
-
-kbdb can be used as a library in Node.js and Deno
-projects. The main entry point is
-`createWorkerClient()`, which connects to a
-background worker daemon and returns a typed client
-for all knowledge base operations.
-
-## Installation
-
-```
-npm install @dikolab/kbdb       # Node.js
-deno add @dikolab/kbdb          # Deno
-```
-
-## Quick Start
-
-```typescript
-import { createWorkerClient } from '@dikolab/kbdb';
-
-const client = await createWorkerClient({
-   contextPath: '/path/to/.kbdb',
-});
-
-const results = await client.search({
-   query: 'authentication',
-   limit: 10,
-});
-
-console.log(results);
-client.disconnect();
-```
-
-## API Reference
-
-### Factory
-
-#### `createWorkerClient(options?)`
-
-Creates and connects a `WorkerClient` to the worker
-daemon. Resolves the database path, discovers or
-spawns the daemon, and returns a connected client.
-
-```typescript
-function createWorkerClient(
-   options?: WorkerClientOptions,
-): Promise<WorkerClient>;
-```
-
-**Parameters:**
-
-| Name | Type | Description |
-|------|------|-------------|
-| `options.dbPath` | `string?` | Parent directory containing `.kbdb`. The client discovers `.kbdb` inside it. Defaults to `cwd()`. |
-| `options.contextPath` | `string?` | Direct absolute path to the `.kbdb` directory itself. Skips discovery. |
-| `options.autoSpawn` | `boolean?` | Auto-spawn daemon if not running. Default `true`. |
-| `options.connectTimeoutMs` | `number?` | Max ms to wait for daemon. Default `10000`. |
-| `options.requestTimeoutMs` | `number?` | Per-request timeout in milliseconds. Default `30000`. |
-| `options.workerScript` | `string?` | Path to worker daemon script. |
-
-#### `getOrCreateClient(options?)`
-
-Returns a cached `WorkerClient`, creating one on
-first call. Subsequent calls with the same
-`contextPath` return the existing connected client.
-Use this in long-lived processes to avoid spawning
-multiple daemon connections.
-
-```typescript
-function getOrCreateClient(
-   options?: WorkerClientOptions,
-): Promise<WorkerClient>;
-```
-
-#### `clearClientCache()`
-
-Disconnects and removes all cached clients created
-by `getOrCreateClient`. Call during shutdown to
-clean up connections.
-
-```typescript
-function clearClientCache(): void;
-```
-
----
-
-### Search
-
-#### `search(params)`
-
-Queries the knowledge base and returns ranked
-results.
-
-```typescript
-search(
-   params: SearchParams,
-): Promise<PagedSearchResult>
-```
-
-| Param | Type | Description |
-|-------|------|-------------|
-| `params.query` | `string` | Search query string. |
-| `params.mode` | `'sections' \| 'documents' \| 'stats'?` | Entity type to return. Default `'sections'`. |
-| `params.limit` | `number?` | Maximum results to return. |
-| `params.offset` | `number?` | Zero-based result offset for pagination. |
-| `params.algo` | `'lexical' \| 'vector' \| 'hybrid'?` | Search algorithm. Default `'lexical'`. |
-| `params.relaxed` | `boolean?` | Use relaxed (OR) matching when strict returns no results. |
-
----
-
-### Sections
-
-#### `readSections(kbids)`
-
-Fetches sections by their knowledge-base
-identifiers.
-
-```typescript
-readSections(kbids: string[]): Promise<Section[]>
-```
-
-#### `addSection(params)`
-
-Imports content as a new section into the knowledge
-base.
-
-```typescript
-addSection(
-   params: AddSectionParams,
-): Promise<AddSectionResult>
-```
-
-| Param | Type | Description |
-|-------|------|-------------|
-| `params.sectionType` | `string` | Semantic type label for the section. |
-| `params.content` | `string` | Raw text content. |
-| `params.docid` | `string?` | Parent document to attach to. |
-
-#### `updateSection(kbid, params)`
-
-Replaces an existing section's content and metadata.
-
-```typescript
-updateSection(
-   kbid: string,
-   params: AddSectionParams,
-): Promise<string>
-```
-
-Returns the knowledge-base identifier of the
-updated section.
-
-#### `removeSection(kbid)`
-
-Deletes a section from the knowledge base.
-
-```typescript
-removeSection(kbid: string): Promise<RemoveSectionResult>
-```
-
-#### `listByType(sectionType)`
-
-Retrieves all sections matching the given type name.
-
-```typescript
-listByType(sectionType: string): Promise<SectionRecord[]>
-```
-
----
-
-### Documents
-
-#### `groupSections(title, kbids, docType?)`
-
-Creates a document by grouping sections under a
-title.
-
-```typescript
-groupSections(
-   title: string,
-   kbids: string[],
-   docType?: string,
-): Promise<string>
-```
-
-Returns the assigned document identifier.
-
-#### `removeGrouping(docid)`
-
-Removes a document grouping. The sections themselves
-are not deleted.
-
-```typescript
-removeGrouping(docid: string): Promise<void>
-```
-
-#### `retrieveDocument(docid)`
-
-Fetches a full document record including its
-sections.
-
-```typescript
-retrieveDocument(docid: string): Promise<unknown>
-```
-
-#### `listDocumentsByType(docType)`
-
-Retrieves all documents matching the given type
-name.
-
-```typescript
-listDocumentsByType(
-   docType: string,
-): Promise<DocumentManifest[]>
-```
-
----
-
-### Recall
-
-#### `recall(params)`
-
-Retrieves full content and progressive context for
-one or more sections. Use after `search()` to read
-relevant results.
-
-```typescript
-recall(
-   params: RecallParams,
-): Promise<RecallResult | RecallResult[]>
-```
-
-| Param | Type | Description |
-|-------|------|-------------|
-| `params.kbid` | `string?` | Single section identifier. |
-| `params.kbids` | `string[]?` | Batch section identifiers. |
-| `params.depth` | `number?` | Expansion depth (0--3). Default `0`. |
-
-At least one of `kbid` or `kbids` is required.
-Returns a single `RecallResult` for `kbid`, or an
-array for `kbids`. Depth controls progressive
-context expansion: 0 = section content, 1 = adds
-parent documents and back-references, 2 = adds
-siblings and forward references, 3 = adds full text
-of referenced sections.
-
----
-
-### Content
-
-#### `content(ids)`
-
-Composes rendered Markdown from one or more kbid or
-docid identifiers (may be mixed).
-
-```typescript
-content(ids: string[]): Promise<ContentResult>
-```
-
-Returns a `ContentResult` with the composed Markdown
-string and character count.
-
----
-
-### Maintenance
-
-#### `integrityCheck()`
-
-Verifies database integrity and reports all issues
-found.
-
-```typescript
-integrityCheck(): Promise<IntegrityCheckResult>
-```
-
-#### `gc()`
-
-Garbage collects unreferenced sections from the
-database.
-
-```typescript
-gc(): Promise<GcResult>
-```
-
-#### `rebuildIndexes()`
-
-Reconstructs all index files from section files and
-document manifests on disk.
-
-```typescript
-rebuildIndexes(): Promise<RebuildResult>
-```
-
-#### `initDirectory(targetPath)`
-
-Initialises a new `.kbdb` database directory at the
-given path.
-
-```typescript
-initDirectory(targetPath: string): Promise<void>
-```
-
-#### `migrateDatabase(targetPath)`
-
-Runs all pending database migrations for the
-`.kbdb` directory at the given path.
-
-```typescript
-migrateDatabase(
-   targetPath: string,
-): Promise<MigrateResult>
-```
-
-#### `checkVersion(targetPath)`
-
-Checks format version compatibility of the `.kbdb`
-database against the current WASM engine. Read-only.
-
-```typescript
-checkVersion(
-   targetPath: string,
-): Promise<VersionStatus>
-```
-
----
-
-### Cache
-
-#### `invalidate(kbid)`
-
-Evicts a single section from the daemon's in-memory
-cache.
-
-```typescript
-invalidate(kbid: string): Promise<void>
-```
-
-#### `invalidateAll()`
-
-Clears all entries from the daemon's in-memory
-cache.
-
-```typescript
-invalidateAll(): Promise<void>
-```
-
----
-
-### Lifecycle
-
-#### `status()`
-
-Returns diagnostic status including initialization
-state and cache statistics.
-
-```typescript
-status(): Promise<StatusResult>
-```
-
-#### `dbStatus()`
-
-Returns database metadata and status from the
-daemon's cached state.
-
-```typescript
-dbStatus(): Promise<StatusResult>
-```
-
-#### `disconnect()`
-
-Destroys the IPC socket and closes the connection.
-
-```typescript
-disconnect(): void
-```
-
-#### `isConnected`
-
-Read-only getter that returns `true` when the IPC
-socket is connected and healthy.
-
-```typescript
-get isConnected(): boolean;
-```
-
----
-
-## Types
-
-### `WorkerClientOptions`
-
-```typescript
-interface WorkerClientOptions {
-   readonly dbPath?: string;
-   readonly contextPath?: string;
-   readonly autoSpawn?: boolean;
-   readonly connectTimeoutMs?: number;
-   readonly requestTimeoutMs?: number;
-   readonly workerScript?: string;
-}
-```
-
-### `SearchParams`
-
-```typescript
-interface SearchParams {
-   readonly query: string;
-   readonly mode?: 'sections' | 'documents' | 'stats';
-   readonly limit?: number;
-   readonly offset?: number;
-   readonly content?: boolean;
-}
-```
-
-### `PagedSearchResult`
-
-```typescript
-interface PagedSearchResult {
-   readonly items: readonly SearchResult[];
-   readonly total: number;
-   readonly offset: number;
-   readonly limit: number;
-   readonly has_more: boolean;
-   readonly relaxed: boolean;
-}
-```
-
-### `SearchResult`
-
-```typescript
-interface SearchResult {
-   readonly kbid: string;
-   readonly heading: string | null;
-   readonly type: string;
-   readonly docids: readonly string[];
-   readonly score: number;
-   readonly confidence: number;
-   readonly snippet: string;
-   readonly matched_terms: readonly string[];
-   readonly matched_fields: readonly string[];
-   readonly created_at: string;
-}
-```
-
-### `Section`
-
-```typescript
-interface Section {
-   readonly kbid: string;
-   readonly docid?: string;
-   readonly sectionType: string;
-   readonly content: string;
-   readonly size: number;
-   readonly createdAt: number;
-   readonly checksum: string;
-}
-```
-
-### `AddSectionParams`
-
-```typescript
-interface AddSectionParams {
-   readonly sectionType: string;
-   readonly content: string;
-   readonly docid?: string;
-}
-```
-
-### `StatusResult`
-
-```typescript
-interface StatusResult {
-   readonly initialized: boolean;
-   readonly contextPath: string;
-   readonly cacheStats: {
-      readonly contentCount: number;
-      readonly indexCount: number;
-   };
-}
-```
-
----
-
-## How It Works
-
-The client spawns a background worker daemon that
-loads the Rust/WASM search engine. Communication
-between the client and daemon happens over a local
-IPC socket using JSON-RPC 2.0 with newline-delimited
-framing. The daemon auto-starts on first use and
-persists across invocations; if the connection drops,
-the client automatically rediscovers or respawns the
-daemon on the next method call.