@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/modules/typescript/shared/worker-daemon/classes.md

@@ -1,239 +0,0 @@
-# worker-daemon/classes -- WorkerDaemon
-
-*Core daemon class that loads WASM modules, manages
-the IPC server, routes requests, and handles lifecycle
-(startup, idle tracking, graceful shutdown).*
-
-## Source
-
-`src/shared/worker-daemon/classes/`
-
-## Classes
-
-### `WorkerDaemon`
-
-**File:** `worker-daemon.class.ts`
-
-The main daemon class. Orchestrates WASM module
-loading, IPC server creation, request routing, idle
-timeout tracking, and graceful shutdown.
-
-#### Constructor
-
-```ts
-class WorkerDaemon {
-   constructor(
-      ctx: string,
-      targetDir: string,
-      config: DaemonConfig,
-      wasmDir?: string,
-   );
-}
-```
-
-| Parameter   | Type           | Description             |
-|-------------|----------------|-------------------------|
-| `ctx`       | `string`       | Context ID (16-hex)     |
-| `targetDir` | `string`       | Absolute path to the    |
-|             |                | `.kbdb` directory       |
-| `config`    | `DaemonConfig` | Parsed config from      |
-|             |                | `worker.toml`           |
-| `wasmDir`   | `string?`      | Path to WASM build      |
-|             |                | output directory        |
-
-#### Properties
-
-| Property          | Type           | Description            |
-|-------------------|----------------|------------------------|
-| `ctx`             | `string`       | Context ID             |
-| `targetDir`       | `string`       | Database directory     |
-| `config`          | `DaemonConfig` | Daemon config          |
-| `connectionCount` | `number`       | Active IPC connections |
-
-#### Public Methods
-
-##### `checkPreconditions`
-
-```ts
-checkPreconditions(): void;
-```
-
-Validates startup preconditions before WASM module
-loading. Checks that:
-
-1. Database directory exists.
-2. Directory is readable and writable.
-3. `catalog.toml` reports a compatible version.
-
-Throws on any failure. `startDaemon` catches the
-error, prints to stderr, and exits with code 1.
-
-##### `start`
-
-```ts
-async start(): Promise<void>;
-```
-
-Startup sequence:
-
-1. Calls `loadWasmModules(config, wasmDir)` to
-   compile and instantiate the WASM modules.
-2. Initializes the worker context.
-3. Writes the PID file (`kbdb-{ctx}.pid`) to the
-   OS temporary directory.
-4. Calls `createIpcServer(ipcPath, handler)` to
-   bind the IPC socket.
-5. Begins listening for connections and resets the
-   idle timer.
-
-##### `stop`
-
-```ts
-stop(): void;
-```
-
-Graceful shutdown sequence:
-
-1. Clears the idle timer.
-2. Stops accepting new connections.
-3. Purges WASM module references.
-4. Removes the PID file.
-5. Removes the IPC socket file.
-6. Calls `process.exit(0)`.
-
-##### `handleRequest`
-
-```ts
-async handleRequest(
-   request: IpcRequest,
-): Promise<IpcResponse>;
-```
-
-Handles a single JSON-RPC 2.0 request. Validates
-context, resets the idle timer, routes to the
-appropriate handler, and returns a response.
-Requests whose `ctx` param does not match the
-daemon's context ID are rejected with error
-code `-32001`.
-
-##### `incrementConnections` / `decrementConnections`
-
-```ts
-incrementConnections(): void;
-decrementConnections(): void;
-```
-
-Adjust the active connection count. Called by the
-IPC server on connect/disconnect events.
-
-##### `injectModulesForTesting`
-
-```ts
-injectModulesForTesting(modules: WasmModules): void;
-```
-
-Injects pre-built WASM modules for unit testing.
-Must not be called in production code.
-
-#### Private Behavior
-
-##### Idle Timeout Tracking
-
-The daemon maintains an internal idle timer. The
-timer resets on every:
-
-- Client connection.
-- Client disconnection.
-- IPC request received.
-
-When the timer reaches `config.timeoutMs` with no
-active connections, the daemon calls `stop()`.
-
-##### Connection Counting
-
-Tracks the number of active client connections. The
-daemon stays alive regardless of the idle timer as
-long as at least one connection is open.
-
-##### Startup Preconditions
-
-Before WASM modules are loaded, the daemon checks
-three preconditions. Failure causes `startDaemon` to
-print the error to stderr and exit with code 1:
-
-1. Database directory exists.
-2. Directory is readable and writable.
-3. `catalog.toml` version is compatible.
-
-##### Request Routing
-
-Routes incoming JSON-RPC 2.0 requests. Daemon-state
-methods are handled inline; all other methods are
-looked up in `WASM_METHOD_MAP`:
-
-| IPC Method           | Handler / WASM Export          |
-|----------------------|--------------------------------|
-| `status`             | daemon state (busy/idle)       |
-| `idle_at`            | last idle Unix timestamp       |
-| `config_{key}`       | `DaemonConfig` property lookup |
-| `checkVersion`       | catalog version check          |
-| `search`             | `worker_search`                |
-| `recall`             | `worker_recall`                |
-| `addSection`         | `worker_add_section`           |
-| `updateSection`      | `worker_update_section`        |
-| `removeSection`      | `worker_remove_section`        |
-| `readSections`       | `worker_read_sections`         |
-| `groupSections`      | `worker_group_sections`        |
-| `removeGrouping`     | `worker_remove_grouping`       |
-| `retrieveDocument`   | `worker_retrieve_doc`          |
-| `content`            | `worker_compose`               |
-| `integrityCheck`     | `worker_check_integrity`       |
-| `gc`                 | `worker_garbage_collect`       |
-| `rebuildIndexes`     | `worker_rebuild_indexes`       |
-| `initDirectory`      | `worker_init_directory`        |
-| `dbStatus`           | `worker_status`                |
-| `invalidate`         | `worker_invalidate`            |
-| `invalidateAll`      | `worker_invalidate_all`        |
-| `get_result_status`  | `worker_result_status`         |
-| `get_result`         | `worker_get_result`            |
-| `query`              | `worker_query`                 |
-| `listByType` / `list_by_type` | `worker_list_by_type` |
-| `listDocumentsByType` | `worker_list_docs_by_type`   |
-
-Snake_case aliases (`add_section`, `remove_section`,
-etc.) are also accepted and map to the same exports.
-
-##### Context Validation
-
-Every IPC request includes a `ctx` param. The daemon
-validates it against its own context ID before
-processing. Mismatches are rejected with JSON-RPC
-error code `-32001` (context mismatch).
-
-##### `handleListByType(params)`
-
-Handles `listByType` / `list_by_type` IPC requests.
-Receives `{ ctx, sectionType }` and delegates to
-`worker_list_by_type` in the WASM engine, which
-scans the sections directory and returns all records
-whose `type` field matches `sectionType`.
-
-Returns: array of section records filtered by type.
-
-##### `handleListDocumentsByType(params)`
-
-Handles `listDocumentsByType` IPC requests.
-Receives `{ ctx, docType }` and delegates to
-`worker_list_docs_by_type` in the WASM engine,
-which scans the documents directory and returns all
-manifests whose `type` field matches `docType`.
-
-Returns: array of document manifests filtered by
-type.
-
-## Testing
-
-```
-src/shared/worker-daemon/
-└── worker-daemon.spec.ts
-```

package/docs/modules/typescript/shared/worker-daemon/constants.md

@@ -1,37 +0,0 @@
-# worker-daemon/constants -- Daemon Defaults
-
-*Default values for daemon configuration when
-`worker.toml` is missing or incomplete.*
-
-## Source
-
-`src/shared/worker-daemon/constants/`
-
-### `daemon-defaults`
-
-**File:** `daemon-defaults.constant.ts`
-
-```ts
-const DEFAULT_ITEM_LIMIT = 1000;
-const DEFAULT_INDEX_LIMIT = 100;
-const DEFAULT_TIMEOUT_MS = 300_000;
-const DEFAULT_ITEM_TTL_MS = 60_000;
-```
-
-| Constant              | Value     | Description    |
-|-----------------------|-----------|----------------|
-| `DEFAULT_ITEM_LIMIT`  | `1000`    | Max content    |
-|                       |           | objects held   |
-|                       |           | in cache       |
-| `DEFAULT_INDEX_LIMIT` | `100`     | Max index      |
-|                       |           | segments held  |
-|                       |           | in cache       |
-| `DEFAULT_TIMEOUT_MS`  | `300000`  | Daemon idle    |
-|                       |           | timeout in ms  |
-|                       |           | (5 minutes)    |
-| `DEFAULT_ITEM_TTL_MS` | `60000`   | Per-item cache |
-|                       |           | TTL in ms      |
-|                       |           | (1 minute)     |
-
-These constants are used by `readConfig()` to fill
-missing keys when parsing `worker.toml`.

package/docs/modules/typescript/shared/worker-daemon/functions.md

@@ -1,234 +0,0 @@
-# worker-daemon/functions -- Daemon Functions
-
-*Worker entry point, daemon entry point, host I/O
-imports, WASM module loading, IPC server creation,
-and configuration reading.*
-
-## Source
-
-`src/shared/worker-daemon/functions/`
-
-## Functions
-
-### `runWorker`
-
-**File:** `run-worker.function.ts`
-
-Worker daemon entry point extracted from
-`src/worker.ts`. Parses `argv` for the context path
-and the optional `--log-level <level>` argument,
-configures the log level, validates the context path,
-and delegates to `startDaemon`.
-
-Usage: `kbdb-worker <context-path> [--log-level <level>]`
-
-```ts
-async function runWorker(
-   argv: string[],
-   deps: WorkerDeps,
-): Promise<WorkerResult>;
-```
-
-```ts
-interface WorkerDeps {
-   existsSync: (path: string) => boolean;
-   startDaemon: (
-      contextPath: string,
-      wasmDir?: string,
-   ) => Promise<void>;
-   wasmDir?: string;
-}
-
-interface WorkerResult {
-   error?: string;
-   exitCode: number;
-   logLevel?: LogLevel;
-}
-```
-
-`LogLevel` is `'debug' | 'info' | 'warning' | 'error'`.
-The log level is resolved from the `--log-level` CLI
-argument, then the `KBDB_LOG_LEVEL` environment
-variable, then defaults to `'error'`.
-
-Returns a `WorkerResult` without touching process
-globals, making it fully testable.
-
-### `startDaemon`
-
-**File:** `start-daemon.function.ts`
-
-Entry point for the daemon process. Called from
-`runWorker` after path validation.
-
-```ts
-async function startDaemon(
-   contextPath: string,
-   wasmDir?: string,
-): Promise<void>;
-```
-
-1. Calls `readConfig(contextPath)` to load
-   `worker.toml`.
-2. Computes the context ID from `contextPath`
-   (SHA-256 prefix, 16 hex characters).
-3. Creates a new `WorkerDaemon(ctx, contextPath,
-   config, wasmDir)`.
-4. Calls `daemon.checkPreconditions()`. Failures
-   are written to stderr and a crash log, then
-   `process.exit(1)` is called.
-5. Calls `daemon.start()`.
-6. Registers OS signal handlers (`SIGTERM`,
-   `SIGINT`) that call `daemon.stop()`.
-
-### `createHostImports`
-
-**File:** `create-host-imports.function.ts`
-
-Creates the host I/O import functions required by
-`kb-worker.wasm`. All functions access WASM linear
-memory through a deferred `MemoryRef` that is wired
-up after module instantiation.
-
-```ts
-function createHostImports(
-   contextPath: string,
-   memoryRef?: MemoryRef,
-   fs?: FsAdapter,
-   allocator?: AllocatorRef,
-): Record<string, WebAssembly.ImportValue>;
-```
-
-| Parameter     | Description                        |
-|---------------|------------------------------------|
-| `contextPath` | Absolute path to `.kbdb` directory |
-| `memoryRef`   | Deferred WASM memory reference     |
-| `fs`          | Filesystem adapter (default: real  |
-|               | `node:fs` operations)              |
-| `allocator`   | Deferred WASM allocator reference  |
-
-**Host imports provided:**
-
-| Import name       | Description                     |
-|-------------------|---------------------------------|
-| `host_read_file`  | Read a file into WASM memory    |
-| `host_write_file` | Write WASM memory to a file     |
-| `host_rename`     | Rename a file                   |
-| `host_remove_file`| Delete a file                   |
-| `host_list_dir`   | List directory entries          |
-| `host_mkdir`      | Create a directory (recursive)  |
-| `host_file_exists`| Check if a file exists          |
-
-All paths received from WASM are resolved relative
-to `contextPath`.
-
-**Supporting interfaces:**
-
-```ts
-interface FsAdapter {
-   readFileSync(path: string): Buffer;
-   writeFileSync(path: string, data: Uint8Array): void;
-   renameSync(oldPath: string, newPath: string): void;
-   unlinkSync(path: string): void;
-   readdirSync(path: string): string[];
-   mkdirSync(path: string): void;
-   existsSync(path: string): boolean;
-}
-
-interface MemoryRef {
-   memory: WebAssembly.Memory | null;
-}
-
-interface AllocatorRef {
-   alloc: ((size: number) => number) | null;
-}
-```
-
-### `loadWasmModules`
-
-**File:** `load-wasm-modules.function.ts`
-
-Compiles and instantiates the WASM modules used by
-the worker daemon.
-
-```ts
-async function loadWasmModules(
-   config: DaemonConfig,
-   wasmDir?: string,
-): Promise<WasmModules>;
-```
-
-Loading sequence:
-
-1. Builds `createHostImports` with a deferred
-   `MemoryRef` and `AllocatorRef`.
-2. Compiles and instantiates `kb-worker.wasm`,
-   passing host imports.
-3. Wires the exported `memory` and `kbdb_alloc`
-   function back into the deferred refs.
-4. Compiles and instantiates
-   `default-embedding.wasm` -- no host imports
-   needed (pure computation). The module is
-   instantiated with an empty import object.
-
-Returns a `WasmModules` object containing the
-instantiated modules and their shared memory.
-
-### `createIpcServer`
-
-**File:** `create-ipc-server.function.ts`
-
-Creates a local IPC server bound to the given socket
-path and returns the `net.Server` instance.
-
-```ts
-function createIpcServer(
-   ipcPath: string,
-   handler: IpcHandler,
-): net.Server;
-```
-
-Platform behavior:
-
-- **Linux / macOS** -- Unix domain socket.
-- **Windows** -- Named pipe.
-
-Messages are newline-delimited JSON-RPC 2.0. Each
-message is a single line of JSON followed by `\n`.
-
-Incoming requests are parsed and forwarded to
-`handler`. Responses are serialized and written back
-to the connection.
-
-### `readConfig`
-
-**File:** `read-config.function.ts`
-
-Reads daemon configuration from `worker.toml` in
-the `.kbdb` directory.
-
-```ts
-function readConfig(
-   contextPath: string,
-): DaemonConfig;
-```
-
-Reads `${contextPath}/worker.toml`. Missing keys
-are filled with defaults from
-[daemon-defaults.constant.ts](constants.md). If the
-file does not exist, returns a `DaemonConfig` with
-all defaults.
-
-## Testing
-
-Each function has a `.spec.ts` at the module root:
-
-```
-src/shared/worker-daemon/
-├── run-worker.spec.ts
-├── start-daemon.spec.ts
-├── create-host-imports.spec.ts
-├── load-wasm-modules.spec.ts
-├── create-ipc-server.spec.ts
-└── read-config.spec.ts
-```

package/docs/modules/typescript/shared/worker-daemon/module.md

@@ -1,118 +0,0 @@
-# worker-daemon -- Daemon Process Module
-
-*Background process that loads WASM modules once,
-listens on IPC, serves JSON-RPC 2.0 requests, manages
-cache + context + idle timeout. Spawned on demand by
-worker-client, shuts down on idle timeout or OS
-signal.*
-
-## Source
-
-`src/shared/worker-daemon/`
-
-## Responsibility
-
-The worker-daemon module implements a detached
-background process that:
-
-- Loads and compiles the four WASM modules once at
-  startup (`query-parser.wasm`, `fs-database.wasm`,
-  `default-embedding.wasm`, `kb-worker.wasm`).
-- Reads configuration from `worker.toml` in the
-  context directory.
-- Listens on a local IPC socket (Unix domain socket
-  or Windows named pipe) for JSON-RPC 2.0 requests.
-  Configures accepted connections with `SO_KEEPALIVE`
-  (30 s) and listens with backlog 256.
-- Maintains an in-memory LRU cache with TTL expiry
-  for indexes, content objects, and manifests.
-- Tracks active connections and resets the idle timer
-  on every connect, disconnect, or request.
-- Shuts down gracefully on idle timeout or OS signal.
-
-The daemon is **not** a persistent server. It is a
-short-lived process spawned by
-[worker-client](../worker-client/module.md) that
-exists only to amortize WASM compilation cost and
-share cached data across connections.
-
-See [Worker Daemon](../../../../goals/worker-daemon.md)
-for the full behavioral specification.
-
-## Directory Layout
-
-```
-src/shared/worker-daemon/
-├── index.ts
-├── classes/
-│   └── worker-daemon.class.ts
-├── functions/
-│   ├── create-host-imports.function.ts
-│   ├── create-ipc-server.function.ts
-│   ├── load-wasm-modules.function.ts
-│   ├── read-config.function.ts
-│   ├── run-worker.function.ts
-│   └── start-daemon.function.ts
-├── typings/
-│   ├── daemon-config.interface.ts
-│   ├── ipc-handler.interface.ts
-│   └── wasm-modules.interface.ts
-├── constants/
-│   └── daemon-defaults.constant.ts
-└── *.spec.ts
-```
-
-## Exports
-
-- **[classes.md](classes.md)** -- `WorkerDaemon`.
-- **[functions.md](functions.md)** -- `startDaemon`,
-  `runWorker`, `createHostImports`, `loadWasmModules`,
-  `createIpcServer`, `readConfig`.
-- **[typings.md](typings.md)** -- `DaemonConfig`,
-  `IpcHandler`, `IpcRequest`, `IpcResponse`,
-  `IpcErrorDetail`, `WasmModules`.
-- **[constants.md](constants.md)** --
-  `DEFAULT_ITEM_LIMIT`, `DEFAULT_INDEX_LIMIT`,
-  `DEFAULT_TIMEOUT_MS`, `DEFAULT_ITEM_TTL_MS`.
-
-## Startup Sequence
-
-```
-src/worker.ts (entry point)
-  │
-  ├─ startDaemon(contextPath)
-  │    ├─ readConfig(contextPath)
-  │    │    └─ reads worker.toml from .kbdb dir
-  │    ├─ new WorkerDaemon(ctx, config)
-  │    └─ daemon.start()
-  │         ├─ loadWasmModules(config)
-  │         │    ├─ allocate shared WebAssembly.Memory
-  │         │    ├─ compile query-parser.wasm ─┐
-  │         │    ├─ compile fs-database.wasm ──┤
-  │         │    │  (parallel, standalone,     │
-  │         │    │   shared memory)            │
-  │         │    ├─ compile default-embedding ─┤
-  │         │    │  (standalone, no host       │
-  │         │    │   imports, pure computation)│
-  │         │    └─ compile kb-worker.wasm ────┘
-  │         │       (imports both modules)
-  │         ├─ init cache
-  │         ├─ remove stale IPC socket
-  │         ├─ await createIpcServer(ctx, handler)
-  │         │    └─ kb-worker-{ctx}.sock (listening)
-  │         ├─ write PID file
-  │         │    └─ kb-worker-{ctx}.pid
-  │         └─ start idle timer
-  └─ daemon running
-```
-
-## Shutdown Sequence
-
-On idle timeout or OS signal (`SIGTERM`, `SIGINT`):
-
-1. Stop accepting new connections.
-2. Wait for in-flight requests to complete.
-3. Purge all cached data.
-4. Remove PID file from OS temporary directory.
-5. Remove IPC socket file (Unix only).
-6. Exit process.