@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/release-notes/0.2.0.md

@@ -1,147 +0,0 @@
-# 0.2.0
-
-*Minor release -- 2026-06-21*
-
-Adds progressive context recall, paginated search
-results with rich metadata, and a bidirectional
-reference graph. These features give AI agents
-the ability to explore related content across
-sections and documents without loading everything
-at once.
-
----
-
-## New Features
-
-- **Recall with layered context (depths 0--3)** --
-  New `recall` command (CLI), MCP tool, and IPC
-  method. Accepts a `kbid` (or `kbids` for batch)
-  and a `depth` parameter that controls how much
-  context is returned:
-  - Depth 0: section content, heading, type, docids
-  - Depth 1: adds parent documents and
-    back-references
-  - Depth 2: adds sibling sections and forward
-    references
-  - Depth 3: referenced sections include full
-    content
-
-- **Paginated search results** -- Search now
-  returns a `PagedSearchResult` envelope:
-  `{ items, total, offset, limit, has_more }`.
-  New `--offset` / `-o` CLI flag. The `offset`
-  parameter is available in MCP and IPC as well.
-
-- **Rich search result metadata** -- Each search
-  result item now includes `heading`, `type`,
-  `snippet` (KWIC ~150 chars around matched terms),
-  `matched_terms`, and `docids`. Replaces the
-  previous `matches: [{term, frequency}]` format.
-
-- **Bidirectional reference graph** -- New
-  `references.idx` tracks content-marker references
-  (`$(link: ...)`, `$(doclink: ...)`, etc.) between
-  sections. New WASM exports `db_get_forward_refs`
-  and `db_get_back_refs`. Built automatically on
-  `learn`/`update`, cleaned on `unlearn`,
-  regenerated on `rebuild`.
-
----
-
-## Improvements
-
-- **KWIC snippet generation** -- New
-  `extract_kwic_snippet()` in the Rust engine
-  produces ~150-character context windows around
-  matched terms with case-insensitive matching,
-  Unicode-safe boundaries, and ellipsis markers.
-
-- **MCP search strips score** -- The MCP layer
-  omits the `score` field from search results.
-  Rank order is the signal for AI agents; raw
-  scores are kept in CLI and IPC output for
-  debugging.
-
-- **Tool annotations** -- All 20 MCP tool
-  definitions now include `openWorldHint: false`,
-  `readOnlyHint`, `destructiveHint`, and
-  `idempotentHint` annotations per the MCP spec.
-
-- **Text format paging** -- CLI text output now
-  shows a paging footer:
-  `Showing 1--20 of 42 results. Use --offset 20 to see more.`
-
-- **MCP tool descriptions** -- `search` guides
-  agents to use `recall` for full content.
-  `retrieve` updated to prefer `recall`.
-
-- **Documentation included in package** -- Both
-  npm and JSR packages now ship `docs/details/`,
-  `docs/goals/`, `docs/modules/`,
-  `docs/release-notes/`, and `docs/overview.md`.
-
----
-
-## Testing
-
-- **548 e2e assertions** on both Node.js and Deno --
-  all passing with 0 failures and 0 skips.
-
-- **Recall depth testing** -- Cross-referenced
-  test data (sections with `$(link: ...)` markers)
-  validates depth 0--3 field presence across CLI,
-  MCP, and worker IPC.
-
-- **Response shape validation** -- Every field in
-  the public API is asserted: paged search envelope,
-  search item metadata, recall depth fields, skill
-  and agent CRUD responses, MCP prompts protocol,
-  resource templates, and completions.
-
-- **Rust unit tests** -- 21+ new tests for KWIC
-  snippet extraction, matched-terms ordering,
-  paged result pagination, and section matching.
-
----
-
-## Breaking Changes
-
-- **Search result shape changed** -- Results are
-  now wrapped in a `PagedSearchResult` envelope.
-  Code that expected a flat array of results must
-  read `result.items` instead. Each item's
-  `matches` array is replaced by `snippet`,
-  `matched_terms`, `heading`, `type`, and `docids`.
-
-- **Codec exports renamed** -- `decodeScoredResults`
-  is now `decodePagedSearchResult`. `ScoredResult`
-  is now `SectionMatch`.
-
----
-
-## Internal
-
-- Rust engine: reference tracking wired into
-  `db_add_section`, `db_update_section`, and
-  `db_remove_section`. `worker_search` parses
-  4-part format (query/mode/limit/offset).
-  `worker_recall` WASM export added.
-- `content-creator` renamed to `content-composer`
-  across all code and docs.
-- WASM binaries rebuilt (~8KB and ~35KB larger).
-
----
-
-## Upgrade
-
-```sh
-# Node.js
-npm install -g @dikolab/kbdb@0.2.0
-
-# Deno
-deno install -Agf jsr:@dikolab/kbdb/cli
-```
-
-Review the **Breaking Changes** section above if
-you use the library API or parse search results
-programmatically.

package/docs/release-notes/0.3.0.md

@@ -1,89 +0,0 @@
-# 0.3.0
-
-*Minor release -- 2026-06-23*
-
-Adds supersession, relaxed queries, database export,
-tags, and confidence scoring. The Rust engine gains
-source tracking, near-duplicate detection, and
-synonym expansion.
-
----
-
-## New Features
-
-- **Supersession** -- Re-learning from the same
-  source automatically replaces outdated sections
-  instead of duplicating them. Source key identity
-  tracked via SHA-256.
-
-- **Relaxed OR fallback** -- When an exact AND
-  query returns no results, kbdb automatically
-  retries with relaxed OR matching so you always
-  get the best available results.
-
-- **Database export** -- New `export` CLI command
-  and MCP tool. Copies sections, documents, and
-  catalog as a self-contained directory snapshot.
-
-- **Tags and replace on learn** -- `--tags` attaches
-  labels to imported sections for scoped queries.
-  `--replace` supersedes existing sections from the
-  same source in one step.
-
-- **Confidence scoring** -- Search results include
-  a normalized BM25F confidence score and
-  `matched_fields` (heading, body, code) so agents
-  can tell strong matches from weak ones.
-
-- **Near-duplicate detection** -- Jaccard similarity
-  warns when imported content closely matches an
-  existing section, preventing silent drift.
-
-- **Default skills** -- `db init` installs
-  `query-guidance` and `memory-protocol` skills
-  that teach agents how to search and what to store.
-
----
-
-## Improvements
-
-- **Synonym expansion pipeline** in the Rust engine
-  broadens keyword coverage automatically.
-
-- **Matched field tracking** -- Results show which
-  fields (heading, body, code) matched the query.
-
-- **`created_at` timestamps** in search results.
-
-- **Divergent content detection** via integrity
-  check catches contradictions between versions.
-
-- **GC fix** -- Garbage collection now only removes
-  superseded sections, not active ones.
-
-- **Reference graph rebuild** included in `rebuild`
-  command.
-
-- **Writer lock** prevents concurrent WASM writes.
-
-- **Worker daemon startup health check** verifies
-  WASM modules load correctly.
-
----
-
-## Testing
-
-- **517 e2e assertions (Node.js), 514 (Deno)** --
-  all pass with 0 failures.
-
----
-
-## Upgrade
-
-```sh
-# Node.js
-npm install -g @dikolab/kbdb@0.3.0
-
-# Deno
-deno install -Agf jsr:@dikolab/kbdb/cli
-```

package/docs/release-notes/0.3.1.md

@@ -1,121 +0,0 @@
-# 0.3.1
-
-*Patch release -- 2026-06-23*
-
-Fixes MCP request processing reliability, agent
-document persistence, WASM memory safety, and
-eliminates all e2e test skips. Adds the 20th MCP
-tool (`auto-capture-review`).
-
----
-
-## Bug Fixes
-
-- **MCP sequential request processing** -- The MCP
-  server previously dispatched JSON-RPC requests
-  concurrently (fire-and-forget). When multiple
-  requests arrived in a batch, later requests could
-  execute before earlier ones finished -- for
-  example, reading an agent document before it was
-  fully written. Requests are now processed
-  sequentially via a promise chain, ensuring each
-  completes before the next begins.
-
-- **Agent document type persistence** -- Agent
-  documents created via `agent-create` were missing
-  their `doc_type` field, so `listDocumentsByType`
-  could not find them. `handleAgentCreate` now
-  passes `'agent'` as the document type.
-
-- **Agent prompt lookup** -- `prompts/get` for
-  agent prompts treated the agent title as a
-  document ID, which failed because document IDs
-  are content hashes. Now looks up agents by title
-  via `listDocumentsByType('agent')`.
-
-- **WASM `worker_remove_grouping` double-free** --
-  The WASM export returned success without calling
-  `set_return`, leaving stale memory in the return
-  slot. The next `readReturnSlot` call freed
-  already-freed memory, causing a WASM trap. Now
-  sets the return slot to null before returning.
-
-- **WASM defensive `set_return` calls** -- Added
-  null return slot initialization to `worker_init`
-  and `worker_init_directory` to prevent the same
-  stale-memory pattern from surfacing in edge cases.
-
----
-
-## New Features
-
-- **`auto-capture-review` MCP tool** -- Review and
-  approve pending auto-captured knowledge before
-  storage. Accepts `action` parameter with values
-  `list`, `approve`, or `reject`. This brings the
-  MCP server to 20 tools.
-
-- **Hybrid search infrastructure** -- New `--algo`
-  flag (CLI) and `algo` parameter (MCP/IPC) with
-  values `lexical`, `vector`, `hybrid`. Embedding
-  module with pluggable provider interface, vector
-  index binary format (KBVX), cosine similarity,
-  and RRF score fusion. Defaults to `lexical`.
-  Concrete embedding providers are deferred -- a
-  `NullEmbeddingProvider` is used until one is
-  configured.
-
----
-
-## Testing
-
-- **548 e2e assertions** on both Node.js and Deno,
-  all passing with **0 failures and 0 skips** (up
-  from 517/514 with some skips in 0.3.0).
-
-- **E2E test restructuring** -- Test databases
-  isolated by category (CLI, MCP, Worker) instead
-  of sharing a flat directory. Worker recall tests
-  now seed their own cross-reference data instead
-  of depending on CLI test data.
-
-- **Pre-built WASM in Docker** -- E2E test
-  container caches WASM binaries at build time,
-  reducing test startup from ~30 seconds to
-  sub-second.
-
----
-
-## Documentation
-
-- **README.md** reorganized for clarity -- shorter
-  intro, labeled paragraphs ("How search works",
-  "Knowledge stays fresh"), accessible to both
-  technical and non-technical readers.
-
-- **Architecture overview** rewritten -- simplified
-  prose, ASCII architecture diagram, removed stale
-  "Remaining" section (WASM host I/O is fully
-  implemented).
-
-- **MCP server guide** updated with
-  `auto-capture-review` tool in reference tables
-  and annotations summary.
-
-- **Tool counts corrected** across all docs
-  (19 -> 20).
-
-- **Assertion counts updated** across all docs
-  (380+/383 -> 548).
-
----
-
-## Upgrade
-
-```sh
-# Node.js
-npm install -g @dikolab/kbdb@0.3.1
-
-# Deno
-deno install -Agf jsr:@dikolab/kbdb/cli
-```

package/docs/release-notes/0.3.2.md

@@ -1,64 +0,0 @@
-# 0.3.2
-
-*Patch release -- 2026-06-23*
-
-Documentation alignment, hybrid search highlight,
-and readability improvements for non-technical users.
-
----
-
-## Documentation
-
-- **Internal doc links use relative paths** --
-  README.md links to CLI reference, MCP guide,
-  library API, Makefile, release notes, overview,
-  and LICENSE now use relative paths instead of
-  absolute GitLab URLs. External links (NPM, JSR,
-  GitLab homepage) remain absolute.
-
-- **Fixed repository URL in overview.md** --
-  Corrected `tech-slaves-internal` to `tech-slaves`.
-
-- **`--algo` added to CLI reference** -- Search
-  Options table now documents the `--algo` flag with
-  values `lexical`, `vector`, `hybrid` and a new
-  Search Algorithms subsection.
-
-- **Hybrid search section in search-and-ranking.md**
-  -- New section explains lexical, vector, and hybrid
-  algorithms with plain-language descriptions of
-  how each works and when to use it.
-
-- **Beginner summary in search-and-ranking.md** --
-  Three-sentence intro at the top so non-technical
-  readers know what the page covers without reading
-  the full technical deep-dive.
-
-- **Feature lists updated** -- README and overview
-  now include auto-capture, sync/export, and
-  near-duplicate detection. Design Documents section
-  in overview now links to hybrid search,
-  auto-capture, and sync goal docs.
-
-- **Readability pass** -- Simplified jargon across
-  README and overview. "RRF score fusion" becomes
-  "blends keyword matching with AI-powered
-  similarity". Technology stack table simplified.
-  MCP setup examples consolidated from three
-  nearly-identical blocks into one with platform
-  notes.
-
-- **"Vibe coders" added to audience list** in
-  overview.md.
-
----
-
-## Upgrade
-
-```sh
-# Node.js
-npm install -g @dikolab/kbdb@0.3.2
-
-# Deno
-deno install -Agf jsr:@dikolab/kbdb/cli
-```

package/docs/release-notes/0.4.0.md

@@ -1,80 +0,0 @@
-# 0.4.0
-
-*Minor release -- 2026-06-25*
-
-Rust engine refactor, daemon reliability, publishing
-fixes, and documentation accuracy audit.
-
----
-
-## Engine
-
-- **Rust engine refactor** -- extracted specs into
-  separate files, split large modules across all
-  four crates for readability and maintainability.
-
-## Daemon
-
-- **Reliability fixes** -- improved daemon startup,
-  error handling, and shutdown behaviour.
-- **UX improvements** -- clearer error messages and
-  exit codes from the worker daemon.
-
-## CLI
-
-- **Optional `--db`** -- the `--db` flag is now
-  optional. kbdb walks up from the current directory
-  to find `.kbdb`. Set `KBDB_DB_DIR` as an
-  alternative. Resolution order: `--db` flag >
-  `KBDB_DB_DIR` env var > upward traversal from cwd.
-- **Request timeout** -- per-request timeout via
-  `--timeout-ms` flag or `KBDB_DAEMON_TIMEOUT` env
-  var (default 30 000 ms).
-- **Automatic retry** -- on timeout, the CLI
-  respawns the daemon and retries. Configurable via
-  `--retry` flag or `KBDB_RETRY` env var (default 5).
-- **Breaking: `KBDB_DIR` renamed to `KBDB_DB_DIR`**
-  -- update MCP server configs and scripts that
-  reference the old variable name.
-
-## MCP Server
-
-- **Version reports package version** -- the MCP
-  `initialize` response now uses the `VERSION`
-  constant instead of a hardcoded `'0.1.0'` string.
-  Clients see the real server version.
-- **Retry on timeout** -- tool calls and resource
-  reads automatically retry with a fresh daemon
-  connection when the worker times out.
-
-## Packaging
-
-- **Trimmed npm package** -- removed raw TypeScript
-  source files from the npm tarball. Package dropped
-  from 908 files / 6.7 MB to 545 files / 4.5 MB.
-  JSR still includes sources as required.
-
-## Documentation
-
-- **README accuracy audit** -- removed overstated
-  claims about reStructuredText/AsciiDoc parsing
-  and sync feature. Added npm/JSR/license badges
-  and a beta status line.
-- **CLAUDE.md updated** -- corrected outdated
-  statement about the WASM-to-host bridge being
-  stubbed. The bridge is fully implemented and
-  tested.
-- **Docs alignment** -- knowledge-base guide updated
-  to match README on supported file formats.
-
----
-
-## Upgrade
-
-```sh
-# Node.js
-npm install -g @dikolab/kbdb@0.4.0
-
-# Deno
-deno install -Agf jsr:@dikolab/kbdb/cli
-```

package/docs/release-notes/0.4.1.md

@@ -1,149 +0,0 @@
-# 0.4.1
-
-*Patch release -- 2026-06-26*
-
-Built-in TF-IDF embedding provider so hybrid search
-works out of the box with zero configuration.
-Structured logging across the entire TypeScript
-layer with configurable verbosity. Daemon lifecycle
-fixes for reliable shutdown on bare Linux TTY.
-Non-interactive mode for scripts, CI, and automated
-testing.
-
----
-
-## Engine
-
-- **New WASM crate: `kbdb-default-embedding`** --
-  computes 256-dimensional feature-hashed sublinear
-  TF vectors using FNV-1a with the sign trick
-  (Weinberger et al., 2009). Pure computation, no
-  host imports.
-- **Pipeline moved to `kbdb-shared`** -- the 9-stage
-  tokenization pipeline (NFC normalization, Unicode
-  segmentation, camelCase splitting, stop-word
-  removal, synonym expansion, Snowball stemming)
-  moved from `kbdb-query-parser` to `kbdb-shared`.
-  Both `kbdb-query-parser` and
-  `kbdb-default-embedding` now share a single source
-  of truth, keeping TF-IDF vectors token-consistent
-  with the lexical index.
-
-## Daemon
-
-- **Loads `default-embedding.wasm` at startup** --
-  the worker daemon loads the new standalone WASM
-  module alongside the existing `kb-worker.wasm`.
-  If the module is missing, the daemon gracefully
-  falls back to `NullEmbeddingProvider`.
-- **Daemon lifecycle fixes** -- socket-level
-  connection tracking replaces the broken
-  server-level `close` handler, so the idle timer
-  fires correctly. `stop()` force-closes lingering
-  sockets. `worker stop` polls until the daemon
-  actually exits (5 s cap) instead of fire-and-forget
-  SIGTERM. `retryOnTimeout` kills a stuck daemon
-  before spawning a fresh one.
-- **Accepts `--log-level`** -- the daemon parses an
-  optional `--log-level <level>` argument forwarded
-  by the spawner from the `KBDB_LOG_LEVEL` env var.
-
-## CLI
-
-- **`--non-interactive` flag** -- suppresses
-  interactive prompts and uses default values.
-  When set, data commands that find no `.kbdb`
-  directory return an error instead of prompting
-  to create one.
-- **`KBDB_NON_INTERACTIVE` env var** -- same effect
-  as `--non-interactive`. Accepts `1` or `true`.
-  CLI flag takes priority.
-- **Deprecation warnings fixed** -- `--init` and
-  `--migrate` deprecation messages now emit at all
-  log levels. Previously they fired before log
-  configuration and were suppressed.
-
-## Embedding
-
-- **`TfidfEmbeddingProvider`** -- new provider backed
-  by the `kbdb-default-embedding` WASM module.
-  Returns 256-dimensional L2-normalized vectors.
-  Model name: `tfidf-v1`.
-- **New factory default** -- when no `[embedding]`
-  section is present in `worker.toml`, the factory
-  returns `TfidfEmbeddingProvider` instead of
-  `NullEmbeddingProvider`. Hybrid and vector search
-  work without configuration.
-- **Explicit disable** -- setting `provider = "none"`
-  in `worker.toml` still returns
-  `NullEmbeddingProvider`, which rejects hybrid and
-  vector search with a structured error.
-
-## Logging
-
-- **New `log` module** (`src/shared/log/`) --
-  structured logging with four levels (`DEBUG`,
-  `INFO`, `WARNING`, `ERROR`), per-level Unicode
-  icons (▸ℹ⚠✖), per-level ANSI colors, and
-  configurable output targets. Default level is
-  `ERROR` (errors only).
-- **CLI verbose flags** -- `-v` (warnings), `-vv`
-  (info), `-vvv` (debug). `-v` is reassigned from
-  `--version` to `--verbose`; `--version` has no
-  short alias.
-- **`KBDB_LOG_LEVEL` env var** -- accepts `debug`,
-  `info`, `warning`, `error` (case-insensitive).
-  CLI flags take priority over the env var.
-- **Comprehensive lifecycle logging** -- milestone
-  logging at key process points across CLI command
-  runners, MCP server lifecycle, worker daemon IPC
-  dispatch, embedding providers, worker client
-  discovery, and auto-capture. Full IPC lifecycle
-  coverage: daemon startup/shutdown, socket
-  connections, request dispatch, retry logic,
-  daemon discovery, write lock events, and idle
-  timer. Pre-throw error logging at all throw
-  sites. All `console.error` calls replaced with
-  `log()`.
-
-## Documentation
-
-- **Hybrid search docs aligned** -- updated
-  architecture, CLI/MCP surface, and design
-  constraint sections to reflect the new WASM crate
-  and `--algo` flag.
-- **Worker daemon docs updated** -- WASM module
-  count, loading diagram, and distribution section
-  now describe four modules.
-- **Factory docs corrected** -- `type:` references
-  fixed to `provider:`, factory mapping table
-  updated for the new default.
-- **Log module docs added** -- module overview,
-  functions reference, typings reference, and
-  constants reference under
-  `docs/modules/typescript/shared/log/`.
-- **CLI and worker daemon docs updated** -- verbose
-  flags (`-v`/`-vv`/`-vvv`), `--log-level` arg,
-  `KBDB_LOG_LEVEL` env var documented across CLI,
-  worker daemon, and shared module pages.
-- **E2E test harness hardened** -- `run_kbdb`
-  wrapper wraps all `kbdb` calls in `timeout -k 5`
-  with SIGKILL fallback. Timeouts report as test
-  failures. GNU `coreutils` added to Docker image.
-  Non-interactive mode enabled for all E2E tests.
-- **Completed todos and issues archived** -- three
-  todo docs (pipeline move, TF-IDF provider, default
-  wiring) and two issue docs (daemon lifecycle hang,
-  hybrid search errors) moved to `archived/`.
-
----
-
-## Upgrade
-
-```sh
-# Node.js
-npm install -g @dikolab/kbdb@0.4.1
-
-# Deno
-deno install -Agf jsr:@dikolab/kbdb/cli
-```