@dikolab/kbdb 0.5.0 → 0.6.0

package/LICENSING.md

@@ -0,0 +1,58 @@
+# Licensing
+
+`@dikolab/kbdb` is **dual-licensed**.
+
+## 1. Open source -- AGPL-3.0-only
+
+The project is available for free under the
+[GNU Affero General Public License v3.0](./LICENSE) (`AGPL-3.0-only`).
+
+You may use, modify, and redistribute it under the terms of that license.
+In short, the AGPL requires that:
+
+- if you distribute the software (modified or not), you make the
+  corresponding source available under the AGPL, and
+- **if you run a modified version to provide a service over a network**,
+  you must offer that service's users the complete source of your
+  modified version.
+
+The network clause (section 13) is the important one: running kbdb -- or a
+derivative of it -- as part of a hosted or SaaS product triggers the
+source-disclosure obligation.
+
+## 2. Commercial license
+
+If the AGPL's obligations don't work for you -- for example, you want to:
+
+- embed kbdb in a **closed-source** product,
+- offer it (or a derivative) as a **hosted / managed service** without
+  disclosing your source, or
+- receive it under different warranty / support / indemnification terms,
+
+you can purchase a **commercial license** that removes the AGPL copyleft
+obligations.
+
+**Contact:** Diko <diko316@gmail.com>
+Please include your company, the intended use, and expected scale.
+
+---
+
+## Why dual licensing?
+
+The AGPL keeps kbdb open and prevents competitors from taking the code,
+running it as a closed service, and giving nothing back. The commercial
+license is how the project funds continued development. Individuals,
+open-source projects, and internal (non-service) use are fully served by
+the free AGPL edition.
+
+## Contributions
+
+By contributing, you agree to the terms in [CLA.md](./CLA.md), which lets
+the maintainer offer the project under both the AGPL and commercial
+licenses. This is required to keep dual licensing possible.
+
+## Note on older versions
+
+Versions of `@dikolab/kbdb` published at or before `0.5.0` were released
+under the ISC license and remain available under ISC. The dual-license
+terms above apply from `0.6.0` onward.

package/README.md

@@ -2,7 +2,7 @@
 
 [![npm](https://img.shields.io/npm/v/@dikolab/kbdb)](https://www.npmjs.com/package/@dikolab/kbdb)
 [![JSR](https://jsr.io/badges/@dikolab/kbdb)](https://jsr.io/@dikolab/kbdb)
-[![License: ISC](https://img.shields.io/badge/license-ISC-blue.svg)](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/LICENSE)
+[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue.svg)](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/LICENSE)
 
 A file-based knowledge base database -- no external
 server to install.
@@ -10,7 +10,7 @@ server to install.
 [GitLab](https://gitlab.com/diko316/knowledge-base-db)
 | [NPM](https://www.npmjs.com/package/@dikolab/kbdb)
 | [JSR](https://jsr.io/@dikolab/kbdb)
-| [License: ISC](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/LICENSE)
+| [License: AGPL-3.0](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/LICENSE)
 
 > **Status:** Beta -- actively developed. Core
 > features (search, recall, MCP) are stable and
@@ -103,8 +103,12 @@ kbdb learn ./docs --db ./my-kb
 Point it at a folder of Markdown files. kbdb reads
 them, breaks them into sections, and builds a
 search index. Add `--tags design,v2` to tag
-sections for scoping, or `--replace` to update
-existing sections from the same source.
+sections for scoping, `--replace` to update
+existing sections from the same source, or
+`--level 2` to set the hierarchical depth
+(1 = broadest, 6 = narrowest). When learning a
+directory, level is auto-detected from folder
+depth.
 
 **3. Search**
 
@@ -113,8 +117,11 @@ kbdb search "how does auth work" --db ./my-kb
 ```
 
 Results are ranked by relevance with snippets
-showing where your terms matched. Use `--offset`
-to page through large result sets.
+showing where your terms matched. Output defaults
+to **recfile** format (one `field: value` per
+line) for easy grepping. Use `--format json` for
+machine-readable JSON, or `--offset` to page
+through large result sets.
 
 To try hybrid search (keyword + AI similarity):
 
@@ -128,11 +135,11 @@ kbdb search "how does auth work" --algo hybrid \
 > omit it. You can also set `KBDB_DB_DIR` as an
 > environment variable.
 
-> **Scripting:** Use `--non-interactive` or set
+> **Scripting:** Add `--format json` to get
+> structured JSON output for parsing. Use
+> `--non-interactive` or set
 > `KBDB_NON_INTERACTIVE=1` to suppress prompts in
-> scripts and CI pipelines. Commands that need a
-> database will return an error instead of waiting
-> for user input.
+> CI pipelines.
 
 **4. Recall context**
 
@@ -170,7 +177,7 @@ Build, search, and maintain your knowledge store.
 - **Rebuild** indexes if anything goes wrong
 
 See the
-[Knowledge Base Guide](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/knowledge-base.md)
+[Knowledge Base Guide](https://diko316.gitlab.io/knowledge-base-db/details/knowledge-base)
 for the full walkthrough, including export and backup.
 
 ---
@@ -192,9 +199,14 @@ Integrate kbdb with AI agents and custom tools.
   timeout and automatic retry with daemon respawn
 - **Worker daemon** lifecycle management --
   stop and restart the background process
+- **Granular Deno permissions** -- the daemon
+  runs with scoped permissions instead of
+  `--allow-all`
+- **Path confinement** -- the daemon rejects
+  path traversal (`..`) in export/import
 
 See the
-[Agent Tooling Guide](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/agent-tooling.md)
+[Agent Tooling Guide](https://diko316.gitlab.io/knowledge-base-db/details/agent-tooling)
 for MCP setup, skills, agents, and the library API.
 
 ---
@@ -230,7 +242,7 @@ or `dbPath` (the parent directory -- kbdb discovers
 `.kbdb` inside it).
 
 See the
-[Library API Reference](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/library-api.md)
+[Library API Reference](https://diko316.gitlab.io/knowledge-base-db/details/library-api)
 for the full API.
 
 ### Development Setup
@@ -253,7 +265,7 @@ HOST_UMASK=$(umask) docker compose run --rm dev bash
 
 Run `make benchmark` to measure search and rebuild
 latency at scale -- results are written to
-[docs/benchmark/benchmark.md](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/benchmark/benchmark.md)
+[docs/benchmark/benchmark.md](https://diko316.gitlab.io/knowledge-base-db/benchmark/benchmark)
 automatically.
 
 See the [Makefile](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/Makefile) for all available
@@ -271,27 +283,38 @@ build targets.
 
 ## Documentation
 
-- [Knowledge Base Guide](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/knowledge-base.md)
+- [Knowledge Base Guide](https://diko316.gitlab.io/knowledge-base-db/details/knowledge-base)
   -- importing, searching, recall, export
-- [Agent Tooling Guide](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/agent-tooling.md)
+- [Agent Tooling Guide](https://diko316.gitlab.io/knowledge-base-db/details/agent-tooling)
   -- MCP, skills, agents, library API
-- [CLI Reference](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/cli.md) -- full command
+- [CLI Reference](https://diko316.gitlab.io/knowledge-base-db/details/cli) -- full command
   list with examples
-- [MCP Server Guide](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/mcp-server.md) --
+- [MCP Server Guide](https://diko316.gitlab.io/knowledge-base-db/details/mcp-server) --
   setup, tools, environment config
-- [Search and Ranking](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/search-and-ranking.md)
+- [Search and Ranking](https://diko316.gitlab.io/knowledge-base-db/details/search-and-ranking)
   -- how search works under the hood
-- [Storage Architecture](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/details/storage.md) --
+- [Storage Architecture](https://diko316.gitlab.io/knowledge-base-db/details/storage) --
   file formats and directory layout
-- [Benchmark Results](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/benchmark/benchmark.md)
+- [Benchmark Results](https://diko316.gitlab.io/knowledge-base-db/benchmark/benchmark)
   -- search and rebuild latency at scale
-- [Release Notes](https://gitlab.com/diko316/knowledge-base-db/-/tree/main/docs/release-notes)
-- [Architecture Overview](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/docs/overview.md)
+- [Release Notes](https://diko316.gitlab.io/knowledge-base-db/release-notes/)
+- [Architecture Overview](https://diko316.gitlab.io/knowledge-base-db/overview)
 
 ---
 
 ## License
 
-ISC License -- see [LICENSE](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/LICENSE) for details.
+This project is dual-licensed:
+
+- **Open source** under the
+  [GNU Affero General Public License v3.0](https://gitlab.com/diko316/knowledge-base-db/-/blob/main/LICENSE)
+  (`AGPL-3.0-only`)
+- **Commercial license** available for closed-source
+  or SaaS use
+
+Versions `<= 0.5.0` remain under the ISC license.
+
+See [LICENSING.md](./LICENSING.md) for details and
+contact information.
 
 ---

package/README.md.bak

@@ -0,0 +1,320 @@
+# @dikolab/kbdb
+
+[![npm](https://img.shields.io/npm/v/@dikolab/kbdb)](https://www.npmjs.com/package/@dikolab/kbdb)
+[![JSR](https://jsr.io/badges/@dikolab/kbdb)](https://jsr.io/@dikolab/kbdb)
+[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue.svg)](./LICENSE)
+
+A file-based knowledge base database -- no external
+server to install.
+
+[GitLab](https://gitlab.com/diko316/knowledge-base-db)
+| [NPM](https://www.npmjs.com/package/@dikolab/kbdb)
+| [JSR](https://jsr.io/@dikolab/kbdb)
+| [License: AGPL-3.0](./LICENSE)
+
+> **Status:** Beta -- actively developed. Core
+> features (search, recall, MCP) are stable and
+> tested.
+
+---
+
+## What is kbdb?
+
+**kbdb** stores your docs in a plain folder on
+disk and makes them searchable. No external server
+to install. No cloud account. Import your Markdown
+files, and kbdb indexes them automatically.
+Search returns ranked results instantly.
+
+It runs anywhere Node.js or Deno runs, and also
+works as an [MCP](https://modelcontextprotocol.io/)
+server -- giving AI agents like Claude a searchable
+second brain that persists between sessions.
+
+**How search works:** kbdb uses **keyword search** by
+default -- synonyms are expanded, terms are ranked
+by relevance, and headings carry 2× weight in
+scoring. When an exact query finds nothing, kbdb
+automatically loosens the match so you still get the
+best available results.
+
+Want smarter results? Use `--algo hybrid` to blend
+keyword matching with similarity search -- finding
+results even when different words describe the same
+concept. The default **TF-IDF** embedding provider
+works offline with zero setup. Swap it for a
+third-party provider (local ONNX model or remote
+API) in `worker.toml` when you need richer
+embeddings.
+
+**Knowledge stays fresh:** Re-learn a file and kbdb
+replaces the old version automatically.
+Near-duplicate detection warns you before content
+drifts. Integrity checks catch contradictions.
+Confidence scores help agents tell strong matches
+from weak ones.
+
+---
+
+## Getting Started
+
+### What You Need
+
+One of these (pick whichever you already have):
+
+- **Node.js** version 20 or newer --
+  [Download](https://nodejs.org/)
+- **Deno** version 2 or newer --
+  [Download](https://deno.com/)
+
+That's it. No database server. No extra tools.
+
+### Install
+
+**Using Node.js:**
+
+```sh
+npm install -g @dikolab/kbdb
+```
+
+**Using Deno:**
+
+```sh
+deno install -Agf jsr:@dikolab/kbdb/cli
+```
+
+### Try It Out
+
+**1. Create a knowledge base**
+
+```sh
+kbdb db init --db ./my-kb
+```
+
+This creates a `.kbdb` folder that holds all your
+data.
+
+**2. Feed it your docs**
+
+```sh
+kbdb learn ./docs --db ./my-kb
+```
+
+Point it at a folder of Markdown files. kbdb reads
+them, breaks them into sections, and builds a
+search index. Add `--tags design,v2` to tag
+sections for scoping, `--replace` to update
+existing sections from the same source, or
+`--level 2` to set the hierarchical depth
+(1 = broadest, 6 = narrowest). When learning a
+directory, level is auto-detected from folder
+depth.
+
+**3. Search**
+
+```sh
+kbdb search "how does auth work" --db ./my-kb
+```
+
+Results are ranked by relevance with snippets
+showing where your terms matched. Output defaults
+to **recfile** format (one `field: value` per
+line) for easy grepping. Use `--format json` for
+machine-readable JSON, or `--offset` to page
+through large result sets.
+
+To try hybrid search (keyword + AI similarity):
+
+```sh
+kbdb search "how does auth work" --algo hybrid \
+   --db ./my-kb
+```
+
+> **Tip:** `--db` is optional. If you're already
+> inside a directory with a `.kbdb` folder, just
+> omit it. You can also set `KBDB_DB_DIR` as an
+> environment variable.
+
+> **Scripting:** Add `--format json` to get
+> structured JSON output for parsing. Use
+> `--non-interactive` or set
+> `KBDB_NON_INTERACTIVE=1` to suppress prompts in
+> CI pipelines.
+
+**4. Recall context**
+
+```sh
+kbdb recall <kbid> --depth 1 --db ./my-kb
+```
+
+Start with a search result's kbid and expand
+context progressively: depth 0 gives the section
+content, depth 1 adds parent documents and
+back-references, depth 2 adds siblings and forward
+references, depth 3 includes full text of
+referenced sections.
+
+---
+
+## Knowledge Base
+
+Build, search, and maintain your knowledge store.
+
+- **Import** Markdown and plain text files with
+  tags and source tracking
+- **Smart updates** -- re-learning a file replaces
+  the old version instead of duplicating it
+- **Search** with three algorithms: keyword
+  (default), AI similarity, or hybrid (both)
+- **Auto-fallback** -- if your exact query finds
+  nothing, kbdb loosens the match automatically
+- **Recall** sections with progressive context --
+  from a quick summary to full related content
+- **Export** -- snapshot your knowledge base for
+  backup
+- **Verify** database integrity and clean up
+  stale data
+- **Rebuild** indexes if anything goes wrong
+
+See the
+[Knowledge Base Guide](./docs/details/knowledge-base.md)
+for the full walkthrough, including export and backup.
+
+---
+
+## Agent Tooling
+
+Integrate kbdb with AI agents and custom tools.
+
+- **MCP server** with 20 tools -- search, recall,
+  learn, export, and more
+- **Skills** -- store reusable prompt templates
+  with fill-in-the-blank arguments
+- **Agents** -- create AI agent profiles that
+  combine a persona with skills
+- **Auto-capture** -- the MCP server can
+  proactively suggest knowledge to store from
+  your conversations
+- **Daemon resilience** -- configurable request
+  timeout and automatic retry with daemon respawn
+- **Worker daemon** lifecycle management --
+  stop and restart the background process
+- **Granular Deno permissions** -- the daemon
+  runs with scoped permissions instead of
+  `--allow-all`
+- **Path confinement** -- the daemon rejects
+  path traversal (`..`) in export/import
+
+See the
+[Agent Tooling Guide](./docs/details/agent-tooling.md)
+for MCP setup, skills, agents, and the library API.
+
+---
+
+## For Developers
+
+### Library API
+
+Use kbdb programmatically in your Node.js or Deno
+project:
+
+```typescript
+import { createWorkerClient } from '@dikolab/kbdb';
+
+// Spawns a background worker if not already running
+const client = await createWorkerClient({
+   contextPath: '/path/to/.kbdb',
+   requestTimeoutMs: 30_000,
+});
+
+const results = await client.search({
+   query: 'authentication',
+   limit: 10,
+   offset: 0,
+});
+
+console.log(results.items);
+client.disconnect();
+```
+
+Pass `contextPath` (the `.kbdb` directory itself)
+or `dbPath` (the parent directory -- kbdb discovers
+`.kbdb` inside it).
+
+See the
+[Library API Reference](./docs/details/library-api.md)
+for the full API.
+
+### Development Setup
+
+```sh
+git clone https://gitlab.com/diko316/knowledge-base-db.git
+cd knowledge-base-db
+npm install
+npm test
+```
+
+### Docker
+
+A Docker setup is included with all build tools
+(Node.js, Deno, Rust, wasm-pack):
+
+```sh
+HOST_UMASK=$(umask) docker compose run --rm dev bash
+```
+
+Run `make benchmark` to measure search and rebuild
+latency at scale -- results are written to
+[docs/benchmark/benchmark.md](./docs/benchmark/benchmark.md)
+automatically.
+
+See the [Makefile](./Makefile) for all available
+build targets.
+
+### Contributing
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes and add tests
+4. Run `npm test` and `npm run lint`
+5. Open a merge request
+
+---
+
+## Documentation
+
+- [Knowledge Base Guide](./docs/details/knowledge-base.md)
+  -- importing, searching, recall, export
+- [Agent Tooling Guide](./docs/details/agent-tooling.md)
+  -- MCP, skills, agents, library API
+- [CLI Reference](./docs/details/cli.md) -- full command
+  list with examples
+- [MCP Server Guide](./docs/details/mcp-server.md) --
+  setup, tools, environment config
+- [Search and Ranking](./docs/details/search-and-ranking.md)
+  -- how search works under the hood
+- [Storage Architecture](./docs/details/storage.md) --
+  file formats and directory layout
+- [Benchmark Results](./docs/benchmark/benchmark.md)
+  -- search and rebuild latency at scale
+- [Release Notes](./docs/release-notes)
+- [Architecture Overview](./docs/overview.md)
+
+---
+
+## License
+
+This project is dual-licensed:
+
+- **Open source** under the
+  [GNU Affero General Public License v3.0](./LICENSE)
+  (`AGPL-3.0-only`)
+- **Commercial license** available for closed-source
+  or SaaS use
+
+Versions `<= 0.5.0` remain under the ISC license.
+
+See [LICENSING.md](./LICENSING.md) for details and
+contact information.
+
+---