@dikolab/kbdb 0.4.3 → 0.6.0

package/docs/goals/skills.md

@@ -1,810 +0,0 @@
-# Skills -- Reusable AI Agent Capabilities
-
-*Parameterised prompt templates stored in the knowledge
-base that AI agents can discover, retrieve, and invoke
-by name.*
-
-## Goal
-
-Skills are reusable capability definitions that AI
-agents can discover through MCP and invoke by name.
-Each skill is a prompt template with named argument
-placeholders. At invocation time, an AI agent supplies
-argument values and receives a rendered prompt message
-ready for use in its workflow.
-
-Skills are the composable building blocks for agent
-workflows. Rather than hard-coding prompt text into
-agent code, skills are stored in the knowledge base,
-indexed for search, and surfaced through the MCP
-prompts protocol. This makes them discoverable,
-shareable, and updateable independently of any
-consuming agent.
-
-Skills are stored as knowledge base sections with
-`sectionType = 'skill'`. They benefit from the same
-content-addressing, indexing, and search infrastructure
-as regular sections. The skill name and description are
-weighted during indexing so that agents can find
-relevant skills through keyword search.
-
-## Skill Content Schema
-
-### SkillDefinition type
-
-The full definition of a skill as stored in and
-returned from the knowledge base:
-
-```typescript
-interface SkillDefinition {
-   /** Unique human-readable identifier (kebab-case). */
-   readonly name: string;
-   /** Short sentence describing what the skill does. */
-   readonly description: string;
-   /** Declared arguments. Auto-detected when omitted. */
-   readonly arguments?: readonly SkillArgument[];
-   /** Prompt template body with {{argName}} placeholders. */
-   readonly body: string;
-}
-```
-
-### SkillArgument type
-
-A single named argument accepted by a skill template:
-
-```typescript
-interface SkillArgument {
-   /** Argument name matching a {{name}} placeholder. */
-   readonly name: string;
-   /** Value type hint for the MCP client. */
-   readonly type?: 'string' | 'number' | 'boolean';
-   /** Human-readable description for MCP prompt listing. */
-   readonly description?: string;
-   /** Whether the argument must be supplied for render. */
-   readonly required?: boolean;
-}
-```
-
-### Template Syntax
-
-Skill bodies use a simple `{{argName}}` placeholder
-syntax:
-
-- `{{argName}}` is replaced with the supplied argument
-  value during `prompts/get`.
-- Argument names are **auto-detected** by scanning the
-  body for all `{{...}}` patterns. Auto-detected
-  arguments have no type, description, or required
-  constraint unless explicitly declared in
-  `arguments`.
-- When an argument value is not supplied, the
-  placeholder remains in the rendered output
-  unchanged -- no error is raised.
-- No nested templates, conditionals, loops, or
-  escaping mechanism. `{{` not followed by a valid
-  identifier and `}}` is treated as literal text.
-
-### Skill Content Example
-
-Full JSON representation of a skill as stored on disk
-and returned by `skill-get`:
-
-```json
-{
-   "name": "summarize-document",
-   "description": "Summarize a document into key points
-      with configurable depth",
-   "arguments": [
-      {
-         "name": "docid",
-         "type": "string",
-         "description": "Document identifier to summarize",
-         "required": true
-      },
-      {
-         "name": "max_points",
-         "type": "number",
-         "description": "Maximum number of bullet points",
-         "required": false
-      }
-   ],
-   "body": "Summarize the document identified by
-      {{docid}}.\nExtract at most {{max_points}} key
-      points.\nReturn as a bulleted markdown list."
-}
-```
-
-## Storage
-
-Skills are stored as knowledge base sections with
-`sectionType = 'skill'`. No separate file format or
-entity type is used. This means skills:
-
-- Receive a kbid via the same content-addressing scheme
-  as regular sections.
-- Are indexed by the same term-extraction pipeline.
-- Are returned by `search` when the query matches a
-  skill's name, description, or body terms.
-- Are cleaned up by `gc` when no document references
-  them (though skills are never attached to documents
-  by default).
-
-### Section File Format
-
-Skills are persisted in
-`.kbdb/sections/{prefix}/{kbid}.sec`. The file uses the
-same TOML header / JSON body format as all section
-files:
-
-```toml
-kbid = "abc123def456abc123def456ab"
-docids = []
-type = "skill"
-title = "summarize-document"
-description = "Summarize a document into key points
-   with configurable depth"
-size = 423
-token_count = 35
-heading_token_count = 2
-body_token_count = 35
-code_token_count = 0
-created_at = "2026-06-19T10:00:00Z"
-checksum = "sha256:e3b0c44298fc1c149afb..."
----
-{"name":"summarize-document","description":"Summarize a
-document into key points with configurable depth",
-"arguments":[{"name":"docid","type":"string",
-"description":"Document identifier to summarize",
-"required":true},{"name":"max_points","type":"number",
-"description":"Maximum number of bullet points",
-"required":false}],"body":"Summarize the document
-identified by {{docid}}.\nExtract at most
-{{max_points}} key points.\nReturn as a bulleted
-markdown list."}
-```
-
-Header fields specific to skill sections:
-
-| Field       | Source                              |
-|-------------|-------------------------------------|
-| `type`      | Always `"skill"`                    |
-| `title`     | Skill `name` field                  |
-| `description` | Skill `description` field         |
-
-The JSON body below the `---` separator is the full
-serialised `SkillDefinition` object.
-
-### Indexing
-
-Skill sections are indexed using the standard indexing
-weights:
-
-- `title` (the skill name) is indexed with **2.0x
-  heading weight** -- skill names rank higher in
-  keyword searches than body text.
-- `description` is indexed at **1.0x body weight**.
-- `body` text (the template) is indexed at **1.0x
-  body weight**.
-
-This means an agent searching for `"summarize"` will
-find the `summarize-document` skill through its name
-(higher score) or through occurrences of the word in
-its description or body.
-
-### Content-Addressing
-
-The kbid for a skill section is computed using the
-same SHA-256-based content-addressing scheme used for
-all sections. The kbid is derived from the full
-serialised content of the `SkillDefinition` JSON body.
-
-Two skills with identical names but different bodies
-receive different kbids. Two skills with identical
-bodies receive the same kbid (deduplication applies).
-
-## MCP Surface
-
-### Skills as MCP Prompts
-
-Skills are exposed through the MCP prompts protocol
-(see [MCP](mcp.md) for the full protocol
-specification). Each skill appears as an `McpPrompt`
-entry in the `prompts/list` response and is retrievable
-by name via `prompts/get`.
-
-The skill `name` field maps directly to the MCP prompt
-`name` field. MCP clients discover available skills by
-calling `prompts/list` and invoke them by calling
-`prompts/get` with the skill name and argument values.
-
-### prompts/list
-
-The `prompts/list` request returns all skills as
-`McpPrompt` entries. The response includes skills
-alongside agent prompts (agents use an `agent:` prefix;
-see [Agents](agents.md)):
-
-```json
-{
-   "prompts": [
-      {
-         "name": "summarize-document",
-         "description": "Summarize a document into key
-            points with configurable depth",
-         "arguments": [
-            {
-               "name": "docid",
-               "description": "Document identifier
-                  to summarize",
-               "required": true
-            },
-            {
-               "name": "max_points",
-               "description": "Maximum number of
-                  bullet points",
-               "required": false
-            }
-         ]
-      }
-   ]
-}
-```
-
-Skill arguments are mapped from `SkillArgument` to MCP
-prompt arguments. The `type` field is not included in
-the MCP response (MCP prompt arguments are untyped
-strings from the protocol's perspective).
-
-### prompts/get
-
-The `prompts/get` request supplies a skill name and
-optional argument values:
-
-```json
-{
-   "name": "summarize-document",
-   "arguments": {
-      "docid": "DOC-001",
-      "max_points": "5"
-   }
-}
-```
-
-The response is a single user-role message containing
-the rendered template:
-
-```json
-{
-   "messages": [
-      {
-         "role": "user",
-         "content": {
-            "type": "text",
-            "text": "Summarize the document identified
-               by DOC-001.\nExtract at most 5 key
-               points.\nReturn as a bulleted markdown
-               list."
-         }
-      }
-   ]
-}
-```
-
-Argument values are always strings in the MCP protocol.
-Numeric and boolean argument values are passed as
-their string representation and substituted verbatim
-into the template.
-
-### Argument Substitution
-
-Template rendering applies simple string replacement:
-
-1. For each `{{argName}}` placeholder in the body,
-   look up `argName` in the supplied arguments map.
-2. If found, replace the placeholder with the string
-   value.
-3. If not found, leave the placeholder unchanged
-   (no error).
-4. Return the resulting string as the message text.
-
-Substitution is applied left-to-right, scanning for
-`{{` and matching `}}`. Each placeholder is replaced
-independently. Multiple occurrences of the same
-placeholder are each replaced with the same value.
-
-## MCP Tools
-
-### skill-learn
-
-Create a new skill in the knowledge base.
-
-| Parameter     | Type   | Required | Description                  |
-|---------------|--------|----------|------------------------------|
-| `name`        | string | yes      | Skill name (kebab-case)      |
-| `description` | string | yes      | What the skill does          |
-| `arguments`   | array  | no       | Declared argument definitions|
-| `body`        | string | yes      | Template body text           |
-
-**Returns:** `AddSectionResult` containing the assigned
-kbid:
-
-```json
-{
-   "kbid": "abc123def456abc123def456ab",
-   "name": "summarize-document",
-   "arguments": ["docid", "max_points"]
-}
-```
-
-The `arguments` array in the return value lists the
-auto-detected placeholder names found in the body,
-regardless of whether `arguments` was supplied in the
-request.
-
-Maps to client method:
-`client.addSection({ sectionType: 'skill', ... })`
-
-### skill-list
-
-List all skills stored in the knowledge base.
-
-No parameters.
-
-**Returns:** JSON array of skill summary objects:
-
-```json
-[
-   {
-      "kbid": "abc123def456abc123def456ab",
-      "name": "summarize-document",
-      "description": "Summarize a document into key
-         points with configurable depth",
-      "argumentCount": 2
-   }
-]
-```
-
-Maps to client method: `client.listByType('skill')`
-
-### skill-get
-
-Retrieve the full definition of a skill by name or
-kbid.
-
-| Parameter | Type   | Required | Description              |
-|-----------|--------|----------|--------------------------|
-| `name`    | string | no*      | Skill name to look up    |
-| `kbid`    | string | no*      | Skill kbid to look up    |
-
-\* At least one of `name` or `kbid` must be supplied.
-When both are supplied, `kbid` takes precedence.
-
-**Returns:** Full `SkillDefinition` with kbid:
-
-```json
-{
-   "kbid": "abc123def456abc123def456ab",
-   "name": "summarize-document",
-   "description": "Summarize a document into key
-      points with configurable depth",
-   "arguments": [
-      {
-         "name": "docid",
-         "type": "string",
-         "description": "Document identifier",
-         "required": true
-      }
-   ],
-   "body": "Summarize the document identified by
-      {{docid}}.\nExtract at most {{max_points}}
-      key points.\nReturn as a bulleted markdown list."
-}
-```
-
-Maps to client method: `client.getSection(kbid)` or
-`client.listByType('skill')` filtered by name.
-
-### skill-delete
-
-Remove a skill from the knowledge base by kbid.
-
-| Parameter | Type   | Required | Description            |
-|-----------|--------|----------|------------------------|
-| `kbid`    | string | yes      | Skill kbid to remove   |
-
-**Returns:** Removal confirmation:
-
-```json
-{
-   "kbid": "abc123def456abc123def456ab",
-   "removed": true
-}
-```
-
-When the kbid does not exist, `removed` is `false`
-and no error is thrown.
-
-Maps to client method: `client.removeSection(kbid)`
-
-## CLI Commands
-
-### kbdb skill learn
-
-```
-kbdb skill learn [options] --name <name> --body <file|->
-```
-
-Create a new skill in the knowledge base by reading
-the template body from a file or stdin.
-
-**Options:**
-
-| Option          | Short | Required | Description                    |
-|-----------------|-------|----------|--------------------------------|
-| `--name <n>`    |       | yes      | Skill name (kebab-case)        |
-| `--description` |       | no       | Short description of the skill |
-| `--body <file>` |       | yes      | Template body: file path or `-`|
-| `--db <path>`   |       | no       | Path to .kbdb directory        |
-
-**Behaviour:**
-
-Reads the template body from the specified file path
-or from stdin when `--body -` is given. Scans the body
-for `{{argName}}` placeholders to auto-detect argument
-names. Serialises the `SkillDefinition` to JSON and
-calls `addSection` with `sectionType = 'skill'`.
-
-**Output:**
-
-```json
-{
-   "kbid": "abc123def456abc123def456ab",
-   "name": "summarize-document",
-   "arguments": ["docid", "max_points"]
-}
-```
-
-**Delegation chain:**
-```
-src/cli.ts  parseArgs()
-  -> findDb() / promptUser() / runInit()
-  -> createWorkerClient({ contextPath })
-  -> versionGate(client, dbDir)
-  -> dispatchCommand(client, options)
-       -> runSkillLearn(client, options)
-            -> client.addSection({
-                 sectionType: 'skill',
-                 content: serialisedJson,
-                 title: name,
-                 description })
-                 -> IPC -> worker daemon
-                      -> kb-worker.wasm
-                           -> fs-database.wasm (store)
-                           -> query-parser.wasm (index)
-```
-
-**Examples:**
-
-```sh
-# Create a skill from a file
-kbdb skill learn --name summarize-document \
-   --description "Summarize docs into key points" \
-   --body ./prompts/summarize.txt
-
-# Create a skill from stdin
-cat ./prompts/review.txt | kbdb skill learn \
-   --name code-review --body -
-
-# Specify database directory explicitly
-kbdb skill learn --name my-skill --body ./skill.txt \
-   --db /path/to/project
-```
-
-### kbdb skill list
-
-```
-kbdb skill list [options]
-```
-
-List all skills stored in the knowledge base.
-
-**Options:**
-
-| Option        | Short | Required | Description             |
-|---------------|-------|----------|-------------------------|
-| `--db <path>` |       | no       | Path to .kbdb directory |
-
-**Output:**
-
-JSON array of skill summary objects:
-
-```json
-[
-   {
-      "kbid": "abc123def456abc123def456ab",
-      "name": "summarize-document",
-      "description": "Summarize a document into key
-         points with configurable depth",
-      "argumentCount": 2
-   }
-]
-```
-
-Empty array when no skills are stored.
-
-**Delegation chain:**
-```
-src/cli.ts  parseArgs()
-  -> findDb() / promptUser() / runInit()
-  -> createWorkerClient({ contextPath })
-  -> versionGate(client, dbDir)
-  -> dispatchCommand(client, options)
-       -> runSkillList(client, options)
-            -> client.listByType('skill')
-                 -> IPC -> worker daemon
-                      -> kb-worker.wasm
-                           -> fs-database.wasm (scan)
-```
-
-**Examples:**
-
-```sh
-# List all skills in current directory's database
-kbdb skill list
-
-# List skills from a specific database
-kbdb skill list --db /path/to/project
-```
-
-### kbdb skill get
-
-```
-kbdb skill get [options] <name-or-kbid>
-```
-
-Retrieve the full definition of a skill by its name or
-kbid.
-
-**Arguments:**
-
-- `<name-or-kbid>` -- the skill name or 26-character
-  kbid string. Required.
-
-**Options:**
-
-| Option        | Short | Required | Description             |
-|---------------|-------|----------|-------------------------|
-| `--db <path>` |       | no       | Path to .kbdb directory |
-
-**Output:**
-
-Full `SkillDefinition` with kbid:
-
-```json
-{
-   "kbid": "abc123def456abc123def456ab",
-   "name": "summarize-document",
-   "description": "Summarize a document into key points
-      with configurable depth",
-   "arguments": [
-      {
-         "name": "docid",
-         "type": "string",
-         "description": "Document identifier to summarize",
-         "required": true
-      },
-      {
-         "name": "max_points",
-         "type": "number",
-         "description": "Maximum number of bullet points",
-         "required": false
-      }
-   ],
-   "body": "Summarize the document identified by
-      {{docid}}.\nExtract at most {{max_points}} key
-      points.\nReturn as a bulleted markdown list."
-}
-```
-
-**Delegation chain:**
-```
-src/cli.ts  parseArgs()
-  -> findDb() / promptUser() / runInit()
-  -> createWorkerClient({ contextPath })
-  -> versionGate(client, dbDir)
-  -> dispatchCommand(client, options)
-       -> runSkillGet(client, options)
-            -> client.getSection(kbid)    (if kbid)
-            -> client.listByType('skill') (if name)
-                 -> IPC -> worker daemon
-                      -> kb-worker.wasm
-                           -> fs-database.wasm (fetch)
-```
-
-**Examples:**
-
-```sh
-# Get a skill by name
-kbdb skill get summarize-document
-
-# Get a skill by kbid
-kbdb skill get abc123def456abc123def456ab
-
-# Use explicit database path
-kbdb skill get summarize-document --db /path/to/project
-```
-
-### kbdb skill delete
-
-```
-kbdb skill delete [options] <kbid>
-```
-
-Remove a skill from the knowledge base by kbid.
-
-**Arguments:**
-
-- `<kbid>` -- the 26-character kbid of the skill to
-  remove. Required. Names are not accepted to prevent
-  accidental deletion of the wrong skill when duplicate
-  names exist.
-
-**Options:**
-
-| Option        | Short | Required | Description             |
-|---------------|-------|----------|-------------------------|
-| `--db <path>` |       | no       | Path to .kbdb directory |
-
-**Output:**
-
-```json
-{
-   "kbid": "abc123def456abc123def456ab",
-   "removed": true
-}
-```
-
-**Delegation chain:**
-```
-src/cli.ts  parseArgs()
-  -> findDb() / promptUser() / runInit()
-  -> createWorkerClient({ contextPath })
-  -> versionGate(client, dbDir)
-  -> dispatchCommand(client, options)
-       -> runSkillDelete(client, options)
-            -> client.removeSection(kbid)
-                 -> IPC -> worker daemon
-                      -> kb-worker.wasm
-                           -> fs-database.wasm (delete)
-```
-
-**Examples:**
-
-```sh
-# Delete a skill by kbid
-kbdb skill delete abc123def456abc123def456ab
-
-# Delete from a specific database
-kbdb skill delete abc123def456abc123def456ab \
-   --db /path/to/project
-```
-
-## Edge Cases
-
-**Duplicate skill names.** Two or more skills may share
-the same `name` (they receive different kbids because
-their content differs). `skill-get` by name returns the
-most recently created skill with that name. `skill-list`
-returns all entries. This is not an error.
-
-**Empty body.** A skill with an empty `body` string is
-rejected with an `INVALID_PARAMS` error. A prompt
-template must have non-empty content.
-
-**Body without placeholders.** A skill body that
-contains no `{{...}}` patterns is valid. It is a
-fixed prompt with no arguments. The `arguments` array
-is empty and `prompts/get` returns the body unchanged.
-
-**Invalid JSON in section content.** If a skill
-section's stored JSON body cannot be parsed as a valid
-`SkillDefinition`, `skill-get` and `prompts/get` return
-a parse error for that skill. Other skills are
-unaffected.
-
-**Skill referenced by agent then deleted.** When a
-skill is deleted after an agent has been created
-referencing it, the agent's skill reference becomes a
-dangling kbid. The agent is not deleted. See
-[Agents](agents.md) for how dangling references are
-handled at retrieval time.
-
-## Default Skills
-
-### Overview
-
-When a new database is created via `kbdb db init`, a
-set of default skills are automatically installed.
-These skills provide baseline capabilities that
-improve AI agent effectiveness. Default skills are
-stored as regular skill sections (no special treatment
-in the engine) and can be deleted by users if not
-needed.
-
-### `query-guidance` Default Skill
-
-A built-in skill that teaches AI agents how to
-effectively search the knowledge base. This addresses
-the limitation that lexical search requires
-disciplined query generation for good recall.
-
-**Skill definition:**
-
-```json
-{
-   "name": "query-guidance",
-   "description": "How to effectively search this knowledge base. Read this before issuing search queries.",
-   "arguments": [],
-   "body": "This knowledge base uses lexical search (stemmed BM25F), not semantic/vector search. Follow these guidelines for effective retrieval:\n\n1. Issue MULTIPLE short keyword queries rather than one long natural-language question. Each query should target 2-3 specific terms.\n\n2. EXPAND SYNONYMS yourself. \"login\" will not match \"sign-in\" unless both terms appear in the content. Try alternative phrasings: search for \"login\", then \"sign-in\", then \"authentication\".\n\n3. USE BOOLEAN AND PHRASE SYNTAX:\n   - Exact phrases: \"error handling\"\n   - Boolean AND: auth AND token\n   - Boolean OR: cache OR redis\n   - Exclusion: auth -oauth\n\n4. LEAN ON HEADINGS. Headings carry 2x weight in ranking. Search for terms that are likely in section titles.\n\n5. STEMMING IS AUTOMATIC. \"connections\" matches \"connect\", \"running\" matches \"run\". You do not need to try inflected forms.\n\n6. USE RECALL AFTER SEARCH. After finding relevant sections with search, use recall with increasing depth (0-3) to progressively expand context: depth 0 = full content, depth 1 = parent documents + back-references, depth 2 = siblings + forward references, depth 3 = referenced sections.\n\n7. CODE IDENTIFIERS ARE SPLIT. camelCase and snake_case names are decomposed: getUserById becomes searchable as \"get\", \"user\", \"by\", \"id\".\n\n8. PAGINATION. Default limit is 20 results. Use offset for additional pages if total > limit."
-}
-```
-
-### Installation Behaviour
-
-Default skills are installed during `kbdb db init`.
-The `runDbInit` function installs each default skill
-via `client.addSection({ sectionType: 'skill', ... })`
-after creating the database scaffold.
-
-Default skills receive kbids via the same
-content-addressing scheme as user-created skills.
-Re-running `db init` on a fresh database produces
-identical kbids for the same default skill content.
-
-### `memory-protocol` Default Skill
-
-A built-in skill that teaches AI agents *when* and
-*what* to store in the knowledge base. This addresses
-the risk of the brain filling with transient junk or
-failing to capture durable facts.
-
-**Skill definition:**
-
-```json
-{
-   "name": "memory-protocol",
-   "description": "Guidelines for what to store in this knowledge base and when. Read this before using the learn tool.",
-   "arguments": [],
-   "body": "Follow these guidelines to keep the knowledge base accurate and useful:\n\n1. STORE DURABLE FACTS: decisions, architecture choices, configuration values, corrections to prior knowledge, API contracts, and resolved incidents. These are the facts that matter weeks later.\n\n2. DO NOT STORE: transient conversation context, debugging output, ephemeral task state, raw log dumps, or information that changes every session. The brain should contain knowledge, not activity logs.\n\n3. SEARCH BEFORE STORING. Always search the knowledge base before adding new content. If a section already covers the topic, use learn with --replace to update it rather than creating a duplicate.\n\n4. USE --replace FOR CORRECTIONS. When a fact changes (e.g., rate limit updated from 100 to 200 req/min), re-learn the corrected content with --replace so the old version is superseded. Without --replace, both the old and new versions coexist with equal ranking.\n\n5. ATTACH TAGS for scoping. Use tags like project name, domain, or topic (e.g., --tags auth,api,v2) so queries can be filtered by scope later.\n\n6. PREFER RECENT RESULTS. When search returns multiple sections covering the same topic, prefer the one with the most recent created_at timestamp -- it is more likely to reflect the current state.\n\n7. USE RECALL AFTER SEARCH. After finding relevant sections, use recall with increasing depth (0-3) to expand context progressively before making decisions.\n\n8. INCLUDE HEADINGS. Always provide a descriptive title when storing content. Headings carry 2x ranking weight -- omitting them significantly reduces searchability."
-}
-```
-
-### Future Default Skills
-
-Additional default skills may be added in future
-versions (e.g. `content-authoring-tips`,
-`search-syntax-reference`). Each follows the same
-installation pattern and is a regular skill section.
-
-## Design Constraints
-
-- **Skills are plain sections.** No separate entity
-  type, storage schema, or index is introduced. Skills
-  use `sectionType = 'skill'` within the existing
-  section infrastructure.
-- **Default skills are regular sections.** They use
-  the same storage, indexing, and retrieval
-  infrastructure. There is no distinction between a
-  default skill and a user-created skill beyond the
-  installation source.
-- **No skill versioning.** Updating a skill means
-  creating a new skill section (new kbid) and
-  optionally deleting the old one. The kbid changes
-  whenever the content changes.
-- **Template substitution is simple string
-  replacement.** There is no template engine, no
-  conditionals, no loops, no escaping, and no nested
-  templates. `{{argName}}` is the only syntax.
-- **No template inheritance or composition.** Skills
-  do not reference other skills. Each skill is a
-  self-contained prompt template.