@klhapp/skillmux 1.2.0 → 1.3.0

package/docs/concepts.md

@@ -0,0 +1,165 @@
+# Concepts
+
+Skillmux separates three decisions:
+
+| Decision | Choices |
+| --- | --- |
+| Skill delivery | Native core/project pins or routed MCP retrieval |
+| Process location | On the client machine or as a shared HTTP service |
+| Packaging | Skillmux CLI installation or Skillmux server deployment |
+
+Retrieval capability is a fourth, independent concern: lexical, hybrid,
+reranked, or exact retrieval.
+
+These decisions are independent. A local CLI can manage native pins and serve
+stdio MCP at the same time. A shared service uses HTTP MCP and can run local or
+remote inference.
+
+## Canonical vault
+
+The vault is the source collection for Skillmux. Each direct child directory
+represents one skill and contains a `SKILL.md`.
+
+The default path is `~/skills`. Set `vault_path` in
+`~/.config/skillmux/config.toml` when you keep the collection elsewhere.
+
+Skillmux commands interact with the vault in two ways:
+
+- management commands such as `install` and explicit config operations write
+  to documented paths;
+- MCP retrieval reads skill content and stores indexes and audit records under
+  `state_dir`.
+
+MCP delivery reads the current file bytes. It does not copy an indexed
+`SKILL.md` body from the state database.
+
+## Delivery tiers
+
+Skillmux applies three policies to one vault:
+
+| Tier | Scope | Delivery |
+| --- | --- | --- |
+| Core | Each configured target | Native client skill directory |
+| Project | Selected project paths and targets | Project-local native skill directory |
+| Routed | Full indexed vault | MCP on demand |
+
+Core and project skills are **pinned**. `skillmux sync` creates managed
+symlinks for them. Routed skills stay in the vault until an MCP client asks for
+one.
+
+One skill can serve different roles across machines or projects, but the
+shared manifest prevents conflicting core and project assignments. Core stays
+capped at 25 skills to protect client startup context.
+
+Delivery tiers do not select a deployment. A local Skillmux process can serve
+routed skills over stdio, while a shared Skillmux process can serve the same
+vault over HTTP.
+
+## Deployment topologies
+
+| Topology | Process location | Transport | Typical installation |
+| --- | --- | --- | --- |
+| Native management | Client machine | Filesystem links | Skillmux CLI |
+| Local MCP | Beside one client | stdio | Skillmux CLI |
+| Shared MCP | Server or container host | Streamable HTTP | Full Skillmux server image |
+
+The **Skillmux CLI** is available as either the Bun package or the standalone
+Linux executable; both expose the same commands. The **full Skillmux server
+image** is the default shared-service deployment. The **slim image** is an
+advanced variant for configured remote embeddings or intentional lexical-only
+retrieval. The CLI can also serve HTTP, and Docker can serve stdio for clients
+that support a container command. Those combinations use the same MCP tools;
+the table lists the shortest setup for each use case.
+
+The full Docker image bundles GTE-small. The slim image contains no model
+files, so it uses configured remote embeddings or lexical fallback. The Bun
+CLI distributions download and cache GTE-small when local inference
+first loads it; `skillmux models download` prefetches it. Neither Docker image
+bundles a local reranker; configure one remotely when needed.
+
+## Clients and targets
+
+A **client** is a supported product name such as `claude-code` or `codex`.
+Skillmux maps it to the product's skill directory and safe instruction-file
+conventions.
+
+A **target** is a physical directory managed by sync. Several clients can map
+to one target. Gemini CLI, OpenCode, GitHub Copilot, and Windsurf share
+`~/.agents/skills`, so Skillmux deduplicates that directory.
+
+Custom targets let you manage another directory without adding a product
+adapter:
+
+```sh
+skillmux target add custom-agent --dir /srv/custom-agent/skills --yes
+```
+
+## Ownership markers
+
+Each managed target contains a `.skillmux` marker. The marker records the
+target name, vault, schema version, and entries created by Skillmux.
+
+Sync removes only recorded entries. It refuses to adopt an unmarked directory,
+overwrite unmanaged collisions, or treat a local overlay marker as a target
+marker.
+
+Run `skillmux init --dry-run` before changing a target. Read
+[Managing skills](skill-management.md#target-ownership-and-recovery) before
+undoing an adopted target.
+
+## Project groups
+
+A project group connects:
+
+- one or more local project paths;
+- a set of skill IDs;
+- selected targets.
+
+Skillmux materializes each group inside the project using the target's path
+relative to the user's home directory. A shared `skillmux.toml` can list
+checkout paths from several machines. Sync skips paths that do not exist on
+the current machine.
+
+## Local vault overlays
+
+`local_vault_paths` layer machine-specific skill copies over the canonical
+vault. Skillmux checks overlay paths in order, then falls back to `vault_path`.
+
+Use overlays for work in progress or machine-specific variants. Keep portable
+core and project pins in the canonical vault because another machine may not
+have the overlay.
+
+## Inference and retrieval capabilities
+
+Inference location and deployment location use separate settings. Local
+inference runs GTE-small inside the Skillmux process. Remote inference calls
+configured embedding and reranker endpoints. Either inference choice can back
+an HTTP MCP deployment.
+
+Skillmux reports the active retrieval capability:
+
+| Capability | Behavior |
+| --- | --- |
+| `lexical` | SQLite FTS5 and BM25 produce an ordered shortlist |
+| `hybrid` | Reciprocal-rank fusion combines lexical and embedding results |
+| `reranked` | A configured reranker reorders the fused candidates |
+| `exact` | An exact skill ID resolves directly |
+
+A reranker does not enable automatic matches by itself. Skillmux needs
+calibrated `match_score`, `match_margin`, and `candidate_floor` thresholds
+before it returns a semantic result as `matched`.
+
+## Retrieval outcomes
+
+`resolve_skill` returns one of three outcomes:
+
+- `matched`: one skill passed the calibrated policy, so Skillmux delivers its
+  `SKILL.md` body inline;
+- `ambiguous`: Skillmux returns an ordered candidate list and the calling model
+  chooses one with `fetch_skill`;
+- `no_match`: no candidate passed the policy and the agent continues without a
+  skill.
+
+Embedding or reranker failures reduce the active capability. Vault and index
+failures make the server unready because Skillmux can no longer guarantee
+valid retrieval.

package/docs/configuration.md

@@ -1,11 +1,23 @@
 # Configuration
 
-Skillmux defaults to FTS5 plus local GTE-small semantic retrieval. Most users need no config file.
+Skillmux manages one canonical vault and defaults to FTS5 plus GTE-small
+running in the Skillmux process. Most users need no config file.
 
-For detailed CLI command reference, target resolution, and automation
-envelopes, see [`docs/cli.md`](cli.md). For labelled datasets, threshold
-certification, reference values, and the apply lifecycle, see
-[`docs/calibration.md`](calibration.md).
+Deployment and inference use separate terms:
+
+- **local deployment**: Skillmux runs beside a client, usually over stdio;
+- **shared deployment**: Skillmux serves clients over HTTP;
+- **local inference**: the Skillmux process runs the embedding model;
+- **remote inference**: Skillmux calls configured inference endpoints.
+
+A shared HTTP deployment can use local inference. A local stdio deployment can
+use remote inference.
+
+Read [Concepts](concepts.md) for the vault and delivery model. For detailed CLI
+commands, target resolution, and automation envelopes, see
+[CLI reference](cli.md). For labelled datasets, threshold certification,
+reference values, and the apply lifecycle, see
+[Policy calibration](calibration.md).
 
 ## Machine config bootstrap
 
@@ -26,14 +38,21 @@ It validates that the path resolves to a directory with at least one
 `local_vault_paths` unset. `skillmux init --vault ~/skills --yes` uses the
 same bootstrap when the machine config does not exist.
 
-## Local mode
+## Local inference
 
 ```toml
 [inference]
 mode = "local"
 ```
 
-The versioned `gte-small-v1` bundle uses normalized, mean-pooled `Xenova/gte-small` embeddings (384 dimensions), quantized to q8 on CPU. Models are cached in `~/.cache/skillmux/models`. FTS5 and cosine result lists are combined with reciprocal-rank fusion; without a reranker the calling LLM selects from the ordered shortlist.
+The versioned `gte-small-v1` configuration uses normalized, mean-pooled
+`Xenova/gte-small` embeddings with 384 dimensions, quantized to q8 on CPU.
+Skillmux CLI installations download the model when inference first
+loads it and cache it in `~/.cache/skillmux/models`. The full Docker image
+already contains the model.
+
+Skillmux combines FTS5 and cosine result lists with reciprocal-rank fusion.
+Without a reranker, the calling model selects from the ordered shortlist.
 
 Advanced local overrides:
 
@@ -50,9 +69,10 @@ dtype = "q8"
 
 ```
 
-Use `skillmux models download` to prefetch models and `skillmux doctor` to verify readiness.
+Use `skillmux models download` to prefetch the model and `skillmux doctor` to
+verify readiness.
 
-## Remote mode
+## Remote inference
 
 See [`config.remote.example.toml`](../config.remote.example.toml). Embeddings
 must implement the OpenAI-compatible `{ model, input }` contract. Configure the
@@ -137,7 +157,11 @@ Before exposing HTTP beyond localhost, set `hostname` to a reachable interface,
 
 ## Tiers and the manifest
 
-`skillmux init`/`sync` manage an optional second delivery path — pinning a subset of skills as real symlinks inside an agent's own skill directory, instead of routing every request through `resolve_skill`. See the README's [Tiers](../README.md#tiers-routed-vs-pinned) section for the concept and a walkthrough; this is the manifest reference.
+`skillmux init` and `skillmux sync` manage native delivery by pinning selected
+skills as symlinks inside an agent's skill directory. Routed skills remain
+available through `resolve_skill`. Read [Concepts](concepts.md#delivery-tiers)
+for the model and [Managing skills](skill-management.md) for the workflow.
+This section defines the manifest.
 
 ### `skillmux.toml`
 

package/docs/deployment.md

@@ -0,0 +1,250 @@
+# Deployment
+
+Choose a deployment from the client count and inference source:
+
+| Use case | Recommended package | Transport | Inference |
+| --- | --- | --- | --- |
+| Native skill management | Skillmux CLI | Filesystem | None required |
+| Local MCP retrieval | Skillmux CLI | stdio | Downloaded GTE-small |
+| Shared MCP with local inference | Full Docker image | Streamable HTTP | Bundled GTE-small |
+| Shared MCP with remote or lexical retrieval | Slim Docker image (advanced) | Streamable HTTP | Remote endpoint or lexical fallback |
+
+Install the CLI with either the Bun package or standalone Linux executable;
+they expose the same commands and can also serve HTTP. Deploy the full Docker
+image for a shared service by default. Docker can serve stdio when a client
+requires a container command; use slim only for remote or lexical retrieval.
+
+“Local inference” means the model runs in the Skillmux process. It does not
+mean that the MCP client must run on the same machine.
+
+## Skillmux CLI
+
+Run stdio MCP beside one client:
+
+```sh
+skillmux models download
+skillmux index
+skillmux serve
+```
+
+Run an HTTP service without Docker:
+
+```sh
+skillmux serve --transport http --port 3000
+```
+
+The native HTTP server binds `127.0.0.1` by default. Configure authentication
+and a reachable hostname before serving other machines.
+
+## Docker images
+
+Skillmux publishes Linux AMD64 and ARM64 images to GHCR and Docker Hub:
+
+| Variant | GHCR tag | Contents |
+| --- | --- | --- |
+| Full | `ghcr.io/klhq/skillmux:latest` | Runtime plus bundled GTE-small |
+| Slim | `ghcr.io/klhq/skillmux:latest-slim` | Runtime without model files; remote or lexical retrieval |
+
+Docker Hub mirrors the same tags under `docker.io/klhq/skillmux`.
+
+Neither image bundles a local reranker. Configure a remote reranker when your
+retrieval policy needs one.
+
+Use the full image when the service should run embeddings itself:
+
+```sh
+docker run -d \
+  --name skillmux \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  -p 3000:3000 \
+  ghcr.io/klhq/skillmux:latest
+```
+
+The container sets:
+
+- `VAULT_PATH=/vault`;
+- `STATE_DIR=/data`;
+- `PORT=3000`;
+- `RUNNING_IN_DOCKER=true`.
+
+Mount the vault read-only for a retrieval-only service. Run `install`, `init`,
+`sync`, and other filesystem management commands on the host. Containerized
+native management is intentionally rejected: agent directories belong to the
+host CLI, where their symlinks resolve correctly.
+
+## Container command contract
+
+The image separates its executable from its default command. Running an image
+without arguments starts Streamable HTTP MCP on port 3000:
+
+```sh
+docker run ghcr.io/klhq/skillmux:latest
+```
+
+Arguments after the image replace that default, so one-shot maintenance and
+stdio use the same image:
+
+```sh
+docker run --rm -v ~/skills:/vault:ro -v skillmux-data:/data \
+  ghcr.io/klhq/skillmux:latest doctor
+docker run --rm -v ~/skills:/vault:ro -v skillmux-data:/data \
+  ghcr.io/klhq/skillmux:latest index
+```
+
+Docker Compose `command:` and Kubernetes `args:` likewise replace only the
+default command, not the executable.
+
+The supported container commands are `serve`, `index`, `doctor`, `report`,
+`scan`, `skill which`, and read-only `config show`, `config get`,
+`config validate`, `config diff`, and `config status`.
+
+The image rejects host-management commands, including `init`, `sync`,
+`install`, `project`, `target`, `core`, `local-vault`, `models download`,
+context management, calibration, evaluation, and configuration initialization
+or mutation. Install the Skillmux CLI on the host when a command needs to
+manage a local vault or agent directory.
+
+## Slim image
+
+The slim image stays ready in lexical mode without an inference endpoint.
+Configure remote embeddings to enable hybrid retrieval:
+
+```sh
+docker run -d \
+  --name skillmux-slim \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  -p 3000:3000 \
+  -e EMBED_ENDPOINT="https://embedding.example.com/v1/embeddings" \
+  -e EMBED_MODEL="your-embedding-model" \
+  -e EMBED_DIMENSION="1024" \
+  ghcr.io/klhq/skillmux:latest-slim
+```
+
+Set `SKILLMUX_CONFIG` and mount a TOML file when you need reranking, calibrated
+thresholds, server policy, or API-key environment names:
+
+```sh
+docker run -d \
+  --name skillmux-slim \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  -v "$PWD/config.toml:/etc/skillmux/config.toml:ro" \
+  -e SKILLMUX_CONFIG=/etc/skillmux/config.toml \
+  -e EMBEDDING_API_KEY \
+  -e RERANKER_API_KEY \
+  -p 3000:3000 \
+  ghcr.io/klhq/skillmux:latest-slim
+```
+
+Start from [config.remote.example.toml](../config.remote.example.toml).
+
+## Docker over stdio
+
+Some local clients can launch a container as their stdio MCP command:
+
+```sh
+docker run --no-healthcheck -i --rm \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  ghcr.io/klhq/skillmux:latest serve --transport stdio
+```
+
+The container must keep standard input open, so use `-i`. Disable the baked
+HTTP health check for stdio because no HTTP listener is started.
+
+## Expose HTTP safely
+
+The Skillmux CLI binds `127.0.0.1`. Docker binds `0.0.0.0` so
+port mapping works. Before exposing the port beyond a trusted host:
+
+```toml
+[server]
+hostname = "0.0.0.0"
+auth_enabled = true
+auth_token_env = "SKILLMUX_AUTH_TOKEN"
+allowed_origins = []
+
+[server.rate_limit]
+enabled = true
+requests_per_minute = 60
+trust_proxy = false
+```
+
+Set the token in the process environment:
+
+```sh
+export SKILLMUX_AUTH_TOKEN="replace-with-a-long-random-token"
+skillmux serve --transport http
+```
+
+Clients send:
+
+```text
+Authorization: Bearer replace-with-a-long-random-token
+```
+
+`allowed_origins` controls browser CORS requests. Requests without an `Origin`
+header, including MCP clients and curl, do not need a CORS entry.
+
+Keep `trust_proxy = false` unless a trusted reverse proxy overwrites
+`X-Forwarded-For`. A client can spoof that header when it reaches Skillmux
+directly.
+
+## Health and metrics
+
+The HTTP server provides:
+
+| Endpoint | Purpose |
+| --- | --- |
+| `GET /health/live` | Process liveness |
+| `GET /health/ready` | Vault, index, inference, and active capability |
+| `GET /health` | Compatibility alias for liveness |
+| `GET /metrics` | Prometheus text exposition |
+| `GET /stats` | Aggregated routing outcomes for `skillmux report` |
+| `POST /mcp` | Streamable HTTP MCP transport |
+
+The Docker health check calls `/health/ready`.
+
+Prometheus metrics cover request totals, resolve outcomes, latency, errors, and
+rate-limit rejections. Health and metrics do not require bearer authentication,
+but `/stats` does when server authentication is enabled. CORS still applies to
+browser requests.
+
+## Remote administration
+
+Name a deployed server without storing its token:
+
+```sh
+skillmux context add prod \
+  --server https://skillmux.example.com \
+  --token-env SKILLMUX_PROD_ADMIN_TOKEN
+skillmux context use prod
+skillmux config status
+```
+
+The context stores the token environment variable name. Export its value in
+the shell before running admin commands.
+
+Enable the admin API and use a separate admin token in server configuration.
+Read [CLI reference](cli.md#administrative-http-api-adminv1) for routes and
+[Configuration](configuration.md#http-server) for reload behavior.
+
+## Persistent data and backups
+
+Persist `state_dir` to retain the index, audit log, and calibration evidence.
+Skill content remains in the vault and should use its own backup or Git
+workflow.
+
+Treat the state database as sensitive because audit rows can contain raw user
+queries. Stop the process or use SQLite-safe backup tooling before copying a
+live database.
+
+## Native pins with shared retrieval
+
+For the combined topology, use one Git-backed vault source of truth: each
+machine that manages native skills keeps its own checkout and runs the CLI;
+the shared service mounts its own checkout for retrieval. Skillmux does not
+pull, push, replicate, or otherwise keep those vault checkouts fresh—Git and
+your deployment process own that responsibility.