@klhapp/skillmux 1.1.0 → 1.3.0

package/docs/deployment.md

@@ -0,0 +1,250 @@
+# Deployment
+
+Choose a deployment from the client count and inference source:
+
+| Use case | Recommended package | Transport | Inference |
+| --- | --- | --- | --- |
+| Native skill management | Skillmux CLI | Filesystem | None required |
+| Local MCP retrieval | Skillmux CLI | stdio | Downloaded GTE-small |
+| Shared MCP with local inference | Full Docker image | Streamable HTTP | Bundled GTE-small |
+| Shared MCP with remote or lexical retrieval | Slim Docker image (advanced) | Streamable HTTP | Remote endpoint or lexical fallback |
+
+Install the CLI with either the Bun package or standalone Linux executable;
+they expose the same commands and can also serve HTTP. Deploy the full Docker
+image for a shared service by default. Docker can serve stdio when a client
+requires a container command; use slim only for remote or lexical retrieval.
+
+“Local inference” means the model runs in the Skillmux process. It does not
+mean that the MCP client must run on the same machine.
+
+## Skillmux CLI
+
+Run stdio MCP beside one client:
+
+```sh
+skillmux models download
+skillmux index
+skillmux serve
+```
+
+Run an HTTP service without Docker:
+
+```sh
+skillmux serve --transport http --port 3000
+```
+
+The native HTTP server binds `127.0.0.1` by default. Configure authentication
+and a reachable hostname before serving other machines.
+
+## Docker images
+
+Skillmux publishes Linux AMD64 and ARM64 images to GHCR and Docker Hub:
+
+| Variant | GHCR tag | Contents |
+| --- | --- | --- |
+| Full | `ghcr.io/klhq/skillmux:latest` | Runtime plus bundled GTE-small |
+| Slim | `ghcr.io/klhq/skillmux:latest-slim` | Runtime without model files; remote or lexical retrieval |
+
+Docker Hub mirrors the same tags under `docker.io/klhq/skillmux`.
+
+Neither image bundles a local reranker. Configure a remote reranker when your
+retrieval policy needs one.
+
+Use the full image when the service should run embeddings itself:
+
+```sh
+docker run -d \
+  --name skillmux \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  -p 3000:3000 \
+  ghcr.io/klhq/skillmux:latest
+```
+
+The container sets:
+
+- `VAULT_PATH=/vault`;
+- `STATE_DIR=/data`;
+- `PORT=3000`;
+- `RUNNING_IN_DOCKER=true`.
+
+Mount the vault read-only for a retrieval-only service. Run `install`, `init`,
+`sync`, and other filesystem management commands on the host. Containerized
+native management is intentionally rejected: agent directories belong to the
+host CLI, where their symlinks resolve correctly.
+
+## Container command contract
+
+The image separates its executable from its default command. Running an image
+without arguments starts Streamable HTTP MCP on port 3000:
+
+```sh
+docker run ghcr.io/klhq/skillmux:latest
+```
+
+Arguments after the image replace that default, so one-shot maintenance and
+stdio use the same image:
+
+```sh
+docker run --rm -v ~/skills:/vault:ro -v skillmux-data:/data \
+  ghcr.io/klhq/skillmux:latest doctor
+docker run --rm -v ~/skills:/vault:ro -v skillmux-data:/data \
+  ghcr.io/klhq/skillmux:latest index
+```
+
+Docker Compose `command:` and Kubernetes `args:` likewise replace only the
+default command, not the executable.
+
+The supported container commands are `serve`, `index`, `doctor`, `report`,
+`scan`, `skill which`, and read-only `config show`, `config get`,
+`config validate`, `config diff`, and `config status`.
+
+The image rejects host-management commands, including `init`, `sync`,
+`install`, `project`, `target`, `core`, `local-vault`, `models download`,
+context management, calibration, evaluation, and configuration initialization
+or mutation. Install the Skillmux CLI on the host when a command needs to
+manage a local vault or agent directory.
+
+## Slim image
+
+The slim image stays ready in lexical mode without an inference endpoint.
+Configure remote embeddings to enable hybrid retrieval:
+
+```sh
+docker run -d \
+  --name skillmux-slim \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  -p 3000:3000 \
+  -e EMBED_ENDPOINT="https://embedding.example.com/v1/embeddings" \
+  -e EMBED_MODEL="your-embedding-model" \
+  -e EMBED_DIMENSION="1024" \
+  ghcr.io/klhq/skillmux:latest-slim
+```
+
+Set `SKILLMUX_CONFIG` and mount a TOML file when you need reranking, calibrated
+thresholds, server policy, or API-key environment names:
+
+```sh
+docker run -d \
+  --name skillmux-slim \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  -v "$PWD/config.toml:/etc/skillmux/config.toml:ro" \
+  -e SKILLMUX_CONFIG=/etc/skillmux/config.toml \
+  -e EMBEDDING_API_KEY \
+  -e RERANKER_API_KEY \
+  -p 3000:3000 \
+  ghcr.io/klhq/skillmux:latest-slim
+```
+
+Start from [config.remote.example.toml](../config.remote.example.toml).
+
+## Docker over stdio
+
+Some local clients can launch a container as their stdio MCP command:
+
+```sh
+docker run --no-healthcheck -i --rm \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  ghcr.io/klhq/skillmux:latest serve --transport stdio
+```
+
+The container must keep standard input open, so use `-i`. Disable the baked
+HTTP health check for stdio because no HTTP listener is started.
+
+## Expose HTTP safely
+
+The Skillmux CLI binds `127.0.0.1`. Docker binds `0.0.0.0` so
+port mapping works. Before exposing the port beyond a trusted host:
+
+```toml
+[server]
+hostname = "0.0.0.0"
+auth_enabled = true
+auth_token_env = "SKILLMUX_AUTH_TOKEN"
+allowed_origins = []
+
+[server.rate_limit]
+enabled = true
+requests_per_minute = 60
+trust_proxy = false
+```
+
+Set the token in the process environment:
+
+```sh
+export SKILLMUX_AUTH_TOKEN="replace-with-a-long-random-token"
+skillmux serve --transport http
+```
+
+Clients send:
+
+```text
+Authorization: Bearer replace-with-a-long-random-token
+```
+
+`allowed_origins` controls browser CORS requests. Requests without an `Origin`
+header, including MCP clients and curl, do not need a CORS entry.
+
+Keep `trust_proxy = false` unless a trusted reverse proxy overwrites
+`X-Forwarded-For`. A client can spoof that header when it reaches Skillmux
+directly.
+
+## Health and metrics
+
+The HTTP server provides:
+
+| Endpoint | Purpose |
+| --- | --- |
+| `GET /health/live` | Process liveness |
+| `GET /health/ready` | Vault, index, inference, and active capability |
+| `GET /health` | Compatibility alias for liveness |
+| `GET /metrics` | Prometheus text exposition |
+| `GET /stats` | Aggregated routing outcomes for `skillmux report` |
+| `POST /mcp` | Streamable HTTP MCP transport |
+
+The Docker health check calls `/health/ready`.
+
+Prometheus metrics cover request totals, resolve outcomes, latency, errors, and
+rate-limit rejections. Health and metrics do not require bearer authentication,
+but `/stats` does when server authentication is enabled. CORS still applies to
+browser requests.
+
+## Remote administration
+
+Name a deployed server without storing its token:
+
+```sh
+skillmux context add prod \
+  --server https://skillmux.example.com \
+  --token-env SKILLMUX_PROD_ADMIN_TOKEN
+skillmux context use prod
+skillmux config status
+```
+
+The context stores the token environment variable name. Export its value in
+the shell before running admin commands.
+
+Enable the admin API and use a separate admin token in server configuration.
+Read [CLI reference](cli.md#administrative-http-api-adminv1) for routes and
+[Configuration](configuration.md#http-server) for reload behavior.
+
+## Persistent data and backups
+
+Persist `state_dir` to retain the index, audit log, and calibration evidence.
+Skill content remains in the vault and should use its own backup or Git
+workflow.
+
+Treat the state database as sensitive because audit rows can contain raw user
+queries. Stop the process or use SQLite-safe backup tooling before copying a
+live database.
+
+## Native pins with shared retrieval
+
+For the combined topology, use one Git-backed vault source of truth: each
+machine that manages native skills keeps its own checkout and runs the CLI;
+the shared service mounts its own checkout for retrieval. Skillmux does not
+pull, push, replicate, or otherwise keep those vault checkouts fresh—Git and
+your deployment process own that responsibility.

package/docs/getting-started.md

@@ -0,0 +1,239 @@
+# Getting started
+
+Skillmux supports three setup paths. Pick the result you want before choosing
+an installation.
+
+| Goal | Skill delivery | Recommended installation |
+| --- | --- | --- |
+| [Manage native skills](#manage-native-skills) | Managed links in client skill directories | Skillmux CLI |
+| [Add local MCP retrieval](#add-local-mcp-retrieval) | Local stdio MCP | Skillmux CLI |
+| [Run a shared MCP service](#run-a-shared-mcp-service) | Streamable HTTP MCP | Full Docker image |
+
+Native management and local MCP retrieval can run together. Complete both
+recipes if you want pinned skills plus on-demand access to the rest of the
+vault.
+
+## Install the CLI
+
+Use the Bun package on macOS, Linux, or Windows. It requires
+[Bun 1.3 or newer](https://bun.sh/docs/installation):
+
+```sh
+bun add -g @klhapp/skillmux
+skillmux --help
+```
+
+Native target sync needs permission to create directory symlinks on Windows.
+
+Linux users without Bun can install the standalone AMD64 or ARM64 executable
+instead:
+
+```sh
+gh release download --repo klhq/skillmux --pattern 'skillmux-linux-*'
+gh attestation verify skillmux-linux-amd64 --repo klhq/skillmux
+chmod +x skillmux-linux-amd64
+sudo install skillmux-linux-amd64 /usr/local/bin/skillmux
+```
+
+Replace `amd64` with `arm64` on ARM64. The Bun package and standalone Linux
+executable expose the same Skillmux CLI commands.
+
+## Prepare a vault
+
+Skillmux defaults to `~/skills`. Each direct child directory represents one
+skill:
+
+```text
+~/skills/
+├── code-context/
+│   └── SKILL.md
+└── csv-formatter/
+    ├── SKILL.md
+    └── references/
+```
+
+Each `SKILL.md` needs valid Agent Skills frontmatter. With the Skillmux CLI,
+you can install a skill from Git:
+
+```sh
+skillmux install owner/repo
+skillmux install owner/repo/path/to/skill
+```
+
+Or create a small skill:
+
+```sh
+mkdir -p ~/skills/csv-formatter
+cat > ~/skills/csv-formatter/SKILL.md <<'EOF'
+---
+name: CSV Formatter
+description: Convert CSV or spreadsheet data into clean Markdown tables.
+---
+
+# CSV Formatter
+
+Read the first row as headers. Right-align numbers and left-align text.
+EOF
+```
+
+Run `skillmux scan ~/skills` before adopting an existing collection.
+
+## Manage native skills
+
+Run the guided setup on the machine that owns the client skill directories:
+
+```sh
+skillmux init
+```
+
+The planner:
+
+1. validates the vault;
+2. detects clients from filesystem evidence;
+3. asks which skills belong in the core tier;
+4. shows the config, target, instruction, and sync plan;
+5. applies the plan after confirmation.
+
+Skillmux writes machine config under `~/.config/skillmux`, stores tier policy
+in `~/skills/skillmux.toml`, and records its entries in each target's
+`.skillmux` marker. It preserves unmanaged files and existing instruction
+text.
+
+Use explicit flags for automation:
+
+```sh
+skillmux init \
+  --client claude-code \
+  --client codex \
+  --core csv-formatter \
+  --dry-run
+
+skillmux init \
+  --client claude-code \
+  --client codex \
+  --core csv-formatter \
+  --yes
+```
+
+Verify native delivery:
+
+```sh
+skillmux sync
+skillmux doctor
+skillmux skill which csv-formatter
+```
+
+Repeated syncs are idempotent. Add project-specific skills from a repository
+root:
+
+```sh
+skillmux project init
+```
+
+The noninteractive form accepts repeatable client and skill flags:
+
+```sh
+skillmux project init ~/code/my-project \
+  --name my-project \
+  --client claude-code \
+  --client codex \
+  --skill code-context \
+  --yes
+```
+
+Continue with [Managing skills](skill-management.md) for pinning, target
+ownership, overlays, and recovery.
+
+## Add local MCP retrieval
+
+Run this recipe on the same machine as the MCP client. The default inference
+configuration uses quantized GTE-small embeddings on CPU.
+
+Prefetch the model, build the index, and check readiness:
+
+```sh
+skillmux models download
+skillmux index
+skillmux doctor
+```
+
+The model cache lives at `~/.cache/skillmux/models`. If you skip
+`models download`, Skillmux downloads the model when local inference first
+loads it.
+
+Start the stdio server:
+
+```sh
+skillmux serve
+```
+
+Register it with your MCP client:
+
+```json
+{
+  "mcpServers": {
+    "skillmux": {
+      "command": "skillmux",
+      "args": ["serve"]
+    }
+  }
+}
+```
+
+The client launches the process and closes it with the MCP session. Continue
+with [MCP routing](mcp-routing.md) for client behavior and retrieval outcomes.
+
+## Run a shared MCP service
+
+Use the full image when you want local embeddings without configuring an
+external inference endpoint:
+
+```sh
+docker run -d \
+  --name skillmux \
+  -v ~/skills:/vault:ro \
+  -v skillmux-data:/data \
+  -p 3000:3000 \
+  ghcr.io/klhq/skillmux:latest
+```
+
+The full image includes GTE-small and is the shared-service default. The slim
+image is an advanced option for configured remote embeddings or intentional
+lexical-only retrieval; it contains no model files. Neither image includes a
+local reranker. Configure remote embeddings on slim when you need hybrid
+retrieval. Docker Hub publishes the same tags under `docker.io/klhq/skillmux`.
+
+Check the service:
+
+```sh
+curl http://127.0.0.1:3000/health/ready
+```
+
+Register `http://127.0.0.1:3000/mcp` as a Streamable HTTP MCP endpoint. Enable
+authentication before exposing the service beyond a trusted host. Continue
+with [Deployment](deployment.md) for remote inference, network policy,
+monitoring, and backups.
+
+Manage the mounted vault on the host. A retrieval-only container should mount
+it read-only.
+
+## Combine native pins with shared retrieval
+
+Use this topology when users need native core or project pins and also one
+shared MCP endpoint. Keep one Git-backed vault as the source of truth:
+
+- each machine that owns client skill directories keeps its own checkout and
+  runs the Skillmux CLI for `init`, pinning, and `sync`;
+- the shared server mounts its own checkout and serves routed retrieval over
+  HTTP;
+- MCP-only clients connect to the shared server and do not need the CLI.
+
+Skillmux does not pull, push, replicate, or make those checkouts fresh. Git
+and your deployment process own vault replication and freshness.
+
+## Next steps
+
+- [Concepts](concepts.md) explains delivery, deployment, and inference terms.
+- [Configuration](configuration.md) documents local and remote inference.
+- [Troubleshooting](troubleshooting.md) lists common `doctor`, model, and
+  transport failures.

package/docs/mcp-routing.md

@@ -0,0 +1,172 @@
+# MCP routing
+
+Skillmux exposes one vault through two Model Context Protocol tools. Choose a
+transport based on where the process runs:
+
+| Topology | Transport | Typical package |
+| --- | --- | --- |
+| Skillmux beside one client | stdio | Skillmux CLI |
+| Shared Skillmux service | Streamable HTTP | Full Docker image by default; slim for remote or lexical retrieval |
+
+Both transports expose the same `resolve_skill` and `fetch_skill` contract.
+Local native pinning is optional and can run beside stdio MCP.
+
+## Register a stdio server
+
+Start the server:
+
+```sh
+skillmux index
+skillmux doctor
+skillmux serve
+```
+
+Register the command in your MCP client:
+
+```json
+{
+  "mcpServers": {
+    "skillmux": {
+      "command": "skillmux",
+      "args": ["serve"]
+    }
+  }
+}
+```
+
+The default transport is stdio. The process exits when the client closes its
+input stream or sends a termination signal.
+
+## Connect to a shared HTTP service
+
+Start a Streamable HTTP server:
+
+```sh
+skillmux serve --transport http --port 3000
+```
+
+Clients send MCP requests to:
+
+```text
+http://127.0.0.1:3000/mcp
+```
+
+The default host accepts loopback connections only. Configure authentication
+and network exposure before serving other machines. See
+[Deployment](deployment.md#expose-http-safely).
+
+## Tool contract
+
+### `resolve_skill`
+
+Input:
+
+```json
+{
+  "query": "convert this spreadsheet to a Markdown table"
+}
+```
+
+Skillmux returns one outcome:
+
+- `matched`: `structuredContent` contains match metadata and the text content
+  contains the `SKILL.md` body once;
+- `ambiguous`: `structuredContent` contains up to `candidate_limit` candidates;
+- `no_match`: the agent continues with its normal workflow.
+
+### `fetch_skill`
+
+Input:
+
+```json
+{
+  "skill_id": "csv-formatter"
+}
+```
+
+The response contains the current `SKILL.md` body as text content.
+`structuredContent` contains the skill ID, title, content SHA-256, and
+supporting-file paths. Fetch does not depend on an earlier resolve call.
+
+The complete wire contract lives in [schema.json](schema.json).
+
+## Recommended agent behavior
+
+Give the calling client these rules:
+
+1. Call `resolve_skill` when a task may benefit from a specialized workflow.
+2. Follow the delivered skill on `matched`.
+3. On `ambiguous`, choose the best candidate and call `fetch_skill`.
+4. On `no_match`, continue without loading a skill.
+
+Do not treat the first ambiguous candidate as an automatic match. Skillmux
+uses ambiguity to keep the final choice with the calling model when it lacks
+enough confidence.
+
+## Retrieval pipeline
+
+Skillmux builds candidates in stages:
+
+1. SQLite FTS5 ranks lexical matches with BM25.
+2. Local or remote embeddings rank semantic similarity.
+3. Reciprocal-rank fusion combines both lists.
+4. An optional reranker scores the fused candidates.
+5. Calibrated thresholds select `matched`, `ambiguous`, or `no_match`.
+
+The default local inference configuration uses FTS5 and quantized
+`Xenova/gte-small` embeddings. Skillmux CLI installations cache the
+downloaded model under `~/.cache/skillmux/models`; the full Docker image
+includes it. The slim image starts with lexical retrieval and can call an
+OpenAI-compatible embedding endpoint.
+
+Remote inference also supports `jina-v1` or `bifrost-v1` reranker adapters.
+
+Without a reranker, Skillmux returns a shortlist and does not auto-match.
+Without calibrated thresholds, a configured reranker orders the shortlist but
+still does not auto-match.
+
+Read [Configuration](configuration.md#local-inference) for local and remote
+inference settings. Read [Policy calibration](calibration.md) before enabling
+automatic matches.
+
+## Fallback and readiness
+
+Skillmux advertises the capability it can support:
+
+- embedding failure falls back to lexical retrieval;
+- reranker failure preserves the hybrid shortlist;
+- vault or index failure marks the server unready.
+
+Check the active mode:
+
+```sh
+skillmux doctor
+curl http://127.0.0.1:3000/health/ready
+```
+
+A ready server reports `lexical`, `hybrid`, or `reranked` along with skill and
+index status.
+
+## Content integrity
+
+The index stores metadata and retrieval state. Delivery reads the file from
+disk, computes its SHA-256, and returns those current bytes. If indexed
+metadata has gone stale, Skillmux refreshes it before delivery.
+
+Supporting files remain in the vault. `fetch_skill` lists their relative paths
+so the calling workflow can locate them through its configured filesystem
+access.
+
+## Audit data
+
+Each resolve request records:
+
+- timestamp and query;
+- retrieval capability and outcome;
+- candidates with scores;
+- selected skill ID, when present;
+- latency.
+
+Skillmux stores audit rows in the SQLite database under `state_dir`. Use
+`skillmux report` to summarize activity. Treat raw queries as private user
+data when backing up or sharing the database.

package/docs/releasing.md

@@ -39,9 +39,10 @@ The release workflow publishes:
 - `skillmux-linux-arm64`
 - GitHub build provenance attestations when the repository is public
 - Full image to GHCR and Docker Hub: `:<version>`, `:<major>.<minor>`,
-  and `:latest`
+  and `:latest`; this variant includes GTE-small
 - Slim image to GHCR and Docker Hub: `:<version>-slim`,
-  `:<major>.<minor>-slim`, and `:latest-slim`
+  `:<major>.<minor>-slim`, and `:latest-slim`; this variant contains no model
+  files and uses remote embeddings or lexical fallback
 - Multi-architecture `linux/amd64` and `linux/arm64` images with SBOM and
   provenance
 
@@ -86,7 +87,7 @@ Verify the container with a read-only vault mount:
 
 ```bash
 docker run --rm \
-  -v ~/.agents/skills:/vault:ro \
+  -v ~/skills:/vault:ro \
   -p 3000:3000 \
   ghcr.io/klhq/skillmux:latest