@modootoday/pmcp 0.10.0 → 0.12.0

package/README.md

@@ -1,405 +1,142 @@
 # pmcp
 
-Find and use package skills without loading every document into your agent's context.
-`pmcp` reads `SKILL.md` files from installed packages, from monorepo workspaces
-(`--workspace <dir>`: the root and every member its `workspaces` globs name, nested
-workspace roots included), from single package directories (`--package <dir>`), and
-from plugin marketplaces (`--marketplace <dir>`), and exposes six MCP tools: `skill_catalog`, `skill_find`,
-`skill_describe`, `skill_call`, `skill_read` and `skill_bundle`, plus every skill file as a
-`skill://` resource and, on request, one prompt per skill.
+Use the skills your packages already ship, and keep your AI tools' project
+configuration in one place.
 
-## Install
+pmcp discovers `SKILL.md` documents in installed packages, workspace packages
+and plugin marketplaces. Your AI tool can search the catalog and open a skill
+when it needs one, without loading the entire catalog into each conversation.
 
-```sh
-npm install --save-dev @modootoday/pmcp
-```
+Claude Code, Codex, Gemini CLI, Grok Build and Antigravity keep their native
+model calls, account login and permission controls. pmcp supplies skills,
+projects shared configuration into their native formats, and optionally provides
+a local mailbox for exchanging messages.
 
-Register it with an MCP host:
+| What you want to do                                           | Use                            |
+| ------------------------------------------------------------- | ------------------------------ |
+| Find and read package skills from your AI tool                | The MCP skill catalog          |
+| Keep several tools' MCP servers and project assets consistent | `pmcp.toml` and `pmcp project` |
+| Check what an installed tool loads                            | `pmcp doctor`                  |
+| Search across languages using local embeddings                | `pmcp models` and `pmcp index` |
+| Exchange messages between authorized local actors             | The optional mailbox           |
+| Supervise native main and worker terminals locally            | `pmcp harness`                 |
+| Serve a catalog to your own application                       | The HTTP library               |
 
-```json
-{
-  "mcpServers": {
-    "pmcp": { "command": "npx", "args": ["-y", "@modootoday/pmcp"] }
-  }
-}
-```
+## Get started
 
-For Claude Code:
+Requires Node.js 22 or later, or Bun 1.3 or later.
+
+Install pmcp in your project:
 
 ```sh
-claude mcp add pmcp -- npx -y @modootoday/pmcp
+npm install --save-dev @modootoday/pmcp
 ```
 
-## Commands
+For Claude Code, register the MCP server from the project directory:
 
 ```sh
-pmcp list
-pmcp available
-pmcp install --all --dry-run
-pmcp install <skill-package> --yes
-pmcp sync --dry-run
-pmcp login
-pmcp whoami
-pmcp logout
+claude mcp add pmcp -- npx @modootoday/pmcp
 ```
 
-`available` matches the provided catalog against actual installed dependency
-versions locally. `install` and `sync` use npm or Bun, pin exact versions, and
-require confirmation before changing the project's manifest and lockfile.
-`--dry-run` does not install packages. `--catalog <file>` uses a saved catalog
-response instead of fetching one. Registry access is handled by your package
-manager; an OAuth login is not an npm registry token.
+Other MCP hosts can use a stdio entry:
 
-Run `pmcp --help` or `pmcp <command> --help` for options. With no command,
-`pmcp` serves MCP on stdio. Use `--root <node_modules>` to select a local tree
-and repeat `--scope <prefix>` to restrict local package discovery.
+```json
+{
+  "mcpServers": {
+    "pmcp": {
+      "command": "npx",
+      "args": ["@modootoday/pmcp"]
+    }
+  }
+}
+```
 
-## Project config
+Restart or reconnect your tool's MCP connection after changing its configuration.
+Ask your tool to find a skill relevant to your task and open the result.
+Local skill discovery and reading require no pmcp login.
 
-A `pmcp.toml` at or above the working directory sets the catalog once, so a
-host entry needs no flags and works from any package in the repository:
+To include your workspace's own skills, create `pmcp.toml` at the project root:
 
 ```toml
 [catalog]
-workspaces = [".", "packages/*"]
-packages = ["docs"]
-marketplaces = ["vendor/marketplace"]
+workspaces = ["."]
 ```
 
-Paths resolve against the file's directory. Keys are `roots`, `scopes`,
-`workspaces`, `packages` and `marketplaces`, each a list mirroring the flag of
-the same meaning; an unknown key is an error. A flag replaces the file's list
-of its kind. `--config <file>` names another file and `--no-config` ignores it.
+Run `npx @modootoday/pmcp list` to see the local catalog. Paths in the configuration resolve
+relative to `pmcp.toml`; pmcp searches for this file above the working directory.
 
-## Native files for each agent tool
+## Share configuration across your tools
 
-`pmcp project` writes what Claude Code, Codex, Gemini CLI, Grok Build and
-Antigravity read natively, from cross-tool sources under `.agents/`
-(`skills/<name>/SKILL.md`, `agents/<name>/agent.md`, `rules/*.md`) and the rest
-of `pmcp.toml`:
+Declare your tools and MCP servers in the same `pmcp.toml`:
 
 ```toml
 [targets]
 tools = ["claude", "codex", "gemini", "grok", "antigravity"]
 
-[skills.deploy]
-invocation = "user"          # the person starts it; the model is not offered it
-
-[agents.reviewer.claude]
-tools = "Read, Grep"         # fields one tool reads stay out of agent.md
-
-[rules.packages]
-file = ".agent/rules/PACKAGE.RULE.md"   # each package's path-scoped rule
-strip = ["apps"]                        # dropped from derived link names
-
-[mcp.skills]
-command = "node"
-args = ["${PROJECT_ROOT}/node_modules/@modootoday/pmcp/dist/cli.js", "serve"]
+[mcp.docs]
+url = "https://example.com/mcp"
 ```
 
-```sh
-pmcp project              # write
-pmcp project --check      # exit 2 on drift, stale or undeclared outputs
-pmcp project add <name>   # copy a catalog skill or agent into .agents/
-pmcp project remove <name>
-pmcp doctor               # ask each installed tool what it loads
-```
+Replace the example URL with your server, then run:
 
-Rules go to `.claude/rules/<dir>/`, never the top level, because Grok Build
-loads top-level `.claude/rules/*.md` in every session regardless of `paths:`.
-A server that names `${PROJECT_ROOT}` is launched through `sh`, which finds
-`pmcp.toml` at spawn time, so the generated files hold no machine's path.
-`pmcp.lock` records what was written; a file pmcp did not write is never
-overwritten. `doctor` calls no model: it reads `claude mcp list`,
-`codex mcp list`, `gemini skills list`, `gemini mcp list` and
-`grok inspect --json`, and skips a tool that is not installed.
-
-## Package skills
-
-Skills are discovered by filename, including `skills/`, `.agents/skills/`,
-`.agent/skills/`, `.claude/skills/`, `.gemini/skills/`, `.codex/skills/`, and
-`lib/skill/` layouts
-within the discovery depth. A skill needs a frontmatter `description` for search.
-
-`skill_find` accepts a problem description and returns matching skills.
-`skill_call` opens the selected document and lists the reference, script and asset
-files it carries, as text and as resource links. Local MCP reading requires no login.
-
-Files keep their type, as the Agent Skills layout (`references/`, `scripts/`, `assets/`)
-expects: `skill_read` returns text as text, images and audio as image and audio content,
-and other binary files as an embedded resource with its media type; a file whose extension
-names text but whose bytes are not UTF-8 goes out as a blob, so its sha256 stays exact.
-Scripts are returned, never run: running them is the host's job, from a local copy of the
-skill. `skill_bundle` lists every file of a skill as resource links that `resources/read`
-returns, so a host can write the skill into a local skills directory without the bytes
-passing through the model.
-
-`skill_catalog`, `skill_find`, `skill_describe` and `skill_bundle` declare an
-`outputSchema` and return `structuredContent` alongside the same JSON as text.
-Frontmatter is read as YAML (folded and block descriptions, a BOM and CRLF are fine), and
-`metadata` is read for every skill. Loading is lenient, as the client guide asks; what
-breaks the specification is reported, and `pmcp validate [<skill-dir>...]` checks given
-directories or every skill the catalog reaches (name rules and length, description and
-compatibility length, field types, name matching the directory, duplicate names).
-
-`metadata` keeps strings, string lists and maps of those one level down; `skill_describe`
-reports a map as dotted keys, so `requires: {mcp, gpu-gb}` arrives as `requires.mcp` and
-`requires.gpu-gb`. Deeper nesting is dropped.
-
-Output schemas accept additional properties, so a field added later does not break a
-client that cached an earlier tool list. An answer to a call without the newer inputs keeps
-the earlier shape.
-
-Since 0.9.0 a skill can list `metadata.verified-runtimes`, a list drawn from `claude-code`,
-`codex-cli`, `gemini-cli`, `grok-cli` and `antigravity`, meaning its effect was verified on
-that runtime. `skill_find` and `skill_catalog` accept `runtime` and `verifiedOnly`; when either
-is given, items carry `verifiedRuntimes` (omitted when empty). `runtime` never removes a skill: it marks
-each item `verified` and lists verified ones first (within a package group in
-`skill_catalog`). `verifiedOnly` drops the rest, keeping those verified on `runtime`, or on
-any runtime when none is given. Without `runtime` the answer is ordered exactly as before.
-A server can opt in with `rankByClientRuntime: true`: then, when `runtime` is not passed and
-the MCP client's name is recognised (`claude-code`, `codex`, `gemini`, `grok`,
-`antigravity`), that runtime is used for the ordering only, never for filtering.
-`skill_find` then answers with `runtime`, the one the ordering used, or `null`.
-
-`pmcp serve --prompts` also offers one prompt per skill, for hosts that turn prompts into
-commands; the prompt's text is the skill body and its file list, with an optional `task`.
-
-Marketplace plugins also carry commands, agents, hooks and MCP servers.
-Commands are listed as skills. The others are listed under their own kind and
-are left out of `skill_catalog` and `skill_find` unless `kind` asks for them
-(`agent`, `hook`, `mcp`, or `any`); `skill_describe`, `skill_read` and
-`skill_call` work on every kind.
-
-## MCP Skills extension
-
-The server also declares `io.modelcontextprotocol/skills` (SEP-2640): it
-answers `skills/list` and `skills/get` (both with `ttlMs` and `cacheScope`), with
-a sha256 digest and size per file. A skill qualifies when its frontmatter passes the
-Agent Skills specification and its `name` matches its directory, and it stays within
-512 files and 16 MiB; the rest remain reachable through the tools.
-
-Every file of every skill the caller may see is a `skill://pmcp/<package>/<name>/<file>`
-resource: `resources/list` lists each skill's `SKILL.md`, `resources/read` returns any file
-(text, or a blob with its media type), and the template completes skill names and paths.
-List-change notifications and subscriptions are not offered: they need a session, and the
-HTTP handler is stateless.
-
-## Hosting over HTTP
-
-`@modootoday/pmcp/http` serves the same tools and resources over streamable HTTP for a hosted
-catalog. Tools, resources, the Skills extension and prompts all read the same per-caller,
-tier-filtered catalog, so a file of a skill a caller cannot list cannot be read either. The
-extension's `cacheScope` is `private` there by default (`cacheScope` overrides it), and
-`prompts: true` turns prompts on.
-The host owns identity: `authorize(request)` returns the caller or `undefined` (answered 401),
-and `tiersFor(caller)` decides which tiers that caller sees (default `open` and `free`; skills
-without a tier count as `open`). The catalog is read once; call `reload()` after deploying new
-skill releases.
-
-```ts
-import { createSkillHttpHandler } from "@modootoday/pmcp/http";
-
-const skills = createSkillHttpHandler({
-  roots: [],
-  marketplaces: ["/srv/marketplace", "/srv/marketplace-free"],
-  authorize: async (request) =>
-    verifyBearer(request.headers.get("authorization")),
-});
-export default { fetch: (request: Request) => skills.fetch(request) };
+```sh
+npx @modootoday/pmcp project
+npx @modootoday/pmcp project --check
+npx @modootoday/pmcp doctor
 ```
 
-The handler checks requests before they reach MCP:
-
-| Option            | Default | Effect                                                                                                                                                                                                                                  |
-| ----------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `allowedOrigins`  | none    | A request with an `Origin` other than the server's own gets 403. Listed origins are served with CORS headers, and their preflight is answered. `"*"` admits any origin. Requests without `Origin` are not from a browser page and pass. |
-| `publicOrigins`   | none    | The server's own origins as browsers see them, when a proxy ends TLS and the request URL is `http://`. Without it, a same-site page behind such a proxy gets 403.                                                                       |
-| `allowedHosts`    | any     | When set, a `Host` outside the list gets 403 (DNS rebinding).                                                                                                                                                                           |
-| `maxRequestBytes` | 1 MiB   | A larger body gets 413; a declared length is refused before reading.                                                                                                                                                                    |
-| `rateLimit`       | off     | `{ perMinute }` per caller id; past it, 429 with `retry-after`.                                                                                                                                                                         |
+`project` generates native files for the selected tools and records its ownership
+in `pmcp.lock`. It preserves unrelated settings and refuses conflicting unmanaged
+entries. `--check` reports drift without writing files. Each runtime retains its
+own OAuth session.
 
-An MCP config (`.mcp.json`) is never served as written: every `env` and `headers` value is
-replaced with `<redacted>`, whichever entry reaches the file, and listed digests describe
-the redacted bytes.
+The [configuration guide](docs/project-configuration.md) covers importing
+existing servers, environment references, shared skills and rules, shell guards
+and memory metadata.
 
-Pass `semantic: { modelId: "embeddinggemma-2", cacheDir, localOnly: true }` to rank by meaning: the handler encodes
-the catalog at start with `@huggingface/transformers` and the multilingual model, so an
-intent in one language finds a description in another. Requests rank lexically until
-`ready()` settles, and stay lexical when the encoder cannot load (`onError` says why).
+## Everyday commands
 
-## Optional local mailbox
-
-Mailbox is disabled by default. The six catalog tools stay read-only. A launcher
-that supplies a private actor credential adds five tools: mailbox_send,
-mailbox_inbox, mailbox_read, mailbox_ack, and mailbox_sessions. Read and ack
-record durable receipts; peer text is data and does not authorize execution.
+Run these with `npx @modootoday/pmcp`, or with `pmcp` when it is on your PATH:
 
 ```sh
-pmcp mailbox init --config ./pmcp.toml
-pmcp mailbox actor --config ./pmcp.toml --actor reviewer --actor-file /private/reviewer.actor.json
-pmcp serve --config ./pmcp.toml --mailbox-actor-file /private/reviewer.actor.json --mailbox-runtime codex-cli
-```
-
-Initialization preserves existing configuration and creates a project UUID.
-State defaults to XDG_STATE_HOME/pmcp/projects/<project UUID>, outside the
-repository. Credential and binding files contain secrets: keep them private,
-outside source control and container images. Commands do not print credentials.
-An optional mailbox-binding-file records the current attachment for a trusted
-launcher. Replacement requires the exact mailbox-replace-session ID; stale
-sessions are fenced. Canonical runtimes are codex-cli, claude-code, gemini-cli,
-grok-cli, and antigravity; the Antigravity executable is agy.
-
-The mailbox CLI also supports send, inbox, read, ack, sessions, rotate, export,
-backup, and restore. Actor operations accept a private actor-file or existing
-binding-file and JSON input file. Administration requires explicit operator
-intent. Backups and exports are sensitive. Encrypted recovery bundles are
-available through the mailbox API; key custody belongs to the operator.
-
-```ts
-import { MailboxService, MailboxStore } from "@modootoday/pmcp/mailbox";
-
-const store = await MailboxStore.open({
-  enabled: true,
-  projectId,
-  databasePath,
-});
-const credential = await store.registerActor("reviewer");
-const binding = await store.attach(credential, "codex-cli");
-const mailbox = new MailboxService(store, binding);
+pmcp list
+pmcp available
+pmcp install --all --dry-run
+pmcp install <skill-package> --yes
+pmcp sync --dry-run
+pmcp validate
 ```
 
-Pass mailbox to createSkillServer or registerMailboxTools on an existing server.
-For HTTP, provide mailboxFor(caller) to createSkillHttpHandler; the host resolves
-authenticated callers to authorized bindings. Tool arguments cannot choose a
-sender identity. The host owns attachment heartbeat and cleanup. Child actors
-have bounded authority and may reply only to their parent. Generation changes,
-parent expiry, and credential rotation revoke stale child authority.
+`available` checks catalog packages against your installed dependencies.
+`install` and `sync` pin exact versions with npm or Bun and require confirmation
+before changing your manifest and lockfile. `--dry-run` previews the work.
+Use `--catalog <file>` to read a saved catalog response.
 
-The public package contains one reusable local domain. Organization identity,
-fleet policy, enterprise audit retention, native process supervision, provider
-credentials, and automatic runtime input delivery belong to a host or dedicated
-harness. Existing arbitrary terminal sessions are not captured. Elastic License
-2.0 and NOTICE continue to apply.
+`pmcp login`, `whoami` and `logout` manage pmcp service login. Package registry
+access uses your package manager's credentials. Your runtime's provider login
+remains with that runtime.
 
-## Switching embedding models
+Run `pmcp --help` or `pmcp <command> --help` for options.
+With no command, pmcp serves MCP on stdio.
 
-List the built-in presets with `pmcp models` (or `pmcp models --json`). Select a
-model once in your project's `pmcp.toml`; both indexing and the stdio server use
-the same settings:
+## Guides
 
-```toml
-[embedding]
-model = "embeddinggemma-2"
-dimensions = 256
-cache_dir = ".cache/pmcp/models"
-```
+| Guide                                                  | Contents                                                      |
+| ------------------------------------------------------ | ------------------------------------------------------------- |
+| [Project configuration](docs/project-configuration.md) | Catalog sources, native files, MCP import, guards and doctor  |
+| [Skills and MCP reference](docs/skills.md)             | Skill documents, tools, resources, prompts and compatibility  |
+| [Embedding models](docs/embeddings.md)                 | E5, EmbeddingGemma and custom local model selection           |
+| [Local mailbox](docs/mailbox.md)                       | Actor credentials, messages, receipts and library integration |
+| [HTTP hosting](docs/hosting.md)                        | Authentication, catalog access and request controls           |
+| [Native terminal control](docs/terminal-harness.md)    | Owned sessions, explicit input, Docker profiles and recovery  |
+| [Security policy](SECURITY.md)                         | Reporting vulnerabilities                                     |
 
-```sh
-pmcp index
-pmcp serve
-```
+Semantic search is optional and requires `@huggingface/transformers` installed
+separately. Lexical search works without downloading a model. The mailbox is
+disabled until explicitly configured.
 
-Indexing explicitly downloads missing model artifacts. Serving reads only local
-artifacts and an existing, complete index. A missing model, stale catalog or
-incompatible vectors keeps ranking lexical and reports the reason on stderr.
-`pmcp index --dry-run` plans work without loading a model or writing files;
-`pmcp index --local-only` refuses downloads. Use `[embedding] enabled = false`
-or `pmcp serve --lexical` to disable semantic ranking.
-
-| Preset                | Default precision | Supported output dimensions |
-| --------------------- | ----------------- | --------------------------- |
-| `e5-small`            | `q8`              | 384                         |
-| `embeddinggemma-300m` | `q4`              | 768, 512, 256, 128          |
-| `embeddinggemma-2`    | `q4`              | 768, 512, 256, 128          |
-
-E5 remains the default. Gemma presets use qualified, pinned revisions, listed by
-`pmcp models --json`; E5 retains its existing `main` revision for compatibility.
-Gemma dimension reduction truncates and re-normalizes the projected embedding;
-it reduces vector storage, not loaded model memory.
-
-CLI flags override individual config fields: `--model`, `--dtype`,
-`--dimensions`, `--model-revision`, `--model-path`, `--model-cache`,
-`--query-prefix`, `--passage-prefix` and `--pooling`. Omit precision and dimensions
-in the file when you want each model's defaults. Use `--no-config` for an
-independent CLI selection. Config paths resolve relative to the TOML file;
-CLI model paths resolve relative to the working directory.
-
-Query caches also include the encoder identity, so sharing a cache between
-same-width profiles cannot reuse another encoder's query vector. Library hosts
-using openIntentCache must open a cache with the selected output width when
-changing dimensions; pass the encoder's indexKey as its modelId for explicit
-profile-scoped cache metadata.
-
-Each model, revision, precision, output width and encoder specification has its
-own vector identity. Switching preserves other models' indexes; switching back
-reuses unchanged vectors. `--force` re-encodes only the selected model. A local
-`model_path` must contain the declared revision's artifacts; change `revision`
-and reindex after replacing them. The directory's location is not its identity.
-
-Custom Transformers.js feature-extraction models need no source edits:
-
-```toml
-[embedding]
-model = "your-organization/your-onnx-model"
-dtype = "q8"
-revision = "your-pinned-revision"
-
-[embedding.encoder]
-query_prefix = "query: "
-passage_prefix = "passage: "
-pooling = "mean"
-```
-
-Use the model's documented prefixes and pooling (`mean` or `cls`); unknown models
-default to unprefixed mean pooling. A native `sentence_embedding` export may use
-`projected = true`. Explicit dimension reduction is supported for the listed
-presets. Other architectures require compatible Transformers.js exports.
-
-The library API shares the same presets and identity rules:
-
-```js
-import { loadEmbedder, prepareSearch } from "@modootoday/pmcp";
-
-const encoder = await loadEmbedder("embeddinggemma-2", {
-  dimensions: 256,
-  modelPath: "/opt/models/embeddinggemma-2",
-  localOnly: true,
-});
-const search = await prepareSearch(
-  { roots: ["node_modules"] },
-  {
-    indexPath: "node_modules/.cache/pmcp/vectors.sqlite",
-    modelId: "embeddinggemma-2",
-    dimensions: 256,
-    embedder: encoder ?? undefined,
-  },
-);
-```
+## Licence
 
-HTTP hosts use these options under `semantic`; HTTP catalogs are encoded in
-memory at startup rather than reading the CLI's persisted index. Model runtime
-settings are passed to each loader without changing global download or cache
-policy, so hosts can prepare different encoders in one process.
-
-## Runtime and licence
-
-Node.js 22 or later, or Bun 1.3 or later. Ranking is lexical until an index and
-encoder are ready. Semantic
-ranking (the `index` command, or the HTTP handler's `semantic` option) needs
-`@huggingface/transformers` installed separately; it is not part of the default
-installation. The default encoder is `Xenova/multilingual-e5-small` (MIT, about
-130 MB), so vectors indexed with the earlier English-only default are re-encoded.
-
-The optional encoder also supports the ONNX Community EmbeddingGemma 300M and
-EmbeddingGemma 2 exports. It uses their projected sentence embeddings and task
-prefixes. EmbeddingGemma 2 loads its text encoder without vision or audio models.
-EmbeddingGemma was qualified with Transformers.js 4.3.1. Install a version
-supporting the chosen export and keep model
-weights outside the npm package. Regenerate document vectors when changing the
-encoder. Quantization reduces weight storage; application RAM and latency need
-workload measurements. EmbeddingGemma 300M uses the Gemma terms; EmbeddingGemma 2
-is listed under Apache-2.0 in its model card.
-
-Elastic License 2.0. See `LICENSE` and `NOTICE` for terms and dependency notices.
-Documentation: <https://pmcp.build>.
+pmcp is source available under [Elastic License 2.0](LICENSE).
+See [NOTICE](NOTICE) for dependency notices and the embedding guide for model
+licences. Website: [pmcp.build](https://pmcp.build).