@modootoday/pmcp 0.11.0 → 0.13.0

package/docs/embeddings.md

@@ -0,0 +1,125 @@
+# Embedding models
+
+[Back to the overview](../README.md)
+
+## Switching embedding models
+
+List the built-in presets with `pmcp models` (or `pmcp models --json`). Select a
+model once in your project's `pmcp.toml`; both indexing and the stdio server use
+the same settings:
+
+```toml
+[embedding]
+model = "embeddinggemma-2"
+dimensions = 256
+cache_dir = ".cache/pmcp/models"
+```
+
+```sh
+pmcp index
+pmcp serve
+```
+
+Indexing explicitly downloads missing model artifacts. Serving reads only local
+artifacts and an existing, complete index. A missing model, stale catalog or
+incompatible vectors keeps ranking lexical and reports the reason on stderr.
+`pmcp index --dry-run` plans work without loading a model or writing files;
+`pmcp index --local-only` refuses downloads. Use `[embedding] enabled = false`
+or `pmcp serve --lexical` to disable semantic ranking.
+
+| Preset                | Default precision | Supported output dimensions |
+| --------------------- | ----------------- | --------------------------- |
+| `e5-small`            | `q8`              | 384                         |
+| `embeddinggemma-300m` | `q4`              | 768, 512, 256, 128          |
+| `embeddinggemma-2`    | `q4`              | 768, 512, 256, 128          |
+
+E5 remains the default. Gemma presets use qualified, pinned revisions, listed by
+`pmcp models --json`; E5 retains its existing `main` revision for compatibility.
+Gemma dimension reduction truncates and re-normalizes the projected embedding;
+it reduces vector storage, not loaded model memory.
+
+CLI flags override individual config fields: `--model`, `--dtype`,
+`--dimensions`, `--model-revision`, `--model-path`, `--model-cache`,
+`--query-prefix`, `--passage-prefix` and `--pooling`. Omit precision and dimensions
+in the file when you want each model's defaults. Use `--no-config` for an
+independent CLI selection. Config paths resolve relative to the TOML file;
+CLI model paths resolve relative to the working directory.
+
+Query caches also include the encoder identity, so sharing a cache between
+same-width profiles cannot reuse another encoder's query vector. Library hosts
+using openIntentCache must open a cache with the selected output width when
+changing dimensions; pass the encoder's indexKey as its modelId for explicit
+profile-scoped cache metadata.
+
+Each model, revision, precision, output width and encoder specification has its
+own vector identity. Switching preserves other models' indexes; switching back
+reuses unchanged vectors. `--force` re-encodes only the selected model. A local
+`model_path` must contain the declared revision's artifacts; change `revision`
+and reindex after replacing them. The directory's location is not its identity.
+
+Custom Transformers.js feature-extraction models need no source edits:
+
+```toml
+[embedding]
+model = "your-organization/your-onnx-model"
+dtype = "q8"
+revision = "your-pinned-revision"
+
+[embedding.encoder]
+query_prefix = "query: "
+passage_prefix = "passage: "
+pooling = "mean"
+```
+
+Use the model's documented prefixes and pooling (`mean` or `cls`); unknown models
+default to unprefixed mean pooling. A native `sentence_embedding` export may use
+`projected = true`. Explicit dimension reduction is supported for the listed
+presets. Other architectures require compatible Transformers.js exports.
+
+The library API shares the same presets and identity rules:
+
+```js
+import { loadEmbedder, prepareSearch } from "@modootoday/pmcp";
+
+const encoder = await loadEmbedder("embeddinggemma-2", {
+  dimensions: 256,
+  modelPath: "/opt/models/embeddinggemma-2",
+  localOnly: true,
+});
+const search = await prepareSearch(
+  { roots: ["node_modules"] },
+  {
+    indexPath: "node_modules/.cache/pmcp/vectors.sqlite",
+    modelId: "embeddinggemma-2",
+    dimensions: 256,
+    embedder: encoder ?? undefined,
+  },
+);
+```
+
+HTTP hosts use these options under `semantic`; HTTP catalogs are encoded in
+memory at startup rather than reading the CLI's persisted index. Model runtime
+settings are passed to each loader without changing global download or cache
+policy, so hosts can prepare different encoders in one process.
+
+## Runtime and licence
+
+Node.js 22 or later, or Bun 1.3 or later. Ranking is lexical until an index and
+encoder are ready. Semantic
+ranking (the `index` command, or the HTTP handler's `semantic` option) needs
+`@huggingface/transformers` installed separately; it is not part of the default
+installation. The default encoder is `Xenova/multilingual-e5-small` (MIT, about
+130 MB), so vectors indexed with the earlier English-only default are re-encoded.
+
+The optional encoder also supports the ONNX Community EmbeddingGemma 300M and
+EmbeddingGemma 2 exports. It uses their projected sentence embeddings and task
+prefixes. EmbeddingGemma 2 loads its text encoder without vision or audio models.
+EmbeddingGemma was qualified with Transformers.js 4.3.1. Install a version
+supporting the chosen export and keep model
+weights outside the npm package. Regenerate document vectors when changing the
+encoder. Quantization reduces weight storage; application RAM and latency need
+workload measurements. EmbeddingGemma 300M uses the Gemma terms; EmbeddingGemma 2
+is listed under Apache-2.0 in its model card.
+
+Elastic License 2.0. See `LICENSE` and `NOTICE` for terms and dependency notices.
+Documentation: <https://pmcp.build>.

package/docs/hosting.md

@@ -0,0 +1,46 @@
+# Hosting a skill catalog
+
+[Back to the overview](../README.md)
+
+## Hosting over HTTP
+
+`@modootoday/pmcp/http` serves the same tools and resources over streamable HTTP for a hosted
+catalog. Tools, resources, the Skills extension and prompts all read the same per-caller,
+tier-filtered catalog, so a file of a skill a caller cannot list cannot be read either. The
+extension's `cacheScope` is `private` there by default (`cacheScope` overrides it), and
+`prompts: true` turns prompts on.
+The host owns identity: `authorize(request)` returns the caller or `undefined` (answered 401),
+and `tiersFor(caller)` decides which tiers that caller sees (default `open` and `free`; skills
+without a tier count as `open`). The catalog is read once; call `reload()` after deploying new
+skill releases.
+
+```ts
+import { createSkillHttpHandler } from "@modootoday/pmcp/http";
+
+const skills = createSkillHttpHandler({
+  roots: [],
+  marketplaces: ["/srv/marketplace", "/srv/marketplace-free"],
+  authorize: async (request) =>
+    verifyBearer(request.headers.get("authorization")),
+});
+export default { fetch: (request: Request) => skills.fetch(request) };
+```
+
+The handler checks requests before they reach MCP:
+
+| Option            | Default | Effect                                                                                                                                                                                                                                  |
+| ----------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `allowedOrigins`  | none    | A request with an `Origin` other than the server's own gets 403. Listed origins are served with CORS headers, and their preflight is answered. `"*"` admits any origin. Requests without `Origin` are not from a browser page and pass. |
+| `publicOrigins`   | none    | The server's own origins as browsers see them, when a proxy ends TLS and the request URL is `http://`. Without it, a same-site page behind such a proxy gets 403.                                                                       |
+| `allowedHosts`    | any     | When set, a `Host` outside the list gets 403 (DNS rebinding).                                                                                                                                                                           |
+| `maxRequestBytes` | 1 MiB   | A larger body gets 413; a declared length is refused before reading.                                                                                                                                                                    |
+| `rateLimit`       | off     | `{ perMinute }` per caller id; past it, 429 with `retry-after`.                                                                                                                                                                         |
+
+An MCP config (`.mcp.json`) is never served as written: every `env` and `headers` value is
+replaced with `<redacted>`, whichever entry reaches the file, and listed digests describe
+the redacted bytes.
+
+Pass `semantic: { modelId: "embeddinggemma-2", cacheDir, localOnly: true }` to rank by meaning: the handler encodes
+the catalog at start with `@huggingface/transformers` and the multilingual model, so an
+intent in one language finds a description in another. Requests rank lexically until
+`ready()` settles, and stay lexical when the encoder cannot load (`onError` says why).

package/docs/mailbox.md

@@ -0,0 +1,59 @@
+# Local mailbox
+
+[Back to the overview](../README.md)
+
+## Optional local mailbox
+
+Mailbox is disabled by default. The six catalog tools stay read-only. A launcher
+that supplies a private actor credential adds five tools: mailbox_send,
+mailbox_inbox, mailbox_read, mailbox_ack, and mailbox_sessions. Read and ack
+record durable receipts; peer text is data and does not authorize execution.
+
+```sh
+pmcp mailbox init --config ./pmcp.toml
+pmcp mailbox actor --config ./pmcp.toml --actor reviewer --actor-file /private/reviewer.actor.json
+pmcp serve --config ./pmcp.toml --mailbox-actor-file /private/reviewer.actor.json --mailbox-runtime codex-cli
+```
+
+Initialization preserves existing configuration and creates a project UUID.
+State defaults to XDG_STATE_HOME/pmcp/projects/<project UUID>, outside the
+repository. Credential and binding files contain secrets: keep them private,
+outside source control and container images. Commands do not print credentials.
+An optional mailbox-binding-file records the current attachment for a trusted
+launcher. Replacement requires the exact mailbox-replace-session ID; stale
+sessions are fenced. Canonical runtimes are codex-cli, claude-code, gemini-cli,
+grok-cli, and antigravity; the Antigravity executable is agy.
+
+The mailbox CLI also supports send, inbox, read, ack, sessions, rotate, export,
+backup, and restore. Actor operations accept a private actor-file or existing
+binding-file and JSON input file. Administration requires explicit operator
+intent. Backups and exports are sensitive. Encrypted recovery bundles are
+available through the mailbox API; key custody belongs to the operator.
+
+```ts
+import { MailboxService, MailboxStore } from "@modootoday/pmcp/mailbox";
+
+const store = await MailboxStore.open({
+  enabled: true,
+  projectId,
+  databasePath,
+});
+const credential = await store.registerActor("reviewer");
+const binding = await store.attach(credential, "codex-cli");
+const mailbox = new MailboxService(store, binding);
+```
+
+Pass mailbox to createSkillServer or registerMailboxTools on an existing server.
+For HTTP, provide mailboxFor(caller) to createSkillHttpHandler; the host resolves
+authenticated callers to authorized bindings. Tool arguments cannot choose a
+sender identity. The host owns attachment heartbeat and cleanup. Child actors
+have bounded authority and may reply only to their parent. Generation changes,
+parent expiry, and credential rotation revoke stale child authority.
+
+The public package contains one reusable local mailbox domain. Optional owned
+terminal supervision and explicit input delivery are provided separately by the
+[local CLI harness](terminal-harness.md); MCP mailbox tools cannot reach those
+execution paths. Provider credentials and approvals remain with native runtimes.
+Organization identity, fleet policy and enterprise audit retention belong to a
+dedicated host. Existing arbitrary terminal sessions are not captured. Elastic
+License 2.0 and NOTICE continue to apply.

package/docs/project-configuration.md

@@ -0,0 +1,203 @@
+# Project configuration
+
+[Back to the overview](../README.md)
+
+## Project config
+
+A `pmcp.toml` at or above the working directory sets the catalog once, so a
+host entry needs no flags and works from any package in the repository:
+
+```toml
+[catalog]
+workspaces = [".", "packages/*"]
+packages = ["docs"]
+marketplaces = ["vendor/marketplace"]
+```
+
+Paths resolve against the file's directory. Keys are `roots`, `scopes`,
+`workspaces`, `packages` and `marketplaces`, each a list mirroring the flag of
+the same meaning; an unknown key is an error. A flag replaces the file's list
+of its kind. `--config <file>` names another file and `--no-config` ignores it.
+
+## Native files for each agent tool
+
+`pmcp project` writes what Claude Code, Codex, Gemini CLI, Grok Build and
+Antigravity read natively, from cross-tool sources under `.agents/`
+(`skills/<name>/SKILL.md`, `agents/<name>/agent.md`, `rules/*.md`) and the rest
+of `pmcp.toml`:
+
+```toml
+[targets]
+tools = ["claude", "codex", "gemini", "grok", "antigravity"]
+
+[skills.deploy]
+invocation = "user"          # the person starts it; the model is not offered it
+
+[agents.reviewer.claude]
+tools = "Read, Grep"         # fields one tool reads stay out of agent.md
+
+[rules]
+root = "AGENTS.md"
+
+[mcp.skills]
+command = "node"
+args = ["${PROJECT_ROOT}/node_modules/@modootoday/pmcp/dist/cli.js", "serve"]
+```
+
+```sh
+pmcp project              # write
+pmcp project --check      # exit 2 on drift, stale or undeclared outputs
+pmcp project add <name>   # copy a catalog skill or agent into .agents/
+pmcp project remove <name>
+pmcp doctor               # ask each installed tool what it loads
+```
+
+Rules go to `.claude/rules/<dir>/`, never the top level, because Grok Build
+loads top-level `.claude/rules/*.md` in every session regardless of `paths:`.
+A server that names `${PROJECT_ROOT}` is launched through `sh`, which finds
+`pmcp.toml` at spawn time, so the generated files hold no machine's path.
+`pmcp.lock` records what was written; a file pmcp did not write is never
+overwritten. `doctor` calls no model: it reads `claude mcp list`,
+`codex mcp list`, `gemini skills list`, `gemini mcp list` and
+`grok inspect --json`, and skips a tool that is not installed.
+
+## Unified project MCP configuration
+
+Antigravity receives a native project plugin at .agents/plugins/pmcp-mcp with
+plugin.json and mcp_config.json. Projection removes only lock-owned entries
+from the earlier .agents/mcp_config.json artifact and backs up both layouts.
+Antigravity's global mcp list does not enumerate project plugin servers;
+validate the generated plugin with agy plugin validate.
+Its HTTP entries use serverUrl, accepted by the installed plugin validator.
+
+Declare each MCP server once in pmcp.toml. The projector writes the native
+project files for all selected runtimes. A server can narrow its targets:
+
+```toml
+[mcp.docs]
+url = "https://example.com/mcp"
+
+[mcp.private-api]
+url = "https://example.com/private-mcp"
+targets = ["claude", "codex", "gemini", "grok"]
+bearer_token_env_var = "PRIVATE_MCP_TOKEN"
+header_env = { "X-Workspace" = "PRIVATE_MCP_WORKSPACE" }
+
+[mcp.local-service]
+command = "my-mcp-server"
+env_vars = ["MY_SERVICE_KEY"]
+cwd = "${PROJECT_ROOT}"
+
+[mcp.local-service.native.codex]
+startup_timeout_sec = 30
+
+[mcp.local-service.native.gemini]
+timeout = 120000
+```
+
+url means Streamable HTTP; Gemini receives httpUrl. Legacy SSE is not imported
+as HTTP. headers contains literal nonsecret headers, header_env maps header
+names to environment variable names, and bearer_token_env_var references a
+token without writing its value. Each runtime retains its own OAuth session.
+Antigravity HTTP header projection is not qualified and is rejected; explicitly
+limit those servers to supported targets. tools maps to Gemini includeTools and
+Codex enabled_tools; it is not a cross-runtime security boundary. Native timeout
+options retain each runtime's documented names and units. Unknown fields fail.
+
+Import explicitly selected native configurations. Preview is the default:
+
+```sh
+pmcp project import-mcp --source claude:/path/to/native.json
+pmcp project import-mcp --source codex:/path/to/config.toml --name docs --write
+pmcp project
+pmcp project --check
+```
+
+Repeat source and name to select multiple inputs. Conflicting aliases, missing
+requested names, unsupported authentication fields and credential-bearing URLs
+block import. Source contents and credential values are not printed. Sources
+are never rewritten. Review plugin-provided servers separately to avoid
+duplicating a server the runtime already loads.
+
+Literal stdio environment values remain in their source file. The importer
+stores an env_from reference containing file and an RFC 6901 JSON pointer:
+
+```toml
+[mcp.local-service.env_from.MY_SERVICE_KEY]
+file = "~/.config/my-client/settings.json"
+pointer = "/mcpServers/local-service/env/MY_SERVICE_KEY"
+```
+
+The generated entry invokes pmcp project exec-mcp for that declared server.
+It reads only the referenced values and passes them to the child process, using
+inherited stdio. JSON and TOML sources are supported. The source must remain
+available; rotating its value takes effect on the next server launch. A project
+installed CLI is referenced relative to the project; otherwise pmcp must be on
+PATH. Configurations using env_from require this release on every runtime host.
+
+Writes retain private backups in ~/.cache/pmcp/project-backups, outside the
+project. Receipts record file hashes and absent paths. A conflicting unmanaged
+native alias is refused before projection writes. The managed TOML block and
+JSON keys preserve unrelated settings. Inspect the reported backup before
+restoring any file that another process may have edited.
+
+Configuration references: [Codex](https://developers.openai.com/codex/config-reference/),
+[Claude Code](https://code.claude.com/docs/en/mcp),
+[Gemini CLI](https://geminicli.com/docs/tools/mcp-server/),
+and [Grok Build](https://docs.x.ai/build/features/mcp-servers).
+
+## Operator shell guards
+
+Declare a project-owned guard explicitly to project it into Codex, Gemini or
+Antigravity's native hooks. Without a declaration, pmcp generates no hooks.
+
+```toml
+[[hooks.shell]]
+name = "shell-command-policy"
+command = ["node", ".agents/guards/shell-command-policy.mjs"]
+targets = ["codex", "gemini", "antigravity"]
+timeout_ms = 15000
+```
+
+The guard script must exist under .agents/guards/ and cannot resolve outside
+that directory. It receives one JSON object on stdin containing runtime,
+event=before_shell, command and cwd. It returns a JSON object with
+decision=allow or decision=deny and an optional reason. Malformed responses,
+nonzero exits and timeouts deny the shell call. The guard receives PATH and
+LANG rather than the native CLI's credential environment. Non-shell tools are
+outside this contract.
+
+Run pmcp project to write lock-owned adapter and policy artifacts under
+.agents/pmcp-hooks/, plus the native entries in .codex/hooks.json,
+.gemini/settings.json and .agents/hooks.json. Peer entries are retained.
+Edited owned hook entries refuse projection for review. Removing a declaration
+removes its generated entries and artifacts while preserving peer settings.
+Writes retain private backups; pmcp project --check only reports drift.
+
+Guard, policy and adapter fingerprints bind the generated command. Changing
+guard code requires reprojection and native Codex hook review. Generated files
+contain no trust bypass. Grok can load Claude project hooks, so this first scope
+does not generate a second Grok guard. Hooks do not force pmcp catalog use,
+inject reminders or replace the runtime's execution sandbox.
+
+## Native memory metadata
+
+```sh
+pmcp doctor --memory --json
+pmcp doctor --memory --tool codex
+```
+
+This explicit operation reports selected runtimes' native memory configuration
+and path existence. It reads no memory body, index or transcript, writes no
+configuration, connects to no MCP server and makes no provider request.
+Ordinary doctor output is unchanged.
+
+The report distinguishes enabled, disabled, unknown and unsupported. Path
+existence does not establish activation. Where native metadata cannot identify
+a project-specific location, pathScope=base reports only the storage parent;
+the project identifier is not guessed. Supported native home overrides are
+respected. Codex uses its native features listing. Claude and Grok report
+explicit user settings, with active session overrides and unverified project
+trust left unknown. Gemini context filenames and the global GEMINI.md file
+are separate from unexposed memory-tool activation. Antigravity trajectories
+are not reported as a memory store. pmcp does not synchronize or inject memory.

package/docs/skills.md

@@ -0,0 +1,74 @@
+# Skills and MCP reference
+
+[Back to the overview](../README.md)
+
+## Package skills
+
+Skills are discovered by filename, including `skills/`, `.agents/skills/`,
+`.agent/skills/`, `.claude/skills/`, `.gemini/skills/`, `.codex/skills/`, and
+`lib/skill/` layouts
+within the discovery depth. A skill needs a frontmatter `description` for search.
+
+`skill_find` accepts a problem description and returns matching skills.
+`skill_call` opens the selected document and lists the reference, script and asset
+files it carries, as text and as resource links. Local MCP reading requires no login.
+
+Files keep their type, as the Agent Skills layout (`references/`, `scripts/`, `assets/`)
+expects: `skill_read` returns text as text, images and audio as image and audio content,
+and other binary files as an embedded resource with its media type; a file whose extension
+names text but whose bytes are not UTF-8 goes out as a blob, so its sha256 stays exact.
+Scripts are returned, never run: running them is the host's job, from a local copy of the
+skill. `skill_bundle` lists every file of a skill as resource links that `resources/read`
+returns, so a host can write the skill into a local skills directory without the bytes
+passing through the model.
+
+`skill_catalog`, `skill_find`, `skill_describe` and `skill_bundle` declare an
+`outputSchema` and return `structuredContent` alongside the same JSON as text.
+Frontmatter is read as YAML (folded and block descriptions, a BOM and CRLF are fine), and
+`metadata` is read for every skill. Loading is lenient, as the client guide asks; what
+breaks the specification is reported, and `pmcp validate [<skill-dir>...]` checks given
+directories or every skill the catalog reaches (name rules and length, description and
+compatibility length, field types, name matching the directory, duplicate names).
+
+`metadata` keeps strings, string lists and maps of those one level down; `skill_describe`
+reports a map as dotted keys, so `requires: {mcp, gpu-gb}` arrives as `requires.mcp` and
+`requires.gpu-gb`. Deeper nesting is dropped.
+
+Output schemas accept additional properties, so a field added later does not break a
+client that cached an earlier tool list. An answer to a call without the newer inputs keeps
+the earlier shape.
+
+Since 0.9.0 a skill can list `metadata.verified-runtimes`, a list drawn from `claude-code`,
+`codex-cli`, `gemini-cli`, `grok-cli` and `antigravity`, meaning its effect was verified on
+that runtime. `skill_find` and `skill_catalog` accept `runtime` and `verifiedOnly`; when either
+is given, items carry `verifiedRuntimes` (omitted when empty). `runtime` never removes a skill: it marks
+each item `verified` and lists verified ones first (within a package group in
+`skill_catalog`). `verifiedOnly` drops the rest, keeping those verified on `runtime`, or on
+any runtime when none is given. Without `runtime` the answer is ordered exactly as before.
+A server can opt in with `rankByClientRuntime: true`: then, when `runtime` is not passed and
+the MCP client's name is recognised (`claude-code`, `codex`, `gemini`, `grok`,
+`antigravity`), that runtime is used for the ordering only, never for filtering.
+`skill_find` then answers with `runtime`, the one the ordering used, or `null`.
+
+`pmcp serve --prompts` also offers one prompt per skill, for hosts that turn prompts into
+commands; the prompt's text is the skill body and its file list, with an optional `task`.
+
+Marketplace plugins also carry commands, agents, hooks and MCP servers.
+Commands are listed as skills. The others are listed under their own kind and
+are left out of `skill_catalog` and `skill_find` unless `kind` asks for them
+(`agent`, `hook`, `mcp`, or `any`); `skill_describe`, `skill_read` and
+`skill_call` work on every kind.
+
+## MCP Skills extension
+
+The server also declares `io.modelcontextprotocol/skills` (SEP-2640): it
+answers `skills/list` and `skills/get` (both with `ttlMs` and `cacheScope`), with
+a sha256 digest and size per file. A skill qualifies when its frontmatter passes the
+Agent Skills specification and its `name` matches its directory, and it stays within
+512 files and 16 MiB; the rest remain reachable through the tools.
+
+Every file of every skill the caller may see is a `skill://pmcp/<package>/<name>/<file>`
+resource: `resources/list` lists each skill's `SKILL.md`, `resources/read` returns any file
+(text, or a blob with its media type), and the template completes skill names and paths.
+List-change notifications and subscriptions are not offered: they need a session, and the
+HTTP handler is stateless.