caplets 0.9.0 → 0.11.0

package/README.md

@@ -11,6 +11,52 @@ or call that backend's underlying tools or operations.
 This keeps the initial MCP tool list small, makes tool selection easier, and avoids
 flattened tool-name collisions across servers.
 
+## Why It Matters
+
+Large MCP setups make agents worse before they make them better. If every downstream
+server exposes every tool up front, the model starts with a noisy flat list, duplicate
+tool names, and a bigger context surface before it knows which capability matters.
+
+Caplets turns that flat tool wall into progressive disclosure: one capability card first,
+then scoped discovery only after the agent chooses the relevant domain.
+
+## Benchmark Results
+
+In Caplets' reproducible coding-agent benchmark, the same three mock MCP servers are
+exposed two ways: direct flat MCP aggregation versus Caplets progressive disclosure.
+
+| Initial Agent Surface     |   Direct Flat MCP |      Caplets |     Reduction |
+| ------------------------- | ----------------: | -----------: | ------------: |
+| Visible tools             |               106 |            3 |   97.2% fewer |
+| Serialized MCP payload    |      32,090 bytes |  8,400 bytes | 73.8% smaller |
+| Approx. context surface   |      8,023 tokens | 2,100 tokens |   5,923 fewer |
+| Top-level name collisions | 3 duplicate names |            0 |    eliminated |
+
+The important part: Caplets does not remove access to the downstream tools. It hides
+them behind scoped discovery operations like `search_tools`, `get_tool`, and `call_tool`,
+so the agent sees less up front while still being able to reach the same capabilities.
+
+A local OpenCode live benchmark also completed the full benchmark matrix successfully:
+
+| Agent                          | Mode            | Tasks Passed |
+| ------------------------------ | --------------- | -----------: |
+| OpenCode `openai/gpt-5.5-fast` | Direct flat MCP |          2/2 |
+| OpenCode `openai/gpt-5.5-fast` | Caplets         |          2/2 |
+
+Live results are intentionally not committed as product claims because they depend on
+local agent CLIs, credentials, models, providers, and agent behavior. The deterministic
+surface benchmark is the reproducible claim.
+
+See [`docs/benchmarks/coding-agent.md`](docs/benchmarks/coding-agent.md) for methodology,
+limitations, and reproduction commands.
+
+```sh
+pnpm benchmark
+pnpm benchmark:check
+pnpm build
+CAPLETS_BENCH_LIVE=1 pnpm benchmark:live:opencode -- --model openai/gpt-5.5-fast
+```
+
 ## Inspiration
 
 Caplets is a mashup of two ideas that work well separately but leave a gap together:
@@ -28,8 +74,8 @@ the agent chooses that server and asks to search, list, inspect, or call them.
 
 ## What It Does
 
-- Reads downstream MCP server definitions, native OpenAPI endpoint definitions, native GraphQL endpoint definitions, and explicit HTTP API action definitions from the user config file.
-- Registers one generated MCP tool for each enabled MCP server, OpenAPI endpoint, GraphQL endpoint, or HTTP API.
+- Reads downstream MCP server definitions, native OpenAPI endpoint definitions, native GraphQL endpoint definitions, explicit HTTP API action definitions, and curated CLI tool definitions from the user config file.
+- Registers one generated MCP tool for each enabled MCP server, OpenAPI endpoint, GraphQL endpoint, HTTP API, or CLI tools backend.
 - Uses the configured server ID as the generated tool name.
 - Uses the configured `name` and `description` as the capability card shown to agents.
 - Starts downstream MCP servers and loads OpenAPI specs lazily when an operation needs them.
@@ -38,6 +84,7 @@ the agent chooses that server and asks to search, list, inspect, or call them.
 - Converts OpenAPI operations into MCP-style tool metadata and executes HTTP calls directly.
 - Converts configured GraphQL operations into MCP-style tool metadata, and can auto-generate GraphQL tools from schema root query and mutation fields.
 - Converts explicitly configured HTTP actions into MCP-style tool metadata and executes HTTP calls directly.
+- Converts explicitly configured CLI actions into MCP-style tool metadata and executes commands directly without a shell.
 - Preserves downstream tool results instead of rewriting them into a custom format.
 - Redacts secrets from structured errors.
 - Supports static remote auth and OAuth token storage for remote servers.
@@ -172,7 +219,7 @@ the committed schema stays in sync with the Zod config validator.
 
 For richer skill-like cards, add Markdown Caplet files beside `config.json`. Every Caplet
 file must include exactly one executable backend: `mcpServer`, `openapiEndpoint`,
-`graphqlEndpoint`, or `httpApi`;
+`graphqlEndpoint`, `httpApi`, or `cliTools`;
 serverless Caplets are intentionally out of scope.
 
 Top-level files derive the Caplet ID from the filename:
@@ -255,6 +302,26 @@ httpApi:
 # Status API
 ```
 
+CLI-backed Caplet files use `cliTools`:
+
+```md
+---
+name: Repository CLI
+description: Run curated repository workflows through local CLI commands.
+cliTools:
+  cwd: /home/you/project
+  actions:
+    git_status:
+      description: Show concise Git working tree status.
+      command: git
+      args: ["status", "--short"]
+      annotations:
+        readOnlyHint: true
+---
+
+# Repository CLI
+```
+
 Top-level files derive their Caplet ID from the filename. Directory-style Caplets use
 `linear/CAPLET.md`, which is exposed as `linear`; sibling files can be referenced with
 normal Markdown links from `CAPLET.md`.
@@ -264,6 +331,8 @@ This repository includes polished working examples under [`caplets/`](caplets/):
 - `github`: GitHub's official MCP server container, using `GITHUB_PERSONAL_ACCESS_TOKEN`.
 - `linear`: Linear's hosted OAuth MCP endpoint.
 - `context7`: Context7 documentation lookup through `@upstash/context7-mcp`.
+- `repo-cli`: Read-oriented repository CLI workflows through `git` and package scripts.
+- `github-cli`: Read-oriented GitHub workflows through the `gh` CLI.
 
 Install every example from a repo's `caplets/` directory:
 
@@ -304,7 +373,7 @@ caplets init --force
 
 ### Caplet IDs
 
-Each key under `mcpServers`, `openapiEndpoints`, `graphqlEndpoints`, or `httpApis` is the
+Each key under `mcpServers`, `openapiEndpoints`, `graphqlEndpoints`, `httpApis`, or `cliTools` is the
 stable Caplet ID. It becomes the generated MCP tool name exactly, so keep it short and specific:
 
 ```json
@@ -321,7 +390,7 @@ stable Caplet ID. It becomes the generated MCP tool name exactly, so keep it sho
 ```
 
 Caplet IDs must match `^[a-zA-Z0-9_-]{1,64}$` and must be unique across `mcpServers`,
-`openapiEndpoints`, `graphqlEndpoints`, and `httpApis`. Spaces, dots, slashes, colons, and Unicode IDs are rejected.
+`openapiEndpoints`, `graphqlEndpoints`, `httpApis`, and `cliTools`. Spaces, dots, slashes, colons, and Unicode IDs are rejected.
 
 ### Stdio Servers
 
@@ -491,6 +560,49 @@ parsed `body` when present, and `elapsedMs`; non-2xx responses set `isError`, re
 timeouts are enforced, response bodies are capped by `maxResponseBytes` (default `1000000`), and
 errors redact secrets.
 
+### CLI Tools
+
+Use `cliTools` for curated local command-line workflows. Each action is an explicitly configured
+tool; Caplets does not expose arbitrary shell access and always spawns `command` plus `args`
+without shell interpolation.
+
+```json
+{
+  "name": "Repository CLI",
+  "description": "Run curated repository workflows through local CLI commands.",
+  "cwd": "/home/you/project",
+  "timeoutMs": 60000,
+  "maxOutputBytes": 1000000,
+  "actions": {
+    "git_status": {
+      "description": "Show concise Git working tree status.",
+      "command": "git",
+      "args": ["status", "--short"],
+      "annotations": { "readOnlyHint": true }
+    },
+    "run_tests": {
+      "description": "Run the package test script.",
+      "command": "pnpm",
+      "args": ["run", "test"],
+      "timeoutMs": 120000,
+      "annotations": { "readOnlyHint": true }
+    }
+  }
+}
+```
+
+CLI actions can set `inputSchema`, `outputSchema`, `env`, action-level `cwd`, `timeoutMs`,
+`maxOutputBytes`, `output: {"type":"json"}`, and MCP annotations. `$input.field` references are
+supported inside `args`, `env`, and `cwd` strings. Caplets performs basic required-field and
+primitive-type validation before spawning. Results are returned as structured content with
+`exitCode`, `stdout`, `stderr`, and `elapsedMs`; non-zero exits set `isError`.
+
+Generate a reviewable CLI Caplet manifest from a repository:
+
+```sh
+caplets author cli repo-tools --repo . --include git,gh,package --output -
+```
+
 ### Authentication
 
 Remote servers can use:

package/caplets/github-cli/CAPLET.md

@@ -0,0 +1,41 @@
+---
+$schema: https://raw.githubusercontent.com/spiritledsoftware/caplets/main/schemas/caplet.schema.json
+name: GitHub CLI
+description: Inspect GitHub pull requests and issues through curated gh CLI commands.
+tags:
+  - cli
+  - github
+  - code
+cliTools:
+  actions:
+    gh_pr_status:
+      description: Show pull request status for the current branch as JSON.
+      command: gh
+      args:
+        - pr
+        - status
+        - --json
+        - currentBranch
+      output:
+        type: json
+      annotations:
+        readOnlyHint: true
+        openWorldHint: true
+    gh_issue_list:
+      description: List open GitHub issues as JSON.
+      command: gh
+      args:
+        - issue
+        - list
+        - --json
+        - number,title,state,url
+      output:
+        type: json
+      annotations:
+        readOnlyHint: true
+        openWorldHint: true
+---
+
+# GitHub CLI
+
+Use this Caplet to expose read-oriented GitHub workflows through `gh` without giving the agent an unrestricted shell.

package/caplets/repo-cli/CAPLET.md

@@ -0,0 +1,37 @@
+---
+$schema: https://raw.githubusercontent.com/spiritledsoftware/caplets/main/schemas/caplet.schema.json
+name: Repository CLI
+description: Inspect and run common local repository workflows through curated CLI tools.
+tags:
+  - cli
+  - code
+cliTools:
+  actions:
+    git_status:
+      description: Show concise Git working tree status.
+      command: git
+      args:
+        - status
+        - --short
+      annotations:
+        readOnlyHint: true
+    git_current_branch:
+      description: Print the current Git branch name.
+      command: git
+      args:
+        - branch
+        - --show-current
+      annotations:
+        readOnlyHint: true
+    package_test:
+      description: Run the repository test script with pnpm.
+      command: pnpm
+      args:
+        - run
+        - test
+      timeoutMs: 120000
+---
+
+# Repository CLI
+
+Use this Caplet to expose a small, typed set of local repository commands without giving an agent arbitrary shell access.