toolfunnel 0.4.0 → 0.6.0

package/src/extend/package.md

@@ -1,187 +1,187 @@
-# How to package and share units (self-extension)
-
-This is the instruction served by `toolfunnel_howto({ topic: "package" })`. It explains how to bundle
-tools, hooks, and MCP configs into a **shareable package** and how import/export work. Packaging is a
-**byproduct** of the gateway design: because every unit is already structured and
-self-documented, a package is just a folder that lists its units and carries their files.
-
----
-
-## 0. The WHOLE-SETUP path — `tf_pack` (start here)
-
-Since 0.4.0 the fastest way to ship is not per-unit packaging but the **whole setup**: your entire
-config home (tools + scripts, hot/hidden curation, upstream references, policy hooks, identity) is
-already the product. One call packages it into a deployment-ready artifact under `<home>/dist/`:
-
-- `tf_pack { format: "home" }` → a **portable config home**. Zip it, git-init it, or run it
-  directly: `toolfunnel --config-dir <dir>`. The receiving gateway seeds nothing — the pack IS the
-  config, including the hooks: **the gate travels**, enforcing your policy on the recipient's
-  machine regardless of which MCP client they use.
-- `tf_pack { format: "npm", name: "my-mcp" }` → a **publishable npm package**: a generated
-  `package.json` that depends on `toolfunnel` (caret range — DEPEND, never copy the engine), a
-  2-line bin launcher pointing `--config-dir` at the bundled home, and a README stub.
-  `cd dist/my-mcp && npm publish` → your users run `npx my-mcp` and get YOUR server, under YOUR
-  name (the bundled `toolfunnel.json` identity), with YOUR gate.
-
-Declare the runtimes your tools need in the home's `toolfunnel.json` — they travel with the pack
-and the receiving gateway warns at startup about anything missing:
-
-```json
-"requires": [ { "command": "python", "min": "3.10", "why": "the pdf tools" } ]
-```
-
-Audit honesty (both directions): packs spawn commands. Tell your users to read your pack's
-`expose.json`, `tools.register.json`, AND `toolfunnel.json` (its `requires` probes execute at
-gateway startup — bare program names only, the loader rejects anything shell-shaped) — and read
-all three yourself in anything you install.
-
-The per-UNIT packaging below remains the right shape for sharing a single tool/hook/MCP selection
-rather than a whole gateway.
-
-> Phase note: packages are Phase 2 (`packages/` directory). The structure below is what the importer
-> and exporter target; defining it now means the units authored today are already package-ready.
-
----
-
-## 1. What a unit is (and what travels with it)
-
-Each unit is self-contained — this is the whole reason packaging is cheap:
-
-| Unit | Its metadata | Its files |
-|---|---|---|
-| **tool** | one `tools.register.json` entry (`id/name/summary/category/instructions/invoke`) | the script under `tools/scripts/` (for `invoke.type === "script"`) |
-| **hook** | one `hooks.manifest.json` entry (`id/event/matcher/command/script/timeout/description`) + its `hooks.state.json` enabled value | the script under `hooks/scripts/` |
-| **mcp** | one `expose.json` `upstreams[]` entry + its `expose[]` selections | any vendored server files it spawns (must live inside the package) |
-
-A package gathers one or more of these, plus the files they reference.
-
----
-
-## 2. Package layout
-
-A package is a folder under `packages/` containing a manifest and the units' files in a mirrored
-tree so import is a structured copy, not guesswork:
-
-```
-packages/
-  notify-pack/
-    package.json          # the package manifest (below)
-    tools/
-      scripts/
-        msg-digest.sh
-    hooks/
-      scripts/
-        rate-limit.sh
-    mcp/
-      vendor/             # any vendored upstream server files
-```
-
-### `package.json` — the package manifest
-
-A `package.json`-style manifest listing the package's identity and its units (the metadata blocks are
-embedded so import is self-sufficient — it does not need the source register/manifest to reconstruct
-entries):
-
-```json
-{
-  "name": "notify-pack",
-  "version": "1.0.0",
-  "description": "Message digest tool + rate-limit gate + upstream MCP exposure.",
-  "author": "unknown",
-  "units": {
-    "tools": [
-      {
-        "entry": {
-          "id": "msg-digest",
-          "name": "Message Digest",
-          "summary": "Summarises unread chats.",
-          "category": "messaging",
-          "instructions": "Pass { since? }. Returns a per-chat unread digest.",
-          "invoke": { "type": "script", "path": "scripts/msg-digest.sh" }
-        },
-        "files": ["tools/scripts/msg-digest.sh"]
-      }
-    ],
-    "hooks": [
-      {
-        "entry": {
-          "id": "pre-tool-use/rate-limit",
-          "event": "PreToolUse",
-          "matcher": "msg_send_message",
-          "type": "command",
-          "command": "bash \"${HOOKS_DIR}/scripts/rate-limit.sh\"",
-          "script": "scripts/rate-limit.sh",
-          "timeout": 5,
-          "description": "Denies sends past a per-minute cap."
-        },
-        "enabled": true,
-        "files": ["hooks/scripts/rate-limit.sh"]
-      }
-    ],
-    "mcp": [
-      {
-        "upstream": {
-          "id": "notify",
-          "transport": "stdio",
-          "command": "node",
-          "args": ["${PACKAGE_DIR}/mcp/vendor/notify-mcp/server.js"],
-          "enabled": true,
-          "description": "Messaging bridge MCP."
-        },
-        "expose": [
-          { "upstream": "notify", "tool": "list_chats", "as": "msg_list_chats", "category": "messaging", "enabled": true },
-          { "upstream": "notify", "tool": "send_message", "as": "msg_send_message", "category": "messaging", "enabled": true }
-        ],
-        "files": ["mcp/vendor/notify-mcp/server.js"]
-      }
-    ]
-  }
-}
-```
-
-| Field | Meaning |
-|---|---|
-| `name` / `version` / `description` / `author` | Package identity. `name` is the folder name and the import key. |
-| `units.tools[]` | Each has the full register `entry` plus `files` (paths inside the package). |
-| `units.hooks[]` | Each has the full manifest `entry`, its seed `enabled` (for `hooks.state.json`), and `files`. |
-| `units.mcp[]` | Each has an `upstream` block, its `expose[]` selections, and any vendored `files`. |
-
-`${PACKAGE_DIR}` expands to the installed package's absolute path (mirrors `${HOOKS_DIR}` /
-`${TOOLS_DIR}`), so a package's MCP/tool commands resolve wherever it lands.
-
----
-
-## 3. Export — bundling a unit (or several)
-
-To export, gather a unit's metadata and its files into a package folder:
-
-1. Create `packages/<name>/` with a `package.json` manifest.
-2. For each tool: copy its `tools.register.json` entry into `units.tools[].entry`, copy its
-   `tools/scripts/<file>` into the package's `tools/scripts/`, and list it in `files`.
-3. For each hook: copy its `hooks.manifest.json` entry into `units.hooks[].entry`, copy its enabled
-   value from `hooks.state.json` into `enabled`, copy the script into `hooks/scripts/`, list it.
-4. For each MCP: copy the `upstreams[]` block + matching `expose[]` selections, and vendor any server
-   files into `mcp/vendor/`.
-
-The package is then a single folder, self-describing and copy-portable. (Everything stays inside
-`packages/` — the isolation rule still holds.)
-
----
-
-## 4. Import — installing a package
-
-Import is the reverse: **drop a package folder in `packages/` + register its units.**
-
-1. Read `packages/<name>/package.json`.
-2. **Tools:** copy each `units.tools[].files` into `tools/scripts/` and add each `entry` to
-   `tools.register.json` (atomic write via `registry.js`). Reject id collisions or prompt to rename.
-3. **Hooks:** copy each `units.hooks[].files` into `hooks/scripts/` and add each `entry` to
-   `hooks.manifest.json`; write its `enabled` into `hooks.state.json`.
-4. **MCP:** copy vendored `files` into the package's `mcp/vendor/`, add each `upstream` to
-   `expose.json` `upstreams[]` and its `expose[]` selections (path-rewritten to `${PACKAGE_DIR}`).
-
-After import: tools are live immediately (the register is read fresh on every `toolfunnel_list_tools`).
-Hooks take effect once the loader re-scans (next startup, or a reload). Curated-direct MCP tools need
-a reconnect to reach the CLI surface (register-routed tools do not). Verify with `toolfunnel_list_tools`
-and a `run`/run-once test that the gate still fires on the imported units — the PreToolUse gate
-applies to imported tools exactly as it does to local ones.
+# How to package and share units (self-extension)
+
+This is the instruction served by `toolfunnel_howto({ topic: "package" })`. It explains how to bundle
+tools, hooks, and MCP configs into a **shareable package** and how import/export work. Packaging is a
+**byproduct** of the gateway design: because every unit is already structured and
+self-documented, a package is just a folder that lists its units and carries their files.
+
+---
+
+## 0. The WHOLE-SETUP path - `tf_pack` (start here)
+
+Since 0.4.0 the fastest way to ship is not per-unit packaging but the **whole setup**: your entire
+config home (tools + scripts, hot/hidden curation, upstream references, policy hooks, identity) is
+already the product. One call packages it into a deployment-ready artifact under `<home>/dist/`:
+
+- `tf_pack { format: "home" }` → a **portable config home**. Zip it, git-init it, or run it
+  directly: `toolfunnel --config-dir <dir>`. The receiving gateway seeds nothing - the pack IS the
+  config, including the hooks: **the gate travels**, enforcing your policy on the recipient's
+  machine regardless of which MCP client they use.
+- `tf_pack { format: "npm", name: "my-mcp" }` → a **publishable npm package**: a generated
+  `package.json` that depends on `toolfunnel` (caret range - DEPEND, never copy the engine), a
+  2-line bin launcher pointing `--config-dir` at the bundled home, and a README stub.
+  `cd dist/my-mcp && npm publish` → your users run `npx my-mcp` and get YOUR server, under YOUR
+  name (the bundled `toolfunnel.json` identity), with YOUR gate.
+
+Declare the runtimes your tools need in the home's `toolfunnel.json` - they travel with the pack
+and the receiving gateway warns at startup about anything missing:
+
+```json
+"requires": [ { "command": "python", "min": "3.10", "why": "the pdf tools" } ]
+```
+
+Audit honesty (both directions): packs spawn commands. Tell your users to read your pack's
+`expose.json`, `tools.register.json`, AND `toolfunnel.json` (its `requires` probes execute at
+gateway startup - bare program names only, the loader rejects anything shell-shaped) - and read
+all three yourself in anything you install.
+
+The per-UNIT packaging below remains the right shape for sharing a single tool/hook/MCP selection
+rather than a whole gateway.
+
+> Phase note: packages are Phase 2 (`packages/` directory). The structure below is what the importer
+> and exporter target; defining it now means the units authored today are already package-ready.
+
+---
+
+## 1. What a unit is (and what travels with it)
+
+Each unit is self-contained - this is the whole reason packaging is cheap:
+
+| Unit | Its metadata | Its files |
+|---|---|---|
+| **tool** | one `tools.register.json` entry (`id/name/summary/category/instructions/invoke`) | the script under `tools/scripts/` (for `invoke.type === "script"`) |
+| **hook** | one `hooks.manifest.json` entry (`id/event/matcher/command/script/timeout/description`) + its `hooks.state.json` enabled value | the script under `hooks/scripts/` |
+| **mcp** | one `expose.json` `upstreams[]` entry + its `expose[]` selections | any vendored server files it spawns (must live inside the package) |
+
+A package gathers one or more of these, plus the files they reference.
+
+---
+
+## 2. Package layout
+
+A package is a folder under `packages/` containing a manifest and the units' files in a mirrored
+tree so import is a structured copy, not guesswork:
+
+```
+packages/
+  notify-pack/
+    package.json          # the package manifest (below)
+    tools/
+      scripts/
+        msg-digest.sh
+    hooks/
+      scripts/
+        rate-limit.sh
+    mcp/
+      vendor/             # any vendored upstream server files
+```
+
+### `package.json` - the package manifest
+
+A `package.json`-style manifest listing the package's identity and its units (the metadata blocks are
+embedded so import is self-sufficient - it does not need the source register/manifest to reconstruct
+entries):
+
+```json
+{
+  "name": "notify-pack",
+  "version": "1.0.0",
+  "description": "Message digest tool + rate-limit gate + upstream MCP exposure.",
+  "author": "unknown",
+  "units": {
+    "tools": [
+      {
+        "entry": {
+          "id": "msg-digest",
+          "name": "Message Digest",
+          "summary": "Summarises unread chats.",
+          "category": "messaging",
+          "instructions": "Pass { since? }. Returns a per-chat unread digest.",
+          "invoke": { "type": "script", "path": "scripts/msg-digest.sh" }
+        },
+        "files": ["tools/scripts/msg-digest.sh"]
+      }
+    ],
+    "hooks": [
+      {
+        "entry": {
+          "id": "pre-tool-use/rate-limit",
+          "event": "PreToolUse",
+          "matcher": "msg_send_message",
+          "type": "command",
+          "command": "bash \"${HOOKS_DIR}/scripts/rate-limit.sh\"",
+          "script": "scripts/rate-limit.sh",
+          "timeout": 5,
+          "description": "Denies sends past a per-minute cap."
+        },
+        "enabled": true,
+        "files": ["hooks/scripts/rate-limit.sh"]
+      }
+    ],
+    "mcp": [
+      {
+        "upstream": {
+          "id": "notify",
+          "transport": "stdio",
+          "command": "node",
+          "args": ["${PACKAGE_DIR}/mcp/vendor/notify-mcp/server.js"],
+          "enabled": true,
+          "description": "Messaging bridge MCP."
+        },
+        "expose": [
+          { "upstream": "notify", "tool": "list_chats", "as": "msg_list_chats", "category": "messaging", "enabled": true },
+          { "upstream": "notify", "tool": "send_message", "as": "msg_send_message", "category": "messaging", "enabled": true }
+        ],
+        "files": ["mcp/vendor/notify-mcp/server.js"]
+      }
+    ]
+  }
+}
+```
+
+| Field | Meaning |
+|---|---|
+| `name` / `version` / `description` / `author` | Package identity. `name` is the folder name and the import key. |
+| `units.tools[]` | Each has the full register `entry` plus `files` (paths inside the package). |
+| `units.hooks[]` | Each has the full manifest `entry`, its seed `enabled` (for `hooks.state.json`), and `files`. |
+| `units.mcp[]` | Each has an `upstream` block, its `expose[]` selections, and any vendored `files`. |
+
+`${PACKAGE_DIR}` expands to the installed package's absolute path (mirrors `${HOOKS_DIR}` /
+`${TOOLS_DIR}`), so a package's MCP/tool commands resolve wherever it lands.
+
+---
+
+## 3. Export - bundling a unit (or several)
+
+To export, gather a unit's metadata and its files into a package folder:
+
+1. Create `packages/<name>/` with a `package.json` manifest.
+2. For each tool: copy its `tools.register.json` entry into `units.tools[].entry`, copy its
+   `tools/scripts/<file>` into the package's `tools/scripts/`, and list it in `files`.
+3. For each hook: copy its `hooks.manifest.json` entry into `units.hooks[].entry`, copy its enabled
+   value from `hooks.state.json` into `enabled`, copy the script into `hooks/scripts/`, list it.
+4. For each MCP: copy the `upstreams[]` block + matching `expose[]` selections, and vendor any server
+   files into `mcp/vendor/`.
+
+The package is then a single folder, self-describing and copy-portable. (Everything stays inside
+`packages/` - the isolation rule still holds.)
+
+---
+
+## 4. Import - installing a package
+
+Import is the reverse: **drop a package folder in `packages/` + register its units.**
+
+1. Read `packages/<name>/package.json`.
+2. **Tools:** copy each `units.tools[].files` into `tools/scripts/` and add each `entry` to
+   `tools.register.json` (atomic write via `registry.js`). Reject id collisions or prompt to rename.
+3. **Hooks:** copy each `units.hooks[].files` into `hooks/scripts/` and add each `entry` to
+   `hooks.manifest.json`; write its `enabled` into `hooks.state.json`.
+4. **MCP:** copy vendored `files` into the package's `mcp/vendor/`, add each `upstream` to
+   `expose.json` `upstreams[]` and its `expose[]` selections (path-rewritten to `${PACKAGE_DIR}`).
+
+After import: tools are live immediately (the register is read fresh on every `toolfunnel_list_tools`).
+Hooks take effect once the loader re-scans (next startup, or a reload). Curated-direct MCP tools need
+a reconnect to reach the CLI surface (register-routed tools do not). Verify with `toolfunnel_list_tools`
+and a `run`/run-once test that the gate still fires on the imported units - the PreToolUse gate
+applies to imported tools exactly as it does to local ones.

package/src/extend/wrap.md

@@ -0,0 +1,99 @@
+# How to wrap an MCP server (transparent passthrough)
+
+This is the instruction served by `toolfunnel_howto({ topic: "wrap" })`. It explains the
+**passthrough wrap**: making ONE attached upstream MCP's tools the gateway's ENTIRE surface, under
+their original names, with the upstream's own identity presented to the client - so the client
+cannot tell ToolFunnel is in the middle. Use it to keep a legacy server working with modern
+clients (or vice versa - all four era combinations work), or to put a policy gate in front of a
+server you didn't write.
+
+**What the client sees:** the wrapped server's own `serverInfo`, capabilities, instructions,
+tools, results, errors and notifications - verbatim. **What the wrapped server sees** (stdio,
+legacy clients): the real downstream client's identity, mirrored automatically. Mid-call prompts
+(elicitation), resource subscriptions, progress tokens and cancellations are all bridged across
+protocol eras. Wrapped tool calls wait up to 120 s by default, and a tool that reports progress
+keeps its call alive indefinitely; for silent tools that run longer, set `"timeoutMs"` on the
+upstream entry in `mcp/expose.json`.
+
+**What changes for YOU (the agent):** while a wrap is active, ALL ToolFunnel tools - the four
+meta-tools included - are hidden and uncallable. That includes `tf_wrap` itself. Read §3 before
+setting a wrap in-band.
+
+---
+
+## 1. Setting a wrap - three equivalent paths
+
+The wrap is one persisted field (`"passthrough"` in `tools/tools.state.json`); all three paths
+write the same state and survive restarts.
+
+**CLI (recommended for humans):**
+
+```
+toolfunnel wrap <upstreamId>          # wrap - probes the upstream's protocol era and reports it
+toolfunnel wrap <upstreamId> --as <name>   # also present that name in the handshake (serverInfo)
+toolfunnel wrap                       # status - current wrap + attached upstream ids
+toolfunnel wrap --off                 # clear - restore the normal funnel surface
+```
+
+Global flags may sit anywhere: `toolfunnel --config-dir <dir> wrap <upstreamId>` and
+`toolfunnel wrap <upstreamId> --config-dir <dir>` are equivalent.
+
+The wrap persists as one field in `tools/tools.state.json` in the config home -
+`{ "passthrough": "<upstreamId>" }` when set. Editing that file by hand is a valid last-resort
+undo: clearing the `passthrough` key (e.g. writing `{}` if nothing else is in the file) is the
+file-edit equivalent of `wrap --off`; a running host picks the change up live.
+
+**UI:** the MCPs tab → the Wrap button on the upstream's row (Unwrap on the same button while
+active). A banner shows the active wrap.
+
+**In-band (`tf_wrap`, via `toolfunnel_run_tool`):**
+
+```
+{ }                                     → status: { ok, wrapping, upstreams }
+{ "off": true }                         → clear
+{ "upstream": "<id>", "confirm": true } → set (confirm REQUIRED - see §3)
+```
+
+The upstream must already be attached and enabled (`toolfunnel_howto({ topic: "add-mcp" })` covers
+attaching one).
+
+## 2. Verifying a wrap
+
+- `toolfunnel wrap` (or `tf_wrap {}` before wrapping) shows the current state.
+- After wrapping, a client's `initialize`/`server/discover` handshake presents the WRAPPED
+  server's identity, and `tools/list` shows its tools under their original names.
+- The CLI wrap command prints an era report: whether the upstream speaks the modern
+  (2026-07-28) protocol, the legacy family, or both.
+
+## 3. The lockout - why `confirm: true` is required in-band
+
+Setting a wrap HIDES every ToolFunnel tool, including `tf_wrap`. An agent that sets a wrap
+in-band therefore loses the in-band path to undo it. Recovery is always available OUT-of-band:
+
+- `toolfunnel wrap --off` on the CLI, or
+- the UI's Unwrap button.
+
+`tf_wrap` refuses to set a wrap without `confirm: true` precisely so this is a deliberate,
+informed action. Do not set a wrap unless the operator asked for it or you can reach one of the
+two recovery paths.
+
+## 4. Security - what wrapping changes
+
+Wrapping SUSPENDS the path-isolation guard for the wrapped upstream only (a wrap is an explicit
+"this server is my whole surface" declaration - e.g. a filesystem server serving a documents
+folder MUST reach outside the gateway root). The CLI and UI both print a security notice naming
+the outside paths when this applies. Two things stay true regardless:
+
+- **Every call still passes the PreToolUse gate.** Wrapping never bypasses policy.
+- **Funnel-mode upstreams keep the guard** - the suspension is wrap-scoped, nothing else.
+
+To restrict a wrapped server after wrapping: add a PreToolUse hook that blocks by tool/args
+(`toolfunnel_howto({ topic: "add-hook" })`), or disable individual tools (keyed by the surfaced
+name). The manual's "Wrapping & security" section covers the patterns.
+
+## 5. Related settings
+
+- `legacyPin: true` on an upstream (in `mcp/expose.json`, or the UI's "Legacy pin" toggle) pins
+  that upstream to the legacy protocol forever - the era probe is skipped. Opt-in, warns loudly.
+- The gateway's own presented identities (funnel mode / HTTP) are configured in
+  `toolfunnel.json` - see `toolfunnel_howto({ topic: "configure" })`.