toolfunnel 0.7.0 → 0.8.0

package/docs/packaging.md

@@ -1,115 +1,116 @@
-# Packaging: ship what you built
-
-ToolFunnel's config is a folder. That single fact is the whole packaging story: everything you
-build in a gateway - your tools and their scripts, which of them are promoted hot or hidden, the
-upstream MCPs you attached and the tools you chose to expose from them, the policy hooks that gate
-every call, and the identity the server introduces itself with - lives in one directory tree, the
-**config home**. Ship the tree and you have shipped the product.
-
-## The config home
-
-By default the home is the package/clone root - a git clone works exactly as it always did. Point
-the gateway elsewhere and the config lives there instead:
-
-```
-toolfunnel --config-dir D:\my-setup     # flag (wins)
-TOOLFUNNEL_HOME=D:\my-setup toolfunnel  # env var
-```
-
-An empty home is **seeded** on first use from the shipped defaults (the register + management
-scripts, an empty `mcp/expose.json`, the default hooks manifest) and is **never overwritten**
-afterwards - which is the point: with the home outside the package, `npm update toolfunnel`
-can never eat your tools.
-
-```
-<home>/
-  toolfunnel.json         # identity (serverName/serverVersion), port defaults, `requires`
-  tools/
-    tools.register.json   # your tool entries (incl. their inputSchema)
-    tools.state.json      # the hot/hidden/enabled curation - part of the product
-    scripts/              # the tool scripts
-  mcp/expose.json         # upstream references + curated exposure
-  hooks/                  # the policy gate: manifest + hook scripts
-```
-
-## `tf_pack` - one call to a deployment artifact
-
-The `tf_pack` management tool snapshots the LIVE setup into `<home>/dist/<name>/` - always a
-separate location, never the live tree. `auth/` (environment-specific) and `logs/` never ship.
-
-**`{ format: "home" }`** → a portable config home. Zip it, commit it to a repo, or run it as-is:
-
-```
-toolfunnel --config-dir path/to/the-pack
-```
-
-**`{ format: "npm", name: "my-mcp", version: "1.0.0" }`** → a publishable npm package:
-
-```
-dist/my-mcp/
-  package.json      # name, bin, files, dependencies: { "toolfunnel": "^<current version>" }
-  bin/my-mcp.js     # 2 lines: pin --config-dir at ../home, delegate to toolfunnel
-  home/             # the bundled config home (identity rewritten to my-mcp@1.0.0)
-  README.md         # a stub to make your own
-```
-
-Review it, then `cd dist/my-mcp && npm publish`. Your users run `npx my-mcp` and get **your**
-server: your name in the MCP `initialize` handshake, your tools with your schemas, your curation,
-and - the part nobody else can say - **your gate**. The hooks travel inside the pack and enforce
-your policy on the recipient's machine regardless of which MCP client calls it.
-
-### Depend, never copy
-
-The generated package **depends on** `toolfunnel` (caret range) rather than vendoring it. Your own
-runtime dependencies go in **your** `package.json` exactly as with any npm package - npm installs
-the whole tree on `npx my-mcp`; toolfunnel itself adds zero. Why depending matters:
-
-- security and bug fixes reach every wrapped MCP through a normal `npm update` (the pack pins the caret range of the version that built it, so patch releases flow automatically) - no stale forks
-  carrying old bugs;
-- your package stays a few KB of config instead of a copied engine;
-- (and honestly) every install of your MCP counts as a toolfunnel download, which keeps the thing
-  you depend on alive.
-
-Your users can move the bundled home out of `node_modules` (so *their* edits survive *your*
-updates) via the `<NAME>_HOME` env var the generated launcher supports.
-
-## Declaring runtimes: `requires`
-
-Your tools may need runtimes the gateway doesn't (python, git, ffmpeg...). Declare them in the
-home's `toolfunnel.json` and they travel with every pack:
-
-```json
-{
-  "serverName": "my-mcp",
-  "requires": [
-    { "command": "python", "min": "3.10", "why": "the pdf-extract tools" },
-    { "command": "git" }
-  ]
-}
-```
-
-The gateway probes them once at startup and prints one clear stderr line per problem - found
-version vs needed, plus your `why`. Advisory by design: a missing runtime breaks only the tools
-that need it, so the server still starts and everything else keeps working.
-
-## Composite packs
-
-A pack is not just your own scripts. Because `mcp/expose.json` lives in the home, a pack carries
-**upstream references too**: "attach `@example/weather-mcp` (pin the version) and expose exactly
-these four tools under these names." The upstream itself travels as a *reference* - `npx` fetches
-it on the recipient's machine - while your **curation** and your **gate** travel as config. One
-pack = your tools + other people's servers, curated, renamed, and policed, as a single artifact.
-
-## Audit honesty
-
-Packs spawn commands - that is what they are for. Before you run anyone's pack (including one of
-ours), read **three files** - every command it can run is declared in plain JSON:
-
-- `tools/tools.register.json` - the tool invokes (scripts and shell commands);
-- `mcp/expose.json` - the upstream server commands it will spawn;
-- `toolfunnel.json` - the `requires` probes. Note these run **at gateway startup** (each declared
-  program is executed once with its version flag), which is why a `requires` command must be a
-  bare program name - the gateway rejects paths, spaces, and shell characters outright.
-
-Ship your own packs expecting the same scrutiny, and say so in your README (the `tf_pack` stub
-already does).
+# Packaging: ship what you built
+
+ToolFunnel's config is a folder. That single fact is the whole packaging story: everything you
+build in a gateway - your tools and their scripts, which of them are promoted hot or hidden, the
+upstream MCPs you attached and the tools you chose to expose from them, the policy hooks that gate
+every call, and the identity the server introduces itself with - lives in one directory tree, the
+**config home**. Ship the tree and you have shipped the product.
+
+## The config home
+
+By default the home is the package/clone root - a git clone works exactly as it always did. Point
+the gateway elsewhere and the config lives there instead:
+
+```
+toolfunnel --config-dir D:\my-setup     # flag (wins)
+TOOLFUNNEL_HOME=D:\my-setup toolfunnel  # env var
+```
+
+An empty home is **seeded** on first use from the shipped defaults (the register + management
+scripts, an empty `mcp/expose.json`, the default hooks manifest) and is **never overwritten**
+afterwards - which is the point: with the home outside the package, `npm update toolfunnel`
+can never eat your tools. The one thing an update adds is a tool new in that release, appended
+to your register once; a shipped tool you removed is never put back.
+
+```
+<home>/
+  toolfunnel.json         # identity (serverName/serverVersion), port defaults, `requires`
+  tools/
+    tools.register.json   # your tool entries (incl. their inputSchema)
+    tools.state.json      # the hot/hidden/enabled curation - part of the product
+    scripts/              # the tool scripts
+  mcp/expose.json         # upstream references + curated exposure
+  hooks/                  # the policy gate: manifest + hook scripts
+```
+
+## `tf_pack` - one call to a deployment artifact
+
+The `tf_pack` management tool snapshots the LIVE setup into `<home>/dist/<name>/` - always a
+separate location, never the live tree. `auth/` (environment-specific) and `logs/` never ship.
+
+**`{ format: "home" }`** → a portable config home. Zip it, commit it to a repo, or run it as-is:
+
+```
+toolfunnel --config-dir path/to/the-pack
+```
+
+**`{ format: "npm", name: "my-mcp", version: "1.0.0" }`** → a publishable npm package:
+
+```
+dist/my-mcp/
+  package.json      # name, bin, files, dependencies: { "toolfunnel": "^<current version>" }
+  bin/my-mcp.js     # 2 lines: pin --config-dir at ../home, delegate to toolfunnel
+  home/             # the bundled config home (identity rewritten to my-mcp@1.0.0)
+  README.md         # a stub to make your own
+```
+
+Review it, then `cd dist/my-mcp && npm publish`. Your users run `npx my-mcp` and get **your**
+server: your name in the MCP `initialize` handshake, your tools with your schemas, your curation,
+and - the part nobody else can say - **your gate**. The hooks travel inside the pack and enforce
+your policy on the recipient's machine regardless of which MCP client calls it.
+
+### Depend, never copy
+
+The generated package **depends on** `toolfunnel` (caret range) rather than vendoring it. Your own
+runtime dependencies go in **your** `package.json` exactly as with any npm package - npm installs
+the whole tree on `npx my-mcp`; toolfunnel itself adds zero. Why depending matters:
+
+- security and bug fixes reach every wrapped MCP through a normal `npm update` (the pack pins the caret range of the version that built it, so patch releases flow automatically) - no stale forks
+  carrying old bugs;
+- your package stays a few KB of config instead of a copied engine;
+- (and honestly) every install of your MCP counts as a toolfunnel download, which keeps the thing
+  you depend on alive.
+
+Your users can move the bundled home out of `node_modules` (so *their* edits survive *your*
+updates) via the `<NAME>_HOME` env var the generated launcher supports.
+
+## Declaring runtimes: `requires`
+
+Your tools may need runtimes the gateway doesn't (python, git, ffmpeg...). Declare them in the
+home's `toolfunnel.json` and they travel with every pack:
+
+```json
+{
+  "serverName": "my-mcp",
+  "requires": [
+    { "command": "python", "min": "3.10", "why": "the pdf-extract tools" },
+    { "command": "git" }
+  ]
+}
+```
+
+The gateway probes them once at startup and prints one clear stderr line per problem - found
+version vs needed, plus your `why`. Advisory by design: a missing runtime breaks only the tools
+that need it, so the server still starts and everything else keeps working.
+
+## Composite packs
+
+A pack is not just your own scripts. Because `mcp/expose.json` lives in the home, a pack carries
+**upstream references too**: "attach `@example/weather-mcp` (pin the version) and expose exactly
+these four tools under these names." The upstream itself travels as a *reference* - `npx` fetches
+it on the recipient's machine - while your **curation** and your **gate** travel as config. One
+pack = your tools + other people's servers, curated, renamed, and policed, as a single artifact.
+
+## Audit honesty
+
+Packs spawn commands - that is what they are for. Before you run anyone's pack (including one of
+ours), read **three files** - every command it can run is declared in plain JSON:
+
+- `tools/tools.register.json` - the tool invokes (scripts and shell commands);
+- `mcp/expose.json` - the upstream server commands it will spawn;
+- `toolfunnel.json` - the `requires` probes. Note these run **at gateway startup** (each declared
+  program is executed once with its version flag), which is why a `requires` command must be a
+  bare program name - the gateway rejects paths, spaces, and shell characters outright.
+
+Ship your own packs expecting the same scrutiny, and say so in your README (the `tf_pack` stub
+already does).

package/hooks/hooks.manifest.json

@@ -1,27 +1,27 @@
-{
-  "version": 1,
-  "hooks": [
-    {
-      "id": "pre-tool-use/example-deny-dangerous",
-      "event": "PreToolUse",
-      "matcher": "*",
-      "type": "command",
-      "command": "node \"${HOOKS_DIR}/scripts/example-deny-dangerous.js\"",
-      "script": "scripts/example-deny-dangerous.js",
-      "timeout": 10,
-      "enabled": false,
-      "description": "EXAMPLE (disabled by default). A PreToolUse policy gate: denies any tool call whose arguments contain an obviously-destructive shell pattern (rm -rf, mkfs, drive format, fork bomb). Exit 2 denies and blocks the call before it runs; exit 0 allows. Any OTHER exit (a crash, a missing script, a timeout) is a non-blocking error and the call PROCEEDS - write denies as exit 2, never as a bare non-zero. Enable it in the UI Hooks tab (or set enabled:true here) to see the gate bite; edit scripts/example-deny-dangerous.js to encode your own policy."
-    },
-    {
-      "id": "post-tool-use/example-audit",
-      "event": "PostToolUse",
-      "matcher": "*",
-      "type": "command",
-      "command": "node \"${HOOKS_DIR}/scripts/example-audit.js\"",
-      "script": "scripts/example-audit.js",
-      "timeout": 10,
-      "enabled": false,
-      "description": "EXAMPLE (disabled by default). A PostToolUse observer: runs AFTER a tool executes (advisory - it cannot un-run the call) and receives the tool result on stdin. The stub exits 0 (clean pass); replace its body to forward results to your own audit sink, metrics, or SIEM. Enable it in the UI Hooks tab (or set enabled:true here)."
-    }
-  ]
-}
+{
+  "version": 1,
+  "hooks": [
+    {
+      "id": "pre-tool-use/example-deny-dangerous",
+      "event": "PreToolUse",
+      "matcher": "*",
+      "type": "command",
+      "command": "node \"${HOOKS_DIR}/scripts/example-deny-dangerous.js\"",
+      "script": "scripts/example-deny-dangerous.js",
+      "timeout": 10,
+      "enabled": false,
+      "description": "EXAMPLE (disabled by default). A PreToolUse policy gate: denies any tool call whose arguments contain an obviously-destructive shell pattern (rm -rf, mkfs, drive format, fork bomb). Exit 2 denies and blocks the call before it runs; exit 0 allows. Any OTHER exit (a crash, a missing script, a timeout) is a non-blocking error and the call PROCEEDS - write denies as exit 2, never as a bare non-zero. Enable it in the UI Hooks tab (or set enabled:true here) to see the gate bite; edit scripts/example-deny-dangerous.js to encode your own policy."
+    },
+    {
+      "id": "post-tool-use/example-audit",
+      "event": "PostToolUse",
+      "matcher": "*",
+      "type": "command",
+      "command": "node \"${HOOKS_DIR}/scripts/example-audit.js\"",
+      "script": "scripts/example-audit.js",
+      "timeout": 10,
+      "enabled": false,
+      "description": "EXAMPLE (disabled by default). A PostToolUse observer: runs AFTER a tool executes (advisory - it cannot un-run the call) and receives the tool result on stdin. The stub exits 0 (clean pass); replace its body to forward results to your own audit sink, metrics, or SIEM. Enable it in the UI Hooks tab (or set enabled:true here)."
+    }
+  ]
+}

package/hooks/scripts/example-audit.js

@@ -1,21 +1,21 @@
-#!/usr/bin/env node
-'use strict';
-
-/**
- * EXAMPLE PostToolUse observer - DISABLED by default.
- *
- * PostToolUse is ADVISORY: it runs AFTER a tool has executed and CANNOT un-run it - it observes the
- * outcome. The lifecycle event (including the tool result) arrives as JSON on stdin. This stub just
- * exits 0 (a clean pass). Replace the body to forward results to your own audit sink, metrics, an
- * external SIEM, etc. Exit 0 = observed; a non-zero exit on PostToolUse is a non-blocking error.
- *
- * Zero dependencies - Node built-ins only.
- */
-
-let raw = '';
-process.stdin.on('data', (c) => { raw += c; });
-process.stdin.on('end', () => {
-  // let ev = JSON.parse(raw || '{}');  // { tool_name, tool_input, tool_response, ... }
-  // ... forward ev to your audit sink here ...
-  process.exit(0);
-});
+#!/usr/bin/env node
+'use strict';
+
+/**
+ * EXAMPLE PostToolUse observer - DISABLED by default.
+ *
+ * PostToolUse is ADVISORY: it runs AFTER a tool has executed and CANNOT un-run it - it observes the
+ * outcome. The lifecycle event (including the tool result) arrives as JSON on stdin. This stub just
+ * exits 0 (a clean pass). Replace the body to forward results to your own audit sink, metrics, an
+ * external SIEM, etc. Exit 0 = observed; a non-zero exit on PostToolUse is a non-blocking error.
+ *
+ * Zero dependencies - Node built-ins only.
+ */
+
+let raw = '';
+process.stdin.on('data', (c) => { raw += c; });
+process.stdin.on('end', () => {
+  // let ev = JSON.parse(raw || '{}');  // { tool_name, tool_input, tool_response, ... }
+  // ... forward ev to your audit sink here ...
+  process.exit(0);
+});

package/hooks/scripts/example-deny-dangerous.js

@@ -1,39 +1,39 @@
-#!/usr/bin/env node
-'use strict';
-
-/**
- * EXAMPLE PreToolUse gate - DISABLED by default (enable it in hooks/hooks.manifest.json, or via the
- * UI Hooks tab / the per-tool "Pre" toggle on the Tools tab).
- *
- * This is the shape of a real policy gate: it runs INSIDE the gateway, BEFORE a tool executes, and
- * `process.exit(2)` DENIES the call. The lifecycle event arrives as JSON on stdin; here we block any
- * call whose arguments contain an obviously-destructive shell pattern.
- *
- * Block protocol (the simple one): write the reason to stderr and `process.exit(2)`. (A richer JSON
- * protocol is also supported - see docs/MANUAL.pdf section 10.) Exit 0 = allow.
- *
- * EXIT 2, NOT "non-zero" - the difference matters when you write your own. Every OTHER exit (a
- * crash, a syntax error, a missing file, a timeout) is a NON-BLOCKING ERROR and the call proceeds,
- * so `exit(1)` on an error path allows the very call you meant to stop. If your policy cannot
- * reach a decision, exit 2 deliberately. (The gateway does refuse a decision it cannot READ -
- * output past the cap, unparseable JSON - because an unreadable answer is not permission. It
- * cannot do the same for a script that never answered: one broken hook file would take every tool
- * offline.) test/gate-semantics.test.js pins all of this.
- *
- * Zero dependencies - Node built-ins only.
- */
-
-let raw = '';
-process.stdin.on('data', (c) => { raw += c; });
-process.stdin.on('end', () => {
-  let ev = {};
-  try { ev = JSON.parse(raw || '{}'); } catch (_e) { /* malformed payload -> allow (don't block on a parse error) */ }
-  const args = JSON.stringify((ev && ev.tool_input) || {});
-  // A small illustrative denylist: recursive-force delete, mkfs, drive format, fork bomb.
-  const DANGER = /\brm\s+-[a-z]*r[a-z]*f|\bmkfs\b|\bformat\s+[a-z]:|:\(\)\s*\{\s*:/i;
-  if (DANGER.test(args)) {
-    process.stderr.write('example-deny-dangerous: blocked - the arguments contain a destructive pattern.\n');
-    process.exit(2); // DENY - the gateway refuses to run the tool
-  }
-  process.exit(0); // allow
-});
+#!/usr/bin/env node
+'use strict';
+
+/**
+ * EXAMPLE PreToolUse gate - DISABLED by default (enable it in hooks/hooks.manifest.json, or via the
+ * UI Hooks tab / the per-tool "Pre" toggle on the Tools tab).
+ *
+ * This is the shape of a real policy gate: it runs INSIDE the gateway, BEFORE a tool executes, and
+ * `process.exit(2)` DENIES the call. The lifecycle event arrives as JSON on stdin; here we block any
+ * call whose arguments contain an obviously-destructive shell pattern.
+ *
+ * Block protocol (the simple one): write the reason to stderr and `process.exit(2)`. (A richer JSON
+ * protocol is also supported - see docs/MANUAL.pdf section 10.) Exit 0 = allow.
+ *
+ * EXIT 2, NOT "non-zero" - the difference matters when you write your own. Every OTHER exit (a
+ * crash, a syntax error, a missing file, a timeout) is a NON-BLOCKING ERROR and the call proceeds,
+ * so `exit(1)` on an error path allows the very call you meant to stop. If your policy cannot
+ * reach a decision, exit 2 deliberately. (The gateway does refuse a decision it cannot READ -
+ * output past the cap, unparseable JSON - because an unreadable answer is not permission. It
+ * cannot do the same for a script that never answered: one broken hook file would take every tool
+ * offline.) test/gate-semantics.test.js pins all of this.
+ *
+ * Zero dependencies - Node built-ins only.
+ */
+
+let raw = '';
+process.stdin.on('data', (c) => { raw += c; });
+process.stdin.on('end', () => {
+  let ev = {};
+  try { ev = JSON.parse(raw || '{}'); } catch (_e) { /* malformed payload -> allow (don't block on a parse error) */ }
+  const args = JSON.stringify((ev && ev.tool_input) || {});
+  // A small illustrative denylist: recursive-force delete, mkfs, drive format, fork bomb.
+  const DANGER = /\brm\s+-[a-z]*r[a-z]*f|\bmkfs\b|\bformat\s+[a-z]:|:\(\)\s*\{\s*:/i;
+  if (DANGER.test(args)) {
+    process.stderr.write('example-deny-dangerous: blocked - the arguments contain a destructive pattern.\n');
+    process.exit(2); // DENY - the gateway refuses to run the tool
+  }
+  process.exit(0); // allow
+});

package/mcp/expose.example.json

@@ -1,30 +1,30 @@
-{
-  "_README": "SAMPLE config - NOT the live expose.json. This wires the bundled demo upstream at mcp/servers/mock-upstream/server.js so the MCP Manager has a real target. TO USE: either (A) copy this file over mcp/expose.json to pre-seed the host (drop the _README key - the loader ignores it, but copy cleanly), OR (B) leave expose.json empty and add the same upstream + expose entries through the MCP Manager UI (Add upstream -> Discover -> tick to expose). The upstream offers ping, add, echo; this sample exposes only TWO (ping, add) to demonstrate curation. IMPORTANT: the args path MUST resolve INSIDE the gateway root - the aggregator's isolation guard rejects any arg path that escapes the sandbox. A relative path like 'mcp/servers/mock-upstream/server.js' is resolved against the gateway root and stays inside it; 'node' in the command slot is the interpreter (not guarded). Do NOT edit the live mcp/expose.json to point outside the gateway root.",
-  "version": 1,
-  "upstreams": [
-    {
-      "id": "mock",
-      "transport": "stdio",
-      "command": "node",
-      "args": ["mcp/servers/mock-upstream/server.js"],
-      "enabled": true,
-      "description": "Bundled demo upstream MCP (zero-dependency, stdio): ping, add, echo. No network, no fs writes, no process mutation."
-    }
-  ],
-  "expose": [
-    {
-      "upstream": "mock",
-      "tool": "ping",
-      "as": "mock_ping",
-      "category": "demo",
-      "enabled": true
-    },
-    {
-      "upstream": "mock",
-      "tool": "add",
-      "as": "mock_add",
-      "category": "demo",
-      "enabled": true
-    }
-  ]
-}
+{
+  "_README": "SAMPLE config - NOT the live expose.json. This wires the bundled demo upstream at mcp/servers/mock-upstream/server.js so the MCP Manager has a real target. TO USE: either (A) copy this file over mcp/expose.json to pre-seed the host (drop the _README key - the loader ignores it, but copy cleanly), OR (B) leave expose.json empty and add the same upstream + expose entries through the MCP Manager UI (Add upstream -> Discover -> tick to expose). The upstream offers ping, add, echo; this sample exposes only TWO (ping, add) to demonstrate curation. IMPORTANT: the args path MUST resolve INSIDE the gateway root - the aggregator's isolation guard rejects any arg path that escapes the sandbox. A relative path like 'mcp/servers/mock-upstream/server.js' is resolved against the gateway root and stays inside it; 'node' in the command slot is the interpreter (not guarded). Do NOT edit the live mcp/expose.json to point outside the gateway root.",
+  "version": 1,
+  "upstreams": [
+    {
+      "id": "mock",
+      "transport": "stdio",
+      "command": "node",
+      "args": ["mcp/servers/mock-upstream/server.js"],
+      "enabled": true,
+      "description": "Bundled demo upstream MCP (zero-dependency, stdio): ping, add, echo. No network, no fs writes, no process mutation."
+    }
+  ],
+  "expose": [
+    {
+      "upstream": "mock",
+      "tool": "ping",
+      "as": "mock_ping",
+      "category": "demo",
+      "enabled": true
+    },
+    {
+      "upstream": "mock",
+      "tool": "add",
+      "as": "mock_add",
+      "category": "demo",
+      "enabled": true
+    }
+  ]
+}