toolfunnel 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (69) hide show
  1. package/README.md +231 -167
  2. package/bin/toolfunnel.js +372 -147
  3. package/docs/KNOWN_BUGS.md +99 -33
  4. package/docs/MANUAL.pdf +0 -0
  5. package/docs/design.md +70 -0
  6. package/docs/packaging.md +115 -115
  7. package/hooks/hooks.manifest.json +27 -27
  8. package/hooks/scripts/example-audit.js +21 -21
  9. package/hooks/scripts/example-deny-dangerous.js +31 -31
  10. package/mcp/expose.example.json +30 -30
  11. package/mcp/servers/mock-modern/server.js +147 -0
  12. package/mcp/servers/mock-upstream/server.js +192 -12
  13. package/package.json +1 -1
  14. package/src/auth/config.js +174 -174
  15. package/src/auth/install.js +181 -181
  16. package/src/auth/resource-server.js +344 -344
  17. package/src/core/config-home.js +134 -134
  18. package/src/core/events.js +105 -105
  19. package/src/core/hook-engine.js +275 -275
  20. package/src/core/hook-loader.js +791 -791
  21. package/src/core/hook-runner.js +346 -346
  22. package/src/core/logger.js +171 -171
  23. package/src/core/matcher.js +51 -51
  24. package/src/core/metrics.js +75 -75
  25. package/src/core/requires.js +138 -138
  26. package/src/core/server-config.js +98 -77
  27. package/src/core/tool-registry.js +180 -180
  28. package/src/extend/add-hook.md +152 -142
  29. package/src/extend/add-mcp.md +122 -121
  30. package/src/extend/configure.md +87 -0
  31. package/src/extend/create-tool.md +168 -152
  32. package/src/extend/howto.js +102 -98
  33. package/src/extend/package.md +187 -187
  34. package/src/extend/wrap.md +99 -0
  35. package/src/mcp/aggregator.js +1246 -807
  36. package/src/mcp/expose-store.js +460 -430
  37. package/src/mcp/gated-run.js +201 -194
  38. package/src/mcp/http-transport.js +1311 -1012
  39. package/src/mcp/mcp-client.js +907 -530
  40. package/src/mcp/modern.js +372 -0
  41. package/src/mcp/protocol.js +468 -466
  42. package/src/mcp/server.js +2688 -1405
  43. package/src/packages/packager.js +715 -715
  44. package/src/tools/drift.js +138 -138
  45. package/src/tools/registry.js +513 -513
  46. package/src/tools/tool-state.js +170 -104
  47. package/src/ui/public/app.js +1174 -1055
  48. package/src/ui/public/index.html +286 -249
  49. package/src/ui/public/styles.css +535 -535
  50. package/src/ui/server.js +1348 -1190
  51. package/tools/scripts/base64.js +139 -139
  52. package/tools/scripts/danger.js +61 -61
  53. package/tools/scripts/echo.js +39 -39
  54. package/tools/scripts/hash.js +101 -101
  55. package/tools/scripts/json.js +109 -109
  56. package/tools/scripts/text-stats.js +131 -131
  57. package/tools/scripts/tf-env.js +43 -43
  58. package/tools/scripts/tf-hook-add.js +151 -151
  59. package/tools/scripts/tf-hook-set.js +112 -112
  60. package/tools/scripts/tf-list.js +192 -192
  61. package/tools/scripts/tf-log.js +86 -86
  62. package/tools/scripts/tf-mcp-add.js +117 -117
  63. package/tools/scripts/tf-mcp-set.js +121 -102
  64. package/tools/scripts/tf-pack.js +237 -237
  65. package/tools/scripts/tf-tool-add.js +134 -129
  66. package/tools/scripts/tf-tool-set.js +142 -142
  67. package/tools/scripts/tf-wrap.js +103 -0
  68. package/tools/scripts/uuid.js +102 -102
  69. package/tools/tools.register.json +227 -216
@@ -1,142 +1,152 @@
1
- # How to add a hook (self-extension)
2
-
3
- This is the instruction served by `toolfunnel_howto({ topic: "add-hook" })`. It explains how to add a hook
4
- to the Hook Engine: the manifest entry shape, where the script goes, and how the enabled/disabled
5
- state is persisted separately in `hooks.state.json`. The full contract is HOOK_ENGINE.md — this is
6
- the authoring view.
7
-
8
- A hook is exactly two things:
9
- 1. **A manifest entry** in `hooks/hooks.manifest.json` (its config: event, matcher, command, …).
10
- 2. **A script** under `hooks/scripts/` that the entry's `command` runs.
11
-
12
- Activation (on/off) is NOT in the manifest's source of truth at runtime — it lives in an overlay,
13
- `hooks/hooks.state.json` (see §4).
14
-
15
- ---
16
-
17
- ## 1. The manifest entry shape
18
-
19
- `hooks.manifest.json` has `version`, `hooksDir`, and a `hooks` array. Each entry:
20
-
21
- ```json
22
- {
23
- "id": "pre-tool-use/deny-dangerous",
24
- "event": "PreToolUse",
25
- "matcher": "Bash|toolfunnel_run_tool",
26
- "type": "command",
27
- "command": "bash \"${HOOKS_DIR}/scripts/deny-dangerous.sh\"",
28
- "script": "scripts/deny-dangerous.sh",
29
- "timeout": 5,
30
- "enabled": true,
31
- "description": "Denies any tool call whose args contain a destructive pattern (rm -rf, etc)."
32
- }
33
- ```
34
-
35
- | Field | Type | Meaning |
36
- |---|---|---|
37
- | `id` | string | Unique key, format `"<kebab-event>/<script-basename-without-ext>"` (e.g. `pre-tool-use/deny-dangerous`). The loader and `hooks.state.json` key on this. |
38
- | `event` | string | One of the six lifecycle events: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `Stop`, `PreCompact`. |
39
- | `matcher` | string | Regex anchored full-match against `tool_name`. `""`, `undefined`, or `"*"` = always fire. Tool-less events (SessionStart, UserPromptSubmit, Stop, PreCompact) ignore it. Example: `"Bash|Write|Edit"`. |
40
- | `type` | string | `"command"` (the only type today). |
41
- | `command` | string | The command line the runner spawns. `${HOOKS_DIR}` expands to the absolute `hooks` path at load time. |
42
- | `script` | string | The script path relative to `hooksDir`, e.g. `scripts/deny-dangerous.sh`. Used by the manager's open/edit (`readScript`/`writeScript`). |
43
- | `timeout` | number | **Seconds** (the runner multiplies by 1000). The runner kills the child past this. |
44
- | `enabled` | boolean | The manifest's default/seed state. The live value is the overlay in `hooks.state.json` — see §4. |
45
- | `description` | string | What the hook does, for the manager UI. |
46
-
47
- One entry **per hook command** — a settings.json event with N hooks becomes N entries.
48
-
49
- ---
50
-
51
- ## 2. Where the script goes
52
-
53
- ```
54
- hooks/
55
- hooks.manifest.json # config entries (above)
56
- hooks.state.json # enabled/disabled overlay, keyed by hook id
57
- scripts/ # your hook scripts live HERE
58
- deny-dangerous.sh
59
- ```
60
-
61
- (The loader itself — `hook-loader.js` — is engine code and is NOT user content; it lives in
62
- `src/core/`, not in `hooks/`.)
63
-
64
- `writeScript` (the manager's save path) refuses any path that resolves outside
65
- `hooks/scripts/` — defense-in-depth for the isolation rule. Author your script there only;
66
- never point a hook at a script in a live host hooks directory. Clone the behaviour in if you need it.
67
-
68
- ---
69
-
70
- ## 3. What the script returns (the two protocols)
71
-
72
- The runner spawns the command, writes the event payload as JSON on stdin, then reads the result.
73
- Two protocols (HOOK_ENGINE.md §3):
74
-
75
- **A) Exit-code (simple):**
76
- - `exit 0` → success. On `SessionStart`/`UserPromptSubmit`, stdout becomes **injected context**;
77
- on other events stdout is advisory.
78
- - `exit 2` → **blocking**. stderr is the reason.
79
- - any other code → non-blocking error (stderr captured, loop continues; never throws).
80
-
81
- **B) JSON (advanced)** — honored **only on exit 0**, when stdout parses as a JSON object:
82
- ```json
83
- {
84
- "continue": true,
85
- "decision": "block",
86
- "reason": "…",
87
- "hookSpecificOutput": {
88
- "hookEventName": "PreToolUse",
89
- "permissionDecision": "allow" | "deny" | "ask",
90
- "permissionDecisionReason": "…",
91
- "additionalContext": "…"
92
- }
93
- }
94
- ```
95
- - **PreToolUse blocks via `hookSpecificOutput.permissionDecision: "deny"`** — NOT the top-level
96
- `decision` field. `"allow"` passes; `"ask"` is non-blocking in the autonomous host.
97
- - **Top-level `decision: "block"`** is for the other blockable events (UserPromptSubmit, PostToolUse,
98
- Stop, PreCompact). There is no `"approve"`.
99
- - `continue: false` stops the whole loop. `hookSpecificOutput.additionalContext` → injected text.
100
-
101
- This is the mechanism that makes a PreToolUse hook gate `toolfunnel_run_tool`: match its `tool_name`, and
102
- return `permissionDecision: "deny"` to block the call before the tool runs.
103
-
104
- ---
105
-
106
- ## 4. Activation state lives in `hooks.state.json`
107
-
108
- The Hook Manager design splits **what exists** from **what's on**:
109
-
110
- - **The `scripts/` folder is the source of truth for what exists.** On startup the loader scans it
111
- and reconciles with the manifest: newly-found scripts are added as entries (event/matcher inferred
112
- or flagged "unconfigured"); manifest entries whose script is missing are flagged.
113
- - **The manifest carries config** (event, matcher, command, timeout, description, seed `enabled`).
114
- - **`hooks.state.json` is the activation overlay** — an object keyed by hook `id` recording the live
115
- enabled/disabled value:
116
-
117
- ```json
118
- {
119
- "version": 1,
120
- "enabled": {
121
- "pre-tool-use/deny-dangerous": true,
122
- "user-prompt-submit/inject-context": false
123
- }
124
- }
125
- ```
126
-
127
- Toggling a hook writes `hooks.state.json` **atomically** (temp + rename). It survives restarts and is
128
- kept separate from the auto-detected inventory, so re-scanning the folder never clobbers your toggles.
129
- At load, the overlay wins over the manifest's seed `enabled` for any id it lists.
130
-
131
- ---
132
-
133
- ## 5. Steps to add a hook
134
-
135
- 1. Write the script into `hooks/scripts/<name>.sh` (or a node script). Read the stdin JSON
136
- payload; emit one of the two protocols above.
137
- 2. Add the manifest entry: `id` = `"<kebab-event>/<name>"`, set `event`, `matcher`, `command`
138
- (`bash "${HOOKS_DIR}/scripts/<name>.sh"`), `script`, `timeout` (seconds), `description`, and a
139
- seed `enabled`.
140
- 3. Set its live state in `hooks.state.json` via the loader's `setEnabled(id, bool)` (atomic write).
141
- 4. Verify: the manager lists it under its event; a run-once test fires it with a sample payload and
142
- shows the result. For a gate, confirm a matching `toolfunnel_run_tool` call is denied with your reason.
1
+ # How to add a hook (self-extension)
2
+
3
+ This is the instruction served by `toolfunnel_howto({ topic: "add-hook" })`. It explains how to add a hook
4
+ to the Hook Engine: the manifest entry shape, where the script goes, and how the enabled/disabled
5
+ state is persisted separately in `hooks.state.json`. The full contract is HOOK_ENGINE.md - this is
6
+ the authoring view.
7
+
8
+ A hook is exactly two things:
9
+ 1. **A manifest entry** in `hooks/hooks.manifest.json` (its config: event, matcher, command, ...).
10
+ 2. **A script** under `hooks/scripts/` that the entry's `command` runs.
11
+
12
+ Activation (on/off) is NOT in the manifest's source of truth at runtime - it lives in an overlay,
13
+ `hooks/hooks.state.json` (see §4).
14
+
15
+ ---
16
+
17
+ ## 1. The manifest entry shape
18
+
19
+ `hooks.manifest.json` has `version`, `hooksDir`, and a `hooks` array. Each entry:
20
+
21
+ ```json
22
+ {
23
+ "id": "pre-tool-use/deny-dangerous",
24
+ "event": "PreToolUse",
25
+ "matcher": "Bash|toolfunnel_run_tool",
26
+ "type": "command",
27
+ "command": "bash \"${HOOKS_DIR}/scripts/deny-dangerous.sh\"",
28
+ "script": "scripts/deny-dangerous.sh",
29
+ "timeout": 5,
30
+ "enabled": true,
31
+ "description": "Denies any tool call whose args contain a destructive pattern (rm -rf, etc)."
32
+ }
33
+ ```
34
+
35
+ | Field | Type | Meaning |
36
+ |---|---|---|
37
+ | `id` | string | Unique key, format `"<kebab-event>/<script-basename-without-ext>"` (e.g. `pre-tool-use/deny-dangerous`). The loader and `hooks.state.json` key on this. |
38
+ | `event` | string | One of the six lifecycle events: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `Stop`, `PreCompact`. |
39
+ | `matcher` | string | Regex anchored full-match against `tool_name`. `""`, `undefined`, or `"*"` = always fire. Tool-less events (SessionStart, UserPromptSubmit, Stop, PreCompact) ignore it. Example: `"Bash|Write|Edit"`. |
40
+ | `type` | string | `"command"` (the only type today). |
41
+ | `command` | string | The command line the runner spawns. `${HOOKS_DIR}` expands to the absolute `hooks` path at load time. |
42
+ | `script` | string | The script path relative to `hooksDir`, e.g. `scripts/deny-dangerous.sh`. Used by the manager's open/edit (`readScript`/`writeScript`). |
43
+ | `timeout` | number | **Seconds** (the runner multiplies by 1000). The runner kills the child past this. |
44
+ | `enabled` | boolean | The manifest's default/seed state. The live value is the overlay in `hooks.state.json` - see §4. |
45
+ | `description` | string | What the hook does, for the manager UI. |
46
+
47
+ One entry **per hook command** - a settings.json event with N hooks becomes N entries.
48
+
49
+ ---
50
+
51
+ ## 2. Where the script goes
52
+
53
+ ```
54
+ hooks/
55
+ hooks.manifest.json # config entries (above)
56
+ hooks.state.json # enabled/disabled overlay, keyed by hook id
57
+ scripts/ # your hook scripts live HERE
58
+ deny-dangerous.sh
59
+ ```
60
+
61
+ (The loader itself - `hook-loader.js` - is engine code and is NOT user content; it lives in
62
+ `src/core/`, not in `hooks/`.)
63
+
64
+ `writeScript` (the manager's save path) refuses any path that resolves outside
65
+ `hooks/scripts/` - defense-in-depth for the isolation rule. Author your script there only;
66
+ never point a hook at a script in a live host hooks directory. Clone the behaviour in if you need it.
67
+
68
+ ---
69
+
70
+ ## 3. What the script returns (the two protocols)
71
+
72
+ The runner spawns the command, writes the event payload as JSON on stdin, then reads the result.
73
+ The payload a tool event receives on stdin:
74
+
75
+ ```json
76
+ { "tool_name": "note_slow", "tool_input": { "arg": "value" } }
77
+ ```
78
+
79
+ (`PreToolUse` → `tool_name` + `tool_input`; `PostToolUse` adds `tool_response`. Tool-less
80
+ events carry their own fields - `prompt`, `source` - instead.)
81
+
82
+ Two protocols (HOOK_ENGINE.md §3):
83
+
84
+ **A) Exit-code (simple):**
85
+ - `exit 0` → success. On `SessionStart`/`UserPromptSubmit`, stdout becomes **injected context**;
86
+ on other events stdout is advisory.
87
+ - `exit 2` → **blocking**. stderr is the reason - on a blocked tool call the client receives it
88
+ as the result text with `isError: true`.
89
+ - any other code → non-blocking error (stderr captured, loop continues; never throws).
90
+
91
+ **B) JSON (advanced)** - honored **only on exit 0**, when stdout parses as a JSON object:
92
+ ```json
93
+ {
94
+ "continue": true,
95
+ "decision": "block",
96
+ "reason": "...",
97
+ "hookSpecificOutput": {
98
+ "hookEventName": "PreToolUse",
99
+ "permissionDecision": "allow" | "deny" | "ask",
100
+ "permissionDecisionReason": "...",
101
+ "additionalContext": "..."
102
+ }
103
+ }
104
+ ```
105
+ - **PreToolUse blocks via `hookSpecificOutput.permissionDecision: "deny"`** - NOT the top-level
106
+ `decision` field. `"allow"` passes; `"ask"` is non-blocking in the autonomous host.
107
+ - **Top-level `decision: "block"`** is for the other blockable events (UserPromptSubmit, PostToolUse,
108
+ Stop, PreCompact). There is no `"approve"`.
109
+ - `continue: false` stops the whole loop. `hookSpecificOutput.additionalContext` → injected text.
110
+
111
+ This is the mechanism that makes a PreToolUse hook gate `toolfunnel_run_tool`: match its `tool_name`, and
112
+ return `permissionDecision: "deny"` to block the call before the tool runs.
113
+
114
+ ---
115
+
116
+ ## 4. Activation state lives in `hooks.state.json`
117
+
118
+ The Hook Manager design splits **what exists** from **what's on**:
119
+
120
+ - **The `scripts/` folder is the source of truth for what exists.** On startup the loader scans it
121
+ and reconciles with the manifest: newly-found scripts are added as entries (event/matcher inferred
122
+ or flagged "unconfigured"); manifest entries whose script is missing are flagged.
123
+ - **The manifest carries config** (event, matcher, command, timeout, description, seed `enabled`).
124
+ - **`hooks.state.json` is the activation overlay** - an object keyed by hook `id` recording the live
125
+ enabled/disabled value:
126
+
127
+ ```json
128
+ {
129
+ "version": 1,
130
+ "enabled": {
131
+ "pre-tool-use/deny-dangerous": true,
132
+ "user-prompt-submit/inject-context": false
133
+ }
134
+ }
135
+ ```
136
+
137
+ Toggling a hook writes `hooks.state.json` **atomically** (temp + rename). It survives restarts and is
138
+ kept separate from the auto-detected inventory, so re-scanning the folder never clobbers your toggles.
139
+ At load, the overlay wins over the manifest's seed `enabled` for any id it lists.
140
+
141
+ ---
142
+
143
+ ## 5. Steps to add a hook
144
+
145
+ 1. Write the script into `hooks/scripts/<name>.sh` (or a node script). Read the stdin JSON
146
+ payload; emit one of the two protocols above.
147
+ 2. Add the manifest entry: `id` = `"<kebab-event>/<name>"`, set `event`, `matcher`, `command`
148
+ (`bash "${HOOKS_DIR}/scripts/<name>.sh"`), `script`, `timeout` (seconds), `description`, and a
149
+ seed `enabled`.
150
+ 3. Set its live state in `hooks.state.json` via the loader's `setEnabled(id, bool)` (atomic write).
151
+ 4. Verify: the manager lists it under its event; a run-once test fires it with a sample payload and
152
+ shows the result. For a gate, confirm a matching `toolfunnel_run_tool` call is denied with your reason.
@@ -1,121 +1,122 @@
1
- # How to register an upstream MCP (self-extension)
2
-
3
- This is the instruction served by `toolfunnel_howto({ topic: "add-mcp" })`. It explains how to connect an
4
- **upstream MCP server** (a browser-automation MCP, a chat bridge, …) to the aggregator and choose **which of its
5
- tools to expose downstream** to the CLI. All of this is configured in one persisted file:
6
- `mcp/expose.json`.
7
-
8
- The architecture (see the architecture overview, §1): many upstream MCPs each expose many tools; the
9
- aggregator connects to them and the **MCP Manager picks which tools to surface** to the CLI as
10
- *curated-direct* tools — alongside the four lean meta-tools (`toolfunnel_list_tools`,
11
- `toolfunnel_tool_instructions`, `toolfunnel_run_tool`, `toolfunnel_howto`). The CLI sees a small surface; everything
12
- else stays reachable through the register.
13
-
14
- > Phase note: the live aggregator (`src/mcp/aggregator.js`) is Phase 2. This doc defines the
15
- > `expose.json` shape so the config is authorable now and the aggregator reads exactly these fields
16
- > when it lands.
17
-
18
- ---
19
-
20
- ## 1. `expose.json` — the two blocks
21
-
22
- `expose.json` persists (a) the upstream MCP connections and (b) the curated-direct selection drawn
23
- from them:
24
-
25
- ```json
26
- {
27
- "version": 1,
28
- "upstreams": [
29
- {
30
- "id": "chat",
31
- "transport": "stdio",
32
- "command": "node",
33
- "args": ["vendor/chat-mcp/server.js"],
34
- "env": { "CHAT_DB": "..." },
35
- "enabled": true,
36
- "description": "Chat bridge MCP — chats, messages, send."
37
- },
38
- {
39
- "id": "browser",
40
- "transport": "stdio",
41
- "command": "npx",
42
- "args": ["@example/browser-mcp"],
43
- "enabled": false,
44
- "description": "Browser automation MCP."
45
- }
46
- ],
47
- "expose": [
48
- {
49
- "upstream": "chat",
50
- "tool": "list_chats",
51
- "as": "chat_list_chats",
52
- "category": "chat",
53
- "enabled": true
54
- },
55
- {
56
- "upstream": "chat",
57
- "tool": "send_message",
58
- "as": "chat_send_message",
59
- "category": "chat",
60
- "enabled": true
61
- }
62
- ]
63
- }
64
- ```
65
-
66
- ### `upstreams[]` — one entry per upstream MCP
67
-
68
- | Field | Type | Meaning |
69
- |---|---|---|
70
- | `id` | string | Stable key for this upstream. Referenced by `expose[].upstream`. Unique. |
71
- | `transport` | string | How the aggregator connects. `"stdio"` (spawn `command`+`args`) is the baseline. |
72
- | `command` / `args` / `env` | string / string[] / object | The process the aggregator spawns for a `stdio` upstream. Paths must point **inside the project's vendor directory** (clone vendored servers in; never reference servers configured elsewhere on the host). |
73
- | `enabled` | boolean | Whether the aggregator connects to this upstream at startup. |
74
- | `description` | string | Human-facing note for the MCP Manager UI. |
75
-
76
- ### `expose[]` — which upstream tools become curated-direct
77
-
78
- | Field | Type | Meaning |
79
- |---|---|---|
80
- | `upstream` | string | Must match an `upstreams[].id`. |
81
- | `tool` | string | The upstream tool's own name (as the upstream's `tools/list` reports it). |
82
- | `as` | string | The downstream name the CLI sees (namespaced so two upstreams can't collide, e.g. `chat_send_message`). Optional — defaults to `<upstream>_<tool>`. |
83
- | `category` | string | Grouping label, same convention as register entries. |
84
- | `enabled` | boolean | Whether this tool is actually exposed downstream right now. |
85
-
86
- Only the tools listed in `expose[]` with `enabled: true` are surfaced to the CLI as curated-direct
87
- tools. Everything else on the upstream stays connected but hidden — reachable, if needed, by adding
88
- an `expose` entry (no upstream reconnect required to *list*, though see §3 on the CLI surface).
89
-
90
- ---
91
-
92
- ## 2. Steps to register an upstream and expose a tool
93
-
94
- 1. Add an entry to `upstreams[]` with a unique `id`, the `stdio` `command`/`args`, and
95
- `enabled: true`. Vendor the server **inside the project's vendor directory** if it isn't already.
96
- 2. Let the aggregator connect and run the upstream's `tools/list` (the MCP Manager UI surfaces the
97
- discovered tools).
98
- 3. For each tool you want hot, add an `expose[]` entry: `{ upstream, tool, as, category, enabled:true }`.
99
- 4. Leave the long tail unexposed — the model can still reach those via the register/meta-tools if you
100
- later mirror them, but the *curated-direct* surface stays lean by design (the dial in §1 of the
101
- architecture: directly-exposed = the HOT tools; the register meta-tools = the long tail).
102
-
103
- ---
104
-
105
- ## 3. Curated-direct vs. the register (update cost)
106
-
107
- This matters because the two layers refresh differently (see the architecture overview, §1):
108
-
109
- - **Curated-direct tools** (what `expose[]` produces) are negotiated with the CLI **at connect time**.
110
- Changing the exposed set therefore needs a **reconnect**, or a `notifications/tools/list_changed`
111
- message *if the CLI honours it* (VERIFY per host; restart is the safe baseline).
112
- - **Register tools** (the `toolfunnel_list_tools`/`run` path) are **always live** — the model re-reads the
113
- register on every `toolfunnel_list_tools` call, so no reconnect is needed.
114
-
115
- Rule of thumb: expose a handful of HOT, high-reliability tools curated-direct; everything else lives
116
- in the register and is reached through the meta-tools.
117
-
118
- > Safety: a curated-direct upstream tool is still ultimately invoked by the model. When you wire
119
- > upstream execution through the host, route it through the same PreToolUse gate the register's `run`
120
- > uses (see `toolfunnel_howto({ topic: "add-hook" })`) so the safety invariant holds uniformly. The gate
121
- > firing on every execution path is a tested invariant, not an assumption.
1
+ # How to register an upstream MCP (self-extension)
2
+
3
+ This is the instruction served by `toolfunnel_howto({ topic: "add-mcp" })`. It explains how to connect an
4
+ **upstream MCP server** (a browser-automation MCP, a chat bridge, ...) to the aggregator and choose **which of its
5
+ tools to expose downstream** to the CLI. All of this is configured in one persisted file:
6
+ `mcp/expose.json`.
7
+
8
+ The architecture (see the architecture overview, §1): many upstream MCPs each expose many tools; the
9
+ aggregator connects to them and the **MCP Manager picks which tools to surface** to the CLI as
10
+ *curated-direct* tools - alongside the four lean meta-tools (`toolfunnel_list_tools`,
11
+ `toolfunnel_tool_instructions`, `toolfunnel_run_tool`, `toolfunnel_howto`). The CLI sees a small surface; everything
12
+ else stays reachable through the register.
13
+
14
+ > Phase note: the live aggregator (`src/mcp/aggregator.js`) is Phase 2. This doc defines the
15
+ > `expose.json` shape so the config is authorable now and the aggregator reads exactly these fields
16
+ > when it lands.
17
+
18
+ ---
19
+
20
+ ## 1. `expose.json` - the two blocks
21
+
22
+ `expose.json` persists (a) the upstream MCP connections and (b) the curated-direct selection drawn
23
+ from them:
24
+
25
+ ```json
26
+ {
27
+ "version": 1,
28
+ "upstreams": [
29
+ {
30
+ "id": "chat",
31
+ "transport": "stdio",
32
+ "command": "node",
33
+ "args": ["vendor/chat-mcp/server.js"],
34
+ "env": { "CHAT_DB": "..." },
35
+ "enabled": true,
36
+ "description": "Chat bridge MCP - chats, messages, send."
37
+ },
38
+ {
39
+ "id": "browser",
40
+ "transport": "stdio",
41
+ "command": "npx",
42
+ "args": ["@example/browser-mcp"],
43
+ "enabled": false,
44
+ "description": "Browser automation MCP."
45
+ }
46
+ ],
47
+ "expose": [
48
+ {
49
+ "upstream": "chat",
50
+ "tool": "list_chats",
51
+ "as": "chat_list_chats",
52
+ "category": "chat",
53
+ "enabled": true
54
+ },
55
+ {
56
+ "upstream": "chat",
57
+ "tool": "send_message",
58
+ "as": "chat_send_message",
59
+ "category": "chat",
60
+ "enabled": true
61
+ }
62
+ ]
63
+ }
64
+ ```
65
+
66
+ ### `upstreams[]` - one entry per upstream MCP
67
+
68
+ | Field | Type | Meaning |
69
+ |---|---|---|
70
+ | `id` | string | Stable key for this upstream. Referenced by `expose[].upstream`. Unique. |
71
+ | `transport` | string | How the aggregator connects. `"stdio"` (spawn `command`+`args`) is the baseline. |
72
+ | `command` / `args` / `env` | string / string[] / object | The process the aggregator spawns for a `stdio` upstream. Path-shaped **args** must stay **inside the config home** - the isolation guard refuses to connect an upstream whose args escape it. (The `command` itself, an executable, is not guarded; the currently-WRAPPED upstream is exempt in transparent-wrapper mode, with a warning.) |
73
+ | `enabled` | boolean | Whether the aggregator connects to this upstream at startup. |
74
+ | `description` | string | Human-facing note for the MCP Manager UI. |
75
+
76
+ ### `expose[]` - which upstream tools become curated-direct
77
+
78
+ | Field | Type | Meaning |
79
+ |---|---|---|
80
+ | `upstream` | string | Must match an `upstreams[].id`. |
81
+ | `tool` | string | The upstream tool's own name (as the upstream's `tools/list` reports it). |
82
+ | `as` | string | The downstream name the CLI sees (namespaced so two upstreams can't collide, e.g. `chat_send_message`). Optional - defaults to `<upstream>_<tool>`. |
83
+ | `category` | string | Grouping label, same convention as register entries. |
84
+ | `enabled` | boolean | Whether this tool is actually exposed downstream right now. |
85
+
86
+ Only the tools listed in `expose[]` with `enabled: true` are surfaced to the CLI as curated-direct
87
+ tools. Everything else on the upstream stays connected but hidden - reachable, if needed, by adding
88
+ an `expose` entry (no upstream reconnect required to *list*, though see §3 on the CLI surface).
89
+
90
+ ---
91
+
92
+ ## 2. Steps to register an upstream and expose a tool
93
+
94
+ 1. Add an entry to `upstreams[]` with a unique `id`, the `stdio` `command`/`args`, and
95
+ `enabled: true`. Keep the server's path-shaped args **inside the config home** (vendor the
96
+ server in if it isn't already) - the isolation guard refuses args that escape it.
97
+ 2. Let the aggregator connect and run the upstream's `tools/list` (the MCP Manager UI surfaces the
98
+ discovered tools).
99
+ 3. For each tool you want hot, add an `expose[]` entry: `{ upstream, tool, as, category, enabled:true }`.
100
+ 4. Leave the long tail unexposed - the model can still reach those via the register/meta-tools if you
101
+ later mirror them, but the *curated-direct* surface stays lean by design (the dial in §1 of the
102
+ architecture: directly-exposed = the HOT tools; the register meta-tools = the long tail).
103
+
104
+ ---
105
+
106
+ ## 3. Curated-direct vs. the register (update cost)
107
+
108
+ This matters because the two layers refresh differently (see the architecture overview, §1):
109
+
110
+ - **Curated-direct tools** (what `expose[]` produces) are negotiated with the CLI **at connect time**.
111
+ Changing the exposed set therefore needs a **reconnect**, or a `notifications/tools/list_changed`
112
+ message *if the CLI honours it* (VERIFY per host; restart is the safe baseline).
113
+ - **Register tools** (the `toolfunnel_list_tools`/`run` path) are **always live** - the model re-reads the
114
+ register on every `toolfunnel_list_tools` call, so no reconnect is needed.
115
+
116
+ Rule of thumb: expose a handful of HOT, high-reliability tools curated-direct; everything else lives
117
+ in the register and is reached through the meta-tools.
118
+
119
+ > Safety: a curated-direct upstream tool is still ultimately invoked by the model. When you wire
120
+ > upstream execution through the host, route it through the same PreToolUse gate the register's `run`
121
+ > uses (see `toolfunnel_howto({ topic: "add-hook" })`) so the safety invariant holds uniformly. The gate
122
+ > firing on every execution path is a tested invariant, not an assumption.
@@ -0,0 +1,87 @@
1
+ # How to configure ToolFunnel - config files only, no code
2
+
3
+ This is the instruction served by `toolfunnel_howto({ topic: "configure" })`. It is the complete
4
+ no-code setup map: every behaviour of the gateway is driven by a small set of JSON files at the
5
+ **config home**. Editing these files (or driving the UI / the `tf_*` management tools, which write
6
+ the same files) is the entire configuration story - there is nothing to program.
7
+
8
+ ## 1. The config home
9
+
10
+ All files live under one directory, resolved in this order:
11
+
12
+ 1. `--config-dir <dir>` on the command line
13
+ 2. the `TOOLFUNNEL_HOME` environment variable
14
+ 3. the package/repo root (the default for a git clone)
15
+
16
+ On first start with a fresh `--config-dir`, the home is seeded with the standard layout. Every
17
+ file below is read FRESH per request - edits are live without a restart, except `toolfunnel.json`
18
+ (read at boot; restart to apply).
19
+
20
+ ## 2. The files
21
+
22
+ | File | What it controls | Live? |
23
+ |---|---|---|
24
+ | `toolfunnel.json` | Identity + default ports (see §3) | restart |
25
+ | `mcp/expose.json` | Attached upstream MCPs + which of their tools are curated-direct | live |
26
+ | `tools/tools.register.json` | The local tool register (the gateway's own tools) | live |
27
+ | `tools/tools.state.json` | Per-tool overlays: enabled / hot / hidden, and the active wrap (`"passthrough"`). Wrapped example: `{ "passthrough": "acme-notes" }`; cleared: `{}` (the file-edit equivalent of `wrap --off`) | live |
28
+ | `hooks/hooks.manifest.json` | PreToolUse / PostToolUse policy hooks (the gate) | live |
29
+ | `hooks/hooks.state.json` | Per-hook enabled overlay | live |
30
+ | `logs/log.config.json` | Activity log on/off + path | live |
31
+ | `auth/auth.config.json` | OAuth 2.1 resource-server validation (opt-in; needs `toolfunnel install-oauth` once) | live |
32
+
33
+ Shapes: `expose.json` → `toolfunnel_howto({ topic: "add-mcp" })`; register entries →
34
+ `toolfunnel_howto({ topic: "create-tool" })`; hook entries → `toolfunnel_howto({ topic: "add-hook" })`;
35
+ the wrap field → `toolfunnel_howto({ topic: "wrap" })`.
36
+
37
+ ## 3. `toolfunnel.json` - identity & ports
38
+
39
+ Every field optional; an absent or empty file IS the default identity. Blank/omitted fields fall
40
+ back per-field (a broken file never stops the gateway starting).
41
+
42
+ ```json
43
+ {
44
+ "serverName": "my-mcp", // what CLIENTS see in the initialize handshake (default "toolfunnel")
45
+ "serverVersion": "1.0.0", // ...its version (default: package version)
46
+ "clientName": "my-client", // what UPSTREAM servers see from the gateway (default "toolfunnel")
47
+ "clientVersion": "1.0.0", // ...its version
48
+ "httpPort": 9998, // --http default port (default 9998)
49
+ "uiPort": 9777 // --ui default port (default 9777)
50
+ }
51
+ ```
52
+
53
+ Precedence: CLI flag > this file > built-in default. Under a WRAP on stdio, the wrapped upstream
54
+ is shown the real downstream client's identity automatically - `clientName`/`clientVersion` cover
55
+ funnel mode, HTTP, and the boot connection. The UI's **Settings** tab edits this file.
56
+
57
+ ## 4. Worked example - a complete gateway from five files
58
+
59
+ A gateway that attaches one upstream, exposes two of its tools, gates everything, logs activity,
60
+ and introduces itself as `acme-tools`:
61
+
62
+ 1. `toolfunnel.json` → `{ "serverName": "acme-tools", "clientName": "acme-tools" }`
63
+ 2. `mcp/expose.json` → one `upstreams[]` entry (id, `"transport": "stdio"`, command, args,
64
+ `"enabled": true`) + two `expose[]` entries (`{ "upstream", "tool", "enabled": true }`)
65
+ 3. `hooks/hooks.manifest.json` → one PreToolUse entry matching the tools you want policy on
66
+ 4. `logs/log.config.json` → `{ "enabled": true }`
67
+ 5. Start it: `node bin/toolfunnel.js` (stdio) or `--http` / `--ui` as needed.
68
+
69
+ Optional per-upstream flags in `expose.json`: `"legacyPin": true` pins that upstream to the
70
+ legacy MCP protocol permanently (opt-in, warns loudly); `"env"` passes environment variables to
71
+ the spawned server; `"cwd"` sets its working directory; `"timeoutMs"` raises the payload
72
+ timeout for that upstream (default 120000 ms - applies to `tools/call`, `prompts/get` and
73
+ `resources/read`; under a wrap, a tool that reports progress keeps its call alive regardless.
74
+ The 10 s handshake/list window is fixed: it detects dead servers); `"modernOnly": true`
75
+ requires that upstream to speak the modern (2026-07-28) protocol - the connect fails with a
76
+ clear error instead of falling back to legacy (the mirror of `legacyPin`; setting both on one
77
+ upstream is refused). Server side, `"serveLegacy": false` in `toolfunnel.json` makes the
78
+ gateway itself modern-only: legacy clients are refused with a clear error (default: serve
79
+ both eras - only an explicit `false` flips it).
80
+
81
+ ## 5. Where the AI-driven and human paths meet
82
+
83
+ The UI (all five tabs + Settings), the `tf_*` management tools, and hand-editing these files are
84
+ three views of the SAME state - every write goes through the same validated, atomic store code.
85
+ Pick whichever fits: config files for repeatable/scripted setups, the UI for humans, the
86
+ management tools for in-band agent-driven changes (each documented via
87
+ `toolfunnel_tool_instructions`).