toolfunnel 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +231 -167
- package/bin/toolfunnel.js +372 -147
- package/docs/KNOWN_BUGS.md +99 -33
- package/docs/MANUAL.pdf +0 -0
- package/docs/design.md +70 -0
- package/docs/packaging.md +115 -115
- package/hooks/hooks.manifest.json +27 -27
- package/hooks/scripts/example-audit.js +21 -21
- package/hooks/scripts/example-deny-dangerous.js +31 -31
- package/mcp/expose.example.json +30 -30
- package/mcp/servers/mock-modern/server.js +147 -0
- package/mcp/servers/mock-upstream/server.js +192 -12
- package/package.json +1 -1
- package/src/auth/config.js +174 -174
- package/src/auth/install.js +181 -181
- package/src/auth/resource-server.js +344 -344
- package/src/core/config-home.js +134 -134
- package/src/core/events.js +105 -105
- package/src/core/hook-engine.js +275 -275
- package/src/core/hook-loader.js +791 -791
- package/src/core/hook-runner.js +346 -346
- package/src/core/logger.js +171 -171
- package/src/core/matcher.js +51 -51
- package/src/core/metrics.js +75 -75
- package/src/core/requires.js +138 -138
- package/src/core/server-config.js +98 -77
- package/src/core/tool-registry.js +180 -180
- package/src/extend/add-hook.md +152 -142
- package/src/extend/add-mcp.md +122 -121
- package/src/extend/configure.md +87 -0
- package/src/extend/create-tool.md +168 -152
- package/src/extend/howto.js +102 -98
- package/src/extend/package.md +187 -187
- package/src/extend/wrap.md +99 -0
- package/src/mcp/aggregator.js +1246 -807
- package/src/mcp/expose-store.js +460 -430
- package/src/mcp/gated-run.js +201 -194
- package/src/mcp/http-transport.js +1311 -1012
- package/src/mcp/mcp-client.js +907 -530
- package/src/mcp/modern.js +372 -0
- package/src/mcp/protocol.js +468 -466
- package/src/mcp/server.js +2688 -1405
- package/src/packages/packager.js +715 -715
- package/src/tools/drift.js +138 -138
- package/src/tools/registry.js +513 -513
- package/src/tools/tool-state.js +170 -104
- package/src/ui/public/app.js +1174 -1055
- package/src/ui/public/index.html +286 -249
- package/src/ui/public/styles.css +535 -535
- package/src/ui/server.js +1348 -1190
- package/tools/scripts/base64.js +139 -139
- package/tools/scripts/danger.js +61 -61
- package/tools/scripts/echo.js +39 -39
- package/tools/scripts/hash.js +101 -101
- package/tools/scripts/json.js +109 -109
- package/tools/scripts/text-stats.js +131 -131
- package/tools/scripts/tf-env.js +43 -43
- package/tools/scripts/tf-hook-add.js +151 -151
- package/tools/scripts/tf-hook-set.js +112 -112
- package/tools/scripts/tf-list.js +192 -192
- package/tools/scripts/tf-log.js +86 -86
- package/tools/scripts/tf-mcp-add.js +117 -117
- package/tools/scripts/tf-mcp-set.js +121 -102
- package/tools/scripts/tf-pack.js +237 -237
- package/tools/scripts/tf-tool-add.js +134 -129
- package/tools/scripts/tf-tool-set.js +142 -142
- package/tools/scripts/tf-wrap.js +103 -0
- package/tools/scripts/uuid.js +102 -102
- package/tools/tools.register.json +227 -216
package/src/extend/add-hook.md
CHANGED
|
@@ -1,142 +1,152 @@
|
|
|
1
|
-
# How to add a hook (self-extension)
|
|
2
|
-
|
|
3
|
-
This is the instruction served by `toolfunnel_howto({ topic: "add-hook" })`. It explains how to add a hook
|
|
4
|
-
to the Hook Engine: the manifest entry shape, where the script goes, and how the enabled/disabled
|
|
5
|
-
state is persisted separately in `hooks.state.json`. The full contract is HOOK_ENGINE.md
|
|
6
|
-
the authoring view.
|
|
7
|
-
|
|
8
|
-
A hook is exactly two things:
|
|
9
|
-
1. **A manifest entry** in `hooks/hooks.manifest.json` (its config: event, matcher, command,
|
|
10
|
-
2. **A script** under `hooks/scripts/` that the entry's `command` runs.
|
|
11
|
-
|
|
12
|
-
Activation (on/off) is NOT in the manifest's source of truth at runtime
|
|
13
|
-
`hooks/hooks.state.json` (see §4).
|
|
14
|
-
|
|
15
|
-
---
|
|
16
|
-
|
|
17
|
-
## 1. The manifest entry shape
|
|
18
|
-
|
|
19
|
-
`hooks.manifest.json` has `version`, `hooksDir`, and a `hooks` array. Each entry:
|
|
20
|
-
|
|
21
|
-
```json
|
|
22
|
-
{
|
|
23
|
-
"id": "pre-tool-use/deny-dangerous",
|
|
24
|
-
"event": "PreToolUse",
|
|
25
|
-
"matcher": "Bash|toolfunnel_run_tool",
|
|
26
|
-
"type": "command",
|
|
27
|
-
"command": "bash \"${HOOKS_DIR}/scripts/deny-dangerous.sh\"",
|
|
28
|
-
"script": "scripts/deny-dangerous.sh",
|
|
29
|
-
"timeout": 5,
|
|
30
|
-
"enabled": true,
|
|
31
|
-
"description": "Denies any tool call whose args contain a destructive pattern (rm -rf, etc)."
|
|
32
|
-
}
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
| Field | Type | Meaning |
|
|
36
|
-
|---|---|---|
|
|
37
|
-
| `id` | string | Unique key, format `"<kebab-event>/<script-basename-without-ext>"` (e.g. `pre-tool-use/deny-dangerous`). The loader and `hooks.state.json` key on this. |
|
|
38
|
-
| `event` | string | One of the six lifecycle events: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `Stop`, `PreCompact`. |
|
|
39
|
-
| `matcher` | string | Regex anchored full-match against `tool_name`. `""`, `undefined`, or `"*"` = always fire. Tool-less events (SessionStart, UserPromptSubmit, Stop, PreCompact) ignore it. Example: `"Bash|Write|Edit"`. |
|
|
40
|
-
| `type` | string | `"command"` (the only type today). |
|
|
41
|
-
| `command` | string | The command line the runner spawns. `${HOOKS_DIR}` expands to the absolute `hooks` path at load time. |
|
|
42
|
-
| `script` | string | The script path relative to `hooksDir`, e.g. `scripts/deny-dangerous.sh`. Used by the manager's open/edit (`readScript`/`writeScript`). |
|
|
43
|
-
| `timeout` | number | **Seconds** (the runner multiplies by 1000). The runner kills the child past this. |
|
|
44
|
-
| `enabled` | boolean | The manifest's default/seed state. The live value is the overlay in `hooks.state.json`
|
|
45
|
-
| `description` | string | What the hook does, for the manager UI. |
|
|
46
|
-
|
|
47
|
-
One entry **per hook command**
|
|
48
|
-
|
|
49
|
-
---
|
|
50
|
-
|
|
51
|
-
## 2. Where the script goes
|
|
52
|
-
|
|
53
|
-
```
|
|
54
|
-
hooks/
|
|
55
|
-
hooks.manifest.json # config entries (above)
|
|
56
|
-
hooks.state.json # enabled/disabled overlay, keyed by hook id
|
|
57
|
-
scripts/ # your hook scripts live HERE
|
|
58
|
-
deny-dangerous.sh
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
(The loader itself
|
|
62
|
-
`src/core/`, not in `hooks/`.)
|
|
63
|
-
|
|
64
|
-
`writeScript` (the manager's save path) refuses any path that resolves outside
|
|
65
|
-
`hooks/scripts/`
|
|
66
|
-
never point a hook at a script in a live host hooks directory. Clone the behaviour in if you need it.
|
|
67
|
-
|
|
68
|
-
---
|
|
69
|
-
|
|
70
|
-
## 3. What the script returns (the two protocols)
|
|
71
|
-
|
|
72
|
-
The runner spawns the command, writes the event payload as JSON on stdin, then reads the result.
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
1
|
+
# How to add a hook (self-extension)
|
|
2
|
+
|
|
3
|
+
This is the instruction served by `toolfunnel_howto({ topic: "add-hook" })`. It explains how to add a hook
|
|
4
|
+
to the Hook Engine: the manifest entry shape, where the script goes, and how the enabled/disabled
|
|
5
|
+
state is persisted separately in `hooks.state.json`. The full contract is HOOK_ENGINE.md - this is
|
|
6
|
+
the authoring view.
|
|
7
|
+
|
|
8
|
+
A hook is exactly two things:
|
|
9
|
+
1. **A manifest entry** in `hooks/hooks.manifest.json` (its config: event, matcher, command, ...).
|
|
10
|
+
2. **A script** under `hooks/scripts/` that the entry's `command` runs.
|
|
11
|
+
|
|
12
|
+
Activation (on/off) is NOT in the manifest's source of truth at runtime - it lives in an overlay,
|
|
13
|
+
`hooks/hooks.state.json` (see §4).
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. The manifest entry shape
|
|
18
|
+
|
|
19
|
+
`hooks.manifest.json` has `version`, `hooksDir`, and a `hooks` array. Each entry:
|
|
20
|
+
|
|
21
|
+
```json
|
|
22
|
+
{
|
|
23
|
+
"id": "pre-tool-use/deny-dangerous",
|
|
24
|
+
"event": "PreToolUse",
|
|
25
|
+
"matcher": "Bash|toolfunnel_run_tool",
|
|
26
|
+
"type": "command",
|
|
27
|
+
"command": "bash \"${HOOKS_DIR}/scripts/deny-dangerous.sh\"",
|
|
28
|
+
"script": "scripts/deny-dangerous.sh",
|
|
29
|
+
"timeout": 5,
|
|
30
|
+
"enabled": true,
|
|
31
|
+
"description": "Denies any tool call whose args contain a destructive pattern (rm -rf, etc)."
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
| Field | Type | Meaning |
|
|
36
|
+
|---|---|---|
|
|
37
|
+
| `id` | string | Unique key, format `"<kebab-event>/<script-basename-without-ext>"` (e.g. `pre-tool-use/deny-dangerous`). The loader and `hooks.state.json` key on this. |
|
|
38
|
+
| `event` | string | One of the six lifecycle events: `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PostToolUse`, `Stop`, `PreCompact`. |
|
|
39
|
+
| `matcher` | string | Regex anchored full-match against `tool_name`. `""`, `undefined`, or `"*"` = always fire. Tool-less events (SessionStart, UserPromptSubmit, Stop, PreCompact) ignore it. Example: `"Bash|Write|Edit"`. |
|
|
40
|
+
| `type` | string | `"command"` (the only type today). |
|
|
41
|
+
| `command` | string | The command line the runner spawns. `${HOOKS_DIR}` expands to the absolute `hooks` path at load time. |
|
|
42
|
+
| `script` | string | The script path relative to `hooksDir`, e.g. `scripts/deny-dangerous.sh`. Used by the manager's open/edit (`readScript`/`writeScript`). |
|
|
43
|
+
| `timeout` | number | **Seconds** (the runner multiplies by 1000). The runner kills the child past this. |
|
|
44
|
+
| `enabled` | boolean | The manifest's default/seed state. The live value is the overlay in `hooks.state.json` - see §4. |
|
|
45
|
+
| `description` | string | What the hook does, for the manager UI. |
|
|
46
|
+
|
|
47
|
+
One entry **per hook command** - a settings.json event with N hooks becomes N entries.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## 2. Where the script goes
|
|
52
|
+
|
|
53
|
+
```
|
|
54
|
+
hooks/
|
|
55
|
+
hooks.manifest.json # config entries (above)
|
|
56
|
+
hooks.state.json # enabled/disabled overlay, keyed by hook id
|
|
57
|
+
scripts/ # your hook scripts live HERE
|
|
58
|
+
deny-dangerous.sh
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
(The loader itself - `hook-loader.js` - is engine code and is NOT user content; it lives in
|
|
62
|
+
`src/core/`, not in `hooks/`.)
|
|
63
|
+
|
|
64
|
+
`writeScript` (the manager's save path) refuses any path that resolves outside
|
|
65
|
+
`hooks/scripts/` - defense-in-depth for the isolation rule. Author your script there only;
|
|
66
|
+
never point a hook at a script in a live host hooks directory. Clone the behaviour in if you need it.
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## 3. What the script returns (the two protocols)
|
|
71
|
+
|
|
72
|
+
The runner spawns the command, writes the event payload as JSON on stdin, then reads the result.
|
|
73
|
+
The payload a tool event receives on stdin:
|
|
74
|
+
|
|
75
|
+
```json
|
|
76
|
+
{ "tool_name": "note_slow", "tool_input": { "arg": "value" } }
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
(`PreToolUse` → `tool_name` + `tool_input`; `PostToolUse` adds `tool_response`. Tool-less
|
|
80
|
+
events carry their own fields - `prompt`, `source` - instead.)
|
|
81
|
+
|
|
82
|
+
Two protocols (HOOK_ENGINE.md §3):
|
|
83
|
+
|
|
84
|
+
**A) Exit-code (simple):**
|
|
85
|
+
- `exit 0` → success. On `SessionStart`/`UserPromptSubmit`, stdout becomes **injected context**;
|
|
86
|
+
on other events stdout is advisory.
|
|
87
|
+
- `exit 2` → **blocking**. stderr is the reason - on a blocked tool call the client receives it
|
|
88
|
+
as the result text with `isError: true`.
|
|
89
|
+
- any other code → non-blocking error (stderr captured, loop continues; never throws).
|
|
90
|
+
|
|
91
|
+
**B) JSON (advanced)** - honored **only on exit 0**, when stdout parses as a JSON object:
|
|
92
|
+
```json
|
|
93
|
+
{
|
|
94
|
+
"continue": true,
|
|
95
|
+
"decision": "block",
|
|
96
|
+
"reason": "...",
|
|
97
|
+
"hookSpecificOutput": {
|
|
98
|
+
"hookEventName": "PreToolUse",
|
|
99
|
+
"permissionDecision": "allow" | "deny" | "ask",
|
|
100
|
+
"permissionDecisionReason": "...",
|
|
101
|
+
"additionalContext": "..."
|
|
102
|
+
}
|
|
103
|
+
}
|
|
104
|
+
```
|
|
105
|
+
- **PreToolUse blocks via `hookSpecificOutput.permissionDecision: "deny"`** - NOT the top-level
|
|
106
|
+
`decision` field. `"allow"` passes; `"ask"` is non-blocking in the autonomous host.
|
|
107
|
+
- **Top-level `decision: "block"`** is for the other blockable events (UserPromptSubmit, PostToolUse,
|
|
108
|
+
Stop, PreCompact). There is no `"approve"`.
|
|
109
|
+
- `continue: false` stops the whole loop. `hookSpecificOutput.additionalContext` → injected text.
|
|
110
|
+
|
|
111
|
+
This is the mechanism that makes a PreToolUse hook gate `toolfunnel_run_tool`: match its `tool_name`, and
|
|
112
|
+
return `permissionDecision: "deny"` to block the call before the tool runs.
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## 4. Activation state lives in `hooks.state.json`
|
|
117
|
+
|
|
118
|
+
The Hook Manager design splits **what exists** from **what's on**:
|
|
119
|
+
|
|
120
|
+
- **The `scripts/` folder is the source of truth for what exists.** On startup the loader scans it
|
|
121
|
+
and reconciles with the manifest: newly-found scripts are added as entries (event/matcher inferred
|
|
122
|
+
or flagged "unconfigured"); manifest entries whose script is missing are flagged.
|
|
123
|
+
- **The manifest carries config** (event, matcher, command, timeout, description, seed `enabled`).
|
|
124
|
+
- **`hooks.state.json` is the activation overlay** - an object keyed by hook `id` recording the live
|
|
125
|
+
enabled/disabled value:
|
|
126
|
+
|
|
127
|
+
```json
|
|
128
|
+
{
|
|
129
|
+
"version": 1,
|
|
130
|
+
"enabled": {
|
|
131
|
+
"pre-tool-use/deny-dangerous": true,
|
|
132
|
+
"user-prompt-submit/inject-context": false
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Toggling a hook writes `hooks.state.json` **atomically** (temp + rename). It survives restarts and is
|
|
138
|
+
kept separate from the auto-detected inventory, so re-scanning the folder never clobbers your toggles.
|
|
139
|
+
At load, the overlay wins over the manifest's seed `enabled` for any id it lists.
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
143
|
+
## 5. Steps to add a hook
|
|
144
|
+
|
|
145
|
+
1. Write the script into `hooks/scripts/<name>.sh` (or a node script). Read the stdin JSON
|
|
146
|
+
payload; emit one of the two protocols above.
|
|
147
|
+
2. Add the manifest entry: `id` = `"<kebab-event>/<name>"`, set `event`, `matcher`, `command`
|
|
148
|
+
(`bash "${HOOKS_DIR}/scripts/<name>.sh"`), `script`, `timeout` (seconds), `description`, and a
|
|
149
|
+
seed `enabled`.
|
|
150
|
+
3. Set its live state in `hooks.state.json` via the loader's `setEnabled(id, bool)` (atomic write).
|
|
151
|
+
4. Verify: the manager lists it under its event; a run-once test fires it with a sample payload and
|
|
152
|
+
shows the result. For a gate, confirm a matching `toolfunnel_run_tool` call is denied with your reason.
|
package/src/extend/add-mcp.md
CHANGED
|
@@ -1,121 +1,122 @@
|
|
|
1
|
-
# How to register an upstream MCP (self-extension)
|
|
2
|
-
|
|
3
|
-
This is the instruction served by `toolfunnel_howto({ topic: "add-mcp" })`. It explains how to connect an
|
|
4
|
-
**upstream MCP server** (a browser-automation MCP, a chat bridge,
|
|
5
|
-
tools to expose downstream** to the CLI. All of this is configured in one persisted file:
|
|
6
|
-
`mcp/expose.json`.
|
|
7
|
-
|
|
8
|
-
The architecture (see the architecture overview, §1): many upstream MCPs each expose many tools; the
|
|
9
|
-
aggregator connects to them and the **MCP Manager picks which tools to surface** to the CLI as
|
|
10
|
-
*curated-direct* tools
|
|
11
|
-
`toolfunnel_tool_instructions`, `toolfunnel_run_tool`, `toolfunnel_howto`). The CLI sees a small surface; everything
|
|
12
|
-
else stays reachable through the register.
|
|
13
|
-
|
|
14
|
-
> Phase note: the live aggregator (`src/mcp/aggregator.js`) is Phase 2. This doc defines the
|
|
15
|
-
> `expose.json` shape so the config is authorable now and the aggregator reads exactly these fields
|
|
16
|
-
> when it lands.
|
|
17
|
-
|
|
18
|
-
---
|
|
19
|
-
|
|
20
|
-
## 1. `expose.json`
|
|
21
|
-
|
|
22
|
-
`expose.json` persists (a) the upstream MCP connections and (b) the curated-direct selection drawn
|
|
23
|
-
from them:
|
|
24
|
-
|
|
25
|
-
```json
|
|
26
|
-
{
|
|
27
|
-
"version": 1,
|
|
28
|
-
"upstreams": [
|
|
29
|
-
{
|
|
30
|
-
"id": "chat",
|
|
31
|
-
"transport": "stdio",
|
|
32
|
-
"command": "node",
|
|
33
|
-
"args": ["vendor/chat-mcp/server.js"],
|
|
34
|
-
"env": { "CHAT_DB": "..." },
|
|
35
|
-
"enabled": true,
|
|
36
|
-
"description": "Chat bridge MCP
|
|
37
|
-
},
|
|
38
|
-
{
|
|
39
|
-
"id": "browser",
|
|
40
|
-
"transport": "stdio",
|
|
41
|
-
"command": "npx",
|
|
42
|
-
"args": ["@example/browser-mcp"],
|
|
43
|
-
"enabled": false,
|
|
44
|
-
"description": "Browser automation MCP."
|
|
45
|
-
}
|
|
46
|
-
],
|
|
47
|
-
"expose": [
|
|
48
|
-
{
|
|
49
|
-
"upstream": "chat",
|
|
50
|
-
"tool": "list_chats",
|
|
51
|
-
"as": "chat_list_chats",
|
|
52
|
-
"category": "chat",
|
|
53
|
-
"enabled": true
|
|
54
|
-
},
|
|
55
|
-
{
|
|
56
|
-
"upstream": "chat",
|
|
57
|
-
"tool": "send_message",
|
|
58
|
-
"as": "chat_send_message",
|
|
59
|
-
"category": "chat",
|
|
60
|
-
"enabled": true
|
|
61
|
-
}
|
|
62
|
-
]
|
|
63
|
-
}
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
### `upstreams[]`
|
|
67
|
-
|
|
68
|
-
| Field | Type | Meaning |
|
|
69
|
-
|---|---|---|
|
|
70
|
-
| `id` | string | Stable key for this upstream. Referenced by `expose[].upstream`. Unique. |
|
|
71
|
-
| `transport` | string | How the aggregator connects. `"stdio"` (spawn `command`+`args`) is the baseline. |
|
|
72
|
-
| `command` / `args` / `env` | string / string[] / object | The process the aggregator spawns for a `stdio` upstream.
|
|
73
|
-
| `enabled` | boolean | Whether the aggregator connects to this upstream at startup. |
|
|
74
|
-
| `description` | string | Human-facing note for the MCP Manager UI. |
|
|
75
|
-
|
|
76
|
-
### `expose[]`
|
|
77
|
-
|
|
78
|
-
| Field | Type | Meaning |
|
|
79
|
-
|---|---|---|
|
|
80
|
-
| `upstream` | string | Must match an `upstreams[].id`. |
|
|
81
|
-
| `tool` | string | The upstream tool's own name (as the upstream's `tools/list` reports it). |
|
|
82
|
-
| `as` | string | The downstream name the CLI sees (namespaced so two upstreams can't collide, e.g. `chat_send_message`). Optional
|
|
83
|
-
| `category` | string | Grouping label, same convention as register entries. |
|
|
84
|
-
| `enabled` | boolean | Whether this tool is actually exposed downstream right now. |
|
|
85
|
-
|
|
86
|
-
Only the tools listed in `expose[]` with `enabled: true` are surfaced to the CLI as curated-direct
|
|
87
|
-
tools. Everything else on the upstream stays connected but hidden
|
|
88
|
-
an `expose` entry (no upstream reconnect required to *list*, though see §3 on the CLI surface).
|
|
89
|
-
|
|
90
|
-
---
|
|
91
|
-
|
|
92
|
-
## 2. Steps to register an upstream and expose a tool
|
|
93
|
-
|
|
94
|
-
1. Add an entry to `upstreams[]` with a unique `id`, the `stdio` `command`/`args`, and
|
|
95
|
-
`enabled: true`.
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
>
|
|
120
|
-
>
|
|
121
|
-
>
|
|
1
|
+
# How to register an upstream MCP (self-extension)
|
|
2
|
+
|
|
3
|
+
This is the instruction served by `toolfunnel_howto({ topic: "add-mcp" })`. It explains how to connect an
|
|
4
|
+
**upstream MCP server** (a browser-automation MCP, a chat bridge, ...) to the aggregator and choose **which of its
|
|
5
|
+
tools to expose downstream** to the CLI. All of this is configured in one persisted file:
|
|
6
|
+
`mcp/expose.json`.
|
|
7
|
+
|
|
8
|
+
The architecture (see the architecture overview, §1): many upstream MCPs each expose many tools; the
|
|
9
|
+
aggregator connects to them and the **MCP Manager picks which tools to surface** to the CLI as
|
|
10
|
+
*curated-direct* tools - alongside the four lean meta-tools (`toolfunnel_list_tools`,
|
|
11
|
+
`toolfunnel_tool_instructions`, `toolfunnel_run_tool`, `toolfunnel_howto`). The CLI sees a small surface; everything
|
|
12
|
+
else stays reachable through the register.
|
|
13
|
+
|
|
14
|
+
> Phase note: the live aggregator (`src/mcp/aggregator.js`) is Phase 2. This doc defines the
|
|
15
|
+
> `expose.json` shape so the config is authorable now and the aggregator reads exactly these fields
|
|
16
|
+
> when it lands.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## 1. `expose.json` - the two blocks
|
|
21
|
+
|
|
22
|
+
`expose.json` persists (a) the upstream MCP connections and (b) the curated-direct selection drawn
|
|
23
|
+
from them:
|
|
24
|
+
|
|
25
|
+
```json
|
|
26
|
+
{
|
|
27
|
+
"version": 1,
|
|
28
|
+
"upstreams": [
|
|
29
|
+
{
|
|
30
|
+
"id": "chat",
|
|
31
|
+
"transport": "stdio",
|
|
32
|
+
"command": "node",
|
|
33
|
+
"args": ["vendor/chat-mcp/server.js"],
|
|
34
|
+
"env": { "CHAT_DB": "..." },
|
|
35
|
+
"enabled": true,
|
|
36
|
+
"description": "Chat bridge MCP - chats, messages, send."
|
|
37
|
+
},
|
|
38
|
+
{
|
|
39
|
+
"id": "browser",
|
|
40
|
+
"transport": "stdio",
|
|
41
|
+
"command": "npx",
|
|
42
|
+
"args": ["@example/browser-mcp"],
|
|
43
|
+
"enabled": false,
|
|
44
|
+
"description": "Browser automation MCP."
|
|
45
|
+
}
|
|
46
|
+
],
|
|
47
|
+
"expose": [
|
|
48
|
+
{
|
|
49
|
+
"upstream": "chat",
|
|
50
|
+
"tool": "list_chats",
|
|
51
|
+
"as": "chat_list_chats",
|
|
52
|
+
"category": "chat",
|
|
53
|
+
"enabled": true
|
|
54
|
+
},
|
|
55
|
+
{
|
|
56
|
+
"upstream": "chat",
|
|
57
|
+
"tool": "send_message",
|
|
58
|
+
"as": "chat_send_message",
|
|
59
|
+
"category": "chat",
|
|
60
|
+
"enabled": true
|
|
61
|
+
}
|
|
62
|
+
]
|
|
63
|
+
}
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### `upstreams[]` - one entry per upstream MCP
|
|
67
|
+
|
|
68
|
+
| Field | Type | Meaning |
|
|
69
|
+
|---|---|---|
|
|
70
|
+
| `id` | string | Stable key for this upstream. Referenced by `expose[].upstream`. Unique. |
|
|
71
|
+
| `transport` | string | How the aggregator connects. `"stdio"` (spawn `command`+`args`) is the baseline. |
|
|
72
|
+
| `command` / `args` / `env` | string / string[] / object | The process the aggregator spawns for a `stdio` upstream. Path-shaped **args** must stay **inside the config home** - the isolation guard refuses to connect an upstream whose args escape it. (The `command` itself, an executable, is not guarded; the currently-WRAPPED upstream is exempt in transparent-wrapper mode, with a warning.) |
|
|
73
|
+
| `enabled` | boolean | Whether the aggregator connects to this upstream at startup. |
|
|
74
|
+
| `description` | string | Human-facing note for the MCP Manager UI. |
|
|
75
|
+
|
|
76
|
+
### `expose[]` - which upstream tools become curated-direct
|
|
77
|
+
|
|
78
|
+
| Field | Type | Meaning |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| `upstream` | string | Must match an `upstreams[].id`. |
|
|
81
|
+
| `tool` | string | The upstream tool's own name (as the upstream's `tools/list` reports it). |
|
|
82
|
+
| `as` | string | The downstream name the CLI sees (namespaced so two upstreams can't collide, e.g. `chat_send_message`). Optional - defaults to `<upstream>_<tool>`. |
|
|
83
|
+
| `category` | string | Grouping label, same convention as register entries. |
|
|
84
|
+
| `enabled` | boolean | Whether this tool is actually exposed downstream right now. |
|
|
85
|
+
|
|
86
|
+
Only the tools listed in `expose[]` with `enabled: true` are surfaced to the CLI as curated-direct
|
|
87
|
+
tools. Everything else on the upstream stays connected but hidden - reachable, if needed, by adding
|
|
88
|
+
an `expose` entry (no upstream reconnect required to *list*, though see §3 on the CLI surface).
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## 2. Steps to register an upstream and expose a tool
|
|
93
|
+
|
|
94
|
+
1. Add an entry to `upstreams[]` with a unique `id`, the `stdio` `command`/`args`, and
|
|
95
|
+
`enabled: true`. Keep the server's path-shaped args **inside the config home** (vendor the
|
|
96
|
+
server in if it isn't already) - the isolation guard refuses args that escape it.
|
|
97
|
+
2. Let the aggregator connect and run the upstream's `tools/list` (the MCP Manager UI surfaces the
|
|
98
|
+
discovered tools).
|
|
99
|
+
3. For each tool you want hot, add an `expose[]` entry: `{ upstream, tool, as, category, enabled:true }`.
|
|
100
|
+
4. Leave the long tail unexposed - the model can still reach those via the register/meta-tools if you
|
|
101
|
+
later mirror them, but the *curated-direct* surface stays lean by design (the dial in §1 of the
|
|
102
|
+
architecture: directly-exposed = the HOT tools; the register meta-tools = the long tail).
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 3. Curated-direct vs. the register (update cost)
|
|
107
|
+
|
|
108
|
+
This matters because the two layers refresh differently (see the architecture overview, §1):
|
|
109
|
+
|
|
110
|
+
- **Curated-direct tools** (what `expose[]` produces) are negotiated with the CLI **at connect time**.
|
|
111
|
+
Changing the exposed set therefore needs a **reconnect**, or a `notifications/tools/list_changed`
|
|
112
|
+
message *if the CLI honours it* (VERIFY per host; restart is the safe baseline).
|
|
113
|
+
- **Register tools** (the `toolfunnel_list_tools`/`run` path) are **always live** - the model re-reads the
|
|
114
|
+
register on every `toolfunnel_list_tools` call, so no reconnect is needed.
|
|
115
|
+
|
|
116
|
+
Rule of thumb: expose a handful of HOT, high-reliability tools curated-direct; everything else lives
|
|
117
|
+
in the register and is reached through the meta-tools.
|
|
118
|
+
|
|
119
|
+
> Safety: a curated-direct upstream tool is still ultimately invoked by the model. When you wire
|
|
120
|
+
> upstream execution through the host, route it through the same PreToolUse gate the register's `run`
|
|
121
|
+
> uses (see `toolfunnel_howto({ topic: "add-hook" })`) so the safety invariant holds uniformly. The gate
|
|
122
|
+
> firing on every execution path is a tested invariant, not an assumption.
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# How to configure ToolFunnel - config files only, no code
|
|
2
|
+
|
|
3
|
+
This is the instruction served by `toolfunnel_howto({ topic: "configure" })`. It is the complete
|
|
4
|
+
no-code setup map: every behaviour of the gateway is driven by a small set of JSON files at the
|
|
5
|
+
**config home**. Editing these files (or driving the UI / the `tf_*` management tools, which write
|
|
6
|
+
the same files) is the entire configuration story - there is nothing to program.
|
|
7
|
+
|
|
8
|
+
## 1. The config home
|
|
9
|
+
|
|
10
|
+
All files live under one directory, resolved in this order:
|
|
11
|
+
|
|
12
|
+
1. `--config-dir <dir>` on the command line
|
|
13
|
+
2. the `TOOLFUNNEL_HOME` environment variable
|
|
14
|
+
3. the package/repo root (the default for a git clone)
|
|
15
|
+
|
|
16
|
+
On first start with a fresh `--config-dir`, the home is seeded with the standard layout. Every
|
|
17
|
+
file below is read FRESH per request - edits are live without a restart, except `toolfunnel.json`
|
|
18
|
+
(read at boot; restart to apply).
|
|
19
|
+
|
|
20
|
+
## 2. The files
|
|
21
|
+
|
|
22
|
+
| File | What it controls | Live? |
|
|
23
|
+
|---|---|---|
|
|
24
|
+
| `toolfunnel.json` | Identity + default ports (see §3) | restart |
|
|
25
|
+
| `mcp/expose.json` | Attached upstream MCPs + which of their tools are curated-direct | live |
|
|
26
|
+
| `tools/tools.register.json` | The local tool register (the gateway's own tools) | live |
|
|
27
|
+
| `tools/tools.state.json` | Per-tool overlays: enabled / hot / hidden, and the active wrap (`"passthrough"`). Wrapped example: `{ "passthrough": "acme-notes" }`; cleared: `{}` (the file-edit equivalent of `wrap --off`) | live |
|
|
28
|
+
| `hooks/hooks.manifest.json` | PreToolUse / PostToolUse policy hooks (the gate) | live |
|
|
29
|
+
| `hooks/hooks.state.json` | Per-hook enabled overlay | live |
|
|
30
|
+
| `logs/log.config.json` | Activity log on/off + path | live |
|
|
31
|
+
| `auth/auth.config.json` | OAuth 2.1 resource-server validation (opt-in; needs `toolfunnel install-oauth` once) | live |
|
|
32
|
+
|
|
33
|
+
Shapes: `expose.json` → `toolfunnel_howto({ topic: "add-mcp" })`; register entries →
|
|
34
|
+
`toolfunnel_howto({ topic: "create-tool" })`; hook entries → `toolfunnel_howto({ topic: "add-hook" })`;
|
|
35
|
+
the wrap field → `toolfunnel_howto({ topic: "wrap" })`.
|
|
36
|
+
|
|
37
|
+
## 3. `toolfunnel.json` - identity & ports
|
|
38
|
+
|
|
39
|
+
Every field optional; an absent or empty file IS the default identity. Blank/omitted fields fall
|
|
40
|
+
back per-field (a broken file never stops the gateway starting).
|
|
41
|
+
|
|
42
|
+
```json
|
|
43
|
+
{
|
|
44
|
+
"serverName": "my-mcp", // what CLIENTS see in the initialize handshake (default "toolfunnel")
|
|
45
|
+
"serverVersion": "1.0.0", // ...its version (default: package version)
|
|
46
|
+
"clientName": "my-client", // what UPSTREAM servers see from the gateway (default "toolfunnel")
|
|
47
|
+
"clientVersion": "1.0.0", // ...its version
|
|
48
|
+
"httpPort": 9998, // --http default port (default 9998)
|
|
49
|
+
"uiPort": 9777 // --ui default port (default 9777)
|
|
50
|
+
}
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
Precedence: CLI flag > this file > built-in default. Under a WRAP on stdio, the wrapped upstream
|
|
54
|
+
is shown the real downstream client's identity automatically - `clientName`/`clientVersion` cover
|
|
55
|
+
funnel mode, HTTP, and the boot connection. The UI's **Settings** tab edits this file.
|
|
56
|
+
|
|
57
|
+
## 4. Worked example - a complete gateway from five files
|
|
58
|
+
|
|
59
|
+
A gateway that attaches one upstream, exposes two of its tools, gates everything, logs activity,
|
|
60
|
+
and introduces itself as `acme-tools`:
|
|
61
|
+
|
|
62
|
+
1. `toolfunnel.json` → `{ "serverName": "acme-tools", "clientName": "acme-tools" }`
|
|
63
|
+
2. `mcp/expose.json` → one `upstreams[]` entry (id, `"transport": "stdio"`, command, args,
|
|
64
|
+
`"enabled": true`) + two `expose[]` entries (`{ "upstream", "tool", "enabled": true }`)
|
|
65
|
+
3. `hooks/hooks.manifest.json` → one PreToolUse entry matching the tools you want policy on
|
|
66
|
+
4. `logs/log.config.json` → `{ "enabled": true }`
|
|
67
|
+
5. Start it: `node bin/toolfunnel.js` (stdio) or `--http` / `--ui` as needed.
|
|
68
|
+
|
|
69
|
+
Optional per-upstream flags in `expose.json`: `"legacyPin": true` pins that upstream to the
|
|
70
|
+
legacy MCP protocol permanently (opt-in, warns loudly); `"env"` passes environment variables to
|
|
71
|
+
the spawned server; `"cwd"` sets its working directory; `"timeoutMs"` raises the payload
|
|
72
|
+
timeout for that upstream (default 120000 ms - applies to `tools/call`, `prompts/get` and
|
|
73
|
+
`resources/read`; under a wrap, a tool that reports progress keeps its call alive regardless.
|
|
74
|
+
The 10 s handshake/list window is fixed: it detects dead servers); `"modernOnly": true`
|
|
75
|
+
requires that upstream to speak the modern (2026-07-28) protocol - the connect fails with a
|
|
76
|
+
clear error instead of falling back to legacy (the mirror of `legacyPin`; setting both on one
|
|
77
|
+
upstream is refused). Server side, `"serveLegacy": false` in `toolfunnel.json` makes the
|
|
78
|
+
gateway itself modern-only: legacy clients are refused with a clear error (default: serve
|
|
79
|
+
both eras - only an explicit `false` flips it).
|
|
80
|
+
|
|
81
|
+
## 5. Where the AI-driven and human paths meet
|
|
82
|
+
|
|
83
|
+
The UI (all five tabs + Settings), the `tf_*` management tools, and hand-editing these files are
|
|
84
|
+
three views of the SAME state - every write goes through the same validated, atomic store code.
|
|
85
|
+
Pick whichever fits: config files for repeatable/scripted setups, the UI for humans, the
|
|
86
|
+
management tools for in-band agent-driven changes (each documented via
|
|
87
|
+
`toolfunnel_tool_instructions`).
|