toolfunnel 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/README.md +238 -235
  2. package/SECURITY.md +84 -84
  3. package/bin/toolfunnel.js +530 -451
  4. package/docs/KNOWN_BUGS.md +100 -80
  5. package/docs/MANUAL.pdf +0 -0
  6. package/docs/design.md +84 -84
  7. package/docs/packaging.md +116 -115
  8. package/hooks/hooks.manifest.json +27 -27
  9. package/hooks/scripts/example-audit.js +21 -21
  10. package/hooks/scripts/example-deny-dangerous.js +39 -39
  11. package/mcp/expose.example.json +30 -30
  12. package/mcp/servers/mock-modern/server.js +147 -147
  13. package/mcp/servers/mock-upstream/server.js +379 -380
  14. package/package.json +67 -67
  15. package/src/auth/config.js +172 -174
  16. package/src/auth/install.js +181 -181
  17. package/src/auth/resource-server.js +358 -353
  18. package/src/core/atomic-write.js +97 -0
  19. package/src/core/child-reaper.js +61 -61
  20. package/src/core/config-home.js +226 -134
  21. package/src/core/events.js +105 -105
  22. package/src/core/hook-engine.js +304 -329
  23. package/src/core/hook-loader.js +766 -824
  24. package/src/core/hook-runner.js +615 -566
  25. package/src/core/logger.js +295 -297
  26. package/src/core/matcher.js +100 -100
  27. package/src/core/metrics.js +90 -75
  28. package/src/core/output-cap.js +31 -0
  29. package/src/core/rename-migration.js +178 -163
  30. package/src/core/requires.js +138 -138
  31. package/src/core/server-config.js +98 -98
  32. package/src/extend/add-hook.md +155 -152
  33. package/src/extend/add-mcp.md +119 -119
  34. package/src/extend/configure.md +88 -88
  35. package/src/extend/create-tool.md +168 -168
  36. package/src/extend/howto.js +102 -102
  37. package/src/extend/package.md +190 -190
  38. package/src/extend/wrap.md +99 -99
  39. package/src/mcp/aggregator.js +2248 -1674
  40. package/src/mcp/expose-store.js +755 -577
  41. package/src/mcp/gated-run.js +210 -210
  42. package/src/mcp/http-transport.js +1657 -1563
  43. package/src/mcp/mcp-client.js +1174 -1070
  44. package/src/mcp/modern.js +409 -406
  45. package/src/mcp/protocol.js +473 -468
  46. package/src/mcp/server.js +3317 -3112
  47. package/src/mcp/tool-approvals.js +396 -0
  48. package/src/tools/registry.js +607 -661
  49. package/src/tools/tool-state.js +170 -170
  50. package/src/ui/public/app.js +1246 -1187
  51. package/src/ui/public/index.html +286 -286
  52. package/src/ui/public/styles.css +535 -535
  53. package/src/ui/server.js +1742 -1616
  54. package/tools/scripts/base64.js +139 -139
  55. package/tools/scripts/danger.js +61 -61
  56. package/tools/scripts/echo.js +39 -39
  57. package/tools/scripts/hash.js +101 -101
  58. package/tools/scripts/json.js +109 -109
  59. package/tools/scripts/text-stats.js +131 -131
  60. package/tools/scripts/tf-accept-change.js +67 -0
  61. package/tools/scripts/tf-env.js +43 -43
  62. package/tools/scripts/tf-hook-add.js +151 -151
  63. package/tools/scripts/tf-hook-set.js +120 -120
  64. package/tools/scripts/tf-list.js +192 -192
  65. package/tools/scripts/tf-log.js +86 -86
  66. package/tools/scripts/tf-mcp-add.js +130 -117
  67. package/tools/scripts/tf-mcp-set.js +121 -121
  68. package/tools/scripts/tf-pack.js +237 -237
  69. package/tools/scripts/tf-tool-add.js +134 -134
  70. package/tools/scripts/tf-tool-set.js +142 -142
  71. package/tools/scripts/tf-wrap.js +103 -103
  72. package/tools/scripts/uuid.js +102 -102
  73. package/tools/tools.register.json +16 -5
@@ -1,88 +1,88 @@
1
- # How to configure ToolFunnel - config files only, no code
2
-
3
- This is the instruction served by `toolfunnel_howto({ topic: "configure" })`. It is the complete
4
- no-code setup map: every behaviour of the gateway is driven by a small set of JSON files at the
5
- **config home**. Editing these files (or driving the UI / the `tf_*` management tools, which write
6
- the same files) is the entire configuration story - there is nothing to program.
7
-
8
- ## 1. The config home
9
-
10
- All files live under one directory, resolved in this order:
11
-
12
- 1. `--config-dir <dir>` on the command line
13
- 2. the `TOOLFUNNEL_HOME` environment variable
14
- 3. the package/repo root (the default for a git clone)
15
-
16
- On first start with a fresh `--config-dir`, the home is seeded with the standard layout. Every
17
- file below is read FRESH per request - edits are live without a restart, except `toolfunnel.json`
18
- (read at boot; restart to apply).
19
-
20
- ## 2. The files
21
-
22
- | File | What it controls | Live? |
23
- |---|---|---|
24
- | `toolfunnel.json` | Identity + default ports (see §3) | restart |
25
- | `mcp/expose.json` | Attached upstream MCPs + which of their tools are curated-direct | live |
26
- | `tools/tools.register.json` | The local tool register (the gateway's own tools) | live |
27
- | `tools/tools.state.json` | Per-tool overlays: enabled / hot / hidden, and the active wrap (`"passthrough"`). Wrapped example: `{ "passthrough": "acme-notes" }`; cleared: `{}` (the file-edit equivalent of `wrap --off`) | live |
28
- | `hooks/hooks.manifest.json` | PreToolUse / PostToolUse policy hooks (the gate) | live |
29
- | `hooks/hooks.state.json` | Per-hook enabled overlay | live |
30
- | `logs/log.config.json` | Activity log on/off + path | live |
31
- | `auth/auth.config.json` | OAuth 2.1 resource-server validation (opt-in; needs `toolfunnel install-oauth` once) | live |
32
-
33
- Shapes: `expose.json` → `toolfunnel_howto({ topic: "add-mcp" })`; register entries →
34
- `toolfunnel_howto({ topic: "create-tool" })`; hook entries → `toolfunnel_howto({ topic: "add-hook" })`;
35
- the wrap field → `toolfunnel_howto({ topic: "wrap" })`.
36
-
37
- ## 3. `toolfunnel.json` - identity & ports
38
-
39
- Every field optional; an absent or empty file IS the default identity. Blank/omitted fields fall
40
- back per-field (a broken file never stops the gateway starting).
41
-
42
- ```json
43
- {
44
- "serverName": "my-mcp", // what CLIENTS see in the initialize handshake (default "toolfunnel")
45
- "serverVersion": "1.0.0", // ...its version (default: package version)
46
- "clientName": "my-client", // what UPSTREAM servers see from the gateway (default "toolfunnel")
47
- "clientVersion": "1.0.0", // ...its version
48
- "httpPort": 9998, // --http default port (default 9998)
49
- "uiPort": 9777 // --ui default port (default 9777)
50
- }
51
- ```
52
-
53
- Precedence: CLI flag > this file > built-in default. Under a WRAP on stdio, the wrapped upstream
54
- is shown the real downstream client's identity automatically - `clientName`/`clientVersion` cover
55
- funnel mode, HTTP, and the boot connection. The UI's **Settings** tab edits this file.
56
-
57
- ## 4. Worked example - a complete gateway from five files
58
-
59
- A gateway that attaches one upstream, exposes two of its tools, gates everything, logs activity,
60
- and introduces itself as `acme-tools`:
61
-
62
- 1. `toolfunnel.json` → `{ "serverName": "acme-tools", "clientName": "acme-tools" }`
63
- 2. `mcp/expose.json` → one `upstreams[]` entry (id, `"transport": "stdio"`, command, args,
64
- `"enabled": true`) + two `expose[]` entries (`{ "upstream", "tool", "enabled": true }`)
65
- 3. `hooks/hooks.manifest.json` → one PreToolUse entry matching the tools you want policy on
66
- 4. `logs/log.config.json` → `{ "enabled": true }`
67
- 5. Start it: `node bin/toolfunnel.js` (stdio) or `--http` / `--ui` as needed.
68
-
69
- Optional per-upstream flags in `expose.json`: `"legacyPin": true` pins that upstream to the
70
- legacy MCP protocol permanently (opt-in, warns loudly); `"env"` passes environment variables to
71
- the spawned server; `"cwd"` sets its working directory; `"timeoutMs"` raises the payload
72
- timeout for that upstream (default 120000 ms - applies to `tools/call`, `prompts/get` and
73
- `resources/read`; under a wrap, a tool that reports progress keeps its call alive regardless.
74
- the separate 10 s handshake/list window detects dead servers, and `"requestTimeoutMs"` raises
75
- it for a server that needs longer to boot); `"modernOnly": true`
76
- requires that upstream to speak the modern (2026-07-28) protocol - the connect fails with a
77
- clear error instead of falling back to legacy (the mirror of `legacyPin`; setting both on one
78
- upstream is refused). Server side, `"serveLegacy": false` in `toolfunnel.json` makes the
79
- gateway itself modern-only: legacy clients are refused with a clear error (default: serve
80
- both eras - only an explicit `false` flips it).
81
-
82
- ## 5. Where the AI-driven and human paths meet
83
-
84
- The UI (all five tabs + Settings), the `tf_*` management tools, and hand-editing these files are
85
- three views of the SAME state - every write goes through the same validated, atomic store code.
86
- Pick whichever fits: config files for repeatable/scripted setups, the UI for humans, the
87
- management tools for in-band agent-driven changes (each documented via
88
- `toolfunnel_tool_instructions`).
1
+ # How to configure ToolFunnel - config files only, no code
2
+
3
+ This is the instruction served by `toolfunnel_howto({ topic: "configure" })`. It is the complete
4
+ no-code setup map: every behaviour of the gateway is driven by a small set of JSON files at the
5
+ **config home**. Editing these files (or driving the UI / the `tf_*` management tools, which write
6
+ the same files) is the entire configuration story - there is nothing to program.
7
+
8
+ ## 1. The config home
9
+
10
+ All files live under one directory, resolved in this order:
11
+
12
+ 1. `--config-dir <dir>` on the command line
13
+ 2. the `TOOLFUNNEL_HOME` environment variable
14
+ 3. the package/repo root (the default for a git clone)
15
+
16
+ On first start with a fresh `--config-dir`, the home is seeded with the standard layout. Every
17
+ file below is read FRESH per request - edits are live without a restart, except `toolfunnel.json`
18
+ (read at boot; restart to apply).
19
+
20
+ ## 2. The files
21
+
22
+ | File | What it controls | Live? |
23
+ |---|---|---|
24
+ | `toolfunnel.json` | Identity + default ports (see §3) | restart |
25
+ | `mcp/expose.json` | Attached upstream MCPs + which of their tools are curated-direct | live |
26
+ | `tools/tools.register.json` | The local tool register (the gateway's own tools) | live |
27
+ | `tools/tools.state.json` | Per-tool overlays: enabled / hot / hidden, and the active wrap (`"passthrough"`). Wrapped example: `{ "passthrough": "acme-notes" }`; cleared: `{}` (the file-edit equivalent of `wrap --off`) | live |
28
+ | `hooks/hooks.manifest.json` | PreToolUse / PostToolUse policy hooks (the gate) | live |
29
+ | `hooks/hooks.state.json` | Per-hook enabled overlay | live |
30
+ | `logs/log.config.json` | Activity log on/off + path | live |
31
+ | `auth/auth.config.json` | OAuth 2.1 resource-server validation (opt-in; needs `toolfunnel install-oauth` once) | live |
32
+
33
+ Shapes: `expose.json` → `toolfunnel_howto({ topic: "add-mcp" })`; register entries →
34
+ `toolfunnel_howto({ topic: "create-tool" })`; hook entries → `toolfunnel_howto({ topic: "add-hook" })`;
35
+ the wrap field → `toolfunnel_howto({ topic: "wrap" })`.
36
+
37
+ ## 3. `toolfunnel.json` - identity & ports
38
+
39
+ Every field optional; an absent or empty file IS the default identity. Blank/omitted fields fall
40
+ back per-field (a broken file never stops the gateway starting).
41
+
42
+ ```json
43
+ {
44
+ "serverName": "my-mcp", // what CLIENTS see in the initialize handshake (default "toolfunnel")
45
+ "serverVersion": "1.0.0", // ...its version (default: package version)
46
+ "clientName": "my-client", // what UPSTREAM servers see from the gateway (default "toolfunnel")
47
+ "clientVersion": "1.0.0", // ...its version
48
+ "httpPort": 9998, // --http default port (default 9998)
49
+ "uiPort": 9777 // --ui default port (default 9777)
50
+ }
51
+ ```
52
+
53
+ Precedence: CLI flag > this file > built-in default. Under a WRAP on stdio, the wrapped upstream
54
+ is shown the real downstream client's identity automatically - `clientName`/`clientVersion` cover
55
+ funnel mode, HTTP, and the boot connection. The UI's **Settings** tab edits this file.
56
+
57
+ ## 4. Worked example - a complete gateway from five files
58
+
59
+ A gateway that attaches one upstream, exposes two of its tools, gates everything, logs activity,
60
+ and introduces itself as `acme-tools`:
61
+
62
+ 1. `toolfunnel.json` → `{ "serverName": "acme-tools", "clientName": "acme-tools" }`
63
+ 2. `mcp/expose.json` → one `upstreams[]` entry (id, `"transport": "stdio"`, command, args,
64
+ `"enabled": true`) + two `expose[]` entries (`{ "upstream", "tool", "enabled": true }`)
65
+ 3. `hooks/hooks.manifest.json` → one PreToolUse entry matching the tools you want policy on
66
+ 4. `logs/log.config.json` → `{ "enabled": true }`
67
+ 5. Start it: `node bin/toolfunnel.js` (stdio) or `--http` / `--ui` as needed.
68
+
69
+ Optional per-upstream flags in `expose.json`: `"legacyPin": true` pins that upstream to the
70
+ legacy MCP protocol permanently (opt-in, warns loudly); `"env"` passes environment variables to
71
+ the spawned server; `"cwd"` sets its working directory; `"timeoutMs"` raises the payload
72
+ timeout for that upstream (default 120000 ms - applies to `tools/call`, `prompts/get` and
73
+ `resources/read`; under a wrap, a tool that reports progress keeps its call alive regardless.
74
+ the separate 10 s handshake/list window detects dead servers, and `"requestTimeoutMs"` raises
75
+ it for a server that needs longer to boot); `"modernOnly": true`
76
+ requires that upstream to speak the modern (2026-07-28) protocol - the connect fails with a
77
+ clear error instead of falling back to legacy (the mirror of `legacyPin`; setting both on one
78
+ upstream is refused). Server side, `"serveLegacy": false` in `toolfunnel.json` makes the
79
+ gateway itself modern-only: legacy clients are refused with a clear error (default: serve
80
+ both eras - only an explicit `false` flips it).
81
+
82
+ ## 5. Where the AI-driven and human paths meet
83
+
84
+ The UI (all five tabs + Settings), the `tf_*` management tools, and hand-editing these files are
85
+ three views of the SAME state - every write goes through the same validated, atomic store code.
86
+ Pick whichever fits: config files for repeatable/scripted setups, the UI for humans, the
87
+ management tools for in-band agent-driven changes (each documented via
88
+ `toolfunnel_tool_instructions`).
@@ -1,168 +1,168 @@
1
- # How to add a tool (self-extension)
2
-
3
- This is the instruction served by `toolfunnel_howto({ topic: "create-tool" })`. It tells you - the
4
- model - how to add a new tool to the register so it becomes callable through `toolfunnel_run_tool`
5
- (and listable via `toolfunnel_list_tools` / `toolfunnel_tool_instructions`). No restart is needed: a
6
- running gateway watches `tools.register.json` and hot-reloads its register on change (debounced
7
- ~150 ms), the same way `mcp/expose.json` and the hook files reload.
8
-
9
- A tool is exactly two things:
10
- 1. **A register entry** in `tools/tools.register.json` (structured metadata + how to invoke it).
11
- 2. **A script** under `tools/scripts/` that the entry's `invoke` points at (only for
12
- `invoke.type === "script"`; a `shell` tool needs no file).
13
-
14
- ---
15
-
16
- ## 1. The register entry shape
17
-
18
- `tools.register.json` is a JSON file with one top-level array of entries. Each entry:
19
-
20
- ```json
21
- {
22
- "id": "echo-upper",
23
- "name": "Echo Upper",
24
- "summary": "Uppercases the supplied text and echoes it back.",
25
- "category": "demo",
26
- "instructions": "Pass { text: string }. Returns the text uppercased. Use this only as a sanity check of the run path.",
27
- "inputSchema": {
28
- "type": "object",
29
- "properties": { "text": { "type": "string", "description": "The text to uppercase." } },
30
- "required": ["text"],
31
- "additionalProperties": false
32
- },
33
- "invoke": { "type": "script", "path": "scripts/echo-upper.sh" }
34
- }
35
- ```
36
-
37
- A third shape exists - the **reference-mode** tool. It has `"mode": "reference"`, NO `invoke`
38
- and no script: its `instructions` ARE the tool. Running it hands the instructions back to the
39
- caller (still through the PreToolUse gate - a deny withholds them) instead of executing anything
40
- server-side. Use it for run-it-yourself recipes: a CLI invocation the agent should run in its own
41
- environment, a checklist, a procedure.
42
-
43
- ```json
44
- {
45
- "id": "deploy-runbook",
46
- "name": "Deploy Runbook",
47
- "summary": "Step-by-step deploy procedure for this project.",
48
- "mode": "reference",
49
- "instructions": "1. npm test - suite must be green. 2. npm run build. 3. ..."
50
- }
51
- ```
52
-
53
- Field-by-field - these names are fixed (registry.js reads exactly these):
54
-
55
- | Field | Type | Meaning |
56
- |---|---|---|
57
- | `id` | string | Stable unique key, kebab-case. `toolfunnel_run_tool({ name })` and `registry.run(id, args)` look up by this. Never reuse an existing `id`. |
58
- | `name` | string | Human/model-facing display name. Shown in the manager UI. |
59
- | `summary` | string | One line. This is ALL that `toolfunnel_list_tools` returns for the tool - keep it tight and discriminating so the model can pick the right tool from the brief alone. |
60
- | `category` | string | Grouping label (e.g. `demo`, `memory`, `messaging`, `media`). Used by `toolfunnel_list_tools({ category })` to filter. |
61
- | `instructions` | string | Full usage doc returned by `toolfunnel_tool_instructions({ name })`. Describe the args object, what comes back, and any safety caveats. This is the long-tail payload - be complete here, terse in `summary`. |
62
- | `inputSchema` | object (optional) | A JSON Schema for the tool's args. **This is what makes the tool a first-class MCP citizen when promoted hot:** the top-level `tools/list` advertises it VERBATIM, so a connected client knows the exact arg shape without reading `instructions`. Omit it and a hot-promoted tool falls back to a free-form `{ "type": "object" }` (callable, but the client gets no arg hints). Must be an object - the register rejects any other shape at authoring time. |
63
- | `invoke` | object | How `run` executes the tool. See below. |
64
-
65
- ### The `invoke` object - two forms
66
-
67
- ```json
68
- { "type": "script", "path": "scripts/echo-upper.sh" }
69
- ```
70
- - `type: "script"` - run a file under `tools/scripts/`. `path` is **relative to `tools/`**
71
- and MUST stay inside `tools/scripts/` (registry.js path-guards it, same defense-in-depth as the
72
- hook loader's `writeScript`). The args object is delivered to the script (e.g. as a JSON blob on
73
- stdin or argv per the registry's convention) and the script's stdout is the tool output.
74
-
75
- ```json
76
- { "type": "shell", "command": "node \"${TOOLS_DIR}/scripts/echo-upper.js\"" }
77
- ```
78
- - `type: "shell"` - run a command line. `${TOOLS_DIR}` expands to the absolute `tools` path at run
79
- time (mirrors `${HOOKS_DIR}` in the hook manifest). Prefer `script` for anything reusable; reserve
80
- `shell` for thin one-liners.
81
-
82
- > Isolation rule: an `invoke` must only reference scripts/commands that live **inside the gateway's
83
- > own tree**. Never point `invoke` at a live tool that lives outside it (e.g. in a host's external
84
- > tool directory). If you want an external tool's behaviour, clone its script into
85
- > `tools/scripts/` first, then point at the clone.
86
-
87
- ---
88
-
89
- ## 2. Where the script goes
90
-
91
- ```
92
- tools/
93
- tools.register.json # the entries (above)
94
- scripts/ # your tool scripts live HERE and nowhere else
95
- echo-upper.sh
96
- ```
97
-
98
- (`registry.js` - the engine that loads/persists the register and runs list/instructions/run - is
99
- NOT user content. It lives in `src/tools/registry.js` and is not part of this tree.)
100
-
101
- The script reads its arguments (the `args` object passed to `run`) and writes its result to stdout.
102
- Exit non-zero with a message on stderr to signal failure - the registry surfaces that as the tool's
103
- error. Keep scripts pure and side-effect-honest: anything destructive is exactly what the PreToolUse
104
- gate exists to catch (next section).
105
-
106
- ---
107
-
108
- ## 3. How `run` works - and why it is gated
109
-
110
- `toolfunnel_run_tool({ name, args })` does **not** execute the script directly. It calls
111
- `registry.run(id, args)`, which routes the call through the gateway's **hook engine** before - and after -
112
- the tool actually runs:
113
-
114
- ```
115
- toolfunnel_run_tool(name, args)
116
- │
117
- ▼
118
- registry.run(id, args)
119
- │
120
- ├─► hookEngine.fire("PreToolUse", ctx, { tool_name, tool_input: args })
121
- │ └─ if result.blocked → STOP. Return { ok:false, error: <reason> }. Script never runs.
122
- │
123
- ├─► execute the invoke (script/shell) only if NOT blocked
124
- │
125
- └─► hookEngine.fire("PostToolUse", ctx, { tool_name, tool_input: args, tool_response })
126
- └─ feedback only; the tool already ran (cannot un-run it)
127
- ```
128
-
129
- - **The PreToolUse gate is the whole safety case.** A hook denies a call by returning, on exit 0,
130
- `{ "hookSpecificOutput": { "hookEventName": "PreToolUse", "permissionDecision": "deny",
131
- "permissionDecisionReason": "..." } }` - OR by `exit 2` with the reason on stderr. (PreToolUse uses
132
- `permissionDecision`, NOT a top-level `decision` . When that fires,
133
- `run` returns an error result and the script is never spawned.
134
- - `permissionDecision: "allow"` passes; `"ask"` is non-blocking in the autonomous host (no
135
- interactive prompt) - the reason is captured but the tool proceeds.
136
- - The result shape `run` returns matches the meta-tool contract: `{ ok, output, error? }`.
137
-
138
- So adding a tool grants real power, but every invocation is mediated by the same gate that protects
139
- every other tool. If your new tool is dangerous, the correct control is a PreToolUse hook that
140
- matches its `tool_name` (see `toolfunnel_howto({ topic: "add-hook" })`), not omitting the tool.
141
-
142
- ---
143
-
144
- ## 4. Steps to add a tool
145
-
146
- 1. Pick a unique `id` (check `toolfunnel_list_tools` first so you don't collide).
147
- 2. If `invoke.type === "script"`, write the script into `tools/scripts/<id>.sh` (or `.js`).
148
- 3. Add the entry to the `tools.register.json` array (via `registry.js`'s add/edit path so the write
149
- is atomic - never hand-edit and risk a torn file).
150
- 4. Confirm with `toolfunnel_list_tools` (your `summary` appears) and `toolfunnel_tool_instructions({ name })`
151
- (your `instructions` appear).
152
- 5. Test with `toolfunnel_run_tool({ name, args })`. Verify the gate behaves: a benign call returns
153
- `{ ok:true, output }`; if a matching PreToolUse hook should deny it, confirm it does.
154
-
155
- No reconnect is needed - the register is dynamic: the running gateway hot-reloads
156
- `tools.register.json` on change and emits `notifications/tools/list_changed`, exactly like an
157
- `expose.json` or hook edit.
158
-
159
- ---
160
-
161
- ## 5. Promoting your tool to the top-level surface (the "instant MCP" recipe)
162
-
163
- By default your tool lives behind the lean meta-tools (discovered via `toolfunnel_list_tools`).
164
- Set `hot: true` for its id in `tools/tools.state.json` (or via the UI / `tf_tool_set`) and it is
165
- advertised DIRECTLY in the MCP `tools/list` - under its own `id`, with your `summary` as the
166
- description and your `inputSchema` as the advertised schema. Author a real `inputSchema` before
167
- promoting: that schema is the difference between a client calling your tool correctly first time
168
- and it guessing at a free-form object.
1
+ # How to add a tool (self-extension)
2
+
3
+ This is the instruction served by `toolfunnel_howto({ topic: "create-tool" })`. It tells you - the
4
+ model - how to add a new tool to the register so it becomes callable through `toolfunnel_run_tool`
5
+ (and listable via `toolfunnel_list_tools` / `toolfunnel_tool_instructions`). No restart is needed: a
6
+ running gateway watches `tools.register.json` and hot-reloads its register on change (debounced
7
+ ~150 ms), the same way `mcp/expose.json` and the hook files reload.
8
+
9
+ A tool is exactly two things:
10
+ 1. **A register entry** in `tools/tools.register.json` (structured metadata + how to invoke it).
11
+ 2. **A script** under `tools/scripts/` that the entry's `invoke` points at (only for
12
+ `invoke.type === "script"`; a `shell` tool needs no file).
13
+
14
+ ---
15
+
16
+ ## 1. The register entry shape
17
+
18
+ `tools.register.json` is a JSON file with one top-level array of entries. Each entry:
19
+
20
+ ```json
21
+ {
22
+ "id": "echo-upper",
23
+ "name": "Echo Upper",
24
+ "summary": "Uppercases the supplied text and echoes it back.",
25
+ "category": "demo",
26
+ "instructions": "Pass { text: string }. Returns the text uppercased. Use this only as a sanity check of the run path.",
27
+ "inputSchema": {
28
+ "type": "object",
29
+ "properties": { "text": { "type": "string", "description": "The text to uppercase." } },
30
+ "required": ["text"],
31
+ "additionalProperties": false
32
+ },
33
+ "invoke": { "type": "script", "path": "scripts/echo-upper.sh" }
34
+ }
35
+ ```
36
+
37
+ A third shape exists - the **reference-mode** tool. It has `"mode": "reference"`, NO `invoke`
38
+ and no script: its `instructions` ARE the tool. Running it hands the instructions back to the
39
+ caller (still through the PreToolUse gate - a deny withholds them) instead of executing anything
40
+ server-side. Use it for run-it-yourself recipes: a CLI invocation the agent should run in its own
41
+ environment, a checklist, a procedure.
42
+
43
+ ```json
44
+ {
45
+ "id": "deploy-runbook",
46
+ "name": "Deploy Runbook",
47
+ "summary": "Step-by-step deploy procedure for this project.",
48
+ "mode": "reference",
49
+ "instructions": "1. npm test - suite must be green. 2. npm run build. 3. ..."
50
+ }
51
+ ```
52
+
53
+ Field-by-field - these names are fixed (registry.js reads exactly these):
54
+
55
+ | Field | Type | Meaning |
56
+ |---|---|---|
57
+ | `id` | string | Stable unique key, kebab-case. `toolfunnel_run_tool({ name })` and `registry.run(id, args)` look up by this. Never reuse an existing `id`. |
58
+ | `name` | string | Human/model-facing display name. Shown in the manager UI. |
59
+ | `summary` | string | One line. This is ALL that `toolfunnel_list_tools` returns for the tool - keep it tight and discriminating so the model can pick the right tool from the brief alone. Name every mode the tool has (`encode` / `decode`, `sha256` / `md5`): the listing's filter searches the name and summary only, so a mode that lives only in `instructions` or the schema is never found. |
60
+ | `category` | string | Grouping label (e.g. `demo`, `memory`, `messaging`, `media`). Used by `toolfunnel_list_tools({ category })` to filter. |
61
+ | `instructions` | string | Full usage doc returned by `toolfunnel_tool_instructions({ name })`. Describe the args object, what comes back, and any safety caveats. This is the long-tail payload - be complete here, terse in `summary`. |
62
+ | `inputSchema` | object (optional) | A JSON Schema for the tool's args. **This is what makes the tool a first-class MCP citizen when promoted hot:** the top-level `tools/list` advertises it VERBATIM, so a connected client knows the exact arg shape without reading `instructions`. Omit it and a hot-promoted tool falls back to a free-form `{ "type": "object" }` (callable, but the client gets no arg hints). Must be an object - the register rejects any other shape at authoring time. |
63
+ | `invoke` | object | How `run` executes the tool. See below. |
64
+
65
+ ### The `invoke` object - two forms
66
+
67
+ ```json
68
+ { "type": "script", "path": "scripts/echo-upper.sh" }
69
+ ```
70
+ - `type: "script"` - run a file under `tools/scripts/`. `path` is **relative to `tools/`**
71
+ and MUST stay inside `tools/scripts/` (registry.js path-guards it, same defense-in-depth as the
72
+ hook loader's `writeScript`). The args object is delivered to the script (e.g. as a JSON blob on
73
+ stdin or argv per the registry's convention) and the script's stdout is the tool output.
74
+
75
+ ```json
76
+ { "type": "shell", "command": "node \"${TOOLS_DIR}/scripts/echo-upper.js\"" }
77
+ ```
78
+ - `type: "shell"` - run a command line. `${TOOLS_DIR}` expands to the absolute `tools` path at run
79
+ time (mirrors `${HOOKS_DIR}` in the hook manifest). Prefer `script` for anything reusable; reserve
80
+ `shell` for thin one-liners.
81
+
82
+ > Isolation rule: an `invoke` must only reference scripts/commands that live **inside the gateway's
83
+ > own tree**. Never point `invoke` at a live tool that lives outside it (e.g. in a host's external
84
+ > tool directory). If you want an external tool's behaviour, clone its script into
85
+ > `tools/scripts/` first, then point at the clone.
86
+
87
+ ---
88
+
89
+ ## 2. Where the script goes
90
+
91
+ ```
92
+ tools/
93
+ tools.register.json # the entries (above)
94
+ scripts/ # your tool scripts live HERE and nowhere else
95
+ echo-upper.sh
96
+ ```
97
+
98
+ (`registry.js` - the engine that loads/persists the register and runs list/instructions/run - is
99
+ NOT user content. It lives in `src/tools/registry.js` and is not part of this tree.)
100
+
101
+ The script reads its arguments (the `args` object passed to `run`) and writes its result to stdout.
102
+ Exit non-zero with a message on stderr to signal failure - the registry surfaces that as the tool's
103
+ error. Keep scripts pure and side-effect-honest: anything destructive is exactly what the PreToolUse
104
+ gate exists to catch (next section).
105
+
106
+ ---
107
+
108
+ ## 3. How `run` works - and why it is gated
109
+
110
+ `toolfunnel_run_tool({ name, args })` does **not** execute the script directly. It calls
111
+ `registry.run(id, args)`, which routes the call through the gateway's **hook engine** before - and after -
112
+ the tool actually runs:
113
+
114
+ ```
115
+ toolfunnel_run_tool(name, args)
116
+ │
117
+ ▼
118
+ registry.run(id, args)
119
+ │
120
+ ├─► hookEngine.fire("PreToolUse", ctx, { tool_name, tool_input: args })
121
+ │ └─ if result.blocked → STOP. Return { ok:false, error: <reason> }. Script never runs.
122
+ │
123
+ ├─► execute the invoke (script/shell) only if NOT blocked
124
+ │
125
+ └─► hookEngine.fire("PostToolUse", ctx, { tool_name, tool_input: args, tool_response })
126
+ └─ feedback only; the tool already ran (cannot un-run it)
127
+ ```
128
+
129
+ - **The PreToolUse gate is the whole safety case.** A hook denies a call by returning, on exit 0,
130
+ `{ "hookSpecificOutput": { "hookEventName": "PreToolUse", "permissionDecision": "deny",
131
+ "permissionDecisionReason": "..." } }` - OR by `exit 2` with the reason on stderr. (PreToolUse uses
132
+ `permissionDecision`, NOT a top-level `decision` . When that fires,
133
+ `run` returns an error result and the script is never spawned.
134
+ - `permissionDecision: "allow"` passes; `"ask"` is non-blocking in the autonomous host (no
135
+ interactive prompt) - the reason is captured but the tool proceeds.
136
+ - The result shape `run` returns matches the meta-tool contract: `{ ok, output, error? }`.
137
+
138
+ So adding a tool grants real power, but every invocation is mediated by the same gate that protects
139
+ every other tool. If your new tool is dangerous, the correct control is a PreToolUse hook that
140
+ matches its `tool_name` (see `toolfunnel_howto({ topic: "add-hook" })`), not omitting the tool.
141
+
142
+ ---
143
+
144
+ ## 4. Steps to add a tool
145
+
146
+ 1. Pick a unique `id` (check `toolfunnel_list_tools` first so you don't collide).
147
+ 2. If `invoke.type === "script"`, write the script into `tools/scripts/<id>.sh` (or `.js`).
148
+ 3. Add the entry to the `tools.register.json` array (via `registry.js`'s add/edit path so the write
149
+ is atomic - never hand-edit and risk a torn file).
150
+ 4. Confirm with `toolfunnel_list_tools` (your `summary` appears) and `toolfunnel_tool_instructions({ name })`
151
+ (your `instructions` appear).
152
+ 5. Test with `toolfunnel_run_tool({ name, args })`. Verify the gate behaves: a benign call returns
153
+ `{ ok:true, output }`; if a matching PreToolUse hook should deny it, confirm it does.
154
+
155
+ No reconnect is needed - the register is dynamic: the running gateway hot-reloads
156
+ `tools.register.json` on change and emits `notifications/tools/list_changed`, exactly like an
157
+ `expose.json` or hook edit.
158
+
159
+ ---
160
+
161
+ ## 5. Promoting your tool to the top-level surface (the "instant MCP" recipe)
162
+
163
+ By default your tool lives behind the lean meta-tools (discovered via `toolfunnel_list_tools`).
164
+ Set `hot: true` for its id in `tools/tools.state.json` (or via the UI / `tf_tool_set`) and it is
165
+ advertised DIRECTLY in the MCP `tools/list` - under its own `id`, with your `summary` as the
166
+ description and your `inputSchema` as the advertised schema. Author a real `inputSchema` before
167
+ promoting: that schema is the difference between a client calling your tool correctly first time
168
+ and it guessing at a free-form object.