dsh-py-codeact 0.2.2 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +49 -16
  2. package/lib/index.js +232 -67
  3. package/package.json +1 -1
  4. package/py/kernel.py +284 -15
package/README.md CHANGED
@@ -52,24 +52,40 @@ A dunder-named package reads as harness-owned, survives `%reset`, and leaves the
52
52
 
53
53
  ```
54
54
  In [1]: read?
55
- Signature: read(*, file_path: 'str', offset: 'int' = Ellipsis, limit: 'int' = Ellipsis) -> 'str'
56
- Docstring: Read a file from the workspace. Results include line numbers…
55
+ Signature: read(*, file_path: 'str', offset: 'int' = ..., limit: 'int' = ...) -> 'ReadOutput'
56
+ Docstring:
57
+ Read a file from the workspace. Results include line numbers…
58
+
59
+ Parameters:
60
+ file_path: Path to read, resolved by the filesystem backend.
61
+ offset: 1-based first line to return. Defaults to 1.
62
+ limit: Maximum number of lines to return. Defaults to 2000.
57
63
  ```
58
64
 
59
- That is why the prompt block carries signatures only — the descriptions are one `?` away instead of resident in every request. Annotations are rendered host-side with dsh's own exported `jsonSchemaToPy`, so there is no second JSON-Schema mapper to drift.
65
+ The block renders the same picture: one parameter per line, its description as a trailing comment, the tool's own as a docstring so what a model reads in the prompt and what it gets from `?` are the same text, from the same source. Annotations are rendered host-side by dsh's own exported renderers, so there is no second JSON-Schema mapper to drift.
60
66
 
61
67
  The **return** type comes from the tool's own `output` schema, by way of `ctx.tools.sdkSchemas(scope)` — the projection that carries it. It is the one annotation the model cannot recover by reading harder: a wrong argument fails loudly at the call, while an unknown return shape is only discoverable by calling once and printing the result, which costs a whole turn per tool. A tool that declares no output schema still renders `Any`; claiming a type nobody declared would be worse than admitting ignorance.
62
68
 
63
- Each object in that schema whose keys and whose own generated class name Python can take is declared as a named `TypedDict` above the signatures. For a bridged MCP tool the schema described here is the payload, not the transport wrapper — see below; `dict[str, Any]` would say a dict arrives without saying which keys, which is the one thing a return annotation exists to say.
69
+ Every object in that schema is declared as a named `TypedDict` above the signatures, and an output that is a choice of shapes becomes a union of named branches. For a bridged MCP tool the schema described here is the payload, not the transport wrapper — see below; `dict[str, Any]` would say a dict arrives without saying which keys, which is the one thing a return annotation exists to say.
64
70
 
65
71
  ```python
66
- class McpCalendarListEventsOutput(TypedDict):
67
- result: str
72
+ class BashOutput1(TypedDict):
73
+ kind: Literal["background"]
74
+ jobId: str
68
75
 
69
- async def mcp__calendar__list_events(*, calendar_id: str) -> McpCalendarListEventsOutput: ...
76
+ class BashOutput2Stdout(TypedDict):
77
+ text: str
78
+ truncated: bool
79
+
80
+ class BashOutput2(TypedDict):
81
+ kind: Literal["foreground"]
82
+ exitCode: int | None
83
+ stdout: BashOutput2Stdout
84
+
85
+ async def bash(*, command: str, run_in_background: bool = ...) -> BashOutput1 | BashOutput2: ...
70
86
  ```
71
87
 
72
- `jsonSchemaToPy` cannot do this and says so — it is context-free, and naming a `TypedDict` needs the render context `renderToolsSdkPy` supplies. That renderer is not reusable here: it emits a whole document in Code Mode's own `tools.name(args)` contract. So only the object and array branches are handled locally, every leaf still going through `jsonSchemaToPy`a place to hang the names, not a second JSON-Schema mapper. The binding set is resent with every cell, because restrictions and mid-conversation tool changes can move a tool in or out between calls.
88
+ `jsonSchemaToPy` cannot do this and says so — it is context-free, and naming a `TypedDict` needs the render context `renderToolsSdkPy` supplies. With nowhere to hang a declaration it degrades every object to `dict[str, Any]`, which on a stock catalogue is the return type of every tool but one. `renderType`, the context-carrying core, is not exported, so the context is borrowed instead: `renderToolsSdkPy` is called with the parameters stripped — the one input it renders without allocating a class, which makes every class it emits an output class and the block it returns is read back for the declarations and each tool's return text. The `Literal`s, the nested classes, the collision suffixes and the Unicode identifier rules are then dsh's own rather than a second mapper drifting alongside them. The binding set is resent with every cell, because restrictions and mid-conversation tool changes can move a tool in or out between calls.
73
89
 
74
90
  ### The MCP wrapper does not reach the cell
75
91
 
@@ -78,8 +94,8 @@ dsh's MCP client resolves a call to `{ content, structuredContent? }` — the pr
78
94
  So the bridge unwraps, and the signature describes what the cell actually receives:
79
95
 
80
96
  ```python
81
- async def mcp__review__search(*, q: str) -> McpReviewSearchOutput: ... # the payload, not the wrapper
82
- async def mcp__email__ping() -> str: ... # no declared payload: the text blocks, joined
97
+ mcp.review.search(*, q: str) -> McpReviewSearchOutput # the payload, not the wrapper
98
+ mcp.email.ping() -> str # no declared payload: the text blocks, joined
83
99
  ```
84
100
 
85
101
  A server that declares no output schema is the common case in the wild, and its result really is just text — one text block, in every one of the 4142 MCP results measured across six trials, which is why this is a `str` and not a `list[str]` that would cost an `r[0]` at every call site to preserve a boundary that never appears. A result carrying no text block at all resolves to the empty string: an image there is re-attached to the conversation and read on the next step, so there was never anything for the cell to receive. One divergence is possible and belongs to the server: a tool that declares an output schema but omits `structuredContent` on some call resolves to that call's text, where the signature promised the payload type.
@@ -88,19 +104,36 @@ Unwrapping keys off the tool's declared output schema, not off the value that co
88
104
 
89
105
  ### MCP tools work, with nothing special
90
106
 
91
- MCP servers register into the same `ctx.tools` registry as everything else, so they arrive in `__dsh__.tools` like any other binding and dispatch through the same pipeline:
107
+ MCP servers register into the same `ctx.tools` registry as everything else, so they arrive in `__dsh__.tools` like any other binding and dispatch through the same pipeline. They are *presented* grouped, under one `mcp`:
92
108
 
93
109
  ```python
94
- from __dsh__.tools import mcp__gh__github_graphql as gh
110
+ from __dsh__.tools import mcp
95
111
 
96
- data = await gh(query="{ viewer { login } }")
112
+ data = await mcp.gh.github_graphql(query="{ viewer { login } }")
97
113
  ```
98
114
 
115
+ dsh names them `mcp__<server>__<rawName>`, and with a hundred mounted that import line was most of the prompt block while every call site respelled its server. The flat names stay bound — `mcp` is how they are shown, not what they are — so a cell written before this still runs, `import *` still binds them, and only the listings drop them. A name the grouping cannot serve is still shown: dsh hashes a public name that needed normalising and the cut can land before the second `__`, and a raw name that is a true dunder is refused by `__getattr__`; either way the flat name is the only one that works. They are kept out of `dir(__dsh__.tools)` for the same reason the block stopped printing them; a name the grouping cannot reach (dsh hashes a public name that needed normalising, and the cut can land before the second `__`) stays listed, because `mcp` is not another way to say it. The block lists them one module per section — `# __dsh__.tools`, then `# __dsh__.tools.mcp.<server>` — because that is what they are. It used to declare `Protocol` stubs with `self`-taking methods, copied from dsh's own SDK renderer; that renderer describes a singleton object (`tools: Tools`), while this grouping is a real package, so the stub claimed a binding that never happens. `mcp.exa.web_search?` answers `(*, query: str) -> str`, and the block now says the same thing.
116
+
117
+ It is a real package, so a server can be imported as a module — which reads better than `mcp.` at every call site when a cell leans on one server:
118
+
119
+ ```python
120
+ from __dsh__.tools.mcp.calendar import list_events, create_event
121
+ from __dsh__.tools.mcp import calendar # or the server itself
122
+ ```
123
+
124
+ The deep form is why each server gets a `sys.modules` entry of its own: `__getattr__` can serve `from __dsh__.tools.mcp import calendar`, but not `from __dsh__.tools.mcp.calendar import list_events` — the import machinery looks that one up as a module. No meta path finder is needed; registration is enough.
125
+
126
+ `mcp` and its server modules are live views of the catalogue, not snapshots of the cell they were imported in: a restriction or a reconnecting server moves tools in and out between calls, and unlike a single tool the model has no reason to ever import the namespace twice. (A name pulled OUT with `from ... import` is a snapshot, as it is for any Python import.) The name is reserved — a native tool called `mcp` is not bound, nor is one called `ToolCallError`, nor any whose name starts with `_`. The prompt block mirrors all three, because a block that imports a name the kernel never binds is an `ImportError` on the first line the model copies.
127
+
99
128
  dsh's MCP client is explicitly aware of this route — its canonical value "retains the complete JSON MCP blocks and optional structured content for programmatic and Code Mode callers" — and the sub-call logs a `SUBTOOL` row like any other.
100
129
 
101
130
  Worth contrasting: Anthropic's server-side [programmatic tool calling](https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling) is *not* compatible with MCP tools. Owning the bridge host-side is what buys this.
102
131
 
103
- (A tool whose name is not a valid Python identifier cannot be `import`ed, but is still reachable as `getattr(__dsh__.tools, "odd-name")`. dsh's MCP naming `mcp__<server>__<tool>` always is one.)
132
+ A raw MCP name is a routine place to find a hyphen — dsh normalises over `[A-Za-z0-9_-]`, so `-` survives, and `-` is legal in no Python identifier. Such a tool is bound under BOTH spellings: its own, and the `-`→`_` fold that the listings show and the block spells. `mcp.notion.API_patch_block_children(...)` dispatches to `mcp__notion__API-patch-block-children`, and an existing `getattr(mcp.notion, "API-patch-block-children")` keeps working. The fold never displaces a real tool: a server exposing both `a-b` and `a_b` keeps `a_b` meaning `a_b`, and both stay listed.
133
+
134
+ Measured before adding it: on a 20-task benchmark run, 676 of 17030 dispatches (4.0%) went to twelve hyphenated tools, all from one server, and `API-patch-block-children` was the single most-dispatched tool of the whole catalogue — each call site paying for a `getattr`, with no signature in the block to go with it, because an unspellable name was excluded from the listing entirely. Across the 208 distinct tool names those runs dispatched, no fold collided with another fold or shadowed an existing name.
135
+
136
+ (Folding fixes only what a substitution can reach. A keyword (`class`) or a digit-leading name (`123tool`) is unspellable for reasons no rewrite fixes, and those keep the `getattr` route — `getattr(__dsh__.tools, "class")` — which the block still points at when there is one.)
104
137
 
105
138
  ## Exclusive mode
106
139
 
@@ -174,7 +207,7 @@ Sub-dispatches carry the outer execution's `parent` token, so they re-enter the
174
207
  - **Redundant-import hints.** When an `import` rebinds a name to the object it already held, the result carries `` `json` is already imported in this session — no need to re-import it. `` A model driving a persistent REPL re-imports constantly; telling it is cheaper than letting it burn a line every cell. (Implemented with a `dict` subclass that watches top-level `STORE_NAME` and checks the preceding opcode was `IMPORT_NAME`/`IMPORT_FROM`, so `x = x` does not trip it.)
175
208
  - **Readable reprs.** `objprint` + IPython's `pretty` for objects whose own `__repr__` is `object.__repr__` — an agent reading values needs structure, not `<Foo object at 0x…>`.
176
209
  - **Tagged observations.** `<stdout>`, `<stderr>`, `<return>`, `<traceback>`, `<note>` — with four things possibly present at once, the model needs to know which is which. A plain successful value stays bare.
177
- - **Introspection over prompt text.** `read?` for one tool's full description, `dir(__dsh__.tools)` for the whole list, `%whos` for its own bindings.
210
+ - **Introspection over prompt text.** `read?` for one tool's full description, `dir(__dsh__.tools)` for the list — `mcp` rather than the hundred flat names under it, matching what the block printed — then `dir(mcp)` and `dir(mcp.<server>)` for those, and `%whos` for its own bindings. `__all__` is left alone: it is what `import *` binds, not what the model is shown — so it carries every flat name the listings drop, and `ToolCallError`, which no listing shows because it is not a tool but which `except ToolCallError` needs after an `import *`. What it introspects matches what the block showed it: the listing carries `mcp` rather than the hundred flat names under it, and an optional parameter renders `= ...` there as it does here — `inspect.signature` uses `repr`, and `repr(...)` is `Ellipsis`, which is what `read?` used to say.
178
211
 
179
212
  ## Cancellation
180
213
 
@@ -187,7 +220,7 @@ A pure CPU loop (`while True: pass`) never reaches an await point. After `hardIn
187
220
  - **Native writes are not captured.** stdout/stderr are captured at the Python level, so `print` is captured but a subprocess writing to fd 1 is not. Use `subprocess.run(..., capture_output=True)`, or `%run`. (Anything that does reach fd 1/2 — including IPython's own colored traceback, deliberately routed there — is retained only for the crash message.)
188
221
  - **Scope is not optional.** The visible tool set comes from `ctx.tools.sdkSchemas(scope)` — the scope being the agent. Omitting it yields the *global* view, which in a preset composition holds only host-registered tools; the preset's own `read`/`bash`/`edit` live in the agent scope and vanish. The prompt section reads `assembly.scope`, and the kernel's name list is resent with every cell (restrictions and mid-conversation tool changes can move a tool in or out between calls).
189
222
  - **Pick a `toolName` nothing else answers to.** With an MCP IPython server also mounted, a model told to "use the python tool" reaches for `mcp__py__ipython_execute_code` — which has no `tools` binding — and then reports that your tool does not exist.
190
- - **A shape Python cannot name stays vague.** A field whose key is not a valid identifier degrades its own class back to `dict[str, Any]` rather than emitting a body that will not parse — the tool stays callable and only that one annotation goes quiet. The same guard covers the class name itself, which is not local damage: a tool named `123tool` would otherwise emit `class 123toolOutput`, a SyntaxError that takes the whole block with it. A schema with no declared properties still renders `Any`; claiming a type nobody declared would be worse.
223
+ - **A shape Python cannot name stays vague.** A field whose key is not a valid identifier degrades its own class back to `dict[str, Any]` rather than emitting a body that will not parse — the tool stays callable and only that one annotation goes quiet. A parameter named `self` is now just a parameter: every tool renders as a module-level function, so no enclosing signature has spent that name. A tool name Python refuses gets a class under one it accepts (`123tool` `Tool123toolOutput`), rather than the `class 123toolOutput` that would be a SyntaxError taking the whole block with it; that tool has no `async def` line, but it is still bound, and `123tool?` names the same class. A schema with no declared properties still renders `Any`; claiming a type nobody declared would be worse.
191
224
  - **Cold start.** The PEP 723 environment is resolved on first use (`uv python find --script`). Warm afterwards; pass `python` to skip it.
192
225
 
193
226
  The interpreter is then spawned **directly**, never behind `uv run --script`. A wrapper stays in the process tree as the interpreter's parent: when it exits first, the interpreter is reparented to init, the handle the host holds reports an exit, and a perfectly live kernel looks dead — so the next cell respawns and the session's state vanishes with an `[the interpreter was restarted]` notice nothing actually caused. `alive` is likewise tracked from the exit event rather than read off `proc.killed`, which Node sets on any `kill()` call, including a signal the process survived.
package/lib/index.js CHANGED
@@ -12,10 +12,12 @@
12
12
 
13
13
  import { CallId } from '@deepseek-ai/dsh-llm/brand'
14
14
  import { contentHasImage, createUserMessage } from '@deepseek-ai/dsh-llm'
15
- import { defineTool, jsonSchemaToPy } from '@deepseek-ai/dsh-tools'
15
+ import { defineTool, jsonSchemaToPy, renderToolsSdkPy } from '@deepseek-ai/dsh-tools'
16
16
  import { KERNEL_PY, PythonKernel } from './kernel.js'
17
17
 
18
18
  /** Same prompt band as Code Mode's `tools:sdk`: tool guidance is 100–199. */
19
+ const TOOLS_MODULE = '__dsh__.tools'
20
+
19
21
  const SDK_SECTION_ORDER = 150
20
22
 
21
23
  /**
@@ -88,33 +90,58 @@ export function mcpPayload(value) {
88
90
  return 'structuredContent' in value ? { value: value.structuredContent } : { value: contentText(value.content) }
89
91
  }
90
92
 
93
+ /** dsh's schema subset, from `assertSupportedJsonSchema`: eight constraint keywords plus four annotations, enforced whole-tree and all-or-nothing. */
94
+ const SCHEMA_SUBSET = new Set(['type', 'oneOf', 'properties', 'required', 'additionalProperties', 'items', 'enum', 'const', 'description', 'title', 'default', 'examples'])
95
+
96
+ /**
97
+ * Rewrite a raw schema into {@link SCHEMA_SUBSET}, so a keyword that carries no type information cannot cost a field the type sitting right beside it.
98
+ *
99
+ * The subset is validated whole-tree and rejection is total, so ONE unrecognised keyword anywhere collapses the entire annotation: `{"type": "string", "minLength": 1}` renders `Any` where `{"type": "string"}` renders `str`. Those keywords are what Pydantic and FastMCP emit for `Field(min_length=1)`, `float` and `str | None`, and `$schema` sits on the root of most generated schemas — so this is not an exotic case, it is every MCP server built on one. A required search query arriving as `query: Any` tells the model nothing while looking like it did.
100
+ *
101
+ * Drop whatever the subset cannot take — by name, and for `additionalProperties` by form — and rewrite `anyOf` to `oneOf`, the one rejected keyword that DOES carry type information and which means exactly what a union annotation means. Deliberately not a second JSON-Schema mapper: it decides nothing about types, it only removes reasons to reject, which is why the rewrite stands down where a `oneOf` is already declared. Nor can it lose an annotation that renders today, since a node dsh already accepts has no key to drop and no `anyOf` to rewrite.
102
+ */
103
+ function narrowed(node) {
104
+ if (typeof node !== 'object' || node === null || Array.isArray(node)) return node
105
+ const rewrite = (key, value) => {
106
+ if (key === 'items') return narrowed(value)
107
+ if (key === 'oneOf' && Array.isArray(value)) return value.map((branch) => narrowed(branch))
108
+ if (key === 'properties' && typeof value === 'object' && value !== null) return Object.fromEntries(Object.entries(value).map(([name, child]) => [name, narrowed(child)]))
109
+ return value
110
+ }
111
+ // A name in the subset is not enough: dsh takes `additionalProperties` only as a BOOLEAN, so the schema-valued form Pydantic emits for `dict[str, str]` is rejected however clean its child is — narrowing that child repairs nothing, and dropping it costs the annotation nothing dsh could have expressed.
112
+ const keep = ([key, value]) => SCHEMA_SUBSET.has(key) && (key !== 'additionalProperties' || typeof value === 'boolean')
113
+ // Only where no `oneOf` is declared. With one already there, dropping `anyOf` loses no type information, and substituting its branches for the declared union would be this function deciding a type rather than removing a reason to reject.
114
+ const source = 'anyOf' in node && !('oneOf' in node) ? { ...node, oneOf: node.anyOf } : node
115
+ return Object.fromEntries(Object.entries(source).filter(keep).map(([key, value]) => [key, rewrite(key, value)]))
116
+ }
117
+
91
118
  /**
92
- * Project one tool schema onto the wire spec the kernel builds a binding from: dsh's description becomes the function's docstring, its parameters become a keyword-only signature. Annotations are rendered here with dsh's own `jsonSchemaToPy`, so the kernel needs no second JSON-Schema mapper.
119
+ * Project one tool schema onto the wire spec the kernel builds a binding from: dsh's description becomes the function's docstring, its parameters become a keyword-only signature. Parameter annotations are rendered here with dsh's own `jsonSchemaToPy`, and the return type comes from {@link renderOutputTypes}, so the kernel needs no JSON-Schema mapper of its own.
93
120
  *
94
121
  * The return type comes from the tool's own `output` schema — the shape of the canonical value the call actually resolves to. It is the one annotation a model cannot recover by reading harder: a parameter it gets wrong fails loudly at the call, while an unknown return shape is only discoverable by calling once and printing the result, which is a whole extra turn per tool.
95
122
  */
96
- function toolSpec(schema, declarations) {
123
+ function toolSpec(schema, returns) {
97
124
  const parameters = schema.parameters ?? {}
98
125
  const properties = parameters.properties ?? {}
99
126
  const required = new Set(Array.isArray(parameters.required) ? parameters.required : [])
100
127
  return {
101
128
  name: schema.name,
102
129
  doc: schema.description,
103
- // `output` is absent only if the caller passed a `schemas()` projection; `sdkSchemas()` always carries it.
104
- // An MCP tool is annotated by what the CELL receives, which is the payload the bridge unwraps
105
- // to — never the transport wrapper. Describing the wrapper is accurate and useless: the model
106
- // then hand-writes `r["structuredContent"]["result"]`, and calls once just to learn that.
107
- returns: schema.output === undefined ? 'Any' : mcpReturn(schema, declarations),
130
+ // Absent only if the render dropped this tool, which it does not: `renderOutputTypes` emits one
131
+ // entry per schema, and a schema with no `output` at all comes back as `Any` from there.
132
+ returns: returns.get(schema.name) ?? 'Any',
108
133
  params: Object.entries(properties).map(([name, node]) => ({
109
134
  name,
110
- type: jsonSchemaToPy(node),
135
+ type: jsonSchemaToPy(narrowed(node)),
111
136
  required: required.has(name),
137
+ // What the parameter MEANS, which its type cannot say — `queries` is `list[str]` either way, and only the prose says 1–4 of them. Rendered beside the parameter as a comment AND carried to the kernel, so `read?` agrees with the block instead of being the poorer of the two. `undefined` drops out of `JSON.stringify`, so a parameter without one costs the wire and `specsKey` nothing.
138
+ doc: node?.description,
112
139
  })),
113
140
  }
114
141
  }
115
142
 
116
143
  /**
117
- * Every visible tool's spec, plus the `TypedDict` bodies their return annotations name.
144
+ * @internal exported for the suite, which runs the block it renders. Every visible tool's spec, plus the `TypedDict` declarations their return annotations name.
118
145
  *
119
146
  * Both consumers go through here so they cannot disagree: the prompt block declares the classes,
120
147
  * and the kernel sends the same `returns` text on to `inspect.Signature`, so `read?` shows the
@@ -122,26 +149,41 @@ function toolSpec(schema, declarations) {
122
149
  * second argument is the INDEX — a number, so a default parameter never fires and the accumulator
123
150
  * is silently a `0`.
124
151
  */
125
- function toolSpecs(schemas) {
126
- const declarations = new Map()
127
- const sorted = [...schemas].sort(byName)
152
+ export function toolSpecs(schemas) {
153
+ // Whatever the kernel refuses to bind, this side must not advertise. `build_bindings` drops every
154
+ // `_`-leading name plus `set(vars(ToolsModule)) | set(vars(types.ModuleType)) | {"ToolCallError",
155
+ // "mcp"}`, whose only members that do not already start with `_` are those last two — so the whole
156
+ // reservation mirrors as the three lines below. Only `mcp` was carried across, and the block
157
+ // happily wrote `from __dsh__.tools import ToolCallError, ToolCallError, _private, read`: an
158
+ // `ImportError` on a name nothing binds, and a duplicate that resolves to the exception class the
159
+ // instructions tell the model to catch, so `await ToolCallError(...)` raises an unexplainable
160
+ // `TypeError`. Dropped here rather than at the render, which is downstream of the declarations:
161
+ // one with an object output still had its `TypedDict` emitted, referenced by nothing.
162
+ const sorted = [...schemas].filter((schema) => !RESERVED_NAMES.has(schema.name) && !schema.name.startsWith('_')).sort(byName)
128
163
  // Which tools DECLARE the wrapper. Unwrapping keys off this rather than the returned value:
129
164
  // a tool whose own payload happens to be `{content: [...]}` is not wrapping anything, and
130
165
  // replacing its value with the joined text would change what it returns with nothing to say so.
131
166
  const envelopes = new Set(sorted.filter((schema) => mcpPayloadSchema(schema.output) !== null).map((schema) => schema.name))
132
- return { specs: sorted.map((schema) => toolSpec(schema, declarations)), declarations, envelopes }
167
+ const { returns, declarations } = renderOutputTypes(sorted)
168
+ return { specs: sorted.map((schema) => toolSpec(schema, returns)), declarations, envelopes }
133
169
  }
134
170
 
135
- /** The annotation for one tool's output: an MCP payload if it is an envelope, else the schema as declared. */
136
- function mcpReturn(schema, declarations) {
137
- const payload = mcpPayloadSchema(schema.output)
138
- if (payload === null) return declareType(schema.output, `${pascal(schema.name)}Output`, declarations)
139
- // An envelope with no declared payload resolves to the text blocks joined, so it really is a `str`.
140
- return payload === undefined ? 'str' : declareType(payload, `${pascal(schema.name)}Output`, declarations)
171
+ /**
172
+ * `mcp__calendar__list_events` -> `{ server: 'calendar', tool: 'list_events' }`, matching the
173
+ * kernel's own split. Only the first two separators are consumed: dsh's name is
174
+ * `mcp__<serverName>__<rawName>` and a raw name may itself contain `__`.
175
+ */
176
+ function splitMcp(name) {
177
+ if (!name.startsWith('mcp__')) return null
178
+ const rest = name.slice(5)
179
+ const at = rest.indexOf('__')
180
+ if (at <= 0 || at + 2 >= rest.length) return null
181
+ return { server: rest.slice(0, at), tool: rest.slice(at + 2) }
141
182
  }
142
183
 
143
184
  /** Deterministic (lexicographic) order, so an unchanged tool set renders byte-identically. */
144
- const byName = (a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)
185
+ const by = (key) => (a, b) => (key(a) < key(b) ? -1 : key(a) > key(b) ? 1 : 0)
186
+ const byName = by((spec) => spec.name)
145
187
 
146
188
  /** The kernel's own wording for a shell whose previous cell has not finished. Matched, not parsed: it is the one failure that leaves the bindings uninstalled. */
147
189
  const KERNEL_BUSY = 'kernel busy: a previous cell is still running'
@@ -188,45 +230,92 @@ export function needsRestartNotice(session, toolName, plugin) {
188
230
  /** `keyword.kwlist + keyword.softkwlist` for the `requires-python = ">=3.12"` the kernel pins. */
189
231
  const PY_KEYWORDS = new Set(['False', 'None', 'True', '_', 'and', 'as', 'assert', 'async', 'await', 'break', 'case', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'match', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'type', 'while', 'with', 'yield'])
190
232
 
233
+ /**
234
+ * The name Python can take for a raw MCP name, or the name itself when it already is one.
235
+ *
236
+ * dsh normalises over `[A-Za-z0-9_-]`, so `-` survives — and `-` is legal in no Python identifier.
237
+ * `API-patch-block-children` was therefore reachable only through `getattr`, at every call site and
238
+ * with no signature in this block at all; it was the most-dispatched tool of a 20-task benchmark
239
+ * run. The kernel binds the fold alongside the raw name (`spellable` in `py/kernel.py`), so this
240
+ * side may spell it. Only `-` folds: a keyword or a digit-leading name is unspellable for reasons
241
+ * no substitution fixes, and those keep the `getattr` route.
242
+ */
243
+ const fold = (name) => name.replace(/-/g, '_')
244
+
245
+ /**
246
+ * The name a server is SHOWN under: its fold, unless another server already owns that spelling.
247
+ *
248
+ * Folding unconditionally emitted two `# __dsh__.tools.mcp.a_b` sections for a catalogue holding
249
+ * both `a-b` and `a_b`, one of them spelling its calls against the wrong server. Mirrors
250
+ * `canonical` in `py/kernel.py`, which keeps the two modules distinct for the same reason.
251
+ */
252
+ const serverName = (server, grouped) => (fold(server) !== server && grouped.has(fold(server)) ? server : fold(server))
253
+
191
254
  /** A name this block can put in an `import`, a `def`, or a `TypedDict` field. */
192
255
  const isUsableName = (name) => /^[A-Za-z_][A-Za-z0-9_]*$/.test(name) && !PY_KEYWORDS.has(name)
193
256
 
194
257
  /** `mcp__calendar__list_events` -> `McpCalendarListEvents`, for naming that tool's declarations. */
195
258
  const pascal = (name) => name.split(/[_\-]+/).filter(Boolean).map((part) => part[0].toUpperCase() + part.slice(1)).join('')
196
259
 
197
- /** `dict[str, Any] | dict[str, Any]` is what a `oneOf` of two objects collapses to once both branches degrade to the same text. Rendering the duplicate says there is a choice to make where there is none. */
198
- const dedupeUnion = (expr) => [...new Set(expr.split(' | '))].join(' | ')
260
+ /** Names `build_bindings` refuses that do not already start with `_`. Mirrored from `py/kernel.py`; the two halves are checked against each other in the suite. */
261
+ const RESERVED_NAMES = new Set(['ToolCallError', 'mcp'])
262
+
263
+ /** Every `typing` symbol this block can emit, in the order the import line wants them. Alphabetical, so the filtered list needs no sort. */
264
+ const TYPING_SYMBOLS = ['Any', 'Literal', 'NotRequired', 'Protocol', 'TypedDict']
265
+
266
+ /** The three fixed landmarks in a `renderToolsSdkPy` block: what precedes the class declarations, what follows them, and what closes the member list. */
267
+ const ERROR_STUB = 'class ToolCallError(Exception):\n toolName: str\n\n'
268
+ const PROTOCOL_HEAD = 'class Tools(Protocol):\n'
269
+ const PROTOCOL_TAIL = '\n\ntools: Tools'
199
270
 
200
271
  /**
201
- * Render one output schema, declaring a named `TypedDict` for every object shape it contains.
272
+ * dsh's own Code Mode render, read back for the two things `jsonSchemaToPy` cannot produce: a NAMED
273
+ * `TypedDict` per output object, and the union of named branches a `oneOf` output resolves to.
274
+ *
275
+ * `jsonSchemaToPy` is context-free and says so — "naming a `TypedDict` requires the render context
276
+ * that `renderToolsSdkPy` supplies" — so with nowhere to hang a declaration it degrades every object
277
+ * to `dict[str, Any]`. That is not an edge case: on a stock dsh catalogue it is the return type of
278
+ * every tool but one, which erases exactly what a return annotation is here to say. `bash` resolves
279
+ * to one of two shapes discriminated by a `kind` literal; flattened, the model cannot see there is a
280
+ * discriminator, so it calls once and prints the result to find out — the turn this annotation
281
+ * exists to remove. `renderType`, the context-carrying core, is not exported and its `src/` is not
282
+ * shipped, so `renderToolsSdkPy` is the only door to it.
202
283
  *
203
- * `jsonSchemaToPy` alone cannot do this and says so: it is context-free, and "naming a `TypedDict` requires the render context that `renderToolsSdkPy` supplies" — with nowhere to put a class declaration it degrades every object to `dict[str, Any]`. That erases the one thing an MCP return needs to say. dsh's MCP client resolves a call to the ENVELOPE, `{ content, structuredContent }`, and declares exactly that as the tool's `output`; flattened to `dict[str, Any]` the model is not told the envelope exists, so it reaches for the payload directly, gets nothing, and spends a turn printing the result to find the wrapper the very cost the return annotation is here to remove.
284
+ * The context is therefore borrowed rather than rebuilt. Parameters are stripped before the call —
285
+ * `{}` is the one input that renders as `Any` without allocating a class, so every class in the
286
+ * returned block belongs to an OUTPUT — and descriptions are dropped so the `Tools` body is exactly
287
+ * one line per tool. MCP envelopes are unwrapped first, because the cell receives the payload and
288
+ * annotating the transport wrapper is accurate and useless: the model then hand-writes
289
+ * `r["structuredContent"]["result"]`, and calls once just to learn that.
204
290
  *
205
- * Only the object and array branches are handled here; every leaf still goes through `jsonSchemaToPy`, so there is no second JSON-Schema mapper, only a place to hang the names. A field name Python cannot take degrades that one class back to `dict[str, Any]` rather than emitting a class body that will not parse.
291
+ * Reading generated text back is the seam, and it buys the alternative's absence: the `Literal`s,
292
+ * nested classes, collision suffixes and Unicode identifier rules stay dsh's own instead of a second
293
+ * JSON-Schema mapper drifting alongside them. Should a future dsh reshape the block, the lookups
294
+ * miss and every tool falls back to `Any` — a visible annotation the suite asserts against, not a
295
+ * silently wrong one.
296
+ *
297
+ * @returns the return annotation per tool name, and the class declarations they reference as text.
206
298
  */
207
- function declareType(schema, baseName, declarations) {
208
- if (schema === null || typeof schema !== 'object') return jsonSchemaToPy(schema)
209
- if (schema.type === 'array' && schema.items !== null && typeof schema.items === 'object') {
210
- return `list[${declareType(schema.items, `${baseName}Item`, declarations)}]`
299
+ function renderOutputTypes(sorted) {
300
+ const declared = (schema) => {
301
+ const payload = mcpPayloadSchema(schema.output)
302
+ // An envelope with no declared payload resolves to the text blocks joined, so it really is a `str`.
303
+ return narrowed(payload === null ? schema.output : (payload ?? { type: 'string' }))
211
304
  }
212
- const properties = schema.type === 'object' ? (schema.properties ?? {}) : {}
213
- const names = Object.keys(properties)
214
- // `baseName` too, not just the fields: a tool name may start with a digit or carry a character
215
- // `pascal` does not split on, and `123toolOutput` is a SyntaxError that takes the WHOLE block
216
- // with it including the tools that were fine. It bites hardest for a tool that is not even
217
- // importable, whose class nothing would have referenced.
218
- if (names.length === 0 || !isUsableName(baseName) || !names.every(isUsableName)) return dedupeUnion(jsonSchemaToPy(schema))
219
- const required = new Set(Array.isArray(schema.required) ? schema.required : [])
220
- const body = names.map((name) => {
221
- const type = declareType(properties[name], `${baseName}${pascal(name)}`, declarations)
222
- return ` ${name}: ${required.has(name) ? type : `NotRequired[${type}]`}`
223
- })
224
- // Two tools whose names differ only in separators pascal-case to the same string; keep the first and let the second carry a suffix rather than silently serving one tool the other's shape. Keyed by name, valued by body, so an identical shape reuses the class instead of declaring it twice.
225
- const declared = body.join('\n')
226
- let name = baseName
227
- for (let n = 2; declarations.has(name) && declarations.get(name) !== declared; n++) name = `${baseName}${n}`
228
- declarations.set(name, declared)
229
- return name
305
+ const text = renderToolsSdkPy(sorted.map((schema) => ({ name: schema.name, parameters: {}, output: declared(schema) })))
306
+ const at = text.indexOf(ERROR_STUB)
307
+ const to = text.indexOf(PROTOCOL_HEAD, at)
308
+ const returns = new Map()
309
+ for (const line of text.slice(to + PROTOCOL_HEAD.length, text.indexOf(PROTOCOL_TAIL, to)).split('\n')) {
310
+ // Two shapes, because dsh routes a name Python cannot take to a subscript COMMENT rather than a
311
+ // method. Both are read: this block reaches such a tool through `getattr`, so its return type is
312
+ // as real as any other's.
313
+ const method = / {4}async def ([^(]+)\(self, args: Any\) -> (.+): \.\.\.$/.exec(line)
314
+ if (method !== null) { returns.set(method[1], method[2]); continue }
315
+ const subscript = / {4}# tools\[(".*")\]\(args: Any\) -> (.+)$/.exec(line)
316
+ if (subscript !== null) returns.set(JSON.parse(subscript[1]), subscript[2])
317
+ }
318
+ return { returns, declarations: text.slice(at + ERROR_STUB.length, to).trimEnd() }
230
319
  }
231
320
 
232
321
  /**
@@ -237,28 +326,87 @@ export function renderToolsSection(schemas) {
237
326
  // MCP tools are named `mcp__<server>__<rawName>` over a `[A-Za-z0-9_-]` alphabet, so hyphens are routine — and legal nowhere in `import` or `def`. One such tool used to make the ENTIRE block a SyntaxError, so not a single line in it could be copied, with nothing saying why.
238
327
  //
239
328
  // A keyword is the same failure wearing a legal shape: `class` and `None` match the identifier pattern and then break `import` and `def` just as hard. `_` and the soft keywords (`match`, `case`, `type`) are in the list because they are legal identifiers everywhere EXCEPT where this block puts them.
240
- const importable = specs.filter((spec) => isUsableName(spec.name))
241
- const awkward = specs.filter((spec) => !isUsableName(spec.name))
329
+ // MCP tools are grouped under one `mcp` binding instead of being listed individually. With a
330
+ // hundred of them the import line was most of this block, and `mcp__calendar__list_events`
331
+ // carried its server in the name at every call site; `mcp.calendar.list_events` says the same
332
+ // thing once. The flat names stay bound — this is how they are PRESENTED, not what they are.
333
+ const grouped = new Map()
334
+ const plain = []
335
+ for (const spec of specs) {
336
+ const parts = splitMcp(spec.name)
337
+ if (parts === null) { plain.push(spec); continue }
338
+ if (!grouped.has(parts.server)) grouped.set(parts.server, [])
339
+ grouped.get(parts.server).push({ ...spec, tool: parts.tool, server: parts.server })
340
+ }
341
+ const importable = plain.filter((spec) => isUsableName(spec.name))
342
+ const awkward = plain.filter((spec) => !isUsableName(spec.name))
343
+ // A server or tool whose name Python cannot take keeps its `getattr` route, one level deeper.
344
+ const servers = [...grouped].filter(([server]) => isUsableName(serverName(server, grouped))).sort(by(([server]) => server))
345
+ const oddMcp = [...grouped].flatMap(([server, tools]) =>
346
+ (isUsableName(serverName(server, grouped)) ? tools.filter((t) => !isUsableName(fold(t.tool))) : tools).map((t) => ({ server, tool: t.tool, usableServer: isUsableName(serverName(server, grouped)) })))
347
+ // Every tool renders as a module-level function now, so there is no `self` to splice in front and
348
+ // no reserved name a parameter could collide with — `async def list(*, self: str)` is ordinary
349
+ // Python. That collision guard, and the `head` parameter it read, went with the Protocol stubs.
350
+ // `async def f(*, ) -> T` is still a SyntaxError, which is why a parameterless tool emits no `*`.
351
+ // Escaped for the block to stay COMPILABLE, which matters more here than fidelity: one description carrying a `\"\"\"` or ending in a backslash would close its own docstring and take every tool below it with it — the failure class where one tool invalidates the whole block. Neither appears in a live catalogue; a future MCP server is not bound by that.
352
+ const docstring = (doc) => {
353
+ if (!doc) return [' ...']
354
+ const text = doc.trim().replaceAll('\\', '\\\\').replaceAll('"""', '\\"\\"\\"')
355
+ const lines = text.split('\n')
356
+ return lines.length === 1 ? [` """${lines[0]}"""`] : [' """', ...lines.map((line) => ` ${line}`.trimEnd()), ' """']
357
+ }
358
+ // A `#` comment cannot span lines, so a description that carries newlines is collapsed rather than emitted: left alone its second line parses as code.
359
+ const trailing = (doc) => (doc ? ` # ${doc.trim().replace(/\s*\n\s*/g, ' ')}` : '')
360
+ /** @returns the lines to emit, and — separately — the type expressions they spell, which is what the `typing` import is derived from. Kept apart because prose is now emitted too: a description mentioning "Any file" must not import `Any`. */
242
361
  const signature = (spec) => {
362
+ const body = docstring(spec.doc)
243
363
  // The same rule applies one level down, and used to not be applied at all: `file-path` is routine for MCP tools, and one such PARAMETER made every other tool's signature unusable too. The binding still takes it — the kernel folds unnameable parameters into `**kwargs` — so the tool stays importable and only its signature goes vague; `name?` still shows the real one.
244
- if (!spec.params.every((p) => isUsableName(p.name))) return `async def ${spec.name}(**kwargs: Any) -> ${spec.returns}: ... # parameter names are not all valid Python; see ${spec.name}?`
245
- const fields = spec.params.map((p) => (p.required ? `${p.name}: ${p.type}` : `${p.name}: ${p.type} = ...`))
246
- // `async def f(*, ) -> T` is a SyntaxError: a `*` needs at least one name after it, and a parameterless tool takes no keywords at all.
247
- return `async def ${spec.name}(${fields.length === 0 ? '' : `*, ${fields.join(', ')}`}) -> ${spec.returns}: ...`
364
+ if (!spec.params.every((p) => isUsableName(p.name))) {
365
+ return { lines: [`async def ${spec.name}(**kwargs: Any) -> ${spec.returns}: # not every parameter name can be spelled here; see ${spec.name}?`, ...body], types: ['Any', spec.returns] }
366
+ }
367
+ const types = [spec.returns, ...spec.params.map((p) => p.type)]
368
+ // `async def f(*, ) -> T` is still a SyntaxError, which is why a parameterless tool emits no `*`.
369
+ if (spec.params.length === 0) return { lines: [`async def ${spec.name}() -> ${spec.returns}:`, ...body], types }
370
+ // One parameter per line, so each can carry what it MEANS beside what it is: `queries` is `list[str]` either way, and only the comment says 1–4 of them.
371
+ const fields = spec.params.map((p) => ` ${p.name}: ${p.type}${p.required ? '' : ' = ...'},${trailing(p.doc)}`)
372
+ return { lines: [`async def ${spec.name}(`, ' *,', ...fields, `) -> ${spec.returns}:`, ...body], types }
248
373
  }
249
374
  // `ToolCallError` leads the import because the instructions tell the model to catch it; without it here the natural `except ToolCallError` NameErrors on the failure path, masking the tool failure it was meant to handle. It also keeps the line valid when no tool is importable.
250
375
  // The declarations are stubs, not runtime objects — nothing constructs them — so the import line
251
376
  // lists exactly what the render used. `Any` has always been reachable from a signature
252
377
  // (`**kwargs: Any`, and any degraded annotation) and was never imported at all.
253
- const signatures = importable.map(signature)
254
- const classes = [...declarations].flatMap(([name, body]) => ['', `class ${name}(TypedDict):`, ...body.split('\n')])
255
- const typing = [
256
- // Whole identifiers: `AnyReportOutput` is a class name, not a use of `Any`, and a bare
257
- // `includes` imports a symbol a tool can spell into existence without ever needing it.
258
- signatures.concat(classes).some((line) => /\bAny\b/.test(line)) && 'Any',
259
- classes.some((line) => line.includes('NotRequired[')) && 'NotRequired',
260
- declarations.size > 0 && 'TypedDict',
261
- ].filter(Boolean)
378
+ // One section per MODULE, because that is what these are. `mcp` and each server under it are real
379
+ // modules (`__dsh__.tools.mcp.exa`), so their tools are plain functions: attribute access on a
380
+ // module goes nowhere near the descriptor protocol and binds nothing. The block used to render
381
+ // them as `class _McpExa(Protocol)` with `async def web_search(self, ...)`, on a comment claiming
382
+ // `mcp` was an INSTANCE true before the grouping became a package, false ever since, and the
383
+ // rendering was never revisited. `mcp.exa.web_search?` shows `(*, query: str) -> str` with no
384
+ // `self`, so the block was contradicting the interpreter the model can just ask.
385
+ const listing = (header, specs) => (specs.length === 0 ? [] : ['', `# ${header}`, ...specs])
386
+ const nativeSigs = importable.map((spec) => signature(spec))
387
+ // A blank line between definitions, or a docstring runs straight into the next `async def`.
388
+ const spaced = (blocks) => blocks.flatMap((block, index) => (index === 0 ? block : ['', ...block]))
389
+ const nativeBlock = listing(TOOLS_MODULE, spaced(nativeSigs.map((sig) => sig.lines)))
390
+ // Commented, and spelled the way the call site spells them, because these names are NOT bound at
391
+ // the top level — only `mcp` is. A bare `async def read(...)` under a server header claims a
392
+ // top-level `read` that does not exist, and worse, it BINDS one: three tools named `read` (a
393
+ // native one and two servers') left the last stub shadowing the imported function, so the block
394
+ // executed as written broke the very tool its first section had just declared.
395
+ // A server whose every tool name Python refuses gets no section: nothing here would be callable
396
+ // as written, and the `getattr` line below is where those tools actually live.
397
+ const mcpSigs = servers.flatMap(([server, tools]) => tools.filter((t) => isUsableName(fold(t.tool))).sort(by((t) => t.tool))
398
+ .map((t) => ({ server, ...signature({ ...t, name: fold(t.tool) }) })))
399
+ const mcpBlock = servers.flatMap(([server]) =>
400
+ listing(`${TOOLS_MODULE}.mcp.${serverName(server, grouped)}`, spaced(mcpSigs.filter((sig) => sig.server === server)
401
+ .map((sig) => sig.lines.map((line, index) => `# ${index === 0 ? line.replace(/^async def /, `mcp.${serverName(server, grouped)}.`) : line}`.trimEnd())))))
402
+ const signatures = nativeBlock.concat(mcpBlock)
403
+ const classes = declarations === '' ? [] : ['', ...declarations.split('\n')]
404
+ // Whatever the emitted lines actually spell, rather than a condition per symbol: the conditions
405
+ // were three and the symbols four, so a `Literal` — which a `const` output or parameter renders
406
+ // and nothing here predicts — reached the block with no import and made it a `NameError`.
407
+ // Whole identifiers only: `AnyReportOutput` is a class name, not a use of `Any`.
408
+ const spelled = classes.concat(nativeSigs.concat(mcpSigs).flatMap((sig) => sig.types))
409
+ const typing = TYPING_SYMBOLS.filter((symbol) => spelled.some((line) => new RegExp(`\\b${symbol}\\b`).test(line)))
262
410
  const lines = [
263
411
  INSTRUCTIONS,
264
412
  '',
@@ -266,23 +414,40 @@ export function renderToolsSection(schemas) {
266
414
  '',
267
415
  '```python',
268
416
  ...(typing.length === 0 ? [] : [`from typing import ${typing.join(', ')}`]),
269
- `from __dsh__.tools import ToolCallError${importable.map((spec) => `, ${spec.name}`).join('')}`,
417
+ // `grouped`, not `servers`: a server whose own name Python refuses has no Protocol stub, but its
418
+ // tools are still reached through `mcp` — the `getattr` line below names it. Keyed on the stubs,
419
+ // a catalogue of nothing but such servers advertised `getattr(getattr(mcp, …))` without ever
420
+ // importing `mcp`. The kernel binds it whenever an MCP tool exists, which is this condition.
421
+ `from __dsh__.tools import ToolCallError${grouped.size === 0 ? '' : ', mcp'}${importable.map((spec) => `, ${spec.name}`).join('')}`,
270
422
  ...classes,
271
- '',
272
423
  ...signatures,
273
424
  '```',
274
425
  ]
426
+ if (oddMcp.length > 0) {
427
+ lines.push('', `Under \`mcp\`, but not valid Python identifiers — reach these with \`getattr\`: ${oddMcp.map(({ server, tool, usableServer }) => (usableServer ? `\`getattr(mcp.${server}, ${JSON.stringify(tool)})\`` : `\`getattr(getattr(mcp, ${JSON.stringify(server)}), ${JSON.stringify(tool)})\``)).join(', ')}.`)
428
+ }
275
429
  if (awkward.length > 0) {
276
430
  lines.push('', `Not valid Python identifiers — reach these with \`getattr\`: ${awkward.map((spec) => `\`getattr(__dsh__.tools, ${JSON.stringify(spec.name)})\``).join(', ')}.`)
277
431
  }
278
- lines.push('', 'Each is a real function: `name?` shows its full description, `dir(__dsh__.tools)` lists them.')
432
+ // The section headers ARE the module paths now, so the sentence that used to advertise the import
433
+ // form says nothing the block does not already show. It was measured at zero uses across 22 trials
434
+ // and 6300 cells while it was a trailing note; whether a header does better is the open question,
435
+ // but paying for both is not.
436
+ const example = servers.flatMap(([server, tools]) => tools.filter((t) => isUsableName(fold(t.tool))).map((t) => [serverName(server, grouped), fold(t.tool)])).at(0)
437
+ if (example !== undefined) {
438
+ lines.push('', `A section header is the module: \`from ${TOOLS_MODULE}.mcp.${example[0]} import ${example[1]}\` and \`mcp.${example[0]}.${example[1]}\` reach the same function.`)
439
+ }
440
+ // Naming the second level matters once the grouping exists: `dir(__dsh__.tools)` shows `mcp`,
441
+ // not the hundred tools under it, so a model told only the first level reads a real catalogue as
442
+ // a broken mount.
443
+ lines.push('', `Each is a real function: \`name?\` shows its full description, \`dir(__dsh__.tools)\` lists them${servers.length === 0 ? '' : ', and `dir(mcp)` / `dir(mcp.<server>)` the ones under `mcp`'}.`)
279
444
  return lines.join('\n')
280
445
  }
281
446
 
282
447
  /** Flatten model-facing content blocks to the text a program-visible error carries. */
283
448
  function contentText(content) {
284
449
  if (!Array.isArray(content)) return ''
285
- return content.filter((block) => block?.type === 'text').map((block) => block.text).join('\n')
450
+ return content.filter((block) => block?.type === 'text').map((block) => block.text).join('\n\n')
286
451
  }
287
452
 
288
453
  /**
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-py-codeact",
3
- "version": "0.2.2",
3
+ "version": "0.3.0",
4
4
  "description": "CodeAct agent loop for the DeepSeek Harness: a persistent IPython session as the model's action space, with harness tools bridged in as a virtual `__dsh__.tools` module.",
5
5
  "license": "MIT",
6
6
  "repository": {
package/py/kernel.py CHANGED
@@ -2,7 +2,7 @@
2
2
  # /// script
3
3
  # requires-python = ">=3.12"
4
4
  # dependencies = [
5
- # "ipython~=9.16.0",
5
+ # "ipython~=9.17.0",
6
6
  # "objprint~=0.3.0",
7
7
  # ]
8
8
  # # Checked with `TY_UV=scripts ty check py/kernel.py` — that prefix is what hands ty this script's venv. A `ty.toml` would be found and then silently ignored for PEP 723 scripts (astral-sh/ty#4083), so any ty config has to live here.
@@ -48,6 +48,7 @@ import contextlib
48
48
  import inspect
49
49
  import io
50
50
  import json
51
+ import keyword
51
52
  import select
52
53
  import sys
53
54
  import traceback
@@ -216,6 +217,13 @@ def _make_binding(bridge: Bridge, spec):
216
217
  call.__name__ = name if name.isidentifier() else "call"
217
218
  call.__qualname__ = f"__dsh__.tools.{name}"
218
219
  call.__doc__ = spec.get("doc") or None
220
+ # The same prose the block renders beside each parameter, repeated here so `read?` is not the poorer view: the block is read once a turn, while `?` is what a model reaches for to re-check ONE tool without scrolling back. Same source, so the two cannot drift.
221
+ documented = [p for p in spec.get("params") or [] if p.get("doc")]
222
+ if documented:
223
+ # Continuation lines indented, or a description carrying its own newlines reads as the next parameter's.
224
+ section = "Parameters:\n" + "\n".join(f" {p['name']}: {p['doc'].replace(chr(10), chr(10) + ' ')}" for p in documented)
225
+ call.__doc__ = f"{call.__doc__}\n\n{section}" if call.__doc__ else section
226
+ # A rebind builds NEW callables, so a name already pulled out with `from __dsh__.tools import read` keeps the docstring it was imported with. `__dsh__.tools.read?` is the authoritative view after a schema revision. True of the signature and return annotation too, and deliberate: the shell's namespace is the model's, and a rebind that reached into it could swap a binding under a cell mid-await.
219
227
  # Per-parameter, not one suppress around the whole thing: a single exotic name (`class`, `file-path` — hyphens are routine for MCP tools) used to discard the ENTIRE signature, so `read?` showed `(**kwargs)` while the prompt showed the full parameter list, with no error either way. Anything unrenderable is folded into `**kwargs` so the picture stays honest.
220
228
  params, dropped = [], False
221
229
  for p in spec.get("params") or []:
@@ -238,6 +246,12 @@ def _make_binding(bridge: Bridge, spec):
238
246
  return call
239
247
 
240
248
 
249
+ def bound_tools() -> dict:
250
+ """The calling shell's catalogue, or nothing outside a cell."""
251
+ session = _current_session.get()
252
+ return {} if session is None else session.bindings
253
+
254
+
241
255
  class ToolsModule(types.ModuleType):
242
256
  """`__dsh__.tools` — the bridged tool surface of the CALLING shell.
243
257
 
@@ -245,38 +259,286 @@ class ToolsModule(types.ModuleType):
245
259
 
246
260
  def __init__(self) -> None:
247
261
  super().__init__("__dsh__.tools", "Harness tools, bridged into this session as awaitables.")
262
+ self.__path__ = [] # a package, so `__dsh__.tools.mcp` resolves under it
248
263
  self.ToolCallError = ToolCallError
249
264
 
250
- @staticmethod
251
- def _bindings():
252
- session = _current_session.get()
253
- return {} if session is None else session.bindings
254
-
255
265
  def __getattr__(self, name): # only reached when the attribute is absent
256
266
  if name.startswith("__"):
257
267
  raise AttributeError(name) # import/introspection probing — never answer with a tool
258
- bindings = ToolsModule._bindings()
268
+ bindings = bound_tools()
259
269
  if name in bindings:
260
270
  return bindings[name]
261
- available = ", ".join(sorted(bindings)) or "(none)"
271
+ # The shown listing, not the bound one: a typo used to push the whole flat catalogue into
272
+ # the trajectory at the one moment the model is guaranteed to be reading it.
273
+ available = ", ".join(sorted(listed_tools())) or "(none)"
262
274
  raise AttributeError(f"no such tool: {name!r}. Available: {available}")
263
275
 
264
276
  def __dir__(self):
265
- return sorted(ToolsModule._bindings())
277
+ return sorted(listed_tools())
278
+
279
+ # NOT `listed_tools()`: `__all__` is what `from __dsh__.tools import *` BINDS, and narrowing it
280
+ # left `mcp__gh__ok` undefined in a cell that used to work. Display is `__dir__`'s job.
281
+ #
282
+ # `ToolCallError` is here and in no listing, for the same reason: it is not a tool, so showing it
283
+ # among them would be noise — but the instructions tell the model to catch it, and a cell that
284
+ # reached for `import *` instead of the block's import line lost the failure path it was told to
285
+ # handle, to a `NameError` raised from inside the `except` clause meant to explain the failure.
286
+ @property
287
+ def __all__(self):
288
+ return sorted([*bound_tools(), "ToolCallError"])
289
+
290
+ def __repr__(self) -> str:
291
+ # The cell's trailing expression is echoed back, so ending on `__dsh__.tools` is the
292
+ # cheapest "what do I have" move there is — and it used to re-emit the whole flat catalogue.
293
+ return f"<module '__dsh__.tools': {', '.join(sorted(listed_tools())) or 'no tools bound'}>"
294
+
295
+
296
+ MCP_PREFIX = "mcp__"
297
+
298
+
299
+ def split_mcp(name: str) -> tuple[str, str] | None:
300
+ """`mcp__calendar__list_events` -> `("calendar", "list_events")`.
301
+
302
+ dsh names every MCP tool `mcp__<serverName>__<rawName>`, and a raw name may itself contain
303
+ `__` — `split("__")` would tear such a tool apart and file it under a server that does not
304
+ exist, so only the first two separators are ever consumed.
305
+
306
+ PRESENTATION ONLY. dsh's own naming contract says the public name "is never parsed to recover"
307
+ the raw one, and it is right: a name that needed normalizing, or that ran past 64 characters,
308
+ becomes `<truncated>_<12 hex of sha256>`, and the cut can land anywhere — including before the
309
+ second `__`. That case returns `None` here and the tool simply stays flat, reachable under its
310
+ full public name, which is the only name dispatch ever uses.
311
+ """
312
+ if not name.startswith(MCP_PREFIX):
313
+ return None
314
+ server, sep, raw = name[len(MCP_PREFIX) :].partition("__")
315
+ return (server, raw) if sep and server and raw else None
316
+
317
+
318
+ def servable(name: str) -> tuple[str, str] | None:
319
+ """`(server, tool)` when `mcp.<server>.<tool>` can actually ANSWER, else `None`.
320
+
321
+ Not `split_mcp` alone. Two ways a name splits cleanly and the grouping still cannot serve it:
322
+ `McpModule.__getattr__` refuses a true dunder — import and introspection probing (`__path__`,
323
+ `__all__`, `__spec__`) all wear that shape — and dsh hashes a public name that needed
324
+ normalising, where the cut can land before the second `__` and `split_mcp` returns `None`.
325
+
326
+ The one predicate for the whole file, because the listing and the lookup have to agree: hiding
327
+ a flat name on the strength of the split alone left `mcp__gh____weird__` in no listing the
328
+ model ever reads, while `mcp.gh.__weird__` raised `AttributeError`.
329
+ """
330
+ parts = split_mcp(name)
331
+ if parts is None or (parts[1].startswith("__") and parts[1].endswith("__")):
332
+ return None
333
+ return parts
334
+
335
+
336
+ def listed_tools() -> dict:
337
+ """What the model is SHOWN: `dir()`, `repr()`, and the `Available:` list of a failed lookup.
338
+
339
+ Not `__all__` — that is the star-import BINDING contract, and narrowing it unbound every flat
340
+ name from `from __dsh__.tools import *`, which is a regression on running code rather than a
341
+ quieter listing.
342
+
343
+ Every flat `mcp__server__tool` name stays bound; showing them contradicted the prompt block,
344
+ which stopped printing them. Of 103 entries 86 were flat MCP names, and a model asked to
345
+ introspect its own tools filtered them out by hand. A name the grouping cannot serve stays
346
+ shown — it is then the only name that works.
347
+ """
348
+ return {name: call for name, call in bound_tools().items() if servable(name) is None}
349
+
350
+
351
+ def spellable(raw: str) -> str | None:
352
+ """The name Python can take for `raw`, or `None` when `raw` already is one.
353
+
354
+ dsh normalises a tool name over `[A-Za-z0-9_-]`, so `-` survives — and `-` is legal in no Python
355
+ identifier. `API-patch-block-children` was therefore reachable only as
356
+ `getattr(mcp.notion, "API-patch-block-children")`, at every call site, with no signature in the
357
+ prompt block to go with it; it was the single most-dispatched tool of a 20-task benchmark run,
358
+ and 4.0% of all dispatches went to twelve tools shaped like it.
359
+
360
+ Only `-` is folded. A keyword (`class`) or a digit-leading name (`123tool`) is unspellable for
361
+ reasons no substitution fixes, and inventing a spelling for those would mean inventing a name.
362
+ """
363
+ folded = raw.replace("-", "_")
364
+ # `isidentifier()` alone is not the host's rule: it accepts keywords, and the block's
365
+ # `isUsableName` refuses them. A raw `-` folds to `_`, a SOFT keyword — the kernel would have
366
+ # bound and listed `mcp.srv._` while the block routed the same tool to `getattr(mcp.srv, "-")`.
367
+ # The two halves have to agree on what is spellable; the suite checks that they do.
368
+ if folded == raw or not folded.isidentifier() or keyword.iskeyword(folded) or keyword.issoftkeyword(folded):
369
+ return None
370
+ return folded
371
+
372
+
373
+ def mcp_servers(bindings: dict) -> dict[str, dict]:
374
+ """Group the flat `mcp__server__tool` bindings into `{server: {tool: call}}`.
375
+
376
+ The flat names stay bound too. They are what dsh dispatches on and what an older session may
377
+ already have imported; dropping them to tidy the surface would break a cell mid-conversation.
378
+
379
+ A hyphenated raw name is reachable under BOTH spellings: its own, so an existing
380
+ `getattr(mcp.srv, "a-b")` keeps working, and its folded alias, which is what the listings show.
381
+ The alias never displaces a real tool — a server exposing both `a-b` and `a_b` keeps `a_b`
382
+ meaning `a_b`, and `a-b` stays getattr-only rather than quietly answering for its neighbour.
383
+ """
384
+ servers: dict[str, dict] = {}
385
+ for name, call in bindings.items():
386
+ if (parts := split_mcp(name)) is not None:
387
+ servers.setdefault(parts[0], {})[parts[1]] = call
388
+ for tools in servers.values():
389
+ alias(tools)
390
+ alias(servers)
391
+ return servers
392
+
393
+
394
+ def alias(members: dict) -> None:
395
+ """Add each hyphenated key's folded spelling, in place, without displacing a real one.
396
+
397
+ `setdefault`, so a name that already exists keeps its own value: a server exposing `a-b` beside
398
+ `a_b` gets no alias for `a-b`, and both stay reachable under their own spellings.
399
+ """
400
+ for raw in [name for name in members if spellable(name) is not None]:
401
+ members.setdefault(spellable(raw), members[raw])
402
+
403
+
404
+ def canonical(server: str, servers: dict) -> str:
405
+ """The module name for `server`: its fold, unless another server already owns that spelling.
406
+
407
+ Folding unconditionally made `a-b` and `a_b` ONE module — `mcp_server_module` is keyed by name,
408
+ so both root entries resolved to the fold's module and the hyphenated server's tools became
409
+ unreachable, through either spelling. This is the same rule the tool level applies by identity,
410
+ stated directly because a server's value here is a module built on demand, not a shared callable.
411
+ """
412
+ folded = spellable(server)
413
+ return server if folded is None or servers.get(folded) is not servers.get(server) else folded
414
+
415
+
416
+ def listed(members: dict) -> dict:
417
+ """What a listing SHOWS: the folded spelling stands in for the name it was folded from.
418
+
419
+ The same split as `bound_tools` / `listed_tools` one level up, for the same reason — showing
420
+ both spellings would report twice as many tools as the server has, and showing only the raw one
421
+ would name something the model cannot type.
422
+ """
423
+ # Identity, not membership: `alias` maps a fold to the SAME callable, so `members[fold] is value`
424
+ # is what distinguishes a name's own alias from a DIFFERENT tool that happens to own that
425
+ # spelling. A server exposing both `a-b` and `a_b` keeps both listed — dropping the hyphenated
426
+ # one there would hide a real tool behind its neighbour.
427
+ return {name: value for name, value in members.items() if (fold := spellable(name)) is None or members.get(fold) is not value}
428
+
429
+
430
+ MCP_MODULE = "__dsh__.tools.mcp"
431
+
432
+
433
+ def mcp_members(module_name: str) -> dict:
434
+ """One level of the `mcp` tree, resolved against the catalogue in force NOW.
435
+
436
+ Deliberately not a method: a non-dunder attribute on the class would shadow a tool or a server
437
+ of that name, and a raw MCP name is the server's to choose — `_private` is a legal one.
438
+ """
439
+ servers = mcp_servers({name: call for name, call in bound_tools().items() if servable(name) is not None})
440
+ if module_name == MCP_MODULE:
441
+ # Keyed by the FOLDED name so a hyphenated server and its fold resolve to one module object,
442
+ # not two: `listed` tells an alias from a real neighbour by identity, and two objects would
443
+ # leave both spellings in `dir(mcp)`. Only the fold reaches `sys.modules`, which is the only
444
+ # spelling `from __dsh__.tools.mcp.<server> import …` could have used anyway.
445
+ return {server: mcp_server_module(canonical(server, servers)) for server in servers}
446
+ # `removeprefix`, not `rpartition`: a raw server name is not guaranteed dot-free, and taking
447
+ # the last segment of one would look up a server that does not exist and resolve it empty.
448
+ return servers.get(module_name.removeprefix(f"{MCP_MODULE}."), {})
449
+
450
+
451
+ class McpModule(types.ModuleType):
452
+ """`__dsh__.tools.mcp`, and one of these per server under it.
453
+
454
+ Real modules, so every import form the model might reach for resolves the way it does for
455
+ `__dsh__.tools` itself — including `from __dsh__.tools.mcp.calendar import list_events`, which
456
+ needs a `sys.modules` entry of its own (the shallower forms can be served by `__getattr__`,
457
+ that one cannot).
458
+
459
+ Their CONTENTS still come from the ContextVar, because `sys.modules` is process-global while
460
+ the catalogue is per shell — and because a restriction or a reconnecting server moves tools in
461
+ and out between cells. A name pulled OUT with `from ... import` is a snapshot, as it is for any
462
+ Python import; the module itself stays live.
463
+ """
464
+
465
+ def __getattr__(self, name):
466
+ # True dunders only. The guard is here to refuse import and introspection probing
467
+ # (`__path__`, `__all__`, `__spec__`, `__deepcopy__`), which always ends in `__` too —
468
+ # and a leading-`__` raw tool name is the server's to choose, so `startswith` alone
469
+ # advertised `mcp.<server>.__weird` in `dir()` and then refused the call.
470
+ if name.startswith("__") and name.endswith("__"):
471
+ raise AttributeError(name)
472
+ members = mcp_members(self.__name__)
473
+ if name not in members:
474
+ available = ", ".join(sorted(listed(members))) or "(none)"
475
+ raise AttributeError(f"no such tool: {self.__name__.removeprefix('__dsh__.tools.')}.{name}. Available: {available}")
476
+ return members[name]
477
+
478
+ def __setattr__(self, name, value):
479
+ # One module per server for the whole PROCESS, so a write here would shadow that name for
480
+ # every other agent in it — permanently, and invisibly to `dir()`, which keeps reporting
481
+ # the tool it no longer reaches. The old per-call `Namespace` made this a local mistake.
482
+ if not (name.startswith("__") and name.endswith("__")):
483
+ raise AttributeError(f"{self.__name__.removeprefix('__dsh__.tools.')} belongs to the harness and is shared by every agent in this process — bind your own name instead of writing to it")
484
+ super().__setattr__(name, value)
485
+
486
+ def __dir__(self):
487
+ return sorted(listed(mcp_members(self.__name__)))
266
488
 
489
+ # Star-import reads `__all__` (or `vars()`), never `__getattr__` or `__dir__`, and nothing ever
490
+ # lands in these modules' `__dict__` — so without this `from __dsh__.tools.mcp.x import *`
491
+ # succeeded and bound nothing. `ToolsModule` carries the same property for the same reason.
267
492
  @property
268
493
  def __all__(self):
269
- return sorted(ToolsModule._bindings())
494
+ return sorted(listed(mcp_members(self.__name__)))
270
495
 
271
496
  def __repr__(self) -> str:
272
- return f"<module '__dsh__.tools': {', '.join(sorted(ToolsModule._bindings())) or 'no tools bound'}>"
497
+ members = listed(mcp_members(self.__name__))
498
+ return f"<module {self.__name__!r}: {', '.join(sorted(members)) or 'empty'}>"
499
+
500
+
501
+ MCP_ROOT = McpModule(MCP_MODULE, "MCP tools, one module per server.")
502
+ MCP_ROOT.__path__ = [] # a package, like `__dsh__` and `__dsh__.tools`, so its server modules resolve
503
+ sys.modules[MCP_MODULE] = MCP_ROOT
504
+
505
+
506
+ def mcp_server_module(server: str) -> McpModule:
507
+ """The one module for `server`, made on first ask.
508
+
509
+ The single construction site, so a module that went missing — a cell can `del sys.modules[…]`
510
+ — comes back instead of leaving `dir(mcp)` raising `KeyError` for the rest of the session.
511
+ """
512
+ name = f"{MCP_MODULE}.{server}"
513
+ if not isinstance(module := sys.modules.get(name), McpModule):
514
+ module = McpModule(name, f"Tools bridged from the `{server}` MCP server.")
515
+ sys.modules[name] = module
516
+ return module
517
+
518
+
519
+ def install_mcp_modules(bindings: dict) -> None:
520
+ """Register a module per visible MCP server.
521
+
522
+ Eager, because the deep import form never reaches `mcp_members`: `from __dsh__.tools.mcp.x
523
+ import y` is resolved by the import machinery against `sys.modules`, before any attribute
524
+ lookup happens.
525
+
526
+ `sys.modules` only ever gains entries: a server another shell can see costs this one an unused
527
+ module, while removing it would break an import that shell is mid-conversation with. What a
528
+ shell can actually reach is decided by `mcp_members`, not by what is registered.
529
+ """
530
+ for server in mcp_servers(bindings):
531
+ mcp_server_module(server)
273
532
 
274
533
 
275
534
  def build_bindings(bridge: Bridge, specs) -> dict:
276
535
  """Project one agent's visible tools into awaitables for its shell."""
277
- # A tool named `ToolCallError` or `_bindings` would be shadowed by the module's own attributes, and one named `_rebind` used to overwrite a bound method outright. dsh's own SDK renderer refuses `_`-leading tool names for exactly this collision class.
278
- reserved = set(vars(ToolsModule)) | set(vars(types.ModuleType)) | {"ToolCallError"}
279
- return {spec["name"]: _make_binding(bridge, spec) for spec in specs if not spec["name"].startswith("_") and spec["name"] not in reserved}
536
+ # A tool named `ToolCallError` would be shadowed by the module's own attributes, and one named `_rebind` used to overwrite a bound method outright. dsh's own SDK renderer refuses `_`-leading tool names for exactly this collision class.
537
+ # `mcp` joins them, and unconditionally: it used to be bound and then overwritten by the namespace, so the tool was uncallable anyway — but only when an MCP server happened to be mounted. A name that means the namespace in one catalogue and a tool in the next is worse than one that always means the same thing.
538
+ reserved = set(vars(ToolsModule)) | set(vars(types.ModuleType)) | {"ToolCallError", "mcp"}
539
+ flat = {spec["name"]: _make_binding(bridge, spec) for spec in specs if not spec["name"].startswith("_") and spec["name"] not in reserved}
540
+ # `mcp` only when something is under it: an empty namespace in `dir()` reads as a broken mount.
541
+ return {**flat, "mcp": MCP_ROOT} if mcp_servers(flat) else flat
280
542
 
281
543
 
282
544
  def install_bridge_modules() -> ToolsModule:
@@ -386,7 +648,7 @@ class Session:
386
648
  history = Config()
387
649
  history.HistoryAccessor.hist_file = ":memory:"
388
650
  self.shell = InteractiveShell(user_ns=namespace, config=history)
389
- self.bindings = build_bindings(bridge, specs)
651
+ self.rebind(bridge, specs)
390
652
  self.sinks: list = [None, None] # the Capped buffers of the cell in flight
391
653
  install_bridge_modules()
392
654
 
@@ -434,7 +696,14 @@ class Session:
434
696
  self.sinks[:] = previous
435
697
 
436
698
  def rebind(self, bridge: Bridge, specs) -> None:
699
+ """Swap in a catalogue — a restriction or a reconnecting server moves tools between cells.
700
+
701
+ The one place the `sys.modules` registration lives, so `build_bindings` stays the pure
702
+ projection its name promises and no caller can produce bindings the import machinery
703
+ cannot follow.
704
+ """
437
705
  self.bindings = build_bindings(bridge, specs)
706
+ install_mcp_modules(self.bindings)
438
707
 
439
708
  def format_exc(self) -> str:
440
709
  """IPython's own traceback, rendered without ANSI colors."""