dsh-py-codeact 0.2.1 → 0.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +38 -13
  2. package/lib/index.js +140 -17
  3. package/package.json +1 -1
  4. package/py/kernel.py +201 -15
package/README.md CHANGED
@@ -52,7 +52,7 @@ A dunder-named package reads as harness-owned, survives `%reset`, and leaves the
52
52
 
53
53
  ```
54
54
  In [1]: read?
55
- Signature: read(*, file_path: 'str', offset: 'int' = Ellipsis, limit: 'int' = Ellipsis) -> 'str'
55
+ Signature: read(*, file_path: 'str', offset: 'int' = ..., limit: 'int' = ...) -> 'str'
56
56
  Docstring: Read a file from the workspace. Results include line numbers…
57
57
  ```
58
58
 
@@ -60,36 +60,61 @@ That is why the prompt block carries signatures only — the descriptions are on
60
60
 
61
61
  The **return** type comes from the tool's own `output` schema, by way of `ctx.tools.sdkSchemas(scope)` — the projection that carries it. It is the one annotation the model cannot recover by reading harder: a wrong argument fails loudly at the call, while an unknown return shape is only discoverable by calling once and printing the result, which costs a whole turn per tool. A tool that declares no output schema still renders `Any`; claiming a type nobody declared would be worse than admitting ignorance.
62
62
 
63
- Each object in that schema whose keys — and whose own generated class name — Python can take is declared as a named `TypedDict` above the signatures, because for an MCP tool the flat form is not merely vague — it is the whole message. dsh's MCP client resolves a call to the **envelope** `{ content, structuredContent }` and declares exactly that as the tool's output, so a model told only `dict[str, Any]` is not told the envelope exists: it reaches for the payload directly, gets nothing, and spends the turn printing the result to find the wrapper — the exact cost this annotation is here to remove.
63
+ Each object in that schema whose keys — and whose own generated class name — Python can take is declared as a named `TypedDict` above the signatures. For a bridged MCP tool the schema described here is the payload, not the transport wrapper see below; `dict[str, Any]` would say a dict arrives without saying which keys, which is the one thing a return annotation exists to say.
64
64
 
65
65
  ```python
66
- class McpCalendarListEventsOutputStructuredContent(TypedDict):
67
- result: str
68
-
69
66
  class McpCalendarListEventsOutput(TypedDict):
70
- content: list[Any]
71
- structuredContent: McpCalendarListEventsOutputStructuredContent
67
+ result: str
72
68
 
73
- async def mcp__calendar__list_events(*, calendar_id: str) -> McpCalendarListEventsOutput: ...
69
+ class _McpCalendar(Protocol):
70
+ async def list_events(self, *, calendar_id: str) -> McpCalendarListEventsOutput: ...
74
71
  ```
75
72
 
76
73
  `jsonSchemaToPy` cannot do this and says so — it is context-free, and naming a `TypedDict` needs the render context `renderToolsSdkPy` supplies. That renderer is not reusable here: it emits a whole document in Code Mode's own `tools.name(args)` contract. So only the object and array branches are handled locally, every leaf still going through `jsonSchemaToPy` — a place to hang the names, not a second JSON-Schema mapper. The binding set is resent with every cell, because restrictions and mid-conversation tool changes can move a tool in or out between calls.
77
74
 
75
+ ### The MCP wrapper does not reach the cell
76
+
77
+ dsh's MCP client resolves a call to `{ content, structuredContent? }` — the protocol's block array plus the server's own payload — and declares exactly that as the tool's output. That wrapper is transport, not API: `content`'s text duplicates the payload, and an image inside it is re-attached to the conversation separately, so a cell can do nothing with it. Handed the wrapper, a model writes `r["structuredContent"]["result"]` — and spends a call discovering it has to.
78
+
79
+ So the bridge unwraps, and the signature describes what the cell actually receives:
80
+
81
+ ```python
82
+ async def mcp__review__search(*, q: str) -> McpReviewSearchOutput: ... # the payload, not the wrapper
83
+ async def mcp__email__ping() -> str: ... # no declared payload: the text blocks, joined
84
+ ```
85
+
86
+ A server that declares no output schema is the common case in the wild, and its result really is just text — one text block, in every one of the 4142 MCP results measured across six trials, which is why this is a `str` and not a `list[str]` that would cost an `r[0]` at every call site to preserve a boundary that never appears. A result carrying no text block at all resolves to the empty string: an image there is re-attached to the conversation and read on the next step, so there was never anything for the cell to receive. One divergence is possible and belongs to the server: a tool that declares an output schema but omits `structuredContent` on some call resolves to that call's text, where the signature promised the payload type.
87
+
88
+ Unwrapping keys off the tool's declared output schema, not off the value that comes back: a tool whose own payload happens to be `{content: [...]}` is not wrapping anything, and replacing its value with joined text would change what that tool returns with nothing to say so. This deliberately differs from Code Mode, which hands the whole envelope to `tools.name(args)`.
89
+
78
90
  ### MCP tools work, with nothing special
79
91
 
80
- MCP servers register into the same `ctx.tools` registry as everything else, so they arrive in `__dsh__.tools` like any other binding and dispatch through the same pipeline:
92
+ MCP servers register into the same `ctx.tools` registry as everything else, so they arrive in `__dsh__.tools` like any other binding and dispatch through the same pipeline. They are *presented* grouped, under one `mcp`:
81
93
 
82
94
  ```python
83
- from __dsh__.tools import mcp__gh__github_graphql as gh
95
+ from __dsh__.tools import mcp
84
96
 
85
- data = await gh(query="{ viewer { login } }")
97
+ data = await mcp.gh.github_graphql(query="{ viewer { login } }")
86
98
  ```
87
99
 
100
+ dsh names them `mcp__<server>__<rawName>`, and with a hundred mounted that import line was most of the prompt block while every call site respelled its server. The flat names stay bound — `mcp` is how they are shown, not what they are — so a cell written before this still runs, `import *` still binds them, and only the listings drop them. A name the grouping cannot serve is still shown: dsh hashes a public name that needed normalising and the cut can land before the second `__`, and a raw name that is a true dunder is refused by `__getattr__`; either way the flat name is the only one that works. They are kept out of `dir(__dsh__.tools)` for the same reason the block stopped printing them; a name the grouping cannot reach (dsh hashes a public name that needed normalising, and the cut can land before the second `__`) stays listed, because `mcp` is not another way to say it. The block declares the grouping as `Protocol` stubs, one per server, which is the shape dsh's own SDK renderer uses for the same problem.
101
+
102
+ It is a real package, so a server can be imported as a module — which reads better than `mcp.` at every call site when a cell leans on one server:
103
+
104
+ ```python
105
+ from __dsh__.tools.mcp.calendar import list_events, create_event
106
+ from __dsh__.tools.mcp import calendar # or the server itself
107
+ ```
108
+
109
+ The deep form is why each server gets a `sys.modules` entry of its own: `__getattr__` can serve `from __dsh__.tools.mcp import calendar`, but not `from __dsh__.tools.mcp.calendar import list_events` — the import machinery looks that one up as a module. No meta path finder is needed; registration is enough.
110
+
111
+ `mcp` and its server modules are live views of the catalogue, not snapshots of the cell they were imported in: a restriction or a reconnecting server moves tools in and out between calls, and unlike a single tool the model has no reason to ever import the namespace twice. (A name pulled OUT with `from ... import` is a snapshot, as it is for any Python import.) The name is reserved — a native tool called `mcp` is not bound.
112
+
88
113
  dsh's MCP client is explicitly aware of this route — its canonical value "retains the complete JSON MCP blocks and optional structured content for programmatic and Code Mode callers" — and the sub-call logs a `SUBTOOL` row like any other.
89
114
 
90
115
  Worth contrasting: Anthropic's server-side [programmatic tool calling](https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling) is *not* compatible with MCP tools. Owning the bridge host-side is what buys this.
91
116
 
92
- (A tool whose name is not a valid Python identifier cannot be `import`ed, but is still reachable as `getattr(__dsh__.tools, "odd-name")`. dsh's MCP naming `mcp__<server>__<tool>` always is one.)
117
+ (A tool whose name is not a valid Python identifier cannot be `import`ed, but is still reachable with `getattr` — `getattr(__dsh__.tools, "odd-name")`, or one level deeper for an MCP tool, `getattr(mcp.notion, "API-patch-block-children")`. A raw MCP name is a routine place to find a hyphen.)
93
118
 
94
119
  ## Exclusive mode
95
120
 
@@ -163,7 +188,7 @@ Sub-dispatches carry the outer execution's `parent` token, so they re-enter the
163
188
  - **Redundant-import hints.** When an `import` rebinds a name to the object it already held, the result carries `` `json` is already imported in this session — no need to re-import it. `` A model driving a persistent REPL re-imports constantly; telling it is cheaper than letting it burn a line every cell. (Implemented with a `dict` subclass that watches top-level `STORE_NAME` and checks the preceding opcode was `IMPORT_NAME`/`IMPORT_FROM`, so `x = x` does not trip it.)
164
189
  - **Readable reprs.** `objprint` + IPython's `pretty` for objects whose own `__repr__` is `object.__repr__` — an agent reading values needs structure, not `<Foo object at 0x…>`.
165
190
  - **Tagged observations.** `<stdout>`, `<stderr>`, `<return>`, `<traceback>`, `<note>` — with four things possibly present at once, the model needs to know which is which. A plain successful value stays bare.
166
- - **Introspection over prompt text.** `read?` for one tool's full description, `dir(__dsh__.tools)` for the whole list, `%whos` for its own bindings.
191
+ - **Introspection over prompt text.** `read?` for one tool's full description, `dir(__dsh__.tools)` for the list — `mcp` rather than the hundred flat names under it, matching what the block printed — then `dir(mcp)` and `dir(mcp.<server>)` for those, and `%whos` for its own bindings. `__all__` is left alone: it is what `import *` binds, not what the model is shown. What it introspects matches what the block showed it: the listing carries `mcp` rather than the hundred flat names under it, and an optional parameter renders `= ...` there as it does here — `inspect.signature` uses `repr`, and `repr(...)` is `Ellipsis`, which is what `read?` used to say.
167
192
 
168
193
  ## Cancellation
169
194
 
package/lib/index.js CHANGED
@@ -62,6 +62,32 @@ Your action space is Python. Each call runs one cell in a **persistent IPython s
62
62
 
63
63
  The available tools:`
64
64
 
65
+ /**
66
+ * The wrapper dsh's MCP client puts around every result: `{ content, structuredContent? }`, with
67
+ * `content` the protocol's block array and `structuredContent` the server's own payload. Detected
68
+ * by shape rather than by an `mcp__` name, because the shape IS the declared contract — the client
69
+ * builds exactly this schema, `additionalProperties: false` and all.
70
+ *
71
+ * @returns the payload's schema when the server declares one, `null` when this is not an envelope,
72
+ * and `undefined` for an envelope carrying only text.
73
+ */
74
+ function mcpPayloadSchema(schema) {
75
+ if (schema?.type !== 'object' || schema.additionalProperties !== false) return null
76
+ const keys = Object.keys(schema.properties ?? {})
77
+ if (keys.length !== 2 || !keys.includes('content') || !keys.includes('structuredContent')) return null
78
+ if (schema.properties.content?.type !== 'array') return null
79
+ return (schema.required ?? []).includes('structuredContent') ? schema.properties.structuredContent : undefined
80
+ }
81
+
82
+ /** @internal exported for the suite. The same wrapper at run time. A value is only unwrapped when it carries nothing but the wrapper's own keys. */
83
+ export function mcpPayload(value) {
84
+ if (value === null || typeof value !== 'object' || Array.isArray(value)) return undefined
85
+ const keys = Object.keys(value)
86
+ if (!keys.includes('content') || !Array.isArray(value.content)) return undefined
87
+ if (keys.some((k) => k !== 'content' && k !== 'structuredContent')) return undefined
88
+ return 'structuredContent' in value ? { value: value.structuredContent } : { value: contentText(value.content) }
89
+ }
90
+
65
91
  /**
66
92
  * Project one tool schema onto the wire spec the kernel builds a binding from: dsh's description becomes the function's docstring, its parameters become a keyword-only signature. Annotations are rendered here with dsh's own `jsonSchemaToPy`, so the kernel needs no second JSON-Schema mapper.
67
93
  *
@@ -75,7 +101,10 @@ function toolSpec(schema, declarations) {
75
101
  name: schema.name,
76
102
  doc: schema.description,
77
103
  // `output` is absent only if the caller passed a `schemas()` projection; `sdkSchemas()` always carries it.
78
- returns: schema.output === undefined ? 'Any' : declareType(schema.output, `${pascal(schema.name)}Output`, declarations),
104
+ // An MCP tool is annotated by what the CELL receives, which is the payload the bridge unwraps
105
+ // to — never the transport wrapper. Describing the wrapper is accurate and useless: the model
106
+ // then hand-writes `r["structuredContent"]["result"]`, and calls once just to learn that.
107
+ returns: schema.output === undefined ? 'Any' : mcpReturn(schema, declarations),
79
108
  params: Object.entries(properties).map(([name, node]) => ({
80
109
  name,
81
110
  type: jsonSchemaToPy(node),
@@ -95,11 +124,41 @@ function toolSpec(schema, declarations) {
95
124
  */
96
125
  function toolSpecs(schemas) {
97
126
  const declarations = new Map()
98
- return { specs: [...schemas].sort(byName).map((schema) => toolSpec(schema, declarations)), declarations }
127
+ // The kernel reserves `mcp` for the namespace, so a tool of that name is never bound. Dropped
128
+ // here rather than at the render, which is downstream of the declarations: one with an object
129
+ // output still had its `TypedDict` emitted, referenced by nothing.
130
+ const sorted = [...schemas].filter((schema) => schema.name !== 'mcp').sort(byName)
131
+ // Which tools DECLARE the wrapper. Unwrapping keys off this rather than the returned value:
132
+ // a tool whose own payload happens to be `{content: [...]}` is not wrapping anything, and
133
+ // replacing its value with the joined text would change what it returns with nothing to say so.
134
+ const envelopes = new Set(sorted.filter((schema) => mcpPayloadSchema(schema.output) !== null).map((schema) => schema.name))
135
+ return { specs: sorted.map((schema) => toolSpec(schema, declarations)), declarations, envelopes }
136
+ }
137
+
138
+ /** The annotation for one tool's output: an MCP payload if it is an envelope, else the schema as declared. */
139
+ function mcpReturn(schema, declarations) {
140
+ const payload = mcpPayloadSchema(schema.output)
141
+ if (payload === null) return declareType(schema.output, `${pascal(schema.name)}Output`, declarations)
142
+ // An envelope with no declared payload resolves to the text blocks joined, so it really is a `str`.
143
+ return payload === undefined ? 'str' : declareType(payload, `${pascal(schema.name)}Output`, declarations)
144
+ }
145
+
146
+ /**
147
+ * `mcp__calendar__list_events` -> `{ server: 'calendar', tool: 'list_events' }`, matching the
148
+ * kernel's own split. Only the first two separators are consumed: dsh's name is
149
+ * `mcp__<serverName>__<rawName>` and a raw name may itself contain `__`.
150
+ */
151
+ function splitMcp(name) {
152
+ if (!name.startsWith('mcp__')) return null
153
+ const rest = name.slice(5)
154
+ const at = rest.indexOf('__')
155
+ if (at <= 0 || at + 2 >= rest.length) return null
156
+ return { server: rest.slice(0, at), tool: rest.slice(at + 2) }
99
157
  }
100
158
 
101
159
  /** Deterministic (lexicographic) order, so an unchanged tool set renders byte-identically. */
102
- const byName = (a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)
160
+ const by = (key) => (a, b) => (key(a) < key(b) ? -1 : key(a) > key(b) ? 1 : 0)
161
+ const byName = by((spec) => spec.name)
103
162
 
104
163
  /** The kernel's own wording for a shell whose previous cell has not finished. Matched, not parsed: it is the one failure that leaves the bindings uninstalled. */
105
164
  const KERNEL_BUSY = 'kernel busy: a previous cell is still running'
@@ -195,45 +254,105 @@ export function renderToolsSection(schemas) {
195
254
  // MCP tools are named `mcp__<server>__<rawName>` over a `[A-Za-z0-9_-]` alphabet, so hyphens are routine — and legal nowhere in `import` or `def`. One such tool used to make the ENTIRE block a SyntaxError, so not a single line in it could be copied, with nothing saying why.
196
255
  //
197
256
  // A keyword is the same failure wearing a legal shape: `class` and `None` match the identifier pattern and then break `import` and `def` just as hard. `_` and the soft keywords (`match`, `case`, `type`) are in the list because they are legal identifiers everywhere EXCEPT where this block puts them.
198
- const importable = specs.filter((spec) => isUsableName(spec.name))
199
- const awkward = specs.filter((spec) => !isUsableName(spec.name))
200
- const signature = (spec) => {
257
+ // MCP tools are grouped under one `mcp` binding instead of being listed individually. With a
258
+ // hundred of them the import line was most of this block, and `mcp__calendar__list_events`
259
+ // carried its server in the name at every call site; `mcp.calendar.list_events` says the same
260
+ // thing once. The flat names stay bound — this is how they are PRESENTED, not what they are.
261
+ const grouped = new Map()
262
+ const plain = []
263
+ for (const spec of specs) {
264
+ const parts = splitMcp(spec.name)
265
+ if (parts === null) { plain.push(spec); continue }
266
+ if (!grouped.has(parts.server)) grouped.set(parts.server, [])
267
+ grouped.get(parts.server).push({ ...spec, tool: parts.tool, server: parts.server })
268
+ }
269
+ const importable = plain.filter((spec) => isUsableName(spec.name))
270
+ const awkward = plain.filter((spec) => !isUsableName(spec.name))
271
+ // A server or tool whose name Python cannot take keeps its `getattr` route, one level deeper.
272
+ const servers = [...grouped].filter(([server]) => isUsableName(server)).sort(by(([server]) => server))
273
+ const oddMcp = [...grouped].flatMap(([server, tools]) =>
274
+ (isUsableName(server) ? tools.filter((t) => !isUsableName(t.tool)) : tools).map((t) => ({ server, tool: t.tool, usableServer: isUsableName(server) })))
275
+ // `self` is a parameter here rather than a splice into the finished line, because a Protocol
276
+ // method needs it in front of BOTH shapes below — and `async def f(*, ) -> T` is a SyntaxError:
277
+ // a `*` needs at least one name after it, and a parameterless tool takes no keywords at all.
278
+ // `head` has NO default, so this can never be handed to `.map` — whose second argument is the
279
+ // index, which a default would silently accept and which is not iterable.
280
+ const signature = (spec, head) => {
201
281
  // The same rule applies one level down, and used to not be applied at all: `file-path` is routine for MCP tools, and one such PARAMETER made every other tool's signature unusable too. The binding still takes it — the kernel folds unnameable parameters into `**kwargs` — so the tool stays importable and only its signature goes vague; `name?` still shows the real one.
202
- if (!spec.params.every((p) => isUsableName(p.name))) return `async def ${spec.name}(**kwargs: Any) -> ${spec.returns}: ... # parameter names are not all valid Python; see ${spec.name}?`
282
+ if (!spec.params.every((p) => isUsableName(p.name))) return `async def ${spec.name}(${[...head, '**kwargs: Any'].join(', ')}) -> ${spec.returns}: ... # parameter names are not all valid Python; see ${spec.name}?`
203
283
  const fields = spec.params.map((p) => (p.required ? `${p.name}: ${p.type}` : `${p.name}: ${p.type} = ...`))
204
- // `async def f(*, ) -> T` is a SyntaxError: a `*` needs at least one name after it, and a parameterless tool takes no keywords at all.
205
- return `async def ${spec.name}(${fields.length === 0 ? '' : `*, ${fields.join(', ')}`}) -> ${spec.returns}: ...`
284
+ return `async def ${spec.name}(${[...head, ...(fields.length === 0 ? [] : [`*, ${fields.join(', ')}`])].join(', ')}) -> ${spec.returns}: ...`
206
285
  }
207
286
  // `ToolCallError` leads the import because the instructions tell the model to catch it; without it here the natural `except ToolCallError` NameErrors on the failure path, masking the tool failure it was meant to handle. It also keeps the line valid when no tool is importable.
208
287
  // The declarations are stubs, not runtime objects — nothing constructs them — so the import line
209
288
  // lists exactly what the render used. `Any` has always been reachable from a signature
210
289
  // (`**kwargs: Any`, and any degraded annotation) and was never imported at all.
211
- const signatures = importable.map(signature)
290
+ const signatures = importable.map((spec) => signature(spec, []))
291
+ // A Protocol, not a class of `staticmethod`s: `mcp` is an INSTANCE, so `mcp.calendar.list_events(...)`
292
+ // binds `self` on its own and the stub says exactly what the call site does. It is also the shape
293
+ // dsh's own SDK renderer uses for the same problem.
294
+ const method = (spec) => ` ${signature({ ...spec, name: spec.tool }, ['self'])}`
295
+ const protocols = servers.flatMap(([server, tools]) => {
296
+ const body = tools.filter((t) => isUsableName(t.tool)).sort(by((t) => t.tool)).map(method)
297
+ // A server whose every tool name Python refuses would otherwise emit a class with an empty
298
+ // body — a SyntaxError that takes the whole block with it. The attribute still has to exist:
299
+ // the `getattr` line below reaches its tools through it.
300
+ return ['', `class _Mcp${pascal(server)}(Protocol):`, ...(body.length === 0 ? [' ...'] : body)]
301
+ })
302
+ const mcpBlock = servers.length === 0 ? [] : [
303
+ ...protocols,
304
+ '',
305
+ 'class _Mcp(Protocol):',
306
+ ...servers.map(([server]) => ` ${server}: _Mcp${pascal(server)}`),
307
+ '',
308
+ 'mcp: _Mcp',
309
+ ]
212
310
  const classes = [...declarations].flatMap(([name, body]) => ['', `class ${name}(TypedDict):`, ...body.split('\n')])
213
311
  const typing = [
214
312
  // Whole identifiers: `AnyReportOutput` is a class name, not a use of `Any`, and a bare
215
313
  // `includes` imports a symbol a tool can spell into existence without ever needing it.
216
- signatures.concat(classes).some((line) => /\bAny\b/.test(line)) && 'Any',
314
+ signatures.concat(classes, mcpBlock).some((line) => /\bAny\b/.test(line)) && 'Any',
217
315
  classes.some((line) => line.includes('NotRequired[')) && 'NotRequired',
218
316
  declarations.size > 0 && 'TypedDict',
219
317
  ].filter(Boolean)
318
+ const protocolImport = servers.length === 0 ? [] : ['Protocol']
220
319
  const lines = [
221
320
  INSTRUCTIONS,
222
321
  '',
223
322
  renderEnvironment(),
224
323
  '',
225
324
  '```python',
226
- ...(typing.length === 0 ? [] : [`from typing import ${typing.join(', ')}`]),
227
- `from __dsh__.tools import ToolCallError${importable.map((spec) => `, ${spec.name}`).join('')}`,
325
+ ...(typing.concat(protocolImport).length === 0 ? [] : [`from typing import ${typing.concat(protocolImport).sort().join(', ')}`]),
326
+ // `grouped`, not `servers`: a server whose own name Python refuses has no Protocol stub, but its
327
+ // tools are still reached through `mcp` — the `getattr` line below names it. Keyed on the stubs,
328
+ // a catalogue of nothing but such servers advertised `getattr(getattr(mcp, …))` without ever
329
+ // importing `mcp`. The kernel binds it whenever an MCP tool exists, which is this condition.
330
+ `from __dsh__.tools import ToolCallError${grouped.size === 0 ? '' : ', mcp'}${importable.map((spec) => `, ${spec.name}`).join('')}`,
228
331
  ...classes,
332
+ ...mcpBlock,
229
333
  '',
230
334
  ...signatures,
231
335
  '```',
232
336
  ]
337
+ if (oddMcp.length > 0) {
338
+ lines.push('', `Under \`mcp\`, but not valid Python identifiers — reach these with \`getattr\`: ${oddMcp.map(({ server, tool, usableServer }) => (usableServer ? `\`getattr(mcp.${server}, ${JSON.stringify(tool)})\`` : `\`getattr(getattr(mcp, ${JSON.stringify(server)}), ${JSON.stringify(tool)})\``)).join(', ')}.`)
339
+ }
233
340
  if (awkward.length > 0) {
234
341
  lines.push('', `Not valid Python identifiers — reach these with \`getattr\`: ${awkward.map((spec) => `\`getattr(__dsh__.tools, ${JSON.stringify(spec.name)})\``).join(', ')}.`)
235
342
  }
236
- lines.push('', 'Each is a real function: `name?` shows its full description, `dir(__dsh__.tools)` lists them.')
343
+ // The grouping is a real package, so a server can also be imported as a module — which reads
344
+ // better than `mcp.` at every call site when a cell leans on one server. The example is built
345
+ // from a server that HAS an importable tool, not just the first one: a server whose every tool
346
+ // name Python refuses would have put `import ...` in the line, handing the model a SyntaxError
347
+ // as its example. If no server has one, the `getattr` route above is the only honest advice.
348
+ const example = servers.flatMap(([server, tools]) => tools.filter((t) => isUsableName(t.tool)).map((t) => [server, t.tool])).at(0)
349
+ if (example !== undefined) {
350
+ lines.push('', `Each server is also a module: \`from __dsh__.tools.mcp.${example[0]} import ${example[1]}\` binds its tools directly.`)
351
+ }
352
+ // Naming the second level matters once the grouping exists: `dir(__dsh__.tools)` shows `mcp`,
353
+ // not the hundred tools under it, so a model told only the first level reads a real catalogue as
354
+ // a broken mount.
355
+ lines.push('', `Each is a real function: \`name?\` shows its full description, \`dir(__dsh__.tools)\` lists them${servers.length === 0 ? '' : ', and `dir(mcp)` / `dir(mcp.<server>)` the ones under `mcp`'}.`)
237
356
  return lines.join('\n')
238
357
  }
239
358
 
@@ -383,9 +502,12 @@ export function apply(ctx, config = {}) {
383
502
  exec.deferContext(createUserMessage({ content: outcome.content, source: { kind: 'plugin', plugin: 'dsh-py-codeact' } }))
384
503
  }
385
504
  for (const context of outcome.additionalContexts ?? []) exec.deferContext(context)
505
+ // `=== undefined` rather than `??`: a payload that IS `null` is the server's answer, and
506
+ // falling back to the wrapper there would hand the cell the one shape it was promised not to see.
507
+ const unwrapped = entry.envelopes.get(from)?.has(name) ? mcpPayload(outcome.value) : undefined
386
508
  return outcome.isError
387
509
  ? { ok: false, message: contentText(outcome.content) || 'tool call failed' }
388
- : { ok: true, value: outcome.value }
510
+ : { ok: true, value: unwrapped === undefined ? outcome.value : unwrapped.value }
389
511
  } catch (error) {
390
512
  session?.append('tool/code-dispatch', { ...trace, isError: true, content: [{ type: 'text', text: String(error?.message ?? error) }] })
391
513
  throw error
@@ -415,7 +537,7 @@ export function apply(ctx, config = {}) {
415
537
  } catch { /* it never came up — fall through and replace it */ }
416
538
  }
417
539
  // One entry backs every shell in the conversation tree, so `sent` is keyed BY SHELL. `calls` is the exception: it only has to make `subCallId` unique, so process-wide is right.
418
- const entry = { kernel: undefined, current: new Map(), calls: 0, sent: new Map(), started: undefined }
540
+ const entry = { kernel: undefined, current: new Map(), calls: 0, sent: new Map(), envelopes: new Map(), started: undefined }
419
541
  entry.kernel = new PythonKernel({
420
542
  command: config.command ?? (config.python === undefined ? undefined : [config.python, KERNEL_PY]),
421
543
  cwd: exec.agent?.session?.header?.cwd ?? process.cwd(),
@@ -447,9 +569,10 @@ export function apply(ctx, config = {}) {
447
569
  },
448
570
  // No `isConcurrencySafe`: cells mutate the shell's namespace, so this agent's cells must stay exclusive ordering barriers. That is a per-AGENT constraint, not a per-process one — the kernel tracks one in-flight task per shell, so a subagent's cell runs alongside its parent's just fine.
449
571
  async execute(args, exec) {
450
- const { specs } = toolSpecs(visibleSchemas(exec.agent))
572
+ const { specs, envelopes } = toolSpecs(visibleSchemas(exec.agent))
451
573
  const { entry, shell, restarted: wasRestarted } = await kernelFor(exec, specs)
452
574
  entry.current.set(shell, exec)
575
+ entry.envelopes.set(shell, envelopes)
453
576
  try {
454
577
  // Only resend the bindings when the visible set actually moved — an unchanged catalogue is the norm, and it is 30+ schemas on the wire.
455
578
  const key = specsKey(specs)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-py-codeact",
3
- "version": "0.2.1",
3
+ "version": "0.2.3",
4
4
  "description": "CodeAct agent loop for the DeepSeek Harness: a persistent IPython session as the model's action space, with harness tools bridged in as a virtual `__dsh__.tools` module.",
5
5
  "license": "MIT",
6
6
  "repository": {
package/py/kernel.py CHANGED
@@ -2,7 +2,7 @@
2
2
  # /// script
3
3
  # requires-python = ">=3.12"
4
4
  # dependencies = [
5
- # "ipython~=9.16.0",
5
+ # "ipython~=9.17.0",
6
6
  # "objprint~=0.3.0",
7
7
  # ]
8
8
  # # Checked with `TY_UV=scripts ty check py/kernel.py` — that prefix is what hands ty this script's venv. A `ty.toml` would be found and then silently ignored for PEP 723 scripts (astral-sh/ty#4083), so any ty config has to live here.
@@ -238,6 +238,12 @@ def _make_binding(bridge: Bridge, spec):
238
238
  return call
239
239
 
240
240
 
241
+ def bound_tools() -> dict:
242
+ """The calling shell's catalogue, or nothing outside a cell."""
243
+ session = _current_session.get()
244
+ return {} if session is None else session.bindings
245
+
246
+
241
247
  class ToolsModule(types.ModuleType):
242
248
  """`__dsh__.tools` — the bridged tool surface of the CALLING shell.
243
249
 
@@ -245,38 +251,211 @@ class ToolsModule(types.ModuleType):
245
251
 
246
252
  def __init__(self) -> None:
247
253
  super().__init__("__dsh__.tools", "Harness tools, bridged into this session as awaitables.")
254
+ self.__path__ = [] # a package, so `__dsh__.tools.mcp` resolves under it
248
255
  self.ToolCallError = ToolCallError
249
256
 
250
- @staticmethod
251
- def _bindings():
252
- session = _current_session.get()
253
- return {} if session is None else session.bindings
254
-
255
257
  def __getattr__(self, name): # only reached when the attribute is absent
256
258
  if name.startswith("__"):
257
259
  raise AttributeError(name) # import/introspection probing — never answer with a tool
258
- bindings = ToolsModule._bindings()
260
+ bindings = bound_tools()
259
261
  if name in bindings:
260
262
  return bindings[name]
261
- available = ", ".join(sorted(bindings)) or "(none)"
263
+ # The shown listing, not the bound one: a typo used to push the whole flat catalogue into
264
+ # the trajectory at the one moment the model is guaranteed to be reading it.
265
+ available = ", ".join(sorted(listed_tools())) or "(none)"
262
266
  raise AttributeError(f"no such tool: {name!r}. Available: {available}")
263
267
 
264
268
  def __dir__(self):
265
- return sorted(ToolsModule._bindings())
269
+ return sorted(listed_tools())
270
+
271
+ # NOT `listed_tools()`: `__all__` is what `from __dsh__.tools import *` BINDS, and narrowing it
272
+ # left `mcp__gh__ok` undefined in a cell that used to work. Display is `__dir__`'s job.
273
+ @property
274
+ def __all__(self):
275
+ return sorted(bound_tools())
276
+
277
+ def __repr__(self) -> str:
278
+ # The cell's trailing expression is echoed back, so ending on `__dsh__.tools` is the
279
+ # cheapest "what do I have" move there is — and it used to re-emit the whole flat catalogue.
280
+ return f"<module '__dsh__.tools': {', '.join(sorted(listed_tools())) or 'no tools bound'}>"
281
+
282
+
283
+ MCP_PREFIX = "mcp__"
284
+
285
+
286
+ def split_mcp(name: str) -> tuple[str, str] | None:
287
+ """`mcp__calendar__list_events` -> `("calendar", "list_events")`.
288
+
289
+ dsh names every MCP tool `mcp__<serverName>__<rawName>`, and a raw name may itself contain
290
+ `__` — `split("__")` would tear such a tool apart and file it under a server that does not
291
+ exist, so only the first two separators are ever consumed.
292
+
293
+ PRESENTATION ONLY. dsh's own naming contract says the public name "is never parsed to recover"
294
+ the raw one, and it is right: a name that needed normalizing, or that ran past 64 characters,
295
+ becomes `<truncated>_<12 hex of sha256>`, and the cut can land anywhere — including before the
296
+ second `__`. That case returns `None` here and the tool simply stays flat, reachable under its
297
+ full public name, which is the only name dispatch ever uses.
298
+ """
299
+ if not name.startswith(MCP_PREFIX):
300
+ return None
301
+ server, sep, raw = name[len(MCP_PREFIX) :].partition("__")
302
+ return (server, raw) if sep and server and raw else None
303
+
304
+
305
+ def servable(name: str) -> tuple[str, str] | None:
306
+ """`(server, tool)` when `mcp.<server>.<tool>` can actually ANSWER, else `None`.
307
+
308
+ Not `split_mcp` alone. Two ways a name splits cleanly and the grouping still cannot serve it:
309
+ `McpModule.__getattr__` refuses a true dunder — import and introspection probing (`__path__`,
310
+ `__all__`, `__spec__`) all wear that shape — and dsh hashes a public name that needed
311
+ normalising, where the cut can land before the second `__` and `split_mcp` returns `None`.
312
+
313
+ The one predicate for the whole file, because the listing and the lookup have to agree: hiding
314
+ a flat name on the strength of the split alone left `mcp__gh____weird__` in no listing the
315
+ model ever reads, while `mcp.gh.__weird__` raised `AttributeError`.
316
+ """
317
+ parts = split_mcp(name)
318
+ if parts is None or (parts[1].startswith("__") and parts[1].endswith("__")):
319
+ return None
320
+ return parts
321
+
322
+
323
+ def listed_tools() -> dict:
324
+ """What the model is SHOWN: `dir()`, `repr()`, and the `Available:` list of a failed lookup.
325
+
326
+ Not `__all__` — that is the star-import BINDING contract, and narrowing it unbound every flat
327
+ name from `from __dsh__.tools import *`, which is a regression on running code rather than a
328
+ quieter listing.
329
+
330
+ Every flat `mcp__server__tool` name stays bound; showing them contradicted the prompt block,
331
+ which stopped printing them. Of 103 entries 86 were flat MCP names, and a model asked to
332
+ introspect its own tools filtered them out by hand. A name the grouping cannot serve stays
333
+ shown — it is then the only name that works.
334
+ """
335
+ return {name: call for name, call in bound_tools().items() if servable(name) is None}
336
+
337
+
338
+ def mcp_servers(bindings: dict) -> dict[str, dict]:
339
+ """Group the flat `mcp__server__tool` bindings into `{server: {tool: call}}`.
340
+
341
+ The flat names stay bound too. They are what dsh dispatches on and what an older session may
342
+ already have imported; dropping them to tidy the surface would break a cell mid-conversation.
343
+ """
344
+ servers: dict[str, dict] = {}
345
+ for name, call in bindings.items():
346
+ if (parts := split_mcp(name)) is not None:
347
+ servers.setdefault(parts[0], {})[parts[1]] = call
348
+ return servers
349
+
350
+
351
+ MCP_MODULE = "__dsh__.tools.mcp"
352
+
353
+
354
+ def mcp_members(module_name: str) -> dict:
355
+ """One level of the `mcp` tree, resolved against the catalogue in force NOW.
356
+
357
+ Deliberately not a method: a non-dunder attribute on the class would shadow a tool or a server
358
+ of that name, and a raw MCP name is the server's to choose — `_private` is a legal one.
359
+ """
360
+ servers = mcp_servers({name: call for name, call in bound_tools().items() if servable(name) is not None})
361
+ if module_name == MCP_MODULE:
362
+ return {server: mcp_server_module(server) for server in servers}
363
+ # `removeprefix`, not `rpartition`: a raw server name is not guaranteed dot-free, and taking
364
+ # the last segment of one would look up a server that does not exist and resolve it empty.
365
+ return servers.get(module_name.removeprefix(f"{MCP_MODULE}."), {})
366
+
367
+
368
+ class McpModule(types.ModuleType):
369
+ """`__dsh__.tools.mcp`, and one of these per server under it.
370
+
371
+ Real modules, so every import form the model might reach for resolves the way it does for
372
+ `__dsh__.tools` itself — including `from __dsh__.tools.mcp.calendar import list_events`, which
373
+ needs a `sys.modules` entry of its own (the shallower forms can be served by `__getattr__`,
374
+ that one cannot).
375
+
376
+ Their CONTENTS still come from the ContextVar, because `sys.modules` is process-global while
377
+ the catalogue is per shell — and because a restriction or a reconnecting server moves tools in
378
+ and out between cells. A name pulled OUT with `from ... import` is a snapshot, as it is for any
379
+ Python import; the module itself stays live.
380
+ """
381
+
382
+ def __getattr__(self, name):
383
+ # True dunders only. The guard is here to refuse import and introspection probing
384
+ # (`__path__`, `__all__`, `__spec__`, `__deepcopy__`), which always ends in `__` too —
385
+ # and a leading-`__` raw tool name is the server's to choose, so `startswith` alone
386
+ # advertised `mcp.<server>.__weird` in `dir()` and then refused the call.
387
+ if name.startswith("__") and name.endswith("__"):
388
+ raise AttributeError(name)
389
+ members = mcp_members(self.__name__)
390
+ if name not in members:
391
+ available = ", ".join(sorted(members)) or "(none)"
392
+ raise AttributeError(f"no such tool: {self.__name__.removeprefix('__dsh__.tools.')}.{name}. Available: {available}")
393
+ return members[name]
394
+
395
+ def __setattr__(self, name, value):
396
+ # One module per server for the whole PROCESS, so a write here would shadow that name for
397
+ # every other agent in it — permanently, and invisibly to `dir()`, which keeps reporting
398
+ # the tool it no longer reaches. The old per-call `Namespace` made this a local mistake.
399
+ if not (name.startswith("__") and name.endswith("__")):
400
+ raise AttributeError(f"{self.__name__.removeprefix('__dsh__.tools.')} belongs to the harness and is shared by every agent in this process — bind your own name instead of writing to it")
401
+ super().__setattr__(name, value)
402
+
403
+ def __dir__(self):
404
+ return sorted(mcp_members(self.__name__))
266
405
 
406
+ # Star-import reads `__all__` (or `vars()`), never `__getattr__` or `__dir__`, and nothing ever
407
+ # lands in these modules' `__dict__` — so without this `from __dsh__.tools.mcp.x import *`
408
+ # succeeded and bound nothing. `ToolsModule` carries the same property for the same reason.
267
409
  @property
268
410
  def __all__(self):
269
- return sorted(ToolsModule._bindings())
411
+ return sorted(mcp_members(self.__name__))
270
412
 
271
413
  def __repr__(self) -> str:
272
- return f"<module '__dsh__.tools': {', '.join(sorted(ToolsModule._bindings())) or 'no tools bound'}>"
414
+ members = mcp_members(self.__name__)
415
+ return f"<module {self.__name__!r}: {', '.join(sorted(members)) or 'empty'}>"
416
+
417
+
418
+ MCP_ROOT = McpModule(MCP_MODULE, "MCP tools, one module per server.")
419
+ MCP_ROOT.__path__ = [] # a package, like `__dsh__` and `__dsh__.tools`, so its server modules resolve
420
+ sys.modules[MCP_MODULE] = MCP_ROOT
421
+
422
+
423
+ def mcp_server_module(server: str) -> McpModule:
424
+ """The one module for `server`, made on first ask.
425
+
426
+ The single construction site, so a module that went missing — a cell can `del sys.modules[…]`
427
+ — comes back instead of leaving `dir(mcp)` raising `KeyError` for the rest of the session.
428
+ """
429
+ name = f"{MCP_MODULE}.{server}"
430
+ if not isinstance(module := sys.modules.get(name), McpModule):
431
+ module = McpModule(name, f"Tools bridged from the `{server}` MCP server.")
432
+ sys.modules[name] = module
433
+ return module
434
+
435
+
436
+ def install_mcp_modules(bindings: dict) -> None:
437
+ """Register a module per visible MCP server.
438
+
439
+ Eager, because the deep import form never reaches `mcp_members`: `from __dsh__.tools.mcp.x
440
+ import y` is resolved by the import machinery against `sys.modules`, before any attribute
441
+ lookup happens.
442
+
443
+ `sys.modules` only ever gains entries: a server another shell can see costs this one an unused
444
+ module, while removing it would break an import that shell is mid-conversation with. What a
445
+ shell can actually reach is decided by `mcp_members`, not by what is registered.
446
+ """
447
+ for server in mcp_servers(bindings):
448
+ mcp_server_module(server)
273
449
 
274
450
 
275
451
  def build_bindings(bridge: Bridge, specs) -> dict:
276
452
  """Project one agent's visible tools into awaitables for its shell."""
277
- # A tool named `ToolCallError` or `_bindings` would be shadowed by the module's own attributes, and one named `_rebind` used to overwrite a bound method outright. dsh's own SDK renderer refuses `_`-leading tool names for exactly this collision class.
278
- reserved = set(vars(ToolsModule)) | set(vars(types.ModuleType)) | {"ToolCallError"}
279
- return {spec["name"]: _make_binding(bridge, spec) for spec in specs if not spec["name"].startswith("_") and spec["name"] not in reserved}
453
+ # A tool named `ToolCallError` would be shadowed by the module's own attributes, and one named `_rebind` used to overwrite a bound method outright. dsh's own SDK renderer refuses `_`-leading tool names for exactly this collision class.
454
+ # `mcp` joins them, and unconditionally: it used to be bound and then overwritten by the namespace, so the tool was uncallable anyway — but only when an MCP server happened to be mounted. A name that means the namespace in one catalogue and a tool in the next is worse than one that always means the same thing.
455
+ reserved = set(vars(ToolsModule)) | set(vars(types.ModuleType)) | {"ToolCallError", "mcp"}
456
+ flat = {spec["name"]: _make_binding(bridge, spec) for spec in specs if not spec["name"].startswith("_") and spec["name"] not in reserved}
457
+ # `mcp` only when something is under it: an empty namespace in `dir()` reads as a broken mount.
458
+ return {**flat, "mcp": MCP_ROOT} if mcp_servers(flat) else flat
280
459
 
281
460
 
282
461
  def install_bridge_modules() -> ToolsModule:
@@ -386,7 +565,7 @@ class Session:
386
565
  history = Config()
387
566
  history.HistoryAccessor.hist_file = ":memory:"
388
567
  self.shell = InteractiveShell(user_ns=namespace, config=history)
389
- self.bindings = build_bindings(bridge, specs)
568
+ self.rebind(bridge, specs)
390
569
  self.sinks: list = [None, None] # the Capped buffers of the cell in flight
391
570
  install_bridge_modules()
392
571
 
@@ -434,7 +613,14 @@ class Session:
434
613
  self.sinks[:] = previous
435
614
 
436
615
  def rebind(self, bridge: Bridge, specs) -> None:
616
+ """Swap in a catalogue — a restriction or a reconnecting server moves tools between cells.
617
+
618
+ The one place the `sys.modules` registration lives, so `build_bindings` stays the pure
619
+ projection its name promises and no caller can produce bindings the import machinery
620
+ cannot follow.
621
+ """
437
622
  self.bindings = build_bindings(bridge, specs)
623
+ install_mcp_modules(self.bindings)
438
624
 
439
625
  def format_exc(self) -> str:
440
626
  """IPython's own traceback, rendered without ANSI colors."""