dotagents-cli 0.8.2__tar.gz → 0.8.3__tar.gz

{dotagents_cli-0.8.2 → dotagents_cli-0.8.3}/CHANGELOG.md

@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [0.8.3] - 2026-10-30
+
+### Added
+
+- `dotagents mcp` starts the stdio MCP gateway and can forward tools from
+  overlay-declared stdio and Streamable HTTP servers. Forwarded names default
+  to `<skill-or-server-name>.<tool>`; `--prefix` accepts a `{name}` template.
+  `dotagents.mcp.refresh` reloads active overlay declarations and reconnects
+  servers without restarting dotagents.
+  Overlays declare servers and skill mappings in `overlay.toml` under `[mcp]`;
+  Streamable HTTP requires the `mcp-http` extra, and credentials are read from
+  named environment variables.
+
+### Changed
+
+- Requires duho 0.7.9 (its `ToolProvider` serves the forwarded tools), also in
+  the scope venvs and the `.pyz`.
+
 ## [0.8.2] - 2026-10-25
 
 ### Fixed
@@ -1567,7 +1585,8 @@ Patch release: the PATH/POSIX-conversion fix above (the only change since 0.3.0)
   affected module sources (`dotagents.cli`, `duho.presets`) to extracted temp files
   before dispatch; a no-op for a plain install.
 
-[Unreleased]: https://github.com/jose-pr/dotagents/compare/v0.8.2...HEAD
+[Unreleased]: https://github.com/jose-pr/dotagents/compare/v0.8.3...HEAD
+[0.8.3]: https://github.com/jose-pr/dotagents/compare/v0.8.2...v0.8.3
 [0.8.2]: https://github.com/jose-pr/dotagents/compare/v0.8.1...v0.8.2
 [0.8.1]: https://github.com/jose-pr/dotagents/compare/v0.8.0...v0.8.1
 [0.8.0]: https://github.com/jose-pr/dotagents/compare/v0.7.6...v0.8.0

{dotagents_cli-0.8.2 → dotagents_cli-0.8.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: dotagents-cli
-Version: 0.8.2
+Version: 0.8.3
 Summary: The dotagents CLI: install and manage a portable ~/.agents config for AI coding agents
 Project-URL: Homepage, https://github.com/jose-pr/dotagents/
 Project-URL: Repository, https://github.com/jose-pr/dotagents
@@ -24,7 +24,7 @@ Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Typing :: Typed
 Requires-Python: >=3.9
 Requires-Dist: distlib<0.5,>=0.4.0
-Requires-Dist: duho<0.8,>=0.7.5
+Requires-Dist: duho<0.8,>=0.7.9
 Requires-Dist: pathapi<0.2,>=0.1.0
 Requires-Dist: tomli>=1.1.0; python_version < '3.11'
 Provides-Extra: dev
@@ -33,12 +33,15 @@ Requires-Dist: hatchling>=1.27; extra == 'dev'
 Requires-Dist: pathapi-uri<0.2,>=0.1.0; extra == 'dev'
 Requires-Dist: pytest; extra == 'dev'
 Requires-Dist: pyyaml; extra == 'dev'
+Requires-Dist: syshttp<0.2,>=0.1.0; extra == 'dev'
 Provides-Extra: docs
 Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
 Requires-Dist: mkdocs>=1.5; extra == 'docs'
 Requires-Dist: mkdocstrings[python]>=0.24; extra == 'docs'
 Provides-Extra: http
 Requires-Dist: pathapi-uri<0.2,>=0.1.0; extra == 'http'
+Provides-Extra: mcp-http
+Requires-Dist: syshttp<0.2,>=0.1.0; extra == 'mcp-http'
 Provides-Extra: repo
 Requires-Dist: pathapi-uri<0.2,>=0.1.0; extra == 'repo'
 Requires-Dist: repopath[uri]<0.2,>=0.1.0; extra == 'repo'

{dotagents_cli-0.8.2 → dotagents_cli-0.8.3}/docs/guide/commands.md

@@ -27,10 +27,11 @@ otherwise.
 | `path` | Print the bin dirs `env` puts on `PATH` (or, with `--lib`, the lib dirs it puts on `PYTHONPATH`). |
 | `build-pyz` | Build the self-contained `dotagents.pyz` zipapp (from a source checkout). |
 | `about` | `dotagents-cli <version>`, then the packages bundled in the `.pyz` (or installed beside a plain install) with their versions. |
+| `mcp [--prefix TEMPLATE]` | Serve dotagents command tools and active overlay MCP tools over stdio. |
 | `findings` | Per-scope findings queue: `add` / `list` / `show` / `done` / `reopen` / `remove` / `index` / `path`. |
 | `launch` | Start an agent's CLI with the `env` exported and the `context` handed over; everything after `--` goes to the agent. |
 
-That table is the **whole** shipped surface: six built-in commands and two bundled
+That table is the **whole** shipped surface: eight built-in commands and two bundled
 command modules, `findings` and `launch` (discovered from the package itself). Every
 other command is **discovered** from a directory of command modules. The sources, in
 order — a later one overrides a same-named command from an earlier one:
@@ -61,14 +62,20 @@ subcommand, no registration needed; see
 
 ## As tools for an agent
 
-Every command above except `env`, and every discovered one, is also an **MCP tool**.
-Start `dotagents` with `DOTAGENTS_MCP=stdio` in its environment and it serves that
-command set over stdio instead of running a command:
+Every command above except `env` and `mcp`, and every discovered one, is also an
+**MCP tool**. `dotagents mcp` serves that command set over stdio; starting
+`dotagents` with `DOTAGENTS_MCP=stdio` in its environment is the same server:
 
 ```sh
-DOTAGENTS_MCP=stdio dotagents        # register this as a stdio MCP server in your agent
+dotagents mcp                        # register this as a stdio MCP server in your agent
 ```
 
+The same server forwards the tools of the MCP servers that active overlays declare
+(see [Overlays → MCP servers](overlays.md#mcp-servers)), named
+`<skill-or-server-name>.<tool>`; `--prefix 'dotagents.{name}'` puts them under
+`dotagents.`. Calling `dotagents.mcp.refresh` re-reads those declarations without a
+restart and sends `notifications/tools/list_changed`.
+
 - One tool per leaf command, named with dots: `dotagents.findings.add`,
   `dotagents.overlays.list`, and for an overlay's commands `dotagents.<command>`.
 - A tool's input schema is generated from the command's fields; its result is what

{dotagents_cli-0.8.2 → dotagents_cli-0.8.3}/docs/guide/overlays.md

@@ -205,7 +205,8 @@ contract is under [Commands → env](commands.md#env).
 
 The manifest is read by the `overlays` command and the context assembler, with
 `tomllib` (Python 3.11+) or `tomli` when either is installed, and otherwise with a
-small built-in reader that covers these keys. Keys:
+small built-in reader that covers these keys (all but `[mcp]`, which needs one of
+the two). Keys:
 
 | Key | Meaning |
 | --- | --- |
@@ -217,11 +218,58 @@ small built-in reader that covers these keys. Keys:
 | `priority` | Merge order (lower sorts earlier; unprioritized default is 500). |
 | `setup_timeout` | Seconds the setup script may run before it is stopped (default 300; `0` = no limit). |
 | `[scripts]` | Launchers on `PATH`: `name = "path/in/overlay.py"` adds or replaces one, `name = false` drops the one for a `bin/<name>.py`. See [Authoring → Commands on PATH](authoring.md#commands-on-path-binpy-and-scripts). |
+| `[mcp]` | `[[mcp.servers]]` and `[[mcp.skills]]` declare MCP endpoints and skills that expose them. See [MCP servers](#mcp-servers). |
 
 `overlay.toml` is the only manifest. A leading `---` frontmatter block in a
 `CONTEXT.md` (the `priority:` / `scope:` / `dependencies:` an older overlay format
 kept there) is not read, and `context` leaves it out of what it assembles.
 
+## MCP servers
+
+`dotagents mcp` forwards the tools of the MCP servers an overlay declares. Each
+`[[mcp.servers]]` entry has a `name`, a `transport` and an optional `timeout` in
+seconds (default 60):
+
+| `transport` | Keys |
+| --- | --- |
+| `stdio` | `command`, `args`; `env` sets values (not secrets: they sit in the manifest); `env_passthrough` names variables passed on from the environment. |
+| `streamable-http` | `url` (no credentials, no fragment); `headers_env` maps a header name to the variable holding its value. Needs `dotagents-cli[mcp-http]`. |
+
+```toml
+[mcp]
+
+[[mcp.servers]]
+name = "builder"
+transport = "stdio"
+command = "uvx"
+args = ["builder-mcp"]
+env = { BUILDER_MCP = "stdio" }
+env_passthrough = ["BUILDER_TOKEN"]
+timeout = 60
+
+[[mcp.servers]]
+name = "docs"
+transport = "streamable-http"
+url = "https://docs.example/mcp"
+headers_env = { Authorization = "DOCS_TOKEN" }
+timeout = 30
+
+[[mcp.skills]]
+name = "code-search"
+server = "builder"
+```
+
+A `[[mcp.skills]]` entry names a skill of the overlay (`skills/code-search/SKILL.md`
+must exist) and the server behind it; that server's tools are then listed as
+`code-search.<tool>` instead of `builder.<tool>`, once per skill when it backs
+several. `dotagents mcp --prefix 'dotagents.{name}'` puts every forwarded name under
+`dotagents.`.
+
+A stdio server gets `PATH`, `HOME`, `USERPROFILE`, `SYSTEMROOT`, `TEMP` and `TMP`,
+the variables in `env_passthrough`, and `env`; nothing else of the environment. An
+invalid `[mcp]` table in any active overlay stops `dotagents mcp` from starting; a
+server that fails to start is left out with a warning. Only tools are forwarded.
+
 ## Skills
 
 An overlay may ship `skills/<skill-name>/` directories. Publishing symlinks each (or

{dotagents_cli-0.8.2 → dotagents_cli-0.8.3}/pyproject.toml

@@ -35,7 +35,7 @@ classifiers = [
 # `build-pyz` vendors the floor of each of these (src/dotagents/cli/build_pyz.py);
 # raise a floor and move its pin there in the same change.
 dependencies = [
-    "duho>=0.7.5,<0.8",
+    "duho>=0.7.9,<0.8",
     "pathapi>=0.1.0,<0.2",
     # The launchers of overlays' [scripts]: ScriptMaker writes the Windows .exe.
     "distlib>=0.4.0,<0.5",
@@ -51,13 +51,14 @@ dependencies = [
 uri = ["pathapi-uri>=0.1.0,<0.2"]
 # http(s) is part of `uri` now; kept so `dotagents-cli[http]` still installs.
 http = ["pathapi-uri>=0.1.0,<0.2"]
+mcp-http = ["syshttp>=0.1.0,<0.2"]
 sftp = ["pathapi-uri>=0.1.0,<0.2", "sftppath[paramiko]>=0.1.0,<0.2"]
 s3 = ["pathapi-uri>=0.1.0,<0.2", "objpath[s3]>=0.1.0,<0.2"]
 # github:, gitlab: and repo: (a repository's tree at a ref).
 repo = ["pathapi-uri>=0.1.0,<0.2", "repopath[uri]>=0.1.0,<0.2"]
 # YAML overlay-repo registries (dotagents.yaml / *.yml).
 yaml = ["pyyaml"]
-dev = ["build", "hatchling>=1.27", "pytest", "pyyaml", "pathapi-uri>=0.1.0,<0.2"]
+dev = ["build", "hatchling>=1.27", "pytest", "pyyaml", "pathapi-uri>=0.1.0,<0.2", "syshttp>=0.1.0,<0.2"]
 docs = [
     "mkdocs>=1.5",
     "mkdocs-material>=9.5",

{dotagents_cli-0.8.2 → dotagents_cli-0.8.3}/src/dotagents/AGENTS.md

@@ -43,17 +43,35 @@ and an error saying a command is required, and exits 2.
 | `findings add\|list\|show\|done\|reopen\|remove\|index\|path [-g] [--dir D]` | The findings queue at `<store>/findings/` (`--dir` overrides): one markdown file per finding, `done` moves it to `processed/` with a required resolution, `INDEX.md` regenerated on every change. The frontmatter is valid YAML (a value is double-quoted when it must be); `index --requote` fixes values written bare in existing files, touching nothing else. |
 | `about [--json]` | `dotagents-cli <version>`, then `<distribution> <version>` per bundled (`.pyz`) or installed package. |
 | `build-pyz [--out P] [--python SHEBANG] [--duho-version V] [--pathapi-version V] [--extras a,b] [--with-pip]` | Build a zipapp from a source checkout; errors elsewhere. `--with-pip` vendors the pinned pip wheel (`_pipwheel`) as `dotagents/_vendor/`. |
+| `mcp [--prefix TEMPLATE]` | Serve command and active overlay MCP tools over stdio; `{name}` in the template is the skill or server name. |
 
-**Every command but `env` is also an MCP tool.** With `DOTAGENTS_MCP=stdio` in its
-environment, `dotagents` serves the command set — built-in, bundled and discovered —
-over stdio instead of running one: a tool per leaf command, named with dots
+**Every command but `env` and `mcp` is also an MCP tool.** `dotagents mcp` (or
+`DOTAGENTS_MCP=stdio dotagents`, the same server) serves the command set — built-in,
+bundled and discovered — over stdio: a tool per leaf command, named with dots
 (`dotagents.findings.add`, `dotagents.overlays.list`), its input schema generated from
 the command's fields, its result the command's output. `env` is not served: it prints
-resolved, possibly secret values. The set is fixed when the server starts, for the
-scope it starts in. With `AGENT_HELP=1`, `--help` prints one
+resolved, possibly secret values. The command set is fixed when the server starts, for
+the scope it starts in. With `AGENT_HELP=1`, `--help` prints one
 JSON document (`duho/agent-help@1`) for the command tree it is asked about, in place
 of the human text.
 
+The same server forwards the tools of the MCP servers active overlays declare in
+`overlay.toml` (`[[mcp.servers]]`, `[[mcp.skills]]`; keys in
+`docs/guide/overlays.md`). Forwarded names are `<skill-or-server-name>.<tool>`, or
+`--prefix`'s template with `{name}` filled in, plus `.<tool>`. Gotchas:
+
+- An invalid `[mcp]` table in any active overlay stops the server from starting
+  (exit 2, the reason on stderr).
+- Overlay servers start on the first `tools/list`; one that fails to start, or a tool
+  whose name collides with one already listed, is left out with a warning.
+- A stdio server's environment is `PATH`, `HOME`, `USERPROFILE`, `SYSTEMROOT`,
+  `TEMP`, `TMP`, the variables named in `env_passthrough`, then `env`. An HTTP
+  server's credentials come only from the variables `headers_env` names.
+- Streamable HTTP needs the `mcp-http` extra (`syshttp`).
+- `dotagents.mcp.refresh` re-reads the declarations, closes every connection and
+  sends `notifications/tools/list_changed`; the next `tools/list` reconnects.
+- Only tools are forwarded: not resources, prompts, or a server's own requests.
+
 Scope flags: `-g` / `--global` selects the user store; `--agents-dir D` overrides the
 store the command resolves to (the project store for `init` / `overlays` / `findings`
 without `-g`, the user store otherwise). For `env`, `context` and `launch`, `-g` means
@@ -152,7 +170,7 @@ The package's own data; library code, importing nothing from `dotagents.cli`.
   install. The archive also holds the vendored duho and pathapi; `env`
   exports it as `AGENTS_PYLIB`.
 
-## Commands — `dotagents.cli.init`, `dotagents.cli.overlays`, `dotagents.cli.context`, `dotagents.cli.env`, `dotagents.cli.path`, `dotagents.cli.about`, `dotagents.cli.build_pyz`
+## Commands — `dotagents.cli.init`, `dotagents.cli.overlays`, `dotagents.cli.context`, `dotagents.cli.env`, `dotagents.cli.path`, `dotagents.cli.about`, `dotagents.cli.build_pyz`, `dotagents.cli.mcp`
 
 One `duho` command class per command above; fields are the flags, `__call__()`
 returns the exit code.
@@ -176,10 +194,28 @@ returns the exit code.
   `bundle_manifest() -> Optional[dict]`, `installed_packages() -> dict[str, str]`,
   `packages() -> tuple[dict[str, str], Optional[dict]]` (the bundle's packages, else
   the installed ones; the CLI itself in neither).
+- `mcp.Mcp` — `prefix: Optional[str]`; exits 2 when the template uses a placeholder
+  other than `{name}` or does not format to a tool-name segment.
 - `build_pyz`: `EXTRA_REQUIREMENTS` — `{extra: (requirement, ...)}`, what each
-  dotagents-cli extra installs (`uri`, `http`, `sftp`, `s3`, `repo`), kept equal to
-  `pyproject.toml`; `--extras` vendors each requirement at its floor and refuses a
-  name not in it.
+  dotagents-cli extra installs (`uri`, `http`, `mcp-http`, `sftp`, `s3`, `repo`),
+  kept equal to `pyproject.toml`; `--extras` vendors each requirement at its floor
+  and refuses a name not in it.
+
+## `dotagents._mcp_gateway`, `dotagents._mcp_gateway_config`, `dotagents._mcp_gateway_clients`
+
+- `_mcp_gateway.Gateway(notify, load_specs, name_prefix=None)` — the
+  `duho.mcp.ToolProvider` the root's `_mcp_tools_` (`cli._mcp_tools`) builds:
+  `list_tools()` (`dotagents.mcp.refresh` and the forwarded tools; servers start on
+  the first call), `call_tool(name, arguments)` (`None` for a name it does not
+  forward), `close()` (every connection). `load_specs()` runs in the
+  constructor, so a `GatewayConfigError` ends `duho.mcp.serve` with status 2 before
+  the first request. duho's server answers everything else (`-32602` for an unknown
+  tool or bad arguments, bad arguments a tool result from protocol `2025-11-25` on).
+- `_mcp_gateway_config.discover(scope) -> list` — the validated server declarations
+  of the scope's active overlays; raises `GatewayConfigError` (a `ValueError`) on
+  the first invalid one, or on a `[[mcp.skills]]` entry without
+  `skills/<name>/SKILL.md`.
+- `_mcp_gateway_clients` — the stdio and Streamable HTTP clients; private.
 
 ## `dotagents._scope`
 
@@ -276,7 +312,8 @@ returns the exit code.
     `setup_timeout`, `scripts` (the `[scripts]` table as written: name -> path
     relative to the overlay, `/`-separated, plain file-name characters, no `..`,
     or `false`; any other entry is dropped with a warning); a missing or invalid
-    manifest gives empty contributions.
+    manifest gives empty contributions. `[mcp]` is read by
+    `_mcp_gateway_config.discover`, not here.
   - `scripts() -> dict[str, str]` — the scripts that get a launcher
     (`dotagents._launchers`): each top-level `bin/<name>.py` whose stem is a
     `SCRIPT_NAME_RE` name (so not `_private.py`, not `__init__.py`), then
@@ -742,7 +779,7 @@ interpreter; layered project > user > system through a `.pth`. `.pyvenv/` holds
 agents' venvs only (skills, overlay helpers, setup scripts), never a project's own
 code. `AGENTS_NO_PYVENV=1` (or `true`/`yes`/`on`) turns it all off.
 
-- `BASE_REQUIREMENTS = ("duho>=0.7.5,<0.8",)` — always in the user venv; dotagents'
+- `BASE_REQUIREMENTS = ("duho>=0.7.9,<0.8",)` — always in the user venv; dotagents'
   own duho range. `REQUIREMENTS_FILE = "requirements.txt"` (at an overlay's root),
   `PYVENV_DIR = ".pyvenv"`, `STATE_FILE = ".dotagents-venv.json"` (in the venv:
   `{"digest", "requirements", "pth"}`), `PTH_FILE = "dotagents-layers.pth"` (in its

{dotagents_cli-0.8.2 → dotagents_cli-0.8.3}/src/dotagents/__init__.py

@@ -1,3 +1,3 @@
 """dotagents: installable CLI for the dotagents agent-config payload."""
 
-__version__ = "0.8.2"
+__version__ = "0.8.3"

dotagents_cli-0.8.3/src/dotagents/_mcp_gateway.py

@@ -0,0 +1,116 @@
+"""Forward the tools of overlay-declared MCP servers through duho's MCP server."""
+
+from __future__ import annotations
+
+import logging
+
+from dotagents._mcp_gateway_clients import (
+    _HTTPClient,
+    _StdioClient,
+    _prefix_tools,
+    _server_error,
+)
+from dotagents._mcp_gateway_config import GatewayConfigError, discover  # noqa: F401  (re-exported)
+
+_LOG = logging.getLogger("dotagents.mcp")
+_REFRESH_TOOL = "dotagents.mcp.refresh"
+_REFRESH_SPEC = {
+    "name": _REFRESH_TOOL,
+    "description": "Reload active overlay MCP declarations and reconnect their servers.",
+    "inputSchema": {"type": "object", "properties": {}, "additionalProperties": False},
+}
+
+
+def _text_result(text, is_error=False):
+    result = {"content": [{"type": "text", "text": text}]}
+    if is_error:
+        result["isError"] = True
+    return result
+
+
+class Gateway:
+    """A `duho.mcp.ToolProvider` over the servers `load_specs()` declares.
+
+    Servers start on the first `list_tools`; one that fails to start, or a tool
+    whose name is already taken, is left out with a warning.
+    """
+
+    def __init__(self, notify, load_specs, name_prefix=None):
+        self._notify = notify
+        self._load_specs = load_specs
+        self._name_prefix = name_prefix
+        self._specs = load_specs()
+        self._clients = []
+        self._tool_map = {}
+        self._tools = []
+        self._connected = False
+
+    def _connect(self):
+        self._connected = True
+        for spec in self._specs:
+            client = None
+            try:
+                client = _StdioClient(spec) if spec.transport == "stdio" else _HTTPClient(spec)
+                client.start()
+                candidates = [
+                    pair
+                    for expose_as in spec.skills or [spec.name]
+                    for pair in _prefix_tools(
+                        spec, client, spec.prefix(expose_as, self._name_prefix)
+                    )
+                ]
+            except Exception as exc:
+                if client is not None:
+                    client.close()
+                reason = _server_error(exc, spec)[:500]
+                _LOG.warning("MCP server %s.%s unavailable (%s)", spec.overlay, spec.name, reason)
+                continue
+            for tool, remote_name in candidates:
+                name = tool["name"]
+                if name == _REFRESH_TOOL or name in self._tool_map:
+                    _LOG.warning("MCP server %s.%s has a colliding tool name", spec.overlay, spec.name)
+                    continue
+                self._tool_map[name] = (client, remote_name)
+                self._tools.append(tool)
+            self._clients.append(client)
+
+    def list_tools(self):
+        if not self._connected:
+            self._connect()
+        return [_REFRESH_SPEC] + self._tools
+
+    def call_tool(self, name, arguments):
+        if name == _REFRESH_TOOL:
+            return self._refresh()
+        target = self._tool_map.get(name)
+        if target is None:
+            return None
+        client, remote_name = target
+        try:
+            value = client.call(remote_name, arguments)
+        except Exception as exc:
+            return _text_result(_server_error(exc, client.spec), is_error=True)
+        if not isinstance(value, dict) or "content" not in value:
+            return _text_result("tool returned an invalid result", is_error=True)
+        return value
+
+    def _refresh(self):
+        try:
+            specs = self._load_specs()
+        except Exception as exc:
+            return _text_result("Could not reload MCP configuration: %s" % exc, is_error=True)
+        self.close()
+        self._specs = specs
+        self._notify("notifications/tools/list_changed")
+        return _text_result(
+            "Reloaded %d declared MCP server(s); request tools/list to reconnect them."
+            % len(specs)
+        )
+
+    def close(self):
+        for client in self._clients:
+            client.close()
+        self._clients = []
+        self._tool_map = {}
+        self._tools = []
+        self._connected = False