@sigrun/ext-mcp 0.0.0-stage → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,45 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.2.0] - 2026-10-22
+### Added
+- Remote MCP servers: a `servers` entry may be `{url, headers?, exposure?,
+  toolExposure?, timeout?}` instead of a stdio `{command, args}`. Streamable
+  HTTP is the transport (POST per frame, JSON or SSE reply,
+  `mcp-session-id` captured and echoed); a legacy HTTP+SSE server is adopted
+  automatically — whether it 404/405s the POST or (crawl4ai's shape) answers
+  it with an `endpoint` event — so no supergateway-style adapter is needed.
+- `mcp.json` now resolves from the sigrun config dir
+  (`SIGRUN_CONFIG_DIR/mcp.json`) before the home fallback, matching every
+  other sigrun store.
+
+### Fixed
+- SSE parsing handles CRLF line endings (the spec allows them; live servers
+  emit them).
+- Streamable replies stop at the id-matching frame — servers legitimately
+  hold SSE streams open past the reply.
+
+## [0.1.0] - 2026-10-13
+### Added
+- Renamed to `@sigrun/ext-mcp` for the 0.1.0 release (repo stays
+  `sigrun-ai/sigrun-ext-mcp`): published package metadata
+  (`publishConfig` access/provenance), host modules moved to
+  `peerDependencies` mirrored in `devDependencies` at exact pins
+  (`typebox` now declared — it was an undeclared host import),
+  CHANGELOG.md joined the files whitelist, and a manifest hygiene check
+  wired into `npm test`.
+- MCP client over stdio: server discovery, tool registration (`<server>/<tool>`
+  naming), OAuth (PKCE paste-the-URL flow) with a durable token store.
+- The MCP exposure model on pi-durable 1.2.0: per-server
+  `exposure: codemode|deferred|direct|hidden` (default `codemode`),
+  per-tool `toolExposure` overrides (exact over patterns, first wins),
+  `enabled`, and per-request `timeout` with progress-notification reset.
+- The `mcp_servers` prompt section: live-rendered, 4KB whole-line-truncated,
+  gated on the discovery tool being model-offered, with an honest
+  enable-notice line for unreachable servers.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Michael MacLeod
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,109 @@
-# Temporary Holding Version
+# @sigrun/ext-mcp
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Register MCP server tools as pi-durable tools, named `<server>/<tool>`.
+
+Config: the sigrun config dir's `mcp.json` (`SIGRUN_CONFIG_DIR/mcp.json`; a
+bare user install falls back to `~/.config/sigrun/mcp.json`; the
+`SIGRUN_MCP_CONFIG` env override points at any path). Each entry under
+`"servers"` names an MCP server — stdio or remote:
+
+```json
+{
+	"servers": {
+		"github": {
+			"command": "npx",
+			"args": ["-y", "@modelcontextprotocol/server-github"],
+			"exposure": "codemode",
+			"timeout": 120
+		},
+		"search": {
+			"url": "https://mcp.example.test/mcp",
+			"headers": { "Authorization": "Bearer <token>" },
+			"exposure": "direct"
+		}
+	}
+}
+```
+
+## Server fields
+
+- `command`, `args` (stdio) — how to spawn the server.
+- `url` (remote) — the server endpoint (http/https). Streamable HTTP is the
+  transport; a legacy HTTP+SSE server (2024-11-05 shape — GET stream +
+  `endpoint` event + message POSTs) is adopted automatically, so no adapter
+  process is needed.
+- `headers` (remote) — extra request headers (auth etc.), sent on every
+  request.
+- `env` (stdio) — extra environment variables for the child process.
+- `enabled` (default `true`) / `disabled` — skip the server entirely. Set one,
+  never both: a server that is simultaneously on and off is a config error and
+  fails loudly.
+- `timeout` — per-request wire timeout in seconds (default `60`). A request
+  that gets no reply by its deadline rejects. A server that keeps sending
+  `notifications/progress` for a pending request never dies at the deadline
+  while it keeps reporting work — progress resets the deadline.
+- `exposure` — how the server's tools register (see below). Default `"codemode"`.
+- `toolExposure` — per-tool overrides of `exposure`: an object mapping tool
+  names (exact) or `*`-globs (e.g. `"get-*"`) to exposure words. An exact name
+  always wins; among matching patterns the FIRST one in config order wins.
+- `oauth` (stdio) — the authorization-code (PKCE) flow block; see
+  `sigrun-ext-mcp oauth <server>` (the CLI mints and stores a token; the spawn
+  injects it).
+
+## Exposure: how tools reach callers
+
+On pi-durable 1.2.0 semantics, `"exposure"` decides who can call a server's
+tools:
+
+| exposure    | registration              | who can call                          |
+| ----------- | ------------------------- | ------------------------------------- |
+| `codemode`  | `callers: ["tools"]`      | scripts only (never model-offered)    |
+| `deferred`  | `deferred: true`          | the model, once `tool_search` loads it |
+| `direct`    | normal                    | the model (offered when in the set)   |
+| `hidden`    | not registered            | nobody (config entry kept)            |
+
+The `codemode-deferred` word is kept as an alias for `codemode` (pi compat).
+
+### The default flip — a breaking change
+
+**Before 1.2.0, an `mcp.json` server entry with no `exposure` field had its
+tools offered to the model. Now the default is `"codemode"`:** a
+default-exposure server's tools are callable by scripts (codemode), never
+offered to the model directly. To restore the old behavior, set the server
+(or the tool, via `toolExposure`) to `"exposure": "direct"`:
+
+```json
+{ "exposure": "direct" }
+```
+
+The `mcp_servers` system-prompt section lists codemode/deferred servers when
+their discovery tool (`codemode`, `tool_search`) is offered in the request;
+when such servers exist but their tools are unreachable, it appends one
+notice line naming the count and the Settings → default tools fix.
+
+## Tool naming: `<server>/<tool>` (decision)
+
+Tools keep the existing `<server>/<tool>` shape rather than upstream pi's
+`mcp__<server>__<tool>`:
+
+- The upstream codemode section groups tools by `CodemodeNamespace.prefix`
+  (longest match wins), and the namespaces are **config-side** — the host
+  passes the groups, so any naming can namespace trivially: ours groups as
+  `prefix: "<server>/"` with no rename at all.
+- The name-addressed surface makes per-tool replacement and `toolExposure`
+  patterns readable (`"github/*"`), and the existing wire name is already
+  deployed in configs and scripts.
+
+No section-rendering compat requires the upstream spelling; the decision is
+recorded here per the spec's open item.
+
+## Replacement semantics
+
+Replacement works by not installing ours and installing yours: the
+name-addressed tool surface (`<server>/<tool>`) makes per-name replacement
+trivial. Co-installing this package with another that registers the same
+tool names shadows silently (pi-durable resolves cross-extension duplicates
+last-selected-wins) — **do not co-install**. Sigrun's extension loader
+(made the shadowing visible at boot: it `console.warn`s once per duplicate
+tool name, naming the extensions and the last-wins resolution) but the
+warning changes nothing: last-wins stands. Replace, don't co-install.

package/dist/bin.d.ts

@@ -0,0 +1,6 @@
+/**
+ * The package bin entry (`sigrun-ext-mcp` → dist/bin.js): parse-and-run the
+ * CLI, mapping the result to the process exit code. The body lives in
+ * ./cli.ts so tests can exercise it without spawning a process.
+ */
+export {};

package/dist/bin.js

@@ -0,0 +1,9 @@
+/**
+ * The package bin entry (`sigrun-ext-mcp` → dist/bin.js): parse-and-run the
+ * CLI, mapping the result to the process exit code. The body lives in
+ * ./cli.ts so tests can exercise it without spawning a process.
+ */
+import { runCli } from "./cli.js";
+void runCli(process.argv.slice(2)).then((code) => {
+    process.exitCode = code;
+});

package/dist/cli.d.ts

@@ -0,0 +1,29 @@
+/**
+ * The `sigrun-ext-mcp` CLI bin (ruling 4): `sigrun-ext-mcp oauth <server>`
+ * runs the paste-the-URL OAuth flow for one configured server.
+ *
+ * argv surface (parsed minimally — no new deps):
+ *   sigrun-ext-mcp oauth <server> [--paste]
+ *
+ * Default mode: the loopback listener runs AND stdin is read (paste
+ * fallback) — first arrival wins. `--paste` is accepted for hosts that want
+ * the semantics explicit; the flow is identical either way.
+ *
+ * Prints to stdout:
+ *   - the authorize URL (open this in a browser)
+ *   - the redirect_uri (what the AS will redirect to)
+ *   - a paste prompt
+ * Exits 0 with the token stored, or 1 with a clear error (usage / config /
+ * flow / 5-minute timeout).
+ */
+/** Parse argv minimally: `oauth <server> [--paste]`. */
+export declare function parseCliArgs(argv: readonly string[]): {
+    readonly command: "oauth";
+    readonly server: string;
+    readonly paste: boolean;
+};
+/** The CLI body (exported for tests); returns the process exit code. */
+export declare function runCli(argv: readonly string[], io?: {
+    readonly stdout: NodeJS.WriteStream;
+    readonly stderr: NodeJS.WriteStream;
+}): Promise<number>;

package/dist/cli.js

@@ -0,0 +1,109 @@
+/**
+ * The `sigrun-ext-mcp` CLI bin (ruling 4): `sigrun-ext-mcp oauth <server>`
+ * runs the paste-the-URL OAuth flow for one configured server.
+ *
+ * argv surface (parsed minimally — no new deps):
+ *   sigrun-ext-mcp oauth <server> [--paste]
+ *
+ * Default mode: the loopback listener runs AND stdin is read (paste
+ * fallback) — first arrival wins. `--paste` is accepted for hosts that want
+ * the semantics explicit; the flow is identical either way.
+ *
+ * Prints to stdout:
+ *   - the authorize URL (open this in a browser)
+ *   - the redirect_uri (what the AS will redirect to)
+ *   - a paste prompt
+ * Exits 0 with the token stored, or 1 with a clear error (usage / config /
+ * flow / 5-minute timeout).
+ */
+import { createInterface } from "node:readline";
+import { isStdioServer, loadMcpConfig, MCP_CONFIG_ENV, } from "./config.js";
+import { startOauth } from "./oauth.js";
+import { resolveMcpTokenPath } from "./tokens.js";
+/** The flow timeout (matches startOauth's default; explicit so the user's
+ * error message is deterministic). */
+const TIMEOUT_MS = 5 * 60_000;
+/** Print the usage banner. */
+function usage(out) {
+    out.write("usage: sigrun-ext-mcp oauth <server> [--paste]\n");
+    out.write(`reads the MCP config (env ${MCP_CONFIG_ENV} or ~/.config/sigrun/mcp.json);\n` +
+        `the named server needs an oauth block; the token is stored beside the\n` +
+        `config as mcp-tokens.json (0600)\n`);
+}
+/** Parse argv minimally: `oauth <server> [--paste]`. */
+export function parseCliArgs(argv) {
+    const args = [...argv];
+    const head = args.shift();
+    if (head !== "oauth") {
+        throw new Error(`unknown command: ${String(head)}`);
+    }
+    const paste = args.includes("--paste");
+    const positional = args.filter((arg) => arg !== "--paste");
+    if (positional.length !== 1) {
+        throw new Error("expected exactly one server name");
+    }
+    return { command: "oauth", server: positional[0], paste };
+}
+/** Load the config, turning a malformed config into exit-1 (not a crash). */
+function loadMcpConfigSafe(stderr) {
+    try {
+        return loadMcpConfig();
+    }
+    catch (cause) {
+        stderr.write(`error: ${cause instanceof Error ? cause.message : String(cause)}\n`);
+        process.exit(1);
+    }
+}
+/** The CLI body (exported for tests); returns the process exit code. */
+export async function runCli(argv, io = { stdout: process.stdout, stderr: process.stderr }) {
+    let parsed;
+    try {
+        parsed = parseCliArgs(argv);
+    }
+    catch (cause) {
+        usage(io.stderr);
+        io.stderr.write(`error: ${cause instanceof Error ? cause.message : String(cause)}\n`);
+        return 1;
+    }
+    const config = loadMcpConfigSafe(io.stderr);
+    const server = config.servers[parsed.server];
+    if (server === undefined) {
+        io.stderr.write(`error: no server named "${parsed.server}" in the MCP config\n`);
+        return 1;
+    }
+    if (!isStdioServer(server) || server.oauth === undefined) {
+        io.stderr.write(`error: server "${parsed.server}" has no oauth block in the MCP config\n`);
+        return 1;
+    }
+    const flow = await startOauth(parsed.server, server.oauth, {
+        timeoutMs: TIMEOUT_MS,
+    });
+    io.stdout.write(`Authorize URL:\n  ${flow.authorizeUrl}\n`);
+    io.stdout.write(`Redirect URI:\n  ${flow.redirectUri}\n`);
+    io.stdout.write("Open the Authorize URL in a browser and approve. If the redirect does\n" +
+        "not reach this machine, paste the FULL redirected URL here instead:\n");
+    // The paste fallback: every stdin line is fed to the flow; first arrival
+    // wins (the loopback listener or the paste settles the code). pasteUrl
+    // never throws (lines with no code are ignored), so the readline handler
+    // cannot crash the bin.
+    const lines = createInterface({ input: process.stdin });
+    lines.on("line", (line) => {
+        const trimmed = line.trim();
+        if (trimmed !== "")
+            flow.pasteUrl(trimmed);
+    });
+    try {
+        await flow.done;
+        lines.close();
+        io.stdout.write(`Token stored for "${parsed.server}" in:\n  ${resolveMcpTokenPath()}\n`);
+        return 0;
+    }
+    catch (cause) {
+        lines.close();
+        flow.stop();
+        io.stderr.write(`error: ${cause instanceof Error ? cause.message : String(cause)}\n`);
+        return 1;
+    }
+}
+// This module is the CLI body only (pure exports; importable for tests).
+// The bin entry lives in ./bin.ts — dist/bin.js is the package.json "bin".

package/dist/client.d.ts

@@ -0,0 +1,42 @@
+/**
+ * The MCP client: a transport-neutral JSON-RPC 2.0 engine plus two
+ * transports.
+ *
+ * stdio (the original transport): newline-delimited JSON-RPC over a spawned
+ * child's pipes. Remote (streamable HTTP): every request POSTs to the
+ * configured URL; the reply is a JSON body OR an SSE stream. A server that
+ * 404/405s the POST (the pre-2025-03-26 "HTTP+SSE" transport — still common,
+ * e.g. many self-hosted servers) falls back to the legacy shape: a GET
+ * stream for server→client frames and a discovered message endpoint for
+ * client→server POSTs.
+ *
+ * Lifecycle (both transports): `initialize` handshake (protocolVersion
+ * "2025-06-18", clientInfo, capabilities) → `notifications/initialized` →
+ * `tools/list` (discovery) → `tools/call` (execution). Requests correlate
+ * by id; the response frame's `id` must match a pending request (a
+ * notification is never a reply). A connect failure rejects `connect`,
+ * never affecting any other server (per-server isolation).
+ *
+ * Per-request timeout (codemode parity, spec slice 3): every request REJECTS
+ * at its deadline (default 60s; config `timeout` per server) instead of
+ * awaiting a hung server forever. A `notifications/progress` frame naming a
+ * pending request's id (sent as the request's `progressToken`) RESETS that
+ * request's deadline — a server that keeps reporting progress stays alive
+ * as long as it works. A timeout rejects the call but never kills the
+ * transport (per-server isolation: the next request gets a fresh chance).
+ */
+import type { McpServerConfig, McpToolDefinition, McpToolResult } from "./types.js";
+/** The public client surface (what the extension consumes). */
+export interface McpClient {
+    listTools(): Promise<McpToolDefinition[]>;
+    callTool(name: string, args: Record<string, unknown>, timeoutSeconds?: number): Promise<McpToolResult>;
+    /** Tear the connection down (idempotent). */
+    close(): void;
+}
+/**
+ * Spawn/connect one configured MCP server (stdio or remote by config
+ * shape). Throws on spawn/connect failure or a failed handshake —
+ * per-server isolation is the CALLER's policy (the extension connects all
+ * servers with `.catch` per server).
+ */
+export declare function connectMcpServer(name: string, config: McpServerConfig): Promise<McpClient>;