testmcpy 0.9.2__tar.gz → 0.10.3__tar.gz

{testmcpy-0.9.2/testmcpy.egg-info → testmcpy-0.10.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: testmcpy
-Version: 0.9.2
+Version: 0.10.3
 Summary: A comprehensive testing framework for validating LLM tool calling capabilities with MCP services
 Author: Amin Ghadersohi
 License-Expression: Apache-2.0
@@ -94,13 +94,14 @@ Dynamic: license-file
   <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python 3.10+"></a>
   <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License"></a>
   <a href="https://pypi.org/project/testmcpy/"><img src="https://img.shields.io/badge/pypi-testmcpy-blue" alt="PyPI"></a>
+  <a href="https://preset-io.github.io/testmcpy"><img src="https://img.shields.io/badge/docs-preset--io.github.io-7aa2f7" alt="Documentation"></a>
 </p>
 
 ![MCP Explorer — tools, resources, and prompts from a connected MCP service](https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/mcp-explorer.png)
 
 ---
 
-**[Documentation](context/)** | **[Examples](examples/)** | **[Contributing](CONTRIBUTING.md)** | **[Discussions](https://github.com/preset-io/testmcpy/discussions)**
+**[Documentation](https://preset-io.github.io/testmcpy)** | **[Getting Started](https://preset-io.github.io/testmcpy/getting-started)** | **[CLI Reference](https://preset-io.github.io/testmcpy/cli)** | **[Examples](examples/)** | **[Contributing](CONTRIBUTING.md)** | **[Discussions](https://github.com/preset-io/testmcpy/discussions)**
 
 ---
 
@@ -397,6 +398,8 @@ testmcpy run tests/ --model claude-haiku-4-5
 
 ## Commands Reference
 
+The highlights are below — the full reference for all 38 commands lives at **[preset-io.github.io/testmcpy/cli](https://preset-io.github.io/testmcpy/cli)**.
+
 | Command | Description |
 |---------|-------------|
 | **Setup** | |
@@ -412,6 +415,12 @@ testmcpy run tests/ --model claude-haiku-4-5
 | `testmcpy research` | Test LLM tool-calling capabilities |
 | `testmcpy chat` | Interactive chat with MCP tools |
 | `testmcpy compare` | Multi-model comparison |
+| **Quality & Benchmarking** | |
+| `testmcpy bench` | Run a suite across models × profiles × repeats |
+| `testmcpy conformance` | Run the official MCP spec conformance suite |
+| `testmcpy score` | Grade tool surface for LLM usability (0-100, A-F) |
+| `testmcpy scan` | Static security scan of tool metadata (SARIF output) |
+| `testmcpy matrix` / `leaderboard` / `flaky` | Per-test × per-config analytics |
 | **Advanced** | |
 | `testmcpy baseline-save` | Save current test results as a named baseline |
 | `testmcpy baseline-compare` | Compare a run against a saved baseline |
@@ -459,7 +468,7 @@ Environment variables are also supported: `MCP_AUTH_TOKEN`, `MCP_JWT_URL`, `MCP_
 
 ## Web Interface
 
-Optional React-based UI with 14 pages for visual testing and analytics:
+Optional React-based UI for visual testing and analytics — every page is documented at **[preset-io.github.io/testmcpy/web-ui](https://preset-io.github.io/testmcpy/web-ui)**:
 
 ![Test Manager — browse YAML suites, kick off runs, watch results stream in](https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/test-manager.png)
 
@@ -477,11 +486,9 @@ testmcpy serve
 | `/tests` | Test Manager | YAML test browser, execution, results |
 | `/reports` | Reports | All test results, evaluations, cost analysis |
 | `/chat` | Chat Interface | Multi-turn conversation with MCP tools |
-| `/metrics` | Metrics Dashboard | Performance and cost analytics |
-| `/compare` | Run Comparison | Side-by-side model comparison |
-| `/compatibility` | Compatibility Matrix | Tool/model compatibility view |
-| `/mcp-health` | MCP Health | Server health monitoring |
-| `/security` | Security Dashboard | Security analysis |
+| `/performance` | Performance | Per-test matrix and config leaderboard (also serves `/metrics`, `/compare`) |
+| `/servers` | Servers | Health monitoring + cross-server schema compatibility (also serves `/mcp-health`, `/compatibility`) |
+| `/security` | Security Dashboard | Security evaluator results and risk summary |
 | `/generation-history` | Generation History | AI test generation logs |
 | `/auth-debugger` | Auth Debugger | Auth flow debugging |
 | `/config` | Configuration | Settings and environment |
@@ -498,7 +505,19 @@ Access at `http://localhost:8000`.
     <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/auth-debugger.png" alt="Auth Debugger page"><br><sub>Auth Debugger — step through OAuth / JWT / Bearer flows</sub></td>
   </tr>
   <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/metrics.png" alt="Performance matrix page"><br><sub>Performance — per-test results across model and MCP configurations</sub></td>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/compare.png" alt="Leaderboard page"><br><sub>Leaderboard — configs ranked by pass rate, cost-per-pass, latency</sub></td>
+  </tr>
+  <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/security.png" alt="Security Dashboard page"><br><sub>Security Dashboard — security evaluator results and risk summary</sub></td>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/compatibility.png" alt="Schema compatibility page"><br><sub>Schema Compat — cross-server tool schema compatibility matrix</sub></td>
+  </tr>
+  <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/mcp-health.png" alt="Server health page"><br><sub>Servers — MCP server health monitoring</sub></td>
     <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/mcp-profiles.png" alt="MCP Profiles page"><br><sub>MCP Profiles — manage MCP service connections</sub></td>
+  </tr>
+  <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/llm-profiles.png" alt="LLM Profiles page"><br><sub>LLM Profiles — provider configurations with model pricing</sub></td>
     <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/config.png" alt="Configuration page"><br><sub>Configuration — current settings and client snippets</sub></td>
   </tr>
 </table>
@@ -620,7 +639,7 @@ class MyEvaluator(BaseEvaluator):
         )
 ```
 
-See **[Evaluator Reference](context/concepts/evaluators.md)** for complete documentation.
+See the **[Evaluator Reference](https://preset-io.github.io/testmcpy/concepts/evaluators)** and the **[Custom Evaluators guide](https://preset-io.github.io/testmcpy/guides/custom-evaluators)** for complete documentation.
 
 ## Examples
 
@@ -641,7 +660,7 @@ We welcome contributions! Whether it's bug reports, feature requests, documentat
 
 - **Issues**: [Report bugs or request features](https://github.com/preset-io/testmcpy/issues)
 - **Discussions**: [Ask questions and share ideas](https://github.com/preset-io/testmcpy/discussions)
-- **Documentation**: Browse the [context/](context/) directory
+- **Documentation**: [preset-io.github.io/testmcpy](https://preset-io.github.io/testmcpy) (agent-facing source docs live in [context/](context/))
 - **Examples**: Explore [examples/](examples/) for sample code
 
 ## License

{testmcpy-0.9.2 → testmcpy-0.10.3}/README.md

@@ -14,13 +14,14 @@
   <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="Python 3.10+"></a>
   <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License"></a>
   <a href="https://pypi.org/project/testmcpy/"><img src="https://img.shields.io/badge/pypi-testmcpy-blue" alt="PyPI"></a>
+  <a href="https://preset-io.github.io/testmcpy"><img src="https://img.shields.io/badge/docs-preset--io.github.io-7aa2f7" alt="Documentation"></a>
 </p>
 
 ![MCP Explorer — tools, resources, and prompts from a connected MCP service](https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/mcp-explorer.png)
 
 ---
 
-**[Documentation](context/)** | **[Examples](examples/)** | **[Contributing](CONTRIBUTING.md)** | **[Discussions](https://github.com/preset-io/testmcpy/discussions)**
+**[Documentation](https://preset-io.github.io/testmcpy)** | **[Getting Started](https://preset-io.github.io/testmcpy/getting-started)** | **[CLI Reference](https://preset-io.github.io/testmcpy/cli)** | **[Examples](examples/)** | **[Contributing](CONTRIBUTING.md)** | **[Discussions](https://github.com/preset-io/testmcpy/discussions)**
 
 ---
 
@@ -317,6 +318,8 @@ testmcpy run tests/ --model claude-haiku-4-5
 
 ## Commands Reference
 
+The highlights are below — the full reference for all 38 commands lives at **[preset-io.github.io/testmcpy/cli](https://preset-io.github.io/testmcpy/cli)**.
+
 | Command | Description |
 |---------|-------------|
 | **Setup** | |
@@ -332,6 +335,12 @@ testmcpy run tests/ --model claude-haiku-4-5
 | `testmcpy research` | Test LLM tool-calling capabilities |
 | `testmcpy chat` | Interactive chat with MCP tools |
 | `testmcpy compare` | Multi-model comparison |
+| **Quality & Benchmarking** | |
+| `testmcpy bench` | Run a suite across models × profiles × repeats |
+| `testmcpy conformance` | Run the official MCP spec conformance suite |
+| `testmcpy score` | Grade tool surface for LLM usability (0-100, A-F) |
+| `testmcpy scan` | Static security scan of tool metadata (SARIF output) |
+| `testmcpy matrix` / `leaderboard` / `flaky` | Per-test × per-config analytics |
 | **Advanced** | |
 | `testmcpy baseline-save` | Save current test results as a named baseline |
 | `testmcpy baseline-compare` | Compare a run against a saved baseline |
@@ -379,7 +388,7 @@ Environment variables are also supported: `MCP_AUTH_TOKEN`, `MCP_JWT_URL`, `MCP_
 
 ## Web Interface
 
-Optional React-based UI with 14 pages for visual testing and analytics:
+Optional React-based UI for visual testing and analytics — every page is documented at **[preset-io.github.io/testmcpy/web-ui](https://preset-io.github.io/testmcpy/web-ui)**:
 
 ![Test Manager — browse YAML suites, kick off runs, watch results stream in](https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/test-manager.png)
 
@@ -397,11 +406,9 @@ testmcpy serve
 | `/tests` | Test Manager | YAML test browser, execution, results |
 | `/reports` | Reports | All test results, evaluations, cost analysis |
 | `/chat` | Chat Interface | Multi-turn conversation with MCP tools |
-| `/metrics` | Metrics Dashboard | Performance and cost analytics |
-| `/compare` | Run Comparison | Side-by-side model comparison |
-| `/compatibility` | Compatibility Matrix | Tool/model compatibility view |
-| `/mcp-health` | MCP Health | Server health monitoring |
-| `/security` | Security Dashboard | Security analysis |
+| `/performance` | Performance | Per-test matrix and config leaderboard (also serves `/metrics`, `/compare`) |
+| `/servers` | Servers | Health monitoring + cross-server schema compatibility (also serves `/mcp-health`, `/compatibility`) |
+| `/security` | Security Dashboard | Security evaluator results and risk summary |
 | `/generation-history` | Generation History | AI test generation logs |
 | `/auth-debugger` | Auth Debugger | Auth flow debugging |
 | `/config` | Configuration | Settings and environment |
@@ -418,7 +425,19 @@ Access at `http://localhost:8000`.
     <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/auth-debugger.png" alt="Auth Debugger page"><br><sub>Auth Debugger — step through OAuth / JWT / Bearer flows</sub></td>
   </tr>
   <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/metrics.png" alt="Performance matrix page"><br><sub>Performance — per-test results across model and MCP configurations</sub></td>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/compare.png" alt="Leaderboard page"><br><sub>Leaderboard — configs ranked by pass rate, cost-per-pass, latency</sub></td>
+  </tr>
+  <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/security.png" alt="Security Dashboard page"><br><sub>Security Dashboard — security evaluator results and risk summary</sub></td>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/compatibility.png" alt="Schema compatibility page"><br><sub>Schema Compat — cross-server tool schema compatibility matrix</sub></td>
+  </tr>
+  <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/mcp-health.png" alt="Server health page"><br><sub>Servers — MCP server health monitoring</sub></td>
     <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/mcp-profiles.png" alt="MCP Profiles page"><br><sub>MCP Profiles — manage MCP service connections</sub></td>
+  </tr>
+  <tr>
+    <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/llm-profiles.png" alt="LLM Profiles page"><br><sub>LLM Profiles — provider configurations with model pricing</sub></td>
     <td align="center"><img src="https://raw.githubusercontent.com/preset-io/testmcpy/main/docs/screenshots/config.png" alt="Configuration page"><br><sub>Configuration — current settings and client snippets</sub></td>
   </tr>
 </table>
@@ -540,7 +559,7 @@ class MyEvaluator(BaseEvaluator):
         )
 ```
 
-See **[Evaluator Reference](context/concepts/evaluators.md)** for complete documentation.
+See the **[Evaluator Reference](https://preset-io.github.io/testmcpy/concepts/evaluators)** and the **[Custom Evaluators guide](https://preset-io.github.io/testmcpy/guides/custom-evaluators)** for complete documentation.
 
 ## Examples
 
@@ -561,7 +580,7 @@ We welcome contributions! Whether it's bug reports, feature requests, documentat
 
 - **Issues**: [Report bugs or request features](https://github.com/preset-io/testmcpy/issues)
 - **Discussions**: [Ask questions and share ideas](https://github.com/preset-io/testmcpy/discussions)
-- **Documentation**: Browse the [context/](context/) directory
+- **Documentation**: [preset-io.github.io/testmcpy](https://preset-io.github.io/testmcpy) (agent-facing source docs live in [context/](context/))
 - **Examples**: Explore [examples/](examples/) for sample code
 
 ## License

{testmcpy-0.9.2 → testmcpy-0.10.3}/pyproject.toml

@@ -93,7 +93,7 @@ testmcpy = [
 
 [project]
 name = "testmcpy"
-version = "0.9.2"
+version = "0.10.3"
 description = "A comprehensive testing framework for validating LLM tool calling capabilities with MCP services"
 authors = [{name = "Amin Ghadersohi"}]
 license = "Apache-2.0"

{testmcpy-0.9.2 → testmcpy-0.10.3}/testmcpy/cli/commands/wizard.py

@@ -104,40 +104,32 @@ def add_mcp():
     # Step 5: Test Connection
     console.print("\n[bold yellow]Step 5: Test Connection[/bold yellow]")
     if Confirm.ask("Test connection now?", default=True):
-        from testmcpy.mcp_profiles import AuthConfig, MCPServer
-
-        test_url = mcp_url if transport == "sse" else f"stdio://{command}"
-        mcp_server = MCPServer(
-            name=name,
-            mcp_url=test_url,
-            auth=AuthConfig(auth_type=auth_config.get("type", "none")),
-            timeout=timeout,
-            rate_limit_rpm=rate_limit,
-            transport=transport,
-            command=command if transport == "stdio" else None,
-            args=args_str.split() if args_str else None,
-        )
-
         console.print("[dim]Connecting...[/dim]")
         try:
-            from testmcpy.src.mcp_client import MCPClient
-
-            client = MCPClient(
-                mcp_url=mcp_server.mcp_url,
-                auth=mcp_server.auth.to_dict() if mcp_server.auth else None,
-                timeout=mcp_server.timeout,
-                transport=mcp_server.transport,
-                command=mcp_server.command,
-                args=mcp_server.args,
-            )
-            tools = asyncio.run(client.list_tools())
+            from testmcpy.src.mcp_client import MCPClient, MCPError, StdioMCPClient
+
+            if transport == "stdio":
+                client: MCPClient | StdioMCPClient = StdioMCPClient(
+                    command=command, args=args_str.split() if args_str else None
+                )
+            else:
+                client = MCPClient(mcp_url, auth=auth_config)
+
+            async def _test_connection():
+                try:
+                    await client.initialize(timeout=float(timeout))
+                    return await client.list_tools(timeout=float(timeout))
+                finally:
+                    await client.close()
+
+            tools = asyncio.run(_test_connection())
             console.print(f"[green]Connected! Found {len(tools)} tools.[/green]")
             if tools:
                 tool_names = [t.name if hasattr(t, "name") else str(t) for t in tools[:5]]
                 console.print(
                     f"[dim]  Tools: {', '.join(tool_names)}{'...' if len(tools) > 5 else ''}[/dim]"
                 )
-        except (ConnectionError, TimeoutError, OSError, RuntimeError, ValueError) as e:
+        except (MCPError, ConnectionError, TimeoutError, OSError, RuntimeError, ValueError) as e:
             console.print(f"[red]Connection failed: {e}[/red]")
             if not Confirm.ask("Continue anyway?", default=True):
                 raise typer.Abort()

{testmcpy-0.9.2 → testmcpy-0.10.3}/testmcpy/server/routers/mcp_profiles.py

@@ -1,6 +1,7 @@
 """MCP profile management endpoints."""
 
 import copy
+import logging
 import re
 from pathlib import Path
 
@@ -14,7 +15,9 @@ from testmcpy.server.helpers import (
     load_mcp_yaml,
     save_mcp_yaml,
 )
-from testmcpy.src.mcp_client import MCPClient
+from testmcpy.src.mcp_client import MCPClient, MCPError, StdioMCPClient
+
+logger = logging.getLogger(__name__)
 
 router = APIRouter(prefix="/api/mcp", tags=["mcp-profiles"])
 
@@ -87,6 +90,15 @@ class MCPReorderRequest(BaseModel):
     to_index: int
 
 
+class TestConnectionRequest(BaseModel):
+    mcp_url: str
+    transport: str = "sse"
+    command: str | None = None
+    args: list[str] | None = None
+    auth: AuthConfig | None = None
+    timeout: int | None = None
+
+
 # Helper to get the mcp_clients dict from api module
 def get_mcp_clients() -> dict[str, MCPClient]:
     """Get the global mcp_clients dict from api module."""
@@ -872,3 +884,45 @@ async def test_mcp_connection(profile_id: str, mcp_index: int):
         raise
     except Exception as e:
         raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.post("/test-connection")
+async def test_connection_standalone(request: TestConnectionRequest):
+    """Test connection to an MCP server from an inline config.
+
+    Used by the Add MCP wizard to test a server before it is saved to any
+    profile (unlike /profiles/{id}/test-connection/{idx}, which tests a
+    saved MCP).
+    """
+    timeout = float(request.timeout) if request.timeout else 30.0
+
+    if request.transport == "stdio":
+        command = request.command or request.mcp_url.removeprefix("stdio://")
+        if not command:
+            raise HTTPException(status_code=400, detail="command is required for stdio transport")
+        client: MCPClient | StdioMCPClient = StdioMCPClient(command=command, args=request.args)
+    else:
+        if not request.mcp_url:
+            raise HTTPException(status_code=400, detail="mcp_url is required")
+        # AuthConfig already matches the {type, token, ...} dict shape
+        # MCPClient._setup_auth() reads (it tolerates extra keys).
+        auth_dict = request.auth.model_dump(exclude_none=True) if request.auth else {"type": "none"}
+        client = MCPClient(request.mcp_url, auth=auth_dict)
+
+    try:
+        await client.initialize(timeout=timeout)
+        tools = await client.list_tools(timeout=timeout)
+        return {
+            "success": True,
+            "status": "connected",
+            "message": f"Successfully connected — found {len(tools)} tools",
+            "tool_count": len(tools),
+            "tools": [tool.name for tool in tools],
+        }
+    except (MCPError, ConnectionError, TimeoutError, OSError, RuntimeError, ValueError) as e:
+        return {"success": False, "status": "error", "message": f"Connection failed: {str(e)}"}
+    finally:
+        try:
+            await client.close()
+        except (MCPError, ConnectionError, OSError, RuntimeError) as e:
+            logger.warning("Failed to close test client: %s", e)

{testmcpy-0.9.2 → testmcpy-0.10.3}/testmcpy/server/routers/tools.py

@@ -13,7 +13,7 @@ from testmcpy.config import get_config
 from testmcpy.mcp_profiles import load_profile
 from testmcpy.server.state import get_default_mcp_client, get_mcp_clients
 from testmcpy.src.llm_integration import create_llm_provider
-from testmcpy.src.mcp_client import MCPClient, MCPError
+from testmcpy.src.mcp_client import MCPClient, MCPError, MCPToolCall
 
 logger = logging.getLogger(__name__)
 
@@ -1010,16 +1010,25 @@ async def compare_tools(request: ToolCompareRequest):
             start_time = time.time()
 
             # Initialize client
-            client = MCPClient(mcp_url=mcp_config.get_mcp_url(), auth=mcp_config.auth)
+            client = MCPClient(
+                base_url=mcp_config.mcp_url,
+                auth=mcp_config.auth.to_dict() if mcp_config.auth else None,
+            )
             await client.initialize()
 
             # Call the tool
             tool_result = await client.call_tool(
-                name=request.tool_name, arguments=request.parameters
+                MCPToolCall(name=request.tool_name, arguments=request.parameters)
             )
 
-            result["success"] = True
-            result["result"] = tool_result
+            if tool_result.is_error:
+                result["success"] = False
+                result["error"] = tool_result.error_message or "Tool call failed"
+            else:
+                from testmcpy.server.api import _serialize_tool_content
+
+                result["success"] = True
+                result["result"] = _serialize_tool_content(tool_result.content)
             result["duration_ms"] = (time.time() - start_time) * 1000
 
         except Exception as e:
@@ -1029,9 +1038,9 @@ async def compare_tools(request: ToolCompareRequest):
         finally:
             if client:
                 try:
-                    await client.cleanup()
-                except Exception:
-                    pass
+                    await client.close()
+                except (MCPError, ConnectionError, OSError, RuntimeError) as e:
+                    logger.warning("Failed to close comparison client: %s", e)
 
         return result
 
@@ -1116,7 +1125,10 @@ async def debug_tool(tool_name: str, request: ToolDebugRequest):
                     profile_data = load_profile(request.profile)
                     if profile_data and profile_data.mcps:
                         mcp_config = profile_data.mcps[0]
-                        client = MCPClient(mcp_url=mcp_config.get_mcp_url(), auth=mcp_config.auth)
+                        client = MCPClient(
+                            base_url=mcp_config.mcp_url,
+                            auth=mcp_config.auth.to_dict() if mcp_config.auth else None,
+                        )
                         await client.initialize()
                         mcp_clients[client_key] = client
                 except Exception as e:
@@ -1137,7 +1149,15 @@ async def debug_tool(tool_name: str, request: ToolDebugRequest):
 
         # Step 2: Call tool
         try:
-            response = await client.call_tool(tool_name, request.parameters)
+            tool_result = await client.call_tool(
+                MCPToolCall(name=tool_name, arguments=request.parameters)
+            )
+            if tool_result.is_error:
+                raise MCPError(tool_result.error_message or "Tool call failed")
+
+            from testmcpy.server.api import _serialize_tool_content
+
+            response = _serialize_tool_content(tool_result.content)
 
             steps.append(
                 {