aixtools 0.1.11__tar.gz → 0.2.0__tar.gz

{aixtools-0.1.11 → aixtools-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: aixtools
-Version: 0.1.11
+Version: 0.2.0
 Summary: Tools for AI exploration and debugging
 Requires-Python: >=3.11.2
 Description-Content-Type: text/markdown
@@ -18,7 +18,8 @@ Requires-Dist: langchain-openai>=0.3.14
 Requires-Dist: mcp>=1.11.0
 Requires-Dist: mypy>=1.18.2
 Requires-Dist: pandas>=2.2.3
-Requires-Dist: pydantic-ai>=0.4.10
+Requires-Dist: pydantic-evals>=0.4.10
+Requires-Dist: pydantic-ai>=1.0.9
 Requires-Dist: pylint>=3.3.7
 Requires-Dist: rich>=14.0.0
 Requires-Dist: ruff>=0.11.6
@@ -45,20 +46,16 @@ A2A
 - Google SDK Integration for A2A - `aixtools/a2a/google_sdk/`
 - PydanticAI Adapter for Google SDK - `aixtools/a2a/google_sdk/pydantic_ai_adapter/`
 
-Databases
-- Database Integration - `aixtools/db/`
-- Vector Database Support - `aixtools/db/vector_db.py`
-
 Logging & Debugging
-- Log Viewing Application - `aixtools/log_view/`
+- Log Viewing Web Application - `log_view`
 - Object Logging System - `aixtools/logging/`
 - Model Patch Logging - `aixtools/logging/model_patch_logging.py`
 - Log Filtering System - `aixtools/logfilters/`
 - FastMCP Logging - `aixtools/mcp/fast_mcp_log.py`
-- Command Line Interface for Log Viewing - Entry point: `log_view`
 - MCP (Model Context Protocol) Support - `aixtools/logging/mcp_log_models.py`, `aixtools/logging/mcp_logger.py`
 
-Testing & Tools
+Testing Tools & Evals
+- Evaluations - `aixtools/evals/` - Entry point: `evals`
 - Testing Utilities - `aixtools/testing/`
 - Mock Tool System - `aixtools/testing/mock_tool.py`
 - Model Patch Caching - `aixtools/testing/model_patch_cache.py`
@@ -66,6 +63,10 @@ Testing & Tools
 - Tool Recommendation Engine - `aixtools/tools/doctor/tool_recommendation.py`
 - FaultyMCP - `aixtools/mcp/faulty_mcp.py`
 
+Databases
+- Database Integration - `aixtools/db/`
+- Vector Database Support - `aixtools/db/vector_db.py`
+
 Chainlit & HTTP Server
 - Chainlit Integration - `aixtools/app.py`, `aixtools/chainlit.md`
 - Chainlit Utilities - `aixtools/utils/chainlit/`
@@ -81,28 +82,11 @@ Programming utils
 
 ## Installation
 
-### From GitHub
-
 ```bash
 uv add aixtools
 ```
 
-### Development Setup
-
-```bash
-# Create a new project
-uv init MyNewProject
-cd MyNewProject
-
-# Add virtual environment and activate it
-uv venv .venv
-source .venv/bin/activate
-
-# Add this package
-uv add aixtools
-```
-
-### Updating
+**Updating**
 
 ```bash
 uv add --upgrade aixtools
@@ -114,6 +98,7 @@ AIXtools requires environment variables for model providers.
 
 **IMPORTANT:** Create a `.env` file based on [`.env_template`](./.env_template):
 
+Here is an example configuration:
 ```bash
 # Model family (azure, openai, or ollama)
 MODEL_FAMILY=azure
@@ -161,6 +146,33 @@ agent = get_agent(system_prompt="You are a helpful assistant.")
 result, nodes = await run_agent(agent, "Tell me about AI")
 ```
 
+### Node Debugging and Visualization
+
+The [`print_nodes`](aixtools/agents/print_nodes.py) module provides a clean, indented output for easy reading of the node from agent execution.
+
+```python
+from aixtools.agents.print_nodes import print_nodes, print_node
+from aixtools.agents.agent import get_agent, run_agent
+
+agent = get_agent(system_prompt="You are a helpful assistant.")
+result, nodes = await run_agent(agent, "Explain quantum computing")
+# Print all execution nodes for debugging
+print_nodes(nodes)
+```
+
+**Features:**
+- **Node Type Detection**: Automatically handles different node types (`UserPromptNode`, `CallToolsNode`, `ModelRequestNode`, `End`)
+- **Formatted Output**: Provides clean, indented output for easy reading
+- **Tool Call Visualization**: Shows tool names and arguments for tool calls
+- **Text Content Display**: Formats text parts with proper indentation
+- **Model Request Summary**: Shows character count for model requests to avoid verbose output
+
+**Node Types Supported:**
+- `UserPromptNode` - Displays user prompts with indentation
+- `CallToolsNode` - Shows tool calls with names and arguments
+- `ModelRequestNode` - Summarizes model requests with character count
+- `End` - Marks the end of execution (output suppressed by default)
+
 ### Agent Batch Processing
 
 Process multiple agent queries simultaneously with built-in concurrency control and result aggregation.
@@ -299,9 +311,39 @@ with ObjectLogger() as logger:
     logger.log(agent_response)
 ```
 
-### MCP Logger
+### MCP logging
+
+AIXtools provides  MCP support for both client and server implementations with easier logging for debugging pourposes.
+
+**Example:** 
+
+Let's assume we have an MCP server that runs an agent tool.
+
+Note that the `ctx: Context` parameter is passed to `run_agent`, this will enable logging to the MCP client.
+
+```python
+@mcp.tool
+async def my_tool_with_agent(query: str, ctx: Context) -> str:
+    """ A tool that uses an gents to process the query """
+    agent = get_agent()
+    async with get_qb_agent() as agent:
+        ret, nodes = await run_agent(agent=agent, prompt=query, ctx=ctx)    # Enable MCP logging
+        return str(ret)
+```
+
+On the client side, you can create an agent connected to the MCP server, the "nodes" from the MCP server will show on the STDOUT so you can see what's going on the MCP server's agent loop
+
+```python
+mcp = get_mcp_client("http://localhost:8000")   # Get an MCP client with a default log handler that prints to STDOUT
+agent = get_agent(toolsets=[mcp])
+async with agent:
+    # The messages from the MCP server will be printed to the STDOUT
+    ret, nodes = await run_agent(agent, prompt="...")
+```
+
+#### MCP Server Logging
 
-This is an MCP server that can log MCP requests and responses.
+Create MCP servers with built-in logging capabilities.
 
 ```python
 from aixtools.mcp.fast_mcp_log import FastMcpLog
@@ -371,6 +413,77 @@ By default, the "FaultyMCP" includes several tools you can use in your tests:
 - `freeze_server(seconds)` - Simulates server freeze
 - `throw_404_exception()` - Throws HTTP 404 error
 
+### Evals
+
+Run comprehensive Agent/LLM evaluations using the built-in evaluation discovery based on Pydantic-AI framework.
+
+```bash
+# Run all evaluations
+evals
+
+# Run evaluations with filtering
+evals --filter "specific_test"
+
+# Run with verbose output and detailed reporting
+evals --verbose --include-input --include-output --include-reasons
+
+# Specify custom evaluations directory
+evals --evals-dir /path/to/evals
+
+# Set minimum assertions threshold
+evals --min-assertions 0.8
+```
+
+**Command Line Options:**
+- `--evals-dir` - Directory containing eval_*.py files (default: evals)
+- `--filter` - Filter to run only matching evaluations
+- `--include-input` - Include input in report output
+- `--include-output` - Include output in report output
+- `--include-evaluator-failures` - Include evaluator failures in report
+- `--include-reasons` - Include reasons in report output
+- `--min-assertions` - Minimum assertions average required for success (default: 1.0)
+- `--verbose` - Print detailed information about discovery and processing
+
+The evaluation system discovers and runs all Dataset objects from eval_*.py files in the specified directory, similar to test runners but specifically designed for LLM evaluations using pydantic_evals.
+
+**Discovery Mechanism:**
+
+The evaluation framework uses an automatic discovery system that:
+
+1. **File Discovery**: Scans the specified directory for files matching the pattern `eval_*.py`
+2. **Dataset Discovery**: Within each file, looks for variables named `dataset_*` that are instances of `pydantic_evals.Dataset`
+3. **Target Function Discovery**: Automatically finds the first async function in each module that doesn't start with an underscore (`_`) to use as the evaluation target
+4. **Filtering**: Supports filtering by module name, file name, dataset name, or fully qualified name
+
+**Example Evaluation File Structure:**
+```python
+# eval_math_operations.py
+from pydantic_evals import Dataset, Case
+
+# This dataset will be discovered automatically
+dataset_addition = Dataset(
+    name="Addition Tests",
+    cases=[
+        Case(input="What is 2 + 2?", expected="4"),
+        Case(input="What is 10 + 5?", expected="15"),
+    ],
+    evaluators=[...]
+)
+
+# This function will be used as the evaluation target
+async def evaluate_math_agent(input_text: str) -> str:
+    # Your agent evaluation logic here
+    agent = get_agent(system_prompt="You are a math assistant.")
+    result, _ = await run_agent(agent, input_text)
+    return result
+```
+
+The discovery system will:
+- Find `eval_math_operations.py` in the evals directory
+- Discover `dataset_addition` as an evaluation dataset
+- Use `evaluate_math_agent` as the target function for evaluation
+- Run each case through the target function and evaluate results
+
 ## Testing & Tools
 
 AIXtools provides comprehensive testing utilities and diagnostic tools for AI agent development and debugging.
@@ -397,7 +510,49 @@ test_model = AixTestModel()
 
 ### Tool Doctor System
 
-Automated tool analysis and recommendation system for optimizing agent tool usage.
+Automated tool analysis and recommendation system for optimizing agent tool usage and analyzing MCP servers.
+
+#### MCP Tool Doctor
+
+Analyze tools from MCP (Model Context Protocol) servers and receive AI-powered recommendations for improvement.
+
+```python
+from aixtools.tools.doctor.mcp_tool_doctor import tool_doctor_mcp
+from pydantic_ai.mcp import MCPServerStreamableHTTP, MCPServerStdio
+
+# Analyze HTTP MCP server
+recommendations = await tool_doctor_mcp(mcp_url='http://127.0.0.1:8000/mcp')
+for rec in recommendations:
+    print(rec)
+
+# Analyze STDIO MCP server
+server = MCPServerStdio(command='fastmcp', args=['run', 'my_server.py'])
+recommendations = await tool_doctor_mcp(mcp_server=server, verbose=True)
+```
+
+**Command Line Usage:**
+
+```bash
+# Analyze HTTP MCP server (default)
+tool_doctor_mcp
+
+# Analyze specific HTTP MCP server
+tool_doctor_mcp --mcp-url http://localhost:9000/mcp --verbose
+
+# Analyze STDIO MCP server
+tool_doctor_mcp --stdio-command fastmcp --stdio-args run my_server.py --debug
+
+# Available options:
+# --mcp-url URL          URL of HTTP MCP server (default: http://127.0.0.1:8000/mcp)
+# --stdio-command CMD    Command to run STDIO MCP server
+# --stdio-args ARGS      Arguments for STDIO MCP server command
+# --verbose              Enable verbose output
+# --debug                Enable debug output
+```
+
+#### Traditional Tool Doctor
+
+Analyze tool usage patterns from agent logs and get optimization recommendations.
 
 ```python
 from aixtools.tools.doctor.tool_doctor import ToolDoctor

{aixtools-0.1.11 → aixtools-0.2.0}/README.md

@@ -14,20 +14,16 @@ A2A
 - Google SDK Integration for A2A - `aixtools/a2a/google_sdk/`
 - PydanticAI Adapter for Google SDK - `aixtools/a2a/google_sdk/pydantic_ai_adapter/`
 
-Databases
-- Database Integration - `aixtools/db/`
-- Vector Database Support - `aixtools/db/vector_db.py`
-
 Logging & Debugging
-- Log Viewing Application - `aixtools/log_view/`
+- Log Viewing Web Application - `log_view`
 - Object Logging System - `aixtools/logging/`
 - Model Patch Logging - `aixtools/logging/model_patch_logging.py`
 - Log Filtering System - `aixtools/logfilters/`
 - FastMCP Logging - `aixtools/mcp/fast_mcp_log.py`
-- Command Line Interface for Log Viewing - Entry point: `log_view`
 - MCP (Model Context Protocol) Support - `aixtools/logging/mcp_log_models.py`, `aixtools/logging/mcp_logger.py`
 
-Testing & Tools
+Testing Tools & Evals
+- Evaluations - `aixtools/evals/` - Entry point: `evals`
 - Testing Utilities - `aixtools/testing/`
 - Mock Tool System - `aixtools/testing/mock_tool.py`
 - Model Patch Caching - `aixtools/testing/model_patch_cache.py`
@@ -35,6 +31,10 @@ Testing & Tools
 - Tool Recommendation Engine - `aixtools/tools/doctor/tool_recommendation.py`
 - FaultyMCP - `aixtools/mcp/faulty_mcp.py`
 
+Databases
+- Database Integration - `aixtools/db/`
+- Vector Database Support - `aixtools/db/vector_db.py`
+
 Chainlit & HTTP Server
 - Chainlit Integration - `aixtools/app.py`, `aixtools/chainlit.md`
 - Chainlit Utilities - `aixtools/utils/chainlit/`
@@ -50,28 +50,11 @@ Programming utils
 
 ## Installation
 
-### From GitHub
-
 ```bash
 uv add aixtools
 ```
 
-### Development Setup
-
-```bash
-# Create a new project
-uv init MyNewProject
-cd MyNewProject
-
-# Add virtual environment and activate it
-uv venv .venv
-source .venv/bin/activate
-
-# Add this package
-uv add aixtools
-```
-
-### Updating
+**Updating**
 
 ```bash
 uv add --upgrade aixtools
@@ -83,6 +66,7 @@ AIXtools requires environment variables for model providers.
 
 **IMPORTANT:** Create a `.env` file based on [`.env_template`](./.env_template):
 
+Here is an example configuration:
 ```bash
 # Model family (azure, openai, or ollama)
 MODEL_FAMILY=azure
@@ -130,6 +114,33 @@ agent = get_agent(system_prompt="You are a helpful assistant.")
 result, nodes = await run_agent(agent, "Tell me about AI")
 ```
 
+### Node Debugging and Visualization
+
+The [`print_nodes`](aixtools/agents/print_nodes.py) module provides a clean, indented output for easy reading of the node from agent execution.
+
+```python
+from aixtools.agents.print_nodes import print_nodes, print_node
+from aixtools.agents.agent import get_agent, run_agent
+
+agent = get_agent(system_prompt="You are a helpful assistant.")
+result, nodes = await run_agent(agent, "Explain quantum computing")
+# Print all execution nodes for debugging
+print_nodes(nodes)
+```
+
+**Features:**
+- **Node Type Detection**: Automatically handles different node types (`UserPromptNode`, `CallToolsNode`, `ModelRequestNode`, `End`)
+- **Formatted Output**: Provides clean, indented output for easy reading
+- **Tool Call Visualization**: Shows tool names and arguments for tool calls
+- **Text Content Display**: Formats text parts with proper indentation
+- **Model Request Summary**: Shows character count for model requests to avoid verbose output
+
+**Node Types Supported:**
+- `UserPromptNode` - Displays user prompts with indentation
+- `CallToolsNode` - Shows tool calls with names and arguments
+- `ModelRequestNode` - Summarizes model requests with character count
+- `End` - Marks the end of execution (output suppressed by default)
+
 ### Agent Batch Processing
 
 Process multiple agent queries simultaneously with built-in concurrency control and result aggregation.
@@ -268,9 +279,39 @@ with ObjectLogger() as logger:
     logger.log(agent_response)
 ```
 
-### MCP Logger
+### MCP logging
+
+AIXtools provides  MCP support for both client and server implementations with easier logging for debugging pourposes.
+
+**Example:** 
+
+Let's assume we have an MCP server that runs an agent tool.
+
+Note that the `ctx: Context` parameter is passed to `run_agent`, this will enable logging to the MCP client.
+
+```python
+@mcp.tool
+async def my_tool_with_agent(query: str, ctx: Context) -> str:
+    """ A tool that uses an gents to process the query """
+    agent = get_agent()
+    async with get_qb_agent() as agent:
+        ret, nodes = await run_agent(agent=agent, prompt=query, ctx=ctx)    # Enable MCP logging
+        return str(ret)
+```
+
+On the client side, you can create an agent connected to the MCP server, the "nodes" from the MCP server will show on the STDOUT so you can see what's going on the MCP server's agent loop
+
+```python
+mcp = get_mcp_client("http://localhost:8000")   # Get an MCP client with a default log handler that prints to STDOUT
+agent = get_agent(toolsets=[mcp])
+async with agent:
+    # The messages from the MCP server will be printed to the STDOUT
+    ret, nodes = await run_agent(agent, prompt="...")
+```
+
+#### MCP Server Logging
 
-This is an MCP server that can log MCP requests and responses.
+Create MCP servers with built-in logging capabilities.
 
 ```python
 from aixtools.mcp.fast_mcp_log import FastMcpLog
@@ -340,6 +381,77 @@ By default, the "FaultyMCP" includes several tools you can use in your tests:
 - `freeze_server(seconds)` - Simulates server freeze
 - `throw_404_exception()` - Throws HTTP 404 error
 
+### Evals
+
+Run comprehensive Agent/LLM evaluations using the built-in evaluation discovery based on Pydantic-AI framework.
+
+```bash
+# Run all evaluations
+evals
+
+# Run evaluations with filtering
+evals --filter "specific_test"
+
+# Run with verbose output and detailed reporting
+evals --verbose --include-input --include-output --include-reasons
+
+# Specify custom evaluations directory
+evals --evals-dir /path/to/evals
+
+# Set minimum assertions threshold
+evals --min-assertions 0.8
+```
+
+**Command Line Options:**
+- `--evals-dir` - Directory containing eval_*.py files (default: evals)
+- `--filter` - Filter to run only matching evaluations
+- `--include-input` - Include input in report output
+- `--include-output` - Include output in report output
+- `--include-evaluator-failures` - Include evaluator failures in report
+- `--include-reasons` - Include reasons in report output
+- `--min-assertions` - Minimum assertions average required for success (default: 1.0)
+- `--verbose` - Print detailed information about discovery and processing
+
+The evaluation system discovers and runs all Dataset objects from eval_*.py files in the specified directory, similar to test runners but specifically designed for LLM evaluations using pydantic_evals.
+
+**Discovery Mechanism:**
+
+The evaluation framework uses an automatic discovery system that:
+
+1. **File Discovery**: Scans the specified directory for files matching the pattern `eval_*.py`
+2. **Dataset Discovery**: Within each file, looks for variables named `dataset_*` that are instances of `pydantic_evals.Dataset`
+3. **Target Function Discovery**: Automatically finds the first async function in each module that doesn't start with an underscore (`_`) to use as the evaluation target
+4. **Filtering**: Supports filtering by module name, file name, dataset name, or fully qualified name
+
+**Example Evaluation File Structure:**
+```python
+# eval_math_operations.py
+from pydantic_evals import Dataset, Case
+
+# This dataset will be discovered automatically
+dataset_addition = Dataset(
+    name="Addition Tests",
+    cases=[
+        Case(input="What is 2 + 2?", expected="4"),
+        Case(input="What is 10 + 5?", expected="15"),
+    ],
+    evaluators=[...]
+)
+
+# This function will be used as the evaluation target
+async def evaluate_math_agent(input_text: str) -> str:
+    # Your agent evaluation logic here
+    agent = get_agent(system_prompt="You are a math assistant.")
+    result, _ = await run_agent(agent, input_text)
+    return result
+```
+
+The discovery system will:
+- Find `eval_math_operations.py` in the evals directory
+- Discover `dataset_addition` as an evaluation dataset
+- Use `evaluate_math_agent` as the target function for evaluation
+- Run each case through the target function and evaluate results
+
 ## Testing & Tools
 
 AIXtools provides comprehensive testing utilities and diagnostic tools for AI agent development and debugging.
@@ -366,7 +478,49 @@ test_model = AixTestModel()
 
 ### Tool Doctor System
 
-Automated tool analysis and recommendation system for optimizing agent tool usage.
+Automated tool analysis and recommendation system for optimizing agent tool usage and analyzing MCP servers.
+
+#### MCP Tool Doctor
+
+Analyze tools from MCP (Model Context Protocol) servers and receive AI-powered recommendations for improvement.
+
+```python
+from aixtools.tools.doctor.mcp_tool_doctor import tool_doctor_mcp
+from pydantic_ai.mcp import MCPServerStreamableHTTP, MCPServerStdio
+
+# Analyze HTTP MCP server
+recommendations = await tool_doctor_mcp(mcp_url='http://127.0.0.1:8000/mcp')
+for rec in recommendations:
+    print(rec)
+
+# Analyze STDIO MCP server
+server = MCPServerStdio(command='fastmcp', args=['run', 'my_server.py'])
+recommendations = await tool_doctor_mcp(mcp_server=server, verbose=True)
+```
+
+**Command Line Usage:**
+
+```bash
+# Analyze HTTP MCP server (default)
+tool_doctor_mcp
+
+# Analyze specific HTTP MCP server
+tool_doctor_mcp --mcp-url http://localhost:9000/mcp --verbose
+
+# Analyze STDIO MCP server
+tool_doctor_mcp --stdio-command fastmcp --stdio-args run my_server.py --debug
+
+# Available options:
+# --mcp-url URL          URL of HTTP MCP server (default: http://127.0.0.1:8000/mcp)
+# --stdio-command CMD    Command to run STDIO MCP server
+# --stdio-args ARGS      Arguments for STDIO MCP server command
+# --verbose              Enable verbose output
+# --debug                Enable debug output
+```
+
+#### Traditional Tool Doctor
+
+Analyze tool usage patterns from agent logs and get optimization recommendations.
 
 ```python
 from aixtools.tools.doctor.tool_doctor import ToolDoctor

{aixtools-0.1.11 → aixtools-0.2.0}/aixtools/_version.py

@@ -28,7 +28,7 @@ version_tuple: VERSION_TUPLE
 commit_id: COMMIT_ID
 __commit_id__: COMMIT_ID
 
-__version__ = version = '0.1.11'
-__version_tuple__ = version_tuple = (0, 1, 11)
+__version__ = version = '0.2.0'
+__version_tuple__ = version_tuple = (0, 2, 0)
 
-__commit_id__ = commit_id = 'g32157e2f8'
+__commit_id__ = commit_id = 'g9a5eeee69'

{aixtools-0.1.11 → aixtools-0.2.0}/aixtools/agents/agent.py

@@ -5,10 +5,11 @@ Core agent implementation providing model selection and configuration for AI age
 from types import NoneType
 from typing import Any
 
+from fastmcp import Context
 from openai import AsyncAzureOpenAI
 from pydantic_ai import Agent
 from pydantic_ai.models.bedrock import BedrockConverseModel
-from pydantic_ai.models.openai import OpenAIModel
+from pydantic_ai.models.openai import OpenAIChatModel
 from pydantic_ai.providers.bedrock import BedrockProvider
 from pydantic_ai.providers.openai import OpenAIProvider
 from pydantic_ai.settings import ModelSettings
@@ -54,14 +55,14 @@ def _get_model_ollama(model_name=OLLAMA_MODEL_NAME, ollama_url=OLLAMA_URL):
     assert ollama_url, "OLLAMA_URL is not set"
     assert model_name, "Model name is not set"
     provider = OpenAIProvider(base_url=ollama_url)
-    return OpenAIModel(model_name=model_name, provider=provider)
+    return OpenAIChatModel(model_name=model_name, provider=provider)
 
 
 def _get_model_openai(model_name=OPENAI_MODEL_NAME, openai_api_key=OPENAI_API_KEY):
     assert openai_api_key, "OPENAI_API_KEY is not set"
     assert model_name, "Model name is not set"
     provider = OpenAIProvider(api_key=openai_api_key)
-    return OpenAIModel(model_name=model_name, provider=provider)
+    return OpenAIChatModel(model_name=model_name, provider=provider)
 
 
 def _get_model_openai_azure(
@@ -77,7 +78,7 @@ def _get_model_openai_azure(
     client = AsyncAzureOpenAI(
         azure_endpoint=azure_openai_endpoint, api_version=azure_openai_api_version, api_key=azure_openai_api_key
     )
-    return OpenAIModel(model_name=model_name, provider=OpenAIProvider(openai_client=client))
+    return OpenAIChatModel(model_name=model_name, provider=OpenAIProvider(openai_client=client))
 
 
 def _get_model_open_router(
@@ -87,7 +88,7 @@ def _get_model_open_router(
     assert openrouter_api_key, "OPENROUTER_API_KEY is not set"
     assert model_name, "Model name is not set, missing 'OPENROUTER_MODEL_NAME' environment variable?"
     provider = OpenAIProvider(base_url=openrouter_api_url, api_key=openrouter_api_key)
-    return OpenAIModel(model_name, provider=provider)
+    return OpenAIChatModel(model_name, provider=provider)
 
 
 def get_model(model_family=MODEL_FAMILY, model_name=None, **kwargs):
@@ -146,8 +147,22 @@ async def run_agent(  # noqa: PLR0913, pylint: disable=too-many-arguments,too-ma
     debug: bool = False,
     log_model_requests: bool = False,
     parent_logger: ObjectLogger | None = None,
+    ctx: Context | None = None,
 ):
-    """Query the LLM"""
+    """
+    Run the agent with the given prompt and log the execution details.
+    Args:
+        agent (Agent): The PydanticAI agent to run.
+        prompt (str | list[str]): The input prompt(s) for the agent.
+        usage_limits (UsageLimits | None): Optional usage limits for the agent.
+        verbose (bool): If True, enables verbose logging.
+        debug (bool): If True, enables debug logging.
+        log_model_requests (bool): If True, logs model requests and responses.
+        parent_logger (ObjectLogger | None): Optional parent logger for hierarchical logging.
+        ctx (Context | None): Optional FastMCP context for logging messages to the MCP client.
+    Returns:
+        tuple[final_output, nodes]: A tuple containing the agent's final output and a list of all logged nodes.
+    """
     # Results
     nodes, result = [], None
     async with agent.iter(prompt, usage_limits=usage_limits) as agent_run:
@@ -158,7 +173,11 @@ async def run_agent(  # noqa: PLR0913, pylint: disable=too-many-arguments,too-ma
                 agent.model = model_patch_logging(agent.model, agent_logger)
             # Run the agent
             async for node in agent_run:
-                agent_logger.log(node)
+                await agent_logger.log(node)  # Log each node
+                if ctx:
+                    # If we are executing in an MCP server, send info messages to the client for better debugging
+                    server_name = ctx.fastmcp.name
+                    await ctx.info(f"MCP server {server_name}: {node}")
                 nodes.append(node)
             result = agent_run.result
     return result.output if result else None, nodes