aixtools 0.7.8__tar.gz

aixtools-0.7.8/PKG-INFO

@@ -0,0 +1,1121 @@
+Metadata-Version: 2.4
+Name: aixtools
+Version: 0.7.8
+Summary: Tools for AI exploration and debugging
+Requires-Python: >=3.11.2
+Description-Content-Type: text/markdown
+Requires-Dist: a2a-sdk>=0.3.1
+Requires-Dist: alembic>=1.17.1
+Requires-Dist: cachebox>=5.0.1
+Requires-Dist: chainlit>=2.5.5
+Requires-Dist: colorlog>=6.9.0
+Requires-Dist: fasta2a>=0.5.0
+Requires-Dist: fastmcp>=2.13.0
+Requires-Dist: hvac>=2.3.0
+Requires-Dist: ipykernel>=6.29.5
+Requires-Dist: langchain-chroma>=0.2.3
+Requires-Dist: langchain-ollama>=0.3.2
+Requires-Dist: langchain-openai>=0.3.14
+Requires-Dist: markitdown[docx,pdf,pptx,xls,xlsx]>=0.1.3
+Requires-Dist: mcp>=1.20.0
+Requires-Dist: mypy>=1.18.2
+Requires-Dist: pandas>=2.2.3
+Requires-Dist: psycopg2-binary>=2.9.11
+Requires-Dist: pydantic-evals>=0.4.10
+Requires-Dist: pydantic-ai>=1.11.1
+Requires-Dist: pyjwt>=2.10.1
+Requires-Dist: pylint>=3.3.7
+Requires-Dist: rich>=14.0.0
+Requires-Dist: ruff>=0.11.6
+Requires-Dist: sqlalchemy>=2.0.44
+Requires-Dist: streamlit>=1.44.1
+Requires-Dist: tiktoken>=0.9.0
+Requires-Dist: openpyxl>=3.1.5
+Requires-Dist: xlrd>=2.0.2
+Requires-Dist: odfpy>=1.4.1
+Requires-Dist: watchdog>=6.0.0
+Provides-Extra: test
+Requires-Dist: pyyaml; extra == "test"
+Provides-Extra: feature
+Requires-Dist: logfire; extra == "feature"
+
+# AIXtools
+
+AIXtools is a comprehensive Python library for AI agent development, debugging, and deployment. It provides a complete toolkit for building, testing, and monitoring AI agents with support for multiple model providers, advanced logging, and agent-to-agent communication.
+
+## Capabilities
+
+- **[Installation](#installation)**
+- **[Environment Configuration](#environment-configuration)**
+- **[Agents](#agents)** - Core agent functionality
+  - Basic Agent Usage
+  - Agent Development & Management
+  - Agent Batch Processing
+  - Node Debugging and Visualization
+- **[Context Engineering](#context-engineering)** - Transform files into agent-readable content
+  - File Type Processors
+  - Configuration
+  - Processing Examples
+- **[Agent-to-Agent Communication](#a2a-agent-to-agent-communication)** - Inter-agent communication framework
+  - Core Features
+  - Google SDK Integration
+  - Remote Agent Connections
+- **[Testing & Tools](#testing--tools)** - Comprehensive testing utilities
+  - Running Tests
+  - Testing Utilities
+  - Mock Tool System
+  - Model Patch Caching
+  - Agent Mock
+  - FaultyMCP Testing Server
+  - MCP Tool Doctor
+  - Tool Doctor
+  - Evaluations
+- **[Logging & Debugging](#logging--debugging)** - Advanced logging and debugging
+  - Basic Logging
+  - Log Viewing Application
+  - Object Logging
+  - MCP Logging
+- **[Databases](#databases)** - Traditional and vector database support
+- **[Chainlit & HTTP Server](#chainlit--http-server)** - Web interfaces and server framework
+  - Chainlit Integration
+  - HTTP Server Framework
+- **[Programming Utilities](#programming-utilities)** - Essential utilities
+  - Persisted Dictionary
+  - Enum with Description
+  - Context Management
+  - Configuration Management
+  - File Utilities
+  - Chainlit Utilities
+
+## Installation
+
+```bash
+uv add aixtools
+```
+
+**Updating**
+
+```bash
+uv add --upgrade aixtools
+```
+
+## Environment Configuration
+
+AIXtools requires environment variables for model providers.
+
+**IMPORTANT:** Create a `.env` file based on `.env_template`:
+
+Here is an example configuration:
+
+```bash
+# Model family (azure, openai, or ollama)
+MODEL_FAMILY=azure
+MODEL_TIMEOUT=120
+
+# Azure OpenAI
+AZURE_OPENAI_ENDPOINT=https://your_endpoint.openai.azure.com
+AZURE_OPENAI_API_VERSION=2024-06-01
+AZURE_OPENAI_API_KEY=your_secret_key
+AZURE_MODEL_NAME=gpt-4o
+
+# OpenAI
+OPENAI_MODEL_NAME=gpt-4.5-preview
+OPENAI_API_KEY=openai_api_key
+
+# Ollama
+OLLAMA_MODEL_NAME=llama3.2:3b-instruct-fp16
+OLLAMA_LOCAL_URL=http://localhost:11434/v1
+```
+
+## Agents
+
+### Basic Agent Usage
+
+```python
+from aixtools.agents.agent import get_agent, run_agent
+
+async def main():
+    agent = get_agent(system_prompt="You are a helpful assistant.")
+    result, nodes = await run_agent(agent, "Explain quantum computing")
+    print(result)
+```
+
+### Agent Development & Management
+
+The agent system provides a unified interface for creating and managing AI agents across different model providers.
+
+```python
+from aixtools.agents.agent import get_agent, run_agent
+
+# Create an agent with default model
+agent = get_agent(system_prompt="You are a helpful assistant.")
+
+# Run the agent
+result, nodes = await run_agent(agent, "Tell me about AI")
+```
+
+### Node Debugging and Visualization
+
+The `print_nodes` module provides a clean, indented output for easy reading of the node from agent execution.
+
+```python
+from aixtools.agents.print_nodes import print_nodes, print_node
+from aixtools.agents.agent import get_agent, run_agent
+
+agent = get_agent(system_prompt="You are a helpful assistant.")
+result, nodes = await run_agent(agent, "Explain quantum computing")
+# Print all execution nodes for debugging
+print_nodes(nodes)
+```
+
+**Features:**
+- Node Type Detection: Automatically handles different node types (`UserPromptNode`, `CallToolsNode`, `ModelRequestNode`, `End`)
+- Formatted Output: Provides clean, indented output for easy reading
+- Tool Call Visualization: Shows tool names and arguments for tool calls
+- Text Content Display: Formats text parts with proper indentation
+- Model Request Summary: Shows character count for model requests to avoid verbose output
+
+**Node Types Supported:**
+- `UserPromptNode` - Displays user prompts with indentation
+- `CallToolsNode` - Shows tool calls with names and arguments
+- `ModelRequestNode` - Summarizes model requests with character count
+- `End` - Marks the end of execution (output suppressed by default)
+
+### Agent Batch Processing
+
+Process multiple agent queries simultaneously with built-in concurrency control and result aggregation.
+
+```python
+from aixtools.agents.agent_batch import agent_batch, AgentQueryParams
+
+# Create query parameters
+query_parameters = [
+    AgentQueryParams(prompt="What is the meaning of life"),
+    AgentQueryParams(prompt="Who is the prime minister of Canada")
+]
+
+# Run queries in batches
+async for result in agent_batch(query_parameters):
+    print(result)
+```
+
+## Context Engineering
+
+Transform file formats into agent-readable content with enforced size limits to prevent context overflow. The main entry point is the `read_file()` function in `aixtools/agents/context/reader.py`, which provides automatic file type detection and delegates to specialized processors for each file type.
+
+### Basic Usage
+
+The `read_file()` function in `reader.py` is the main interface for processing files. It automatically detects file types and applies appropriate truncation strategies.
+
+```python
+from aixtools.agents.context.reader import read_file
+from pathlib import Path
+
+# Read a file with automatic type detection and truncation
+result = read_file(Path("data.csv"))
+
+if result.success:
+    print(f"File type: {result.file_type}")
+    print(f"Content length: {len(result.content)}")
+    print(f"Truncation info: {result.truncation_info}")
+    print(result.content)
+
+# Optionally specify custom tokenizer and limits
+result = read_file(
+    Path("large_file.json"),
+    max_tokens_per_file=10000,
+    max_total_output=100000
+)
+```
+
+### Architecture
+
+The context engineering system is organized with `reader.py` as the main interface:
+- `reader.py` - Main `read_file()` function with file type detection and processing coordination
+- `config.py` - Configurable size limits and thresholds
+- `processors/` - Specialized processors for each file type (text, code, JSON, CSV, PDF, etc.)
+- `data_models.py` - Data classes for results and metadata
+
+### Supported File Types
+
+- Text files (`.txt`, `.log`, `.md`)
+- Code files (Python, JavaScript, etc.)
+- Structured data (`JSON`, `YAML`, `XML`)
+- Tabular data (`CSV`, `TSV`)
+- Documents (`PDF`, `DOCX`)
+- Spreadsheets (`.xlsx`, `.xls`, `.ods`)
+- Images (`PNG`, `JPEG`, `GIF`, `WEBP`)
+- Audio files
+
+### Key Features
+
+- Automatic file type detection based on MIME types and extensions
+- Token-based truncation with configurable limits per file
+- Intelligent content sampling (head + tail rows for tabular data)
+- Structure-aware truncation for `JSON`, `YAML`, and `XML`
+- Markdown conversion for documents using `markitdown`
+- Binary content support for images with metadata extraction
+- Comprehensive error handling with partial results when possible
+
+### Configuration
+
+All limits are configurable via environment variables:
+
+```bash
+# Output limits
+MAX_TOKENS_PER_FILE=5000
+MAX_TOTAL_OUTPUT=50000
+
+# Text truncation
+MAX_LINES=200
+MAX_LINE_LENGTH=1000
+
+# Tabular truncation
+MAX_COLUMNS=50
+DEFAULT_ROWS_HEAD=20
+DEFAULT_ROWS_MIDDLE=10
+DEFAULT_ROWS_TAIL=10
+MAX_CELL_LENGTH=500
+
+# Images
+MAX_IMAGE_ATTACHMENT_SIZE=2097152  # 2MB
+```
+
+### Processing Examples
+
+The recommended approach is to use the `read_file()` function which automatically handles file type detection and processing. However, you can also use individual processors directly for specific file types.
+
+#### Using read_file() (Recommended)
+
+```python
+from aixtools.agents.context.reader import read_file
+from pathlib import Path
+
+# Process any file type automatically
+result = read_file(Path("data.csv"))
+if result.success:
+    print(result.content)
+
+# Works with all supported types
+pdf_result = read_file(Path("report.pdf"))
+excel_result = read_file(Path("workbook.xlsx"))
+json_result = read_file(Path("config.json"))
+```
+
+#### Processing Tabular Data Directly
+
+```python
+from aixtools.agents.context.processors.tabular import process_tabular
+from pathlib import Path
+
+# Process specific row range from large CSV
+result = process_tabular(
+    file_path=Path("large_data.csv"),
+    start_row=100,
+    end_row=200,
+    max_columns=20,
+    max_cell_length=500
+)
+
+print(f"Rows shown: {result.truncation_info.rows_shown}")
+print(f"Columns shown: {result.truncation_info.columns_shown}")
+```
+
+#### Processing Spreadsheets Directly
+
+```python
+from aixtools.agents.context.processors.spreadsheet import process_spreadsheet
+from pathlib import Path
+
+# Process Excel file with multiple sheets
+result = process_spreadsheet(
+    file_path=Path("workbook.xlsx"),
+    max_sheets=3,
+    max_rows_per_sheet_head=20,
+    max_rows_per_sheet_tail=10
+)
+
+# Content includes all processed sheets with truncation info
+print(result.content)
+```
+
+#### Processing Documents Directly
+
+```python
+from aixtools.agents.context.processors.document import process_document
+from pathlib import Path
+
+# Convert PDF to markdown and truncate
+result = process_document(
+    file_path=Path("report.pdf"),
+    max_lines=200,
+    max_line_length=1000
+)
+
+if result.was_extracted:
+    print("Document successfully converted to markdown")
+    print(result.content)
+```
+
+### Output Format
+
+All processors return consistent output with metadata:
+
+```
+File: data.csv
+Columns: 8 (of 20000 total)
+Rows: 20 (of 1000000 total)
+
+col1,col2,...,col8
+value1,value2,...
+...
+
+Truncated: columns: 8 of 20000, rows: 20 of 1000000, 45 cells
+```
+
+The context engineering system ensures agents receive properly formatted, size-limited content that fits within token budgets while preserving the most relevant information from each file type.
+
+## A2A (Agent-to-Agent Communication)
+
+The `A2A` module provides a comprehensive framework for enabling sophisticated communication between AI agents across different environments and platforms. It includes Google SDK integration, `PydanticAI` adapters, and `FastA2A` application conversion capabilities.
+
+### Core Features
+
+**Agent Application Conversion**
+- Convert `PydanticAI` agents into `FastA2A` applications (deprecated)
+- Support for session metadata extraction and context management
+- Custom worker classes with enhanced data part support
+- Automatic handling of user and session identification
+
+**Remote Agent Connections**
+- Establish connections between agents across different environments
+- Asynchronous message sending with task polling capabilities
+- Terminal state detection and error handling
+- Support for various message types including text, files, and data
+
+**Google SDK Integration**
+- Native integration with Google's A2A SDK
+- Card-based agent representation and discovery
+- `PydanticAI` adapter for seamless Google SDK compatibility
+- Storage and execution management for agent interactions
+
+### Basic Usage
+
+Enable sophisticated agent interactions with Google SDK integration and `PydanticAI` adapters.
+
+```python
+from aixtools.a2a.google_sdk.remote_agent_connection import RemoteAgentConnection
+from aixtools.a2a.app import agent_to_a2a
+
+# Convert a PydanticAI agent to FastA2A application
+a2a_app = agent_to_a2a(
+    agent=my_agent,
+    name="MyAgent",
+    description="A helpful AI assistant",
+    skills=[{"name": "chat", "description": "General conversation"}]
+)
+
+# Connect agents across different environments
+connection = RemoteAgentConnection(card=agent_card, client=a2a_client)
+response = await connection.send_message_with_polling(message)
+```
+
+### Postgres DB Store for A2A agent
+
+See implementation: `aixtools/a2a/google_sdk/store`
+
+#### Alembic
+
+In order to take full control of the database schema management Alembic is used for handling database migrations.
+Thus make sure, that google-sdk Store objects are being created with parameter create_table=False
+```python
+from a2a.server.tasks import DatabaseTaskStore
+
+...
+
+task_store=DatabaseTaskStore(engine=db_engine, create_table=False)
+```
+
+#### Setup of database and applying migrations (manual if needed):
+
+configure POSTGRES_URL env variable
+```.dotenv
+POSTGRES_URL=postgresql+asyncpg://user:password@localhost:5432/a2a_magic_db
+```
+
+
+```shell
+# from scope of your a2a service
+
+#activate your virtual environment
+kzwk877@degfqx35d621DD a2a_magic_service % source .venv/bin/activate
+# set the PATH_TO_ALEMBIC_CONFIG environment variable to point to the alembic configuration directory
+(a2a_magic_service) kzwk877@degfqx35d621DD a2a_magic_service % export PATH_TO_ALEMBIC_CONFIG="$(pwd)/.venv/lib/python3.12/site-packages/aixtools/a2a/google_sdk/store"
+# Make sure that database is existed
+(a2a_magic_service) kzwk877@degfqx35d621DD a2a_magic_service % uv run "${PATH_TO_ALEMBIC_CONFIG}/ensure_database.py"
+2025-11-11 10:08:51.501 WARNING  [root] Looking for '.env' file in default directory
+2025-11-11 10:08:52.750 INFO     [root] Using .env file at '/PATH_TO_A2A_SERVICE/a2a_magic_service/.env'
+2025-11-11 10:08:52.751 INFO     [root] Using MAIN_PROJECT_DIR='/PATH_TO_A2A_SERVICE/a2a_magic_service'
+2025-11-11 10:08:52.752 WARNING  [root] Using         DATA_DIR='/app/data'
+2025-11-11 10:08:52.757 INFO     [__main__] Starting database creation script...
+...
+2025-11-11 10:08:52.821 INFO     [__main__] Creating database 'a2a_magic_db'...
+2025-11-11 10:08:52.904 INFO     [__main__] Database 'a2a_magic_db' created successfully
+...
+2025-11-11 10:08:52.921 INFO     [__main__] Database creation script completed successfully!
+# Apply alembic migrations
+(a2a_magic_service) kzwk877@degfqx35d621DD a2a_magic_service % alembic --config "${PATH_TO_ALEMBIC_CONFIG}/alembic.ini" upgrade head
+2025-11-11 10:11:34.185 WARNING  [root] Looking for '.env' file in default directory
+2025-11-11 10:11:35.046 WARNING  [root] Looking for '.env' file at '/PATH_TO_A2A_SERVICE/a2a_magic_service'
+2025-11-11 10:11:35.047 INFO     [root] Using .env file at '/PATH_TO_A2A_SERVICE/a2a_magic_service/.env'
+2025-11-11 10:11:35.048 INFO     [root] Using MAIN_PROJECT_DIR='/PATH_TO_A2A_SERVICE/a2a_magic_service'
+2025-11-11 10:11:35.049 WARNING  [root] Using         DATA_DIR='/app/data'
+2025-11-11 10:11:35.054 INFO     [env_py] Using database URL for migrations: postgresql://user:password@localhost:5432/a2a_magic_db
+INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
+INFO  [alembic.runtime.migration] Will assume transactional DDL.
+INFO  [alembic.runtime.migration] Running upgrade  -> 68c6975ed20b, Added a2a-sdk Task table
+```
+
+#### Schema modifications
+
+if new schema modifications has been introduced with new versions of a2a sdk suggested way 
+to create new alembic migrations would be:
+
+- launch a2a service with passed parameter to `DatabaseStore` `create_table=True`
+- make sure that all new tables/columns are created in 
+the database (possibly an new request to a2a server needs to be made)
+- create new alembic migration script
+```shell
+(a2a_magic_service) kzwk877@degfqx35d621DD % alembic --config "${PATH_TO_ALEMBIC_CONFIG}/alembic.ini" revision --autogenerate -m "New table introduced"
+```
+- review the generated migration script
+- apply and test
+
+## Databases
+
+### Database Integration
+
+Support for both traditional and vector databases with seamless integration.
+
+```python
+from aixtools.db.database import Database
+from aixtools.db.vector_db import VectorDB
+
+# Traditional database
+db = Database("sqlite:///app.db")
+
+# Vector database for embeddings
+vector_db = VectorDB()
+vector_db.add_documents(documents)
+```
+
+## Logging & Debugging
+
+AixTools provides functionality for logging and debugging.
+
+### Basic Logging and Debugging
+
+```python
+from aixtools.agents.agent import get_agent, run_agent
+
+async def main():
+    # Create an agent
+    agent = get_agent(system_prompt="You are a helpful assistant.")
+
+    # Run agent - logging is automatic via ObjectLogger
+    result, nodes = await run_agent(
+        agent,
+        "Explain quantum computing",
+        debug=True,  # Enable debug logging
+        log_model_requests=True  # Log model requests/responses
+    )
+
+    print(f"Result: {result}")
+    print(f"Logged {len(nodes)} nodes")
+```
+
+### Log Viewing Application
+
+Interactive `Streamlit` application for analyzing logged objects and debugging agent behavior.
+
+**Features:**
+- Log file selection and filtering
+- Node visualization with expand/collapse
+- Export capabilities to `JSON`
+- Regex pattern matching
+- Real-time log monitoring
+
+```bash
+# Run the log viewer
+log_view
+
+# Or specify custom log directory
+log_view /path/to/logs
+```
+
+### Object Logging & Debugging
+
+Advanced logging system with object serialization and visual debugging tools.
+
+```python
+from aixtools.logging.log_objects import ObjectLogger
+
+# Log any pickleable object
+with ObjectLogger() as logger:
+    logger.log({"message": "Hello, world!"})
+    logger.log(agent_response)
+```
+
+### MCP Logging
+
+AIXtools provides MCP support for both client and server implementations with easier logging for debugging purposes.
+
+**Example:**
+
+Let's assume we have an MCP server that runs an agent tool.
+
+Note that the `ctx: Context` parameter is passed to `run_agent()`, this will enable logging to the MCP client.
+
+```python
+@mcp.tool
+async def my_tool_with_agent(query: str, ctx: Context) -> str:
+    """ A tool that uses an gents to process the query """
+    agent = get_agent()
+    async with get_qb_agent() as agent:
+        ret, nodes = await run_agent(agent=agent, prompt=query, ctx=ctx)    # Enable MCP logging
+        return str(ret)
+```
+
+On the client side, you can create an agent connected to the MCP server, the nodes from the MCP server will show on the `STDOUT` so you can see what's going on the MCP server's agent loop.
+
+```python
+mcp = get_mcp_client("http://localhost:8000")   # Get an MCP client with a default log handler that prints to STDOUT
+agent = get_agent(toolsets=[mcp])
+async with agent:
+    # The messages from the MCP server will be printed to the STDOUT
+    ret, nodes = await run_agent(agent, prompt="...")
+```
+
+#### MCP Server Logging
+
+Create MCP servers with built-in logging capabilities.
+
+```python
+from aixtools.mcp.fast_mcp_log import FastMcpLog
+
+# Use FastMCP server with logging
+mcp = FastMcpLog("Demo")
+```
+
+## Testing & Tools
+
+AIXtools provides comprehensive testing utilities and diagnostic tools for AI agent development and debugging.
+
+### Running Tests
+
+Execute the test suite using the provided scripts:
+
+```bash
+# Run all tests
+./scripts/test.sh
+
+# Run unit tests only
+./scripts/test_unit.sh
+
+# Run integration tests only
+./scripts/test_integration.sh
+```
+
+### Testing Utilities
+
+The testing module provides mock tools, model patching, and test utilities for comprehensive agent testing.
+
+```python
+from aixtools.testing.mock_tool import MockTool
+from aixtools.testing.model_patch_cache import ModelPatchCache
+from aixtools.testing.aix_test_model import AixTestModel
+
+# Create mock tools for testing
+mock_tool = MockTool(name="test_tool", response="mock response")
+
+# Use model patch caching for consistent test results
+cache = ModelPatchCache()
+cached_response = cache.get_cached_response("test_prompt")
+
+# Test model for controlled testing scenarios
+test_model = AixTestModel()
+```
+
+### Mock Tool System
+
+Create and manage mock tools for testing agent behavior without external dependencies.
+
+```python
+from aixtools.testing.mock_tool import MockTool
+
+# Create a mock tool with predefined responses
+mock_calculator = MockTool(
+    name="calculator",
+    description="Performs mathematical calculations",
+    response_map={
+        "2+2": "4",
+        "10*5": "50"
+    }
+)
+
+# Use in agent testing
+agent = get_agent(tools=[mock_calculator])
+result = await run_agent(agent, "What is 2+2?")
+```
+
+### Model Patch Caching
+
+Cache model responses for consistent testing and development workflows.
+
+```python
+from aixtools.testing.model_patch_cache import ModelPatchCache
+
+# Initialize cache
+cache = ModelPatchCache(cache_dir="./test_cache")
+
+# Cache responses for specific prompts
+cache.cache_response("test prompt", "cached response")
+
+# Retrieve cached responses
+response = cache.get_cached_response("test prompt")
+```
+
+### Model Patching System
+
+Dynamic model behavior modification for testing and debugging.
+
+```python
+from aixtools.model_patch.model_patch import ModelPatch
+
+# Apply patches to models for testing
+with ModelPatch() as patch:
+    patch.apply_response_override("test response")
+    result = await agent.run("test prompt")
+```
+
+### Agent Mock
+
+Replay previously recorded agent runs without executing the actual agent. Useful for testing, debugging, and creating reproducible test cases.
+
+```python
+from aixtools.testing.agent_mock import AgentMock
+from aixtools.agents.agent import get_agent, run_agent
+
+# Run an agent and capture its execution
+agent = get_agent(system_prompt="You are a helpful assistant.")
+result, nodes = await run_agent(agent, "Explain quantum computing")
+
+# Create a mock agent from the recorded nodes
+agent_mock = AgentMock(nodes=nodes, result_output=result)
+
+# Save the mock for later use
+agent_mock.save(Path("test_data/quantum_mock.pkl"))
+
+# Load and replay the mock agent
+loaded_mock = AgentMock.load(Path("test_data/quantum_mock.pkl"))
+result, nodes = await run_agent(loaded_mock, "any prompt")  # Returns recorded nodes
+```
+
+### FaultyMCP Testing Server
+
+A specialized MCP server designed for testing error handling and resilience in MCP client implementations. `FaultyMCP` simulates various failure scenarios including network errors, server crashes, and random exceptions.
+
+**Features:**
+- Configurable error probabilities for different request types
+- HTTP 404 error injection for `POST`/`DELETE` requests
+- Server crash simulation on `GET` requests
+- Random exception throwing in tool operations
+- MCP-specific error simulation (`ValidationError`, `ResourceError`, etc.)
+- Safe mode for controlled testing
+
+```python
+from aixtools.mcp.faulty_mcp import run_server_on_port, config
+
+# Configure error probabilities
+config.prob_on_post_404 = 0.3      # 30% chance of 404 on POST
+config.prob_on_get_crash = 0.1     # 10% chance of crash on GET
+config.prob_in_list_tools_throw = 0.2  # 20% chance of exception in tools/list
+
+# Run the faulty server
+run_server_on_port()
+```
+
+**Command Line Usage:**
+
+```bash
+# Run with default error probabilities
+python -m aixtools.mcp.faulty_mcp
+
+# Run in safe mode (no errors by default)
+python -m aixtools.mcp.faulty_mcp --safe-mode
+
+# Custom configuration
+python -m aixtools.mcp.faulty_mcp \
+    --port 8888 \
+    --prob-on-post-404 0.2 \
+    --prob-on-get-crash 0.1 \
+    --prob-in-list-tools-throw 0.3
+```
+
+**Available Test Tools:**
+- `add(a, b)` - Reliable addition operation
+- `multiply(a, b)` - Reliable multiplication operation
+- `always_error()` - Always throws an exception
+- `random_throw_exception(a, b, prob)` - Randomly throws exceptions
+- `freeze_server(seconds)` - Simulates server freeze
+- `throw_404_exception()` - Throws HTTP 404 error
+
+### MCP Tool Doctor
+
+Analyze tools from MCP (Model Context Protocol) servers and receive AI-powered recommendations for improvement.
+
+```python
+from aixtools.tools.doctor.mcp_tool_doctor import tool_doctor_mcp
+from pydantic_ai.mcp import MCPServerStreamableHTTP, MCPServerStdio
+
+# Analyze HTTP MCP server
+recommendations = await tool_doctor_mcp(mcp_url='http://127.0.0.1:8000/mcp')
+for rec in recommendations:
+    print(rec)
+
+# Analyze STDIO MCP server
+server = MCPServerStdio(command='fastmcp', args=['run', 'my_server.py'])
+recommendations = await tool_doctor_mcp(mcp_server=server, verbose=True)
+```
+
+**Command Line Usage:**
+
+```bash
+# Analyze HTTP MCP server (default)
+tool_doctor_mcp
+
+# Analyze specific HTTP MCP server
+tool_doctor_mcp --mcp-url http://localhost:9000/mcp --verbose
+
+# Analyze STDIO MCP server
+tool_doctor_mcp --stdio-command fastmcp --stdio-args run my_server.py --debug
+```
+
+**Available options:**
+- `--mcp-url URL` - URL of HTTP MCP server (default: `http://127.0.0.1:8000/mcp`)
+- `--stdio-command CMD` - Command to run STDIO MCP server
+- `--stdio-args ARGS` - Arguments for STDIO MCP server command
+- `--verbose` - Enable verbose output
+- `--debug` - Enable debug output
+
+### Tool Doctor
+
+Analyze tool usage patterns from agent logs and get optimization recommendations.
+
+```python
+from aixtools.tools.doctor.tool_doctor import ToolDoctor
+from aixtools.tools.doctor.tool_recommendation import ToolRecommendation
+
+# Analyze tool usage patterns
+doctor = ToolDoctor()
+analysis = doctor.analyze_tools(agent_logs)
+
+# Get tool recommendations
+recommendation = ToolRecommendation()
+suggestions = recommendation.recommend_tools(agent_context)
+```
+
+### Evaluations
+
+Run comprehensive Agent/LLM evaluations using the built-in evaluation discovery based on `Pydantic-AI` framework with AIXtools enhancements.
+
+```bash
+# Run all evaluations
+python -m aixtools.evals
+
+# Run evaluations with filtering
+python -m aixtools.evals --filter "specific_test"
+
+# Run with verbose output and detailed reporting
+python -m aixtools.evals --verbose --include-input --include-output --include-reasons
+
+# Specify custom evaluations directory
+python -m aixtools.evals --evals-dir /path/to/evals
+
+# Set minimum assertions threshold
+python -m aixtools.evals --min-assertions 0.8
+```
+
+**Command Line Options:**
+- `--evals-dir` - Directory containing `eval_*.py` files (default: `evals`)
+- `--filter` - Filter to run only matching evaluations
+- `--include-input` - Include input in report output (default: `True`)
+- `--include-output` - Include output in report output (default: `True`)
+- `--include-evaluator-failures` - Include evaluator failures in report
+- `--include-reasons` - Include reasons in report output
+- `--min-assertions` - Minimum assertions average required for success (default: `1.0`)
+- `--verbose` - Print detailed information about discovery and processing
+
+The evaluation system discovers and runs all `Dataset` objects from `eval_*.py` files in the specified directory, similar to test runners but specifically designed for LLM evaluations using `pydantic_evals`.
+
+**Discovery Mechanism**
+
+The evaluation framework uses an automatic discovery system:
+
+1. **File Discovery**: Scans the specified directory for files matching the pattern `eval_*.py`
+2. **Dataset Discovery**: Within each file, looks for variables named `dataset_*` that are instances of `pydantic_evals.Dataset`
+3. **Target Function Discovery**: Within the same file looks for function or async function named `target_*`. There must be 1 target function per file.
+4. **Function Discovery**: Looks for functions with specific prefixes:
+   - Functions prefixed with `scorer_*`, `evaluator_*` for custom scorer and evaluator functions that will be used for each dataset in that file
+5. **Filtering**: Supports filtering by module name, file name, dataset name, or fully qualified name
+
+**Example Evaluation File Structure:**
+
+```python
+# eval_math_operations.py
+from pydantic_evals import Dataset, Case
+
+# This dataset will be discovered automatically
+dataset_addition = Dataset(
+    name="Addition Tests",
+    cases=[
+        Case(input="What is 2 + 2?", expected="4"),
+        Case(input="What is 10 + 5?", expected="15"),
+    ],
+    evaluators=[...]
+)
+
+# This function will be used as the evaluation target
+async def target_math_agent(input_text: str) -> str:
+    # Your agent run logic here
+    agent = get_agent(system_prompt="You are a math assistant.")
+    result, _ = await run_agent(agent, input_text)
+    return result
+
+# This function will be used as evaluator for all datasets (optional)
+def evaluator_check_output(ctx: EvaluatorContext) -> bool:
+    # Your result evaluation logic here
+    return ctx.output == ctx.expected_output
+```
+
+The discovery system will:
+- Find `eval_math_operations.py` in the `evals` directory
+- Discover `dataset_addition` as an evaluation dataset
+- Use `evaluate_math_agent` as the target function for evaluation
+- Run each case through the target function and evaluate results
+
+#### Name-Based Discovery
+
+The evaluation system uses name-based discovery for all components:
+
+**Target Functions** (exactly one required per eval file):
+- Purpose: The main function being evaluated - processes inputs and returns outputs
+- Naming: Functions named `target_*` (e.g., `target_my_function`)
+- Signature: `def target_name(inputs: InputType) -> OutputType` or `async def target_name(inputs: InputType) -> OutputType`
+- Example: `async def target_math_agent(input_text: str) -> str`
+
+**Scoring Functions** (optional):
+- Purpose: Determine if evaluation results meet success criteria
+- Naming: Functions named `scorer_*` (e.g., `scorer_custom`)
+- Signature: `def scorer_name(report: EvaluationReport, dataset: AixDataset, min_score: float = 1.0, verbose: bool = False) -> bool`
+- Example: `def scorer_accuracy_threshold(report, dataset, min_score=0.8, verbose=False) -> bool`
+
+**Evaluator Functions** (optional):
+- Purpose: Custom evaluation logic for comparing outputs with expected results
+- Naming: Functions named `evaluator_*` (e.g., `evaluator_check_output`)
+- Signature: `def evaluator_name(ctx: EvaluatorContext) -> EvaluatorOutput` or `async def evaluator_name(ctx: EvaluatorContext) -> EvaluatorOutput`
+- Example: `def evaluator_exact_match(ctx) -> EvaluatorOutput`
+
+This name-based approach works seamlessly with both synchronous and asynchronous functions.
+
+#### Scoring System
+
+The framework includes a custom scoring system with `average_assertions` as the default scorer. This scorer checks if the average assertion score meets a minimum threshold and provides detailed pass/fail reporting.
+
+## Chainlit & HTTP Server
+
+### Chainlit Integration
+
+Ready-to-use `Chainlit` application for interactive agent interfaces.
+
+```python
+# Run the Chainlit app
+# Configuration in aixtools/chainlit.md
+# Main app in aixtools/app.py
+```
+
+### HTTP Server Framework
+
+AIXtools provides an HTTP server framework for deploying agents and tools as web services.
+
+```python
+from aixtools.server.app_mounter import mount_app
+from aixtools.server import create_server
+
+# Create and configure server
+server = create_server()
+
+# Mount applications and endpoints
+mount_app(server, "/agent", agent_app)
+mount_app(server, "/tools", tools_app)
+
+# Run server
+server.run(host="0.0.0.0", port=8000)
+```
+
+**Features:**
+- Application mounting system for modular service composition
+- Integration with `Chainlit` for agent interfaces
+- RESTful API support
+- Middleware support for authentication and logging
+
+## Programming Utilities
+
+AIXtools provides essential programming utilities for configuration management, data persistence, file operations, and context handling.
+
+### Persisted Dictionary
+
+Persistent key-value storage with automatic serialization and file-based persistence.
+
+```python
+from aixtools.utils.persisted_dict import PersistedDict
+
+# Create a persistent dictionary
+cache = PersistedDict("cache.json")
+
+# Store and retrieve data
+cache["user_preferences"] = {"theme": "dark", "language": "en"}
+cache["session_data"] = {"last_login": "2024-01-01"}
+
+# Data is automatically saved to file
+print(cache["user_preferences"])  # Persists across program restarts
+```
+
+### Enum with Description
+
+Enhanced `Enum` classes with built-in descriptions for better documentation and user interfaces.
+
+```python
+from aixtools.utils.enum_with_description import EnumWithDescription
+
+class ModelType(EnumWithDescription):
+    GPT4 = ("gpt-4", "OpenAI GPT-4 model")
+    CLAUDE = ("claude-3", "Anthropic Claude-3 model")
+    LLAMA = ("llama-2", "Meta LLaMA-2 model")
+
+# Access enum values and descriptions
+print(ModelType.GPT4.value)        # "gpt-4"
+print(ModelType.GPT4.description)  # "OpenAI GPT-4 model"
+
+# Get all descriptions
+for model in ModelType:
+    print(f"{model.value}: {model.description}")
+```
+
+### Context Management
+
+Centralized context management for sharing state across components.
+
+```python
+from aixtools.context import Context
+
+# Create and use context
+context = Context()
+context.set("user_id", "12345")
+context.set("session_data", {"preferences": {"theme": "dark"}})
+
+# Retrieve context data
+user_id = context.get("user_id")
+session_data = context.get("session_data")
+
+# Context can be passed between components
+def process_request(ctx: Context):
+    user_id = ctx.get("user_id")
+    # Process with user context
+```
+
+### Configuration Management
+
+Robust configuration handling with environment variable support and validation.
+
+```python
+from aixtools.utils.config import Config
+from aixtools.utils.config_util import load_config
+
+# Load configuration from environment and files
+config = load_config()
+
+# Access configuration values
+model_name = config.get("MODEL_NAME", "gpt-4")
+api_key = config.get("API_KEY")
+timeout = config.get("TIMEOUT", 30, int)
+
+# Configuration with validation
+class AppConfig(Config):
+    model_name: str = "gpt-4"
+    max_tokens: int = 1000
+    temperature: float = 0.7
+
+app_config = AppConfig()
+```
+
+### File Utilities
+
+Enhanced file operations with `Path` support and utility functions.
+
+```python
+from aixtools.utils.files import read_file, write_file, ensure_directory
+from pathlib import Path
+
+# Read and write files with automatic encoding handling
+content = read_file("data.txt")
+write_file("output.txt", "Hello, world!")
+
+# Ensure directories exist
+data_dir = Path("data/logs")
+ensure_directory(data_dir)
+
+# Work with file paths
+config_path = Path("config") / "settings.json"
+if config_path.exists():
+    config_data = read_file(config_path)
+```
+
+### Chainlit Utilities
+
+Specialized utilities for `Chainlit` integration and agent display.
+
+```python
+from aixtools.utils.chainlit.cl_agent_show import show_agent_response
+from aixtools.utils.chainlit.cl_utils import format_message
+
+# Display agent responses in Chainlit
+await show_agent_response(
+    response="Hello, how can I help you?",
+    metadata={"model": "gpt-4", "tokens": 150}
+)
+
+# Format messages for Chainlit display
+formatted_msg = format_message(
+    content="Processing your request...",
+    message_type="info"
+)
+```
+
+### General Utilities
+
+Common utility functions for everyday programming tasks.
+
+```python
+from aixtools.utils.utils import safe_json_loads, timestamp_now, hash_string
+
+# Safe JSON parsing
+data = safe_json_loads('{"key": "value"}', default={})
+
+# Get current timestamp
+now = timestamp_now()
+
+# Generate hash for strings
+file_hash = hash_string("content to hash")
+```
+