agno 2.3.20__tar.gz → 2.3.21__tar.gz

{agno-2.3.20 → agno-2.3.21}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agno
-Version: 2.3.20
+Version: 2.3.21
 Summary: Agno: a lightweight library for building Multi-Agent Systems
 Author-email: Ashpreet Bedi <ashpreet@agno.com>
 Project-URL: homepage, https://agno.com
@@ -415,7 +415,7 @@ Dynamic: license-file
 <div align="center">
   <a href="https://docs.agno.com">Documentation</a>
   <span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
-  <a href="https://docs.agno.com/examples/use-cases/agents/overview">Examples</a>
+  <a href="https://github.com/agno-agi/agno/tree/main/cookbook">Cookbook</a>
   <span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
   <a href="https://www.agno.com/?utm_source=github&utm_medium=readme&utm_campaign=agno-github">Website</a>
   <br />
@@ -423,25 +423,27 @@ Dynamic: license-file
 
 ## What is Agno?
 
-Agno is an incredibly fast multi-agent framework, runtime and control plane.
+Agno is a multi-agent framework, runtime, and control plane. Use it to build private and secure AI products that run in your cloud.
 
-Companies want to build AI products, run them securely in their cloud, and manage their agents with a beautiful UI. This takes more than calling an LLM API, it requires a thoughtfully designed agentic platform.
+- **Build** agents, teams, and workflows with memory, knowledge, guardrails, and 100+ toolkits.
+- **Run** in production with a stateless FastAPI runtime. Horizontally scalable.
+- **Manage** with a control plane that connects directly to your runtime — no data leaves your environment.
 
-Agno provides the unified stack for building, running and managing multi-agent systems:
+## Why Agno?
 
-- **Framework**: Build agents, multi-agent teams and workflows with memory, knowledge, state, guardrails, HITL, context compression, MCP, A2A and 100+ toolkits.
-- **AgentOS Runtime**: Run your multi-agent system in production with a secure, stateless runtime and ready to use integration endpoints.
-- **AgentOS Control Plane**: Test, monitor and manage AgentOS deployments across environments with full operational visibility.
+- **Your cloud, your data:** AgentOS runs entirely in your infrastructure. Zero data leaves your environment.
+- **Production-ready from day one:** Pre-built FastAPI runtime with SSE endpoints, ready to deploy.
+- **Actually fast:** 529× faster than LangGraph, 24× lower memory. Matters at scale.
 
-Checkout the full list of features [here](#features).
+## Getting Started
 
-## Getting started
+New to Agno? Start with the [getting started guide](https://github.com/agno-agi/agno/tree/main/cookbook/00_getting_started).
 
-If you're new to Agno, follow our [quickstart](https://docs.agno.com/get-started/quickstart) to build your first Agent and chat with it using the AgentOS UI.
+Then:
+- Browse the [cookbooks](https://github.com/agno-agi/agno/tree/main/cookbook) for real-world examples
+- Read the [docs](https://docs.agno.com) to learn more.
 
-After that, checkout the [examples gallery](https://docs.agno.com/examples/use-cases/agents/overview) and build real-world applications with Agno.
-
-## Documentation, Community & More Examples
+## Resources
 
 - Docs: <a href="https://docs.agno.com" target="_blank" rel="noopener noreferrer">docs.agno.com</a>
 - Cookbook: <a href="https://github.com/agno-agi/agno/tree/main/cookbook" target="_blank" rel="noopener noreferrer">Cookbook</a>
@@ -497,144 +499,72 @@ When you run the example script shared above, you get a FastAPI app that you can
 
 https://github.com/user-attachments/assets/feb23db8-15cc-4e88-be7c-01a21a03ebf6
 
-## The Complete Agentic Solution
+## Private by Design
 
-Agno provides the complete solution for companies building agentic systems:
+This is the part we care most about.
 
-- The fastest framework for building agents, multi-agent teams and agentic workflows.
-- A ready-to-use FastAPI app that gets you building AI products on day one.
-- A control plane for testing, monitoring and managing your system.
+AgentOS runs in **your** cloud. The control plane UI connects directly to your runtime from your browser. Your data never touches our servers. No retention costs, no vendor lock-in, no compliance headaches.
 
-Agno brings a novel architecture that no other framework provides, your AgentOS runs securely in your cloud, and the control plane connects directly to it from your browser. You don't need to send data to any external services or pay retention costs, you get complete privacy and control.
+This isn't a privacy mode or enterprise add-on. It's how Agno works.
 
 ## Features
 
-Agno is an incredibly feature-rich framework purpose-built for Agent Engineering. Here are some key features:
-
-| **Category**                           | **Feature**                     | **Description**                                                                                                           |
-| -------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
-| **Foundational Principles**            | **Model Agnostic**              | Supports all model providers so you can choose the best model for your use case |
-|                                        | **Type Safe**                   | Enforces structured I/O through input_schema and output_schema for predictable and composable agent behavior. |
-|                                        | **Dynamic Context**             | Inject variables, state, and retrieved data at runtime into context. Compress, summarize and filter context to keep your Agents focused and efficient. |
-|                                        | **Designed for Scale**          | Designed around async execution and long-running tasks for high throughput agent workloads. |
-| **Memory, Knowledge, and Persistence** | **Persistent Storage**          | Give your Agents, Teams, and Workflows a database to persist session history, state, and messages. |
-|                                        | **User Memory**                 | Built in memory layer that helps agents recall user specific context across sessions. |
-|                                        | **Agentic RAG**                 | Connect to 20+ vector stores (called **Knowledge**) with hybrid search, reranking, and chunking out of the box. |
-|                                        | **Culture** | Shared long term collective memory that compounds across agents and time. |
-|                                        | **Ephemeral Context** | In memory scratchpad for short lived reasoning without polluting long term state. |
-| **Execution & Control**                | **Human-in-the-Loop**           | Native support for confirmations, approvals, manual overrides, and external actions. |
-|                                        | **Guardrails**                  | Built-in safeguards for validation, security, and prompt protection.                                                      |
-|                                        | **Agent Lifecycle Hooks**       | Pre and post hooks to validate, enrich, or transform inputs and outputs. |
-|                                        | **MCP Integration**             | First-class support for the Model Context Protocol (MCP) to connect Agents with external systems. |
-|                                        | **A2A Integration**             | First-class support for the Agent to Agent communication protocol (A2A). |
-|                                        | **Toolkits**                    | 100+ built in toolkits with thousands of tools covering data, code, web, and enterprise APIs. |
-| **Runtime & Evaluation**               | **Runtime**                     | Prebuilt FastAPI runtime with SSE compatible endpoints. Production ready from day one. |
-|                                        | **Control Plane (UI)**          | Integrated interface to test, observe, and debug your agents, teams, and workflows in real time. |
-|                                        | **Natively Multimodal**         | Agents can process and generate text, images, audio, video, and files. |
-|                                        | **Evals**                       | Measure Accuracy, Performance, Latency, and Reliability across agents and workflows. |
-|                                        | **Durable Execution**           | Built in support for long running, resumable workflows. |
-| **Security & Privacy**                 | **Private by Design**           | Runs entirely in your cloud. The UI connects directly to your AgentOS from your browser, no data is ever sent externally. |
-|                                        | **Data Governance**             | Your data lives securely in your Agent database, no external data sharing or vendor lock-in.                              |
-|                                        | **Access Control**              | Role-based access (RBAC) and per-agent permissions to protect sensitive contexts and tools.                               |
-
-Every part of Agno is built for real-world deployment — where developer experience meets production performance.
-
-## Setup Your Coding Agent to Use Agno
-
-For LLMs and AI assistants to understand and navigate Agno's documentation, we provide an [llms.txt](https://docs.agno.com/llms.txt) or [llms-full.txt](https://docs.agno.com/llms-full.txt) file. This file is built for AI systems to efficiently parse and reference our documentation.
-
-### IDE Integration
-
-When building Agno agents, using Agno documentation as a source in your IDE is a great way to speed up your development. Here's how to integrate with Cursor:
-
-1. In Cursor, go to the "Cursor Settings" menu.
-2. Find the "Indexing & Docs" section.
-3. Add `https://docs.agno.com/llms-full.txt` to the list of documentation URLs.
-4. Save the changes.
-
-Now, Cursor will have access to the Agno documentation. You can do the same with other IDEs like VSCode, Windsurf etc.
+**Core**
+- Model agnostic — works with OpenAI, Anthropic, Google, local models, whatever
+- Type-safe I/O with `input_schema` and `output_schema`
+- Async-first, built for long-running tasks
+- Natively multimodal (text, images, audio, video, files)
+
+**Memory & Knowledge**
+- Persistent storage for session history and state
+- User memory that persists across sessions
+- Agentic RAG with 20+ vector stores, hybrid search, reranking
+- Culture — shared long-term memory across agents
+
+**Execution**
+- Human-in-the-loop (confirmations, approvals, overrides)
+- Guardrails for validation and security
+- Pre/post hooks for the agent lifecycle
+- First-class MCP and A2A support
+- 100+ built-in toolkits
+
+**Production**
+- Ready-to-use FastAPI runtime
+- Integrated control plane UI
+- Evals for accuracy, performance, latency
+- Durable execution for resumable workflows
+- RBAC and per-agent permissions
 
 ## Performance
 
-If you're building with Agno, you're guaranteed best-in-class performance by default. Our obsession with performance is necessary because even simple AI workflows can spawn hundreds of Agents and because many tasks are long-running -- stateless, horizontal scalability is key for success.
-
-At Agno, we optimize performance across 3 dimensions:
-
-1. **Agent performance:** We optimize static operations (instantiation, memory footprint) and runtime operations (tool calls, memory updates, history management).
-2. **System performance:** The AgentOS API is async by default and has a minimal memory footprint. The system is stateless and horizontally scalable, with a focus on preventing memory leaks. It handles parallel and batch embedding generation during knowledge ingestion, metrics collection in background tasks, and other system-level optimizations.
-3. **Agent reliability and accuracy:** Monitored through evals, which we'll explore later.
-
-### Agent Performance
-
-Let's measure the time it takes to instantiate an Agent and the memory footprint of an Agent. Here are the numbers (last measured in Oct 2025, on an Apple M4 MacBook Pro):
-
-- **Agent instantiation:** ~3μs on average
-- **Memory footprint:** ~6.6Kib on average
-
-We'll show below that Agno Agents instantiate **529× faster than Langgraph**, **57× faster than PydanticAI**, and **70× faster than CrewAI**. Agno Agents also use **24× lower memory than Langgraph**, **4× lower than PydanticAI**, and **10× lower than CrewAI**.
-
-> [!NOTE]
-> Run time performance is bottlenecked by inference and hard to benchmark accurately, so we focus on minimizing overhead, reducing memory usage, and parallelizing tool calls.
-
-### Instantiation Time
+We're obsessive about performance because agent workloads spawn hundreds of instances and run long tasks. Stateless, horizontal scalability isn't optional.
 
-Let's measure instantiation time for an Agent with 1 tool. We'll run the evaluation 1000 times to get a baseline measurement. We'll compare Agno to LangGraph, CrewAI and Pydantic AI.
+**Benchmarks** (Apple M4 MacBook Pro, Oct 2025):
 
-> [!NOTE]
-> The code for this benchmark is available [here](https://github.com/agno-agi/agno/tree/main/cookbook/evals/performance). You should run the evaluation yourself on your own machine, please, do not take these results at face value.
+| Metric | Agno | LangGraph | PydanticAI | CrewAI |
+|--------|------|-----------|------------|--------|
+| Instantiation | **3μs** | 1,587μs (529× slower) | 170μs (57× slower) | 210μs (70× slower) |
+| Memory | **6.6 KiB** | 161 KiB (24× higher) | 29 KiB (4× higher) | 66 KiB (10× higher) |
 
-```shell
-# Setup virtual environment
-./scripts/perf_setup.sh
-source .venvs/perfenv/bin/activate
-
-# Agno
-python cookbook/evals/performance/instantiate_agent_with_tool.py
-
-# LangGraph
-python cookbook/evals/performance/comparison/langgraph_instantiation.py
-# CrewAI
-python cookbook/evals/performance/comparison/crewai_instantiation.py
-# Pydantic AI
-python cookbook/evals/performance/comparison/pydantic_ai_instantiation.py
-```
-
-LangGraph is on the right, **let's start it first and give it a head start**. Then CrewAI and Pydantic AI follow, and finally Agno. Agno obviously finishes first, but let's see by how much.
+Run the benchmarks yourself: [`cookbook/evals/performance`](https://github.com/agno-agi/agno/tree/main/cookbook/evals/performance)
 
 https://github.com/user-attachments/assets/54b98576-1859-4880-9f2d-15e1a426719d
 
-### Memory Usage
-
-To measure memory usage, we use the `tracemalloc` library. We first calculate a baseline memory usage by running an empty function, then run the Agent 1000x times and calculate the difference. This gives a (reasonably) isolated measurement of the memory usage of the Agent.
-
-We recommend running the evaluation yourself on your own machine, and digging into the code to see how it works. If we've made a mistake, please let us know.
-
-### Results
-
-Taking Agno as the baseline, we can see that:
-
-| Metric             | Agno | Langgraph   | PydanticAI | CrewAI     |
-| ------------------ | ---- | ----------- | ---------- | ---------- |
-| **Time (seconds)** | 1×   | 529× slower | 57× slower | 70× slower |
-| **Memory (MiB)**   | 1×   | 24× higher  | 4× higher  | 10× higher |
+## IDE Integration
 
-Exact numbers from the benchmark:
+For AI-assisted development, add our docs to your IDE:
 
-| Metric             | Agno     | Langgraph | PydanticAI | CrewAI   |
-| ------------------ | -------- | --------- | ---------- | -------- |
-| **Time (seconds)** | 0.000003 | 0.001587  | 0.000170   | 0.000210 |
-| **Memory (MiB)**   | 0.006642 | 0.161435  | 0.028712   | 0.065652 |
+**Cursor:** Settings → Indexing & Docs → Add `https://docs.agno.com/llms-full.txt`
 
-> [!NOTE]
-> Agno agents are designed for performance and while we share benchmarks against other frameworks, we should be mindful that accuracy and reliability are more important than speed.
+Works with VSCode, Windsurf, and other AI-enabled editors too.
 
-## Contributions
+## Contributing
 
-We welcome contributions, read our [contributing guide](https://github.com/agno-agi/agno/blob/v2.0/CONTRIBUTING.md) to get started.
+We welcome contributions. See the [contributing guide](https://github.com/agno-agi/agno/blob/v2.0/CONTRIBUTING.md).
 
 ## Telemetry
 
-Agno logs which model an agent used so we can prioritize updates to the most popular providers. You can disable this by setting `AGNO_TELEMETRY=false` in your environment.
+Agno logs which model providers are used so we can prioritize updates. Disable with `AGNO_TELEMETRY=false`.
 
 <p align="left">
   <a href="#top">⬆️ Back to Top</a>

agno-2.3.21/README.md

@@ -0,0 +1,167 @@
+<div align="center" id="top">
+  <a href="https://agno.com">
+    <picture>
+      <source media="(prefers-color-scheme: dark)" srcset="https://agno-public.s3.us-east-1.amazonaws.com/assets/logo-dark.svg">
+      <source media="(prefers-color-scheme: light)" srcset="https://agno-public.s3.us-east-1.amazonaws.com/assets/logo-light.svg">
+      <img src="https://agno-public.s3.us-east-1.amazonaws.com/assets/logo-light.svg" alt="Agno">
+    </picture>
+  </a>
+</div>
+
+<div align="center">
+  <a href="https://docs.agno.com">Documentation</a>
+  <span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+  <a href="https://github.com/agno-agi/agno/tree/main/cookbook">Cookbook</a>
+  <span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
+  <a href="https://www.agno.com/?utm_source=github&utm_medium=readme&utm_campaign=agno-github">Website</a>
+  <br />
+</div>
+
+## What is Agno?
+
+Agno is a multi-agent framework, runtime, and control plane. Use it to build private and secure AI products that run in your cloud.
+
+- **Build** agents, teams, and workflows with memory, knowledge, guardrails, and 100+ toolkits.
+- **Run** in production with a stateless FastAPI runtime. Horizontally scalable.
+- **Manage** with a control plane that connects directly to your runtime — no data leaves your environment.
+
+## Why Agno?
+
+- **Your cloud, your data:** AgentOS runs entirely in your infrastructure. Zero data leaves your environment.
+- **Production-ready from day one:** Pre-built FastAPI runtime with SSE endpoints, ready to deploy.
+- **Actually fast:** 529× faster than LangGraph, 24× lower memory. Matters at scale.
+
+## Getting Started
+
+New to Agno? Start with the [getting started guide](https://github.com/agno-agi/agno/tree/main/cookbook/00_getting_started).
+
+Then:
+- Browse the [cookbooks](https://github.com/agno-agi/agno/tree/main/cookbook) for real-world examples
+- Read the [docs](https://docs.agno.com) to learn more.
+
+## Resources
+
+- Docs: <a href="https://docs.agno.com" target="_blank" rel="noopener noreferrer">docs.agno.com</a>
+- Cookbook: <a href="https://github.com/agno-agi/agno/tree/main/cookbook" target="_blank" rel="noopener noreferrer">Cookbook</a>
+- Community forum: <a href="https://community.agno.com/" target="_blank" rel="noopener noreferrer">community.agno.com</a>
+- Discord: <a href="https://discord.gg/4MtYHHrgA8" target="_blank" rel="noopener noreferrer">discord</a>
+
+## Example
+
+Here's an example of an Agent that connects to an MCP server, manages conversation state in a database, is served using a FastAPI application that you can chat with using the [AgentOS UI](https://os.agno.com).
+
+```python agno_agent.py
+from agno.agent import Agent
+from agno.db.sqlite import SqliteDb
+from agno.models.anthropic import Claude
+from agno.os import AgentOS
+from agno.tools.mcp import MCPTools
+
+# ************* Create Agent *************
+agno_agent = Agent(
+    name="Agno Agent",
+    model=Claude(id="claude-sonnet-4-5"),
+    # Add a database to the Agent
+    db=SqliteDb(db_file="agno.db"),
+    # Add the Agno MCP server to the Agent
+    tools=[MCPTools(transport="streamable-http", url="https://docs.agno.com/mcp")],
+    # Add the previous session history to the context
+    add_history_to_context=True,
+    markdown=True,
+)
+
+
+# ************* Create AgentOS *************
+agent_os = AgentOS(agents=[agno_agent])
+# Get the FastAPI app for the AgentOS
+app = agent_os.get_app()
+
+# ************* Run AgentOS *************
+if __name__ == "__main__":
+    agent_os.serve(app="agno_agent:app", reload=True)
+```
+
+## AgentOS - Production Runtime for Multi-Agent Systems
+
+Building Agents is easy, running them as a secure, scalable service is hard. AgentOS solves this by providing a high performance runtime for serving multi-agent systems in production. Key features include:
+
+1. **Pre-built FastAPI app**: AgentOS includes a ready-to-use FastAPI app for running your agents, teams and workflows. This gives you a significant head start when building an AI product.
+
+2. **Integrated Control Plane**: The [AgentOS UI](https://os.agno.com) connects directly to your runtime, so you can test, monitor and manage your system in real time with full operational visibility.
+
+3. **Private by Design**: AgentOS runs entirely in your cloud, ensuring complete data privacy. No data leaves your environment, making it ideal for security conscious enterprises..
+
+When you run the example script shared above, you get a FastAPI app that you can connect to the [AgentOS UI](https://os.agno.com). Here's what it looks like in action:
+
+https://github.com/user-attachments/assets/feb23db8-15cc-4e88-be7c-01a21a03ebf6
+
+## Private by Design
+
+This is the part we care most about.
+
+AgentOS runs in **your** cloud. The control plane UI connects directly to your runtime from your browser. Your data never touches our servers. No retention costs, no vendor lock-in, no compliance headaches.
+
+This isn't a privacy mode or enterprise add-on. It's how Agno works.
+
+## Features
+
+**Core**
+- Model agnostic — works with OpenAI, Anthropic, Google, local models, whatever
+- Type-safe I/O with `input_schema` and `output_schema`
+- Async-first, built for long-running tasks
+- Natively multimodal (text, images, audio, video, files)
+
+**Memory & Knowledge**
+- Persistent storage for session history and state
+- User memory that persists across sessions
+- Agentic RAG with 20+ vector stores, hybrid search, reranking
+- Culture — shared long-term memory across agents
+
+**Execution**
+- Human-in-the-loop (confirmations, approvals, overrides)
+- Guardrails for validation and security
+- Pre/post hooks for the agent lifecycle
+- First-class MCP and A2A support
+- 100+ built-in toolkits
+
+**Production**
+- Ready-to-use FastAPI runtime
+- Integrated control plane UI
+- Evals for accuracy, performance, latency
+- Durable execution for resumable workflows
+- RBAC and per-agent permissions
+
+## Performance
+
+We're obsessive about performance because agent workloads spawn hundreds of instances and run long tasks. Stateless, horizontal scalability isn't optional.
+
+**Benchmarks** (Apple M4 MacBook Pro, Oct 2025):
+
+| Metric | Agno | LangGraph | PydanticAI | CrewAI |
+|--------|------|-----------|------------|--------|
+| Instantiation | **3μs** | 1,587μs (529× slower) | 170μs (57× slower) | 210μs (70× slower) |
+| Memory | **6.6 KiB** | 161 KiB (24× higher) | 29 KiB (4× higher) | 66 KiB (10× higher) |
+
+Run the benchmarks yourself: [`cookbook/evals/performance`](https://github.com/agno-agi/agno/tree/main/cookbook/evals/performance)
+
+https://github.com/user-attachments/assets/54b98576-1859-4880-9f2d-15e1a426719d
+
+## IDE Integration
+
+For AI-assisted development, add our docs to your IDE:
+
+**Cursor:** Settings → Indexing & Docs → Add `https://docs.agno.com/llms-full.txt`
+
+Works with VSCode, Windsurf, and other AI-enabled editors too.
+
+## Contributing
+
+We welcome contributions. See the [contributing guide](https://github.com/agno-agi/agno/blob/v2.0/CONTRIBUTING.md).
+
+## Telemetry
+
+Agno logs which model providers are used so we can prioritize updates. Disable with `AGNO_TELEMETRY=false`.
+
+<p align="left">
+  <a href="#top">⬆️ Back to Top</a>
+</p>

{agno-2.3.20 → agno-2.3.21}/agno/eval/agent_as_judge.py

@@ -518,8 +518,11 @@ class AgentAsJudgeEval(BaseEval):
         if self.print_summary or print_summary:
             result.print_summary(console)
 
+        # evaluator model info
+        model_id = self.model.id if self.model is not None else None
+        model_provider = self.model.provider if self.model is not None else None
         # Log to DB
-        self._log_eval_to_db(run_id=run_id, result=result)
+        self._log_eval_to_db(run_id=run_id, result=result, model_id=model_id, model_provider=model_provider)
 
         if self.telemetry:
             from agno.api.evals import EvalRunCreate, create_eval_run_telemetry
@@ -613,8 +616,11 @@ class AgentAsJudgeEval(BaseEval):
         if self.print_summary or print_summary:
             result.print_summary(console)
 
+        # evaluator model info
+        model_id = self.model.id if self.model is not None else None
+        model_provider = self.model.provider if self.model is not None else None
         # Log to DB
-        await self._async_log_eval_to_db(run_id=run_id, result=result)
+        await self._async_log_eval_to_db(run_id=run_id, result=result, model_id=model_id, model_provider=model_provider)
 
         if self.telemetry:
             from agno.api.evals import EvalRunCreate, async_create_eval_run_telemetry
@@ -681,8 +687,11 @@ class AgentAsJudgeEval(BaseEval):
         if self.print_summary or print_summary:
             result.print_summary(console)
 
+        # evaluator model info
+        model_id = self.model.id if self.model is not None else None
+        model_provider = self.model.provider if self.model is not None else None
         # Log to DB
-        self._log_eval_to_db(run_id=run_id, result=result)
+        self._log_eval_to_db(run_id=run_id, result=result, model_id=model_id, model_provider=model_provider)
 
         if self.telemetry:
             from agno.api.evals import EvalRunCreate, create_eval_run_telemetry
@@ -748,8 +757,11 @@ class AgentAsJudgeEval(BaseEval):
         if self.print_summary or print_summary:
             result.print_summary(console)
 
+        # evaluator model info
+        model_id = self.model.id if self.model is not None else None
+        model_provider = self.model.provider if self.model is not None else None
         # Log to DB
-        await self._async_log_eval_to_db(run_id=run_id, result=result)
+        await self._async_log_eval_to_db(run_id=run_id, result=result, model_id=model_id, model_provider=model_provider)
 
         if self.telemetry:
             from agno.api.evals import EvalRunCreate, async_create_eval_run_telemetry
@@ -801,15 +813,14 @@ class AgentAsJudgeEval(BaseEval):
         if isinstance(run_output, RunOutput):
             agent_id = run_output.agent_id
             team_id = None
-            model_id = run_output.model
-            model_provider = run_output.model_provider
         elif isinstance(run_output, TeamRunOutput):
             agent_id = None
             team_id = run_output.team_id
-            model_id = run_output.model
-            model_provider = run_output.model_provider
 
-        # Log to DB if we have a valid result (use run_id from result)
+        # evaluator model info
+        model_id = self.model.id if self.model is not None else None
+        model_provider = self.model.provider if self.model is not None else None
+        # Log to DB if we have a valid result
         if result:
             self._log_eval_to_db(
                 run_id=result.run_id,
@@ -841,15 +852,14 @@ class AgentAsJudgeEval(BaseEval):
         if isinstance(run_output, RunOutput):
             agent_id = run_output.agent_id
             team_id = None
-            model_id = run_output.model
-            model_provider = run_output.model_provider
         elif isinstance(run_output, TeamRunOutput):
             agent_id = None
             team_id = run_output.team_id
-            model_id = run_output.model
-            model_provider = run_output.model_provider
 
-        # Log to DB if we have a valid result (use run_id from result)
+        # evaluator model info
+        model_id = self.model.id if self.model is not None else None
+        model_provider = self.model.provider if self.model is not None else None
+        # Log to DB if we have a valid result
         if result:
             await self._async_log_eval_to_db(
                 run_id=result.run_id,

{agno-2.3.20 → agno-2.3.21}/agno/knowledge/embedder/mistral.py

@@ -37,7 +37,7 @@ class MistralEmbedder(Embedder):
             "api_key": self.api_key,
             "endpoint": self.endpoint,
             "max_retries": self.max_retries,
-            "timeout": self.timeout,
+            "timeout_ms": self.timeout * 1000 if self.timeout else None,
         }
         _client_params = {k: v for k, v in _client_params.items() if v is not None}
 

{agno-2.3.20 → agno-2.3.21}/agno/os/routers/evals/evals.py

@@ -144,15 +144,6 @@ def attach_routes(
                 headers=headers,
             )
 
-        # TODO: Delete me:
-        # Filtering out agent-as-judge by default for now,
-        # as they are not supported yet in the AgentOS UI.
-        eval_types = eval_types or [
-            EvalType.ACCURACY,
-            EvalType.PERFORMANCE,
-            EvalType.RELIABILITY,
-        ]
-
         if isinstance(db, AsyncBaseDb):
             db = cast(AsyncBaseDb, db)
             eval_runs, total_count = await db.get_eval_runs(

{agno-2.3.20 → agno-2.3.21}/agno/os/routers/evals/utils.py

@@ -68,15 +68,11 @@ async def run_agent_as_judge_eval(
     if agent:
         agent_response = await agent.arun(eval_run_input.input, stream=False)
         output = str(agent_response.content) if agent_response.content else ""
-        model_id = agent.model.id if agent and agent.model else None
-        model_provider = agent.model.provider if agent and agent.model else None
         agent_id = agent.id
         team_id = None
     elif team:
         team_response = await team.arun(eval_run_input.input, stream=False)
         output = str(team_response.content) if team_response.content else ""
-        model_id = team.model.id if team and team.model else None
-        model_provider = team.model.provider if team and team.model else None
         agent_id = None
         team_id = team.id
     else:
@@ -98,13 +94,17 @@ async def run_agent_as_judge_eval(
     if not result:
         raise HTTPException(status_code=500, detail="Failed to run agent as judge evaluation")
 
+    # Use evaluator's model
+    eval_model_id = agent_as_judge_eval.model.id if agent_as_judge_eval.model is not None else None
+    eval_model_provider = agent_as_judge_eval.model.provider if agent_as_judge_eval.model is not None else None
+
     eval_run = EvalSchema.from_agent_as_judge_eval(
         agent_as_judge_eval=agent_as_judge_eval,
         result=result,
         agent_id=agent_id,
         team_id=team_id,
-        model_id=model_id,
-        model_provider=model_provider,
+        model_id=eval_model_id,
+        model_provider=eval_model_provider,
     )
 
     # Restore original model after eval

{agno-2.3.20 → agno-2.3.21}/agno/run/agent.py

@@ -55,8 +55,11 @@ class RunInput:
             return self.input_content.model_dump_json(exclude_none=True)
         elif isinstance(self.input_content, Message):
             return json.dumps(self.input_content.to_dict())
-        elif isinstance(self.input_content, list) and self.input_content and isinstance(self.input_content[0], Message):
-            return json.dumps([m.to_dict() for m in self.input_content])
+        elif isinstance(self.input_content, list):
+            try:
+                return json.dumps(self.to_dict().get("input_content"))
+            except Exception:
+                return str(self.input_content)
         else:
             return str(self.input_content)
 
@@ -71,22 +74,15 @@ class RunInput:
                 result["input_content"] = self.input_content.model_dump(exclude_none=True)
             elif isinstance(self.input_content, Message):
                 result["input_content"] = self.input_content.to_dict()
-
-            # Handle input_content provided as a list of Message objects
-            elif (
-                isinstance(self.input_content, list)
-                and self.input_content
-                and isinstance(self.input_content[0], Message)
-            ):
-                result["input_content"] = [m.to_dict() for m in self.input_content]
-
-            # Handle input_content provided as a list of dicts
-            elif (
-                isinstance(self.input_content, list) and self.input_content and isinstance(self.input_content[0], dict)
-            ):
-                for content in self.input_content:
-                    # Handle media input
-                    if isinstance(content, dict):
+            elif isinstance(self.input_content, list):
+                serialized_items: List[Any] = []
+                for item in self.input_content:
+                    if isinstance(item, Message):
+                        serialized_items.append(item.to_dict())
+                    elif isinstance(item, BaseModel):
+                        serialized_items.append(item.model_dump(exclude_none=True))
+                    elif isinstance(item, dict):
+                        content = dict(item)
                         if content.get("images"):
                             content["images"] = [
                                 img.to_dict() if isinstance(img, Image) else img for img in content["images"]
@@ -103,7 +99,11 @@ class RunInput:
                             content["files"] = [
                                 file.to_dict() if isinstance(file, File) else file for file in content["files"]
                             ]
-                result["input_content"] = self.input_content
+                        serialized_items.append(content)
+                    else:
+                        serialized_items.append(item)
+
+                result["input_content"] = serialized_items
             else:
                 result["input_content"] = self.input_content