autobyteus 1.1.5__tar.gz → 1.1.7__tar.gz

autobyteus-1.1.7/PKG-INFO

@@ -0,0 +1,204 @@
+Metadata-Version: 2.4
+Name: autobyteus
+Version: 1.1.7
+Summary: Multi-Agent framework
+Home-page: https://github.com/AutoByteus/autobyteus
+Author: Ryan Zheng
+Author-email: ryan.zheng.work@gmail.com
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: aiohttp
+Requires-Dist: anthropic
+Requires-Dist: autobyteus-llm-client==1.1.3
+Requires-Dist: beautifulsoup4
+Requires-Dist: boto3
+Requires-Dist: botocore
+Requires-Dist: brui-core==1.0.9
+Requires-Dist: certifi==2025.4.26
+Requires-Dist: google-api-python-client
+Requires-Dist: google-generativeai
+Requires-Dist: Jinja2
+Requires-Dist: mcp[cli]
+Requires-Dist: mistral_common
+Requires-Dist: mistralai
+Requires-Dist: numpy
+Requires-Dist: ollama
+Requires-Dist: openai
+Requires-Dist: requests
+Requires-Dist: rich
+Requires-Dist: textual
+Requires-Dist: tiktoken
+Requires-Dist: tokenizers
+Provides-Extra: dev
+Requires-Dist: coverage; extra == "dev"
+Requires-Dist: flake8; extra == "dev"
+Requires-Dist: numpy; extra == "dev"
+Requires-Dist: pre-commit; extra == "dev"
+Requires-Dist: black; extra == "dev"
+Requires-Dist: isort; extra == "dev"
+Requires-Dist: gitpython==3.1.31; extra == "dev"
+Requires-Dist: auto-gpt-plugin-template; extra == "dev"
+Requires-Dist: mkdocs; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: asynctest; extra == "dev"
+Requires-Dist: pytest-asyncio; extra == "dev"
+Requires-Dist: pytest-benchmark; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: pytest-integration; extra == "dev"
+Requires-Dist: pytest-mock; extra == "dev"
+Requires-Dist: vcrpy; extra == "dev"
+Requires-Dist: pytest-vcr; extra == "dev"
+Requires-Dist: load_dotenv; extra == "dev"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# Autobyteus
+
+Autobyteus is an open-source, application-first agentic framework for Python. It is designed to help developers build, test, and deploy complex, stateful, and extensible AI agents by providing a robust architecture and a powerful set of tools.
+
+![Autobyteus TUI Dashboard](docs/images/image_1.png)
+
+## Architecture
+
+Autobyteus is built with a modular, event-driven architecture designed for extensibility and clear separation of concerns. The key components are:
+
+-   **Agent Core**: The heart of the system. Each agent is a stateful, autonomous entity that runs as a background process in its own thread, managed by a dedicated `AgentWorker`. This design makes every agent a truly independent entity capable of handling long-running tasks.
+-   **Agent Teams**: The framework provides powerful constructs for building hierarchical multi-agent systems. The `AgentTeam` module allows you to compose teams of individual agents and even nest teams within other teams, enabling sophisticated, real-world organizational structures and delegation patterns.
+-   **Context & Configuration**: Agent behavior is defined through a static configuration (`AgentConfig`) and its dynamic state is managed in `AgentRuntimeState`. These are bundled into a comprehensive `AgentContext` that is passed to all components, providing a single source of truth.
+-   **Event-Driven System**: Agents operate on an internal `asyncio` event loop. User messages, tool results, and internal signals are handled as events, which are processed by dedicated `EventHandlers`. This decouples logic and makes the system highly extensible.
+-   **Pluggable Processors & Hooks**: The framework provides a chain of extension points to inject custom logic at every major step of an agent's reasoning loop. This architecture powers features like flexible tool format parsing. You can customize behavior by implementing:
+    -   **`InputProcessors`**: To modify or enrich user messages *before* they are sent to the LLM.
+    -   **`LLMResponseProcessors`**: To parse the LLM's raw output and extract structured actions, such as tool calls.
+    -   **`ToolExecutionResultProcessors`**: To modify the result from a tool *before* it is sent back to the LLM for the next step of reasoning.
+    -   **`PhaseHooks`**: To run custom code on specific agent lifecycle transitions (e.g., when an agent becomes `IDLE`).
+-   **Context-Aware Tooling**: Tools are first-class citizens that receive the agent's full `AgentContext` during execution. This allows tools to be deeply integrated with the agent's state, configuration, and workspace, enabling more intelligent and powerful actions.
+-   **Tool Approval Flow**: The framework has native support for human-in-the-loop workflows. By setting `auto_execute_tools=False` in the agent's configuration, the agent will pause before executing a tool, emit an event requesting permission, and wait for external approval before proceeding.
+-   **MCP Integration**: The framework has native support for the Model Context Protocol (MCP). This allows agents to discover and use tools from external, language-agnostic tool servers, making the ecosystem extremely flexible and ready for enterprise integration.
+
+## Key Features
+
+#### 📊 Interactive TUI Dashboard
+Launch and monitor your agent teams with our built-in Textual-based TUI.
+-   **Hierarchical View**: See the structure of your team, including sub-teams and their agents.
+-   **Real-Time Status**: Agent and team statuses are updated live, showing you who is idle, thinking, or executing a tool.
+-   **Detailed Logs**: Select any agent to view a detailed, streaming log of their thoughts, actions, and tool interactions.
+-   **Live Task Board**: Watch your team's `TaskBoard` update in real-time as the coordinator publishes a plan and agents complete their tasks.
+
+| TUI - Detailed Agent Log | TUI - Task Board with Completed Task |
+| :---: | :---: |
+| ![Autobyteus Agent Log](docs/images/image_4.png) | ![Autobyteus Task Board](docs/images/image_3.png) |
+
+#### 🏗️ Fluent Team Building
+Define complex agent and team structures with an intuitive, fluent API. The `AgentTeamBuilder` makes composing your team simple and readable.
+
+```python
+# --- From the Multi-Researcher Team Example ---
+research_team = (
+    AgentTeamBuilder(
+        name="MultiSpecialistResearchTeam",
+        description="A team for delegating to multiple specialists."
+    )
+    .set_coordinator(coordinator_config)
+    .add_agent_node(researcher_web_config)
+    .add_agent_node(researcher_db_config)
+    .build()
+)
+```
+
+#### 🔁 Flexible Tool Formatting (JSON & XML)
+Autobyteus intelligently handles tool communication with LLMs while giving you full control.
+-   **Provider-Aware by Default**: The framework automatically generates tool manifests and parses responses in the optimal format for the selected LLM provider (e.g., JSON for OpenAI/Gemini, XML for Anthropic).
+-   **XML Override for Efficiency**: You can set `use_xml_tool_format=True` on an `AgentConfig` or `AgentTeamBuilder` to force the use of XML for tool calls, which can be more efficient and reliable than JSON for complex tool schemas.
+
+#### 📈 Flexible Communication Protocols
+Choose the collaboration pattern that best fits your use case with configurable `TaskNotificationMode`s.
+-   **`AGENT_MANUAL_NOTIFICATION` (Default)**: A traditional approach where a coordinator agent is responsible for creating a plan and then explicitly notifying other agents to begin their work via messages.
+-   **`SYSTEM_EVENT_DRIVEN`**: A more automated approach where the coordinator's only job is to publish a plan to the `TaskBoard`. The framework then monitors the board and automatically notifies agents when their tasks become unblocked, enabling parallel execution and reducing coordinator overhead.
+
+## Requirements
+
+-   **Python Version**: Python 3.11 is the recommended and tested version for this project. Using newer versions of Python may result in dependency conflicts when installing the required packages. For a stable and tested environment, please use Python 3.11.
+
+## Getting Started
+
+### Installation
+
+1.  **Clone the repository:**
+    ```bash
+    git clone https://github.com/your-username/autobyteus.git
+    cd autobyteus
+    ```
+
+2.  **For users:**
+    To install Autobyteus and its core dependencies:
+    ```bash
+    pip install .
+    ```
+
+3.  **For developers:**
+    To install Autobyteus with all development and example dependencies (including the TUI):
+    ```bash
+    pip install -r requirements-dev.txt
+    ```
+
+4.  **Set up Environment Variables:**
+    Create a `.env` file in the root of the project and add your LLM provider API keys:
+    ```
+    # .env
+    OPENAI_API_KEY="sk-..."
+    KIMI_API_KEY="your-kimi-api-key"
+    # etc.
+    ```
+
+### Running the Examples
+
+The best way to experience Autobyteus is to run one of the included examples. The event-driven software engineering team is a great showcase of the framework's capabilities.
+
+```bash
+# Run the event-driven software engineering team example
+python autobyteus/examples/agent_team/event_driven/run_software_engineering_team.py --llm-model gpt-4o
+
+# Run the hierarchical debate team example
+python autobyteus/examples/agent_team/manual_notification/run_debate_team.py --llm-model gpt-4-turbo
+```
+You can see all available models and their identifiers by running an example with the `--help-models` flag.
+
+### Building the Library
+
+To build Autobyteus as a distributable package, follow these steps:
+
+1.  Ensure you have the latest version of `setuptools` and `wheel` installed:
+    ```
+    pip install --upgrade setuptools wheel
+    ```
+
+2.  Build the distribution packages:
+    ```
+    python setup.py sdist bdist_wheel
+    ```
+
+This will create a `dist` directory containing the built `sdist` and `wheel` distributions.
+
+## Contributing
+
+(Add guidelines for contributing to the project)
+
+## License
+
+This project is licensed under the MIT License.

autobyteus-1.1.7/README.md

@@ -0,0 +1,134 @@
+# Autobyteus
+
+Autobyteus is an open-source, application-first agentic framework for Python. It is designed to help developers build, test, and deploy complex, stateful, and extensible AI agents by providing a robust architecture and a powerful set of tools.
+
+![Autobyteus TUI Dashboard](docs/images/image_1.png)
+
+## Architecture
+
+Autobyteus is built with a modular, event-driven architecture designed for extensibility and clear separation of concerns. The key components are:
+
+-   **Agent Core**: The heart of the system. Each agent is a stateful, autonomous entity that runs as a background process in its own thread, managed by a dedicated `AgentWorker`. This design makes every agent a truly independent entity capable of handling long-running tasks.
+-   **Agent Teams**: The framework provides powerful constructs for building hierarchical multi-agent systems. The `AgentTeam` module allows you to compose teams of individual agents and even nest teams within other teams, enabling sophisticated, real-world organizational structures and delegation patterns.
+-   **Context & Configuration**: Agent behavior is defined through a static configuration (`AgentConfig`) and its dynamic state is managed in `AgentRuntimeState`. These are bundled into a comprehensive `AgentContext` that is passed to all components, providing a single source of truth.
+-   **Event-Driven System**: Agents operate on an internal `asyncio` event loop. User messages, tool results, and internal signals are handled as events, which are processed by dedicated `EventHandlers`. This decouples logic and makes the system highly extensible.
+-   **Pluggable Processors & Hooks**: The framework provides a chain of extension points to inject custom logic at every major step of an agent's reasoning loop. This architecture powers features like flexible tool format parsing. You can customize behavior by implementing:
+    -   **`InputProcessors`**: To modify or enrich user messages *before* they are sent to the LLM.
+    -   **`LLMResponseProcessors`**: To parse the LLM's raw output and extract structured actions, such as tool calls.
+    -   **`ToolExecutionResultProcessors`**: To modify the result from a tool *before* it is sent back to the LLM for the next step of reasoning.
+    -   **`PhaseHooks`**: To run custom code on specific agent lifecycle transitions (e.g., when an agent becomes `IDLE`).
+-   **Context-Aware Tooling**: Tools are first-class citizens that receive the agent's full `AgentContext` during execution. This allows tools to be deeply integrated with the agent's state, configuration, and workspace, enabling more intelligent and powerful actions.
+-   **Tool Approval Flow**: The framework has native support for human-in-the-loop workflows. By setting `auto_execute_tools=False` in the agent's configuration, the agent will pause before executing a tool, emit an event requesting permission, and wait for external approval before proceeding.
+-   **MCP Integration**: The framework has native support for the Model Context Protocol (MCP). This allows agents to discover and use tools from external, language-agnostic tool servers, making the ecosystem extremely flexible and ready for enterprise integration.
+
+## Key Features
+
+#### 📊 Interactive TUI Dashboard
+Launch and monitor your agent teams with our built-in Textual-based TUI.
+-   **Hierarchical View**: See the structure of your team, including sub-teams and their agents.
+-   **Real-Time Status**: Agent and team statuses are updated live, showing you who is idle, thinking, or executing a tool.
+-   **Detailed Logs**: Select any agent to view a detailed, streaming log of their thoughts, actions, and tool interactions.
+-   **Live Task Board**: Watch your team's `TaskBoard` update in real-time as the coordinator publishes a plan and agents complete their tasks.
+
+| TUI - Detailed Agent Log | TUI - Task Board with Completed Task |
+| :---: | :---: |
+| ![Autobyteus Agent Log](docs/images/image_4.png) | ![Autobyteus Task Board](docs/images/image_3.png) |
+
+#### 🏗️ Fluent Team Building
+Define complex agent and team structures with an intuitive, fluent API. The `AgentTeamBuilder` makes composing your team simple and readable.
+
+```python
+# --- From the Multi-Researcher Team Example ---
+research_team = (
+    AgentTeamBuilder(
+        name="MultiSpecialistResearchTeam",
+        description="A team for delegating to multiple specialists."
+    )
+    .set_coordinator(coordinator_config)
+    .add_agent_node(researcher_web_config)
+    .add_agent_node(researcher_db_config)
+    .build()
+)
+```
+
+#### 🔁 Flexible Tool Formatting (JSON & XML)
+Autobyteus intelligently handles tool communication with LLMs while giving you full control.
+-   **Provider-Aware by Default**: The framework automatically generates tool manifests and parses responses in the optimal format for the selected LLM provider (e.g., JSON for OpenAI/Gemini, XML for Anthropic).
+-   **XML Override for Efficiency**: You can set `use_xml_tool_format=True` on an `AgentConfig` or `AgentTeamBuilder` to force the use of XML for tool calls, which can be more efficient and reliable than JSON for complex tool schemas.
+
+#### 📈 Flexible Communication Protocols
+Choose the collaboration pattern that best fits your use case with configurable `TaskNotificationMode`s.
+-   **`AGENT_MANUAL_NOTIFICATION` (Default)**: A traditional approach where a coordinator agent is responsible for creating a plan and then explicitly notifying other agents to begin their work via messages.
+-   **`SYSTEM_EVENT_DRIVEN`**: A more automated approach where the coordinator's only job is to publish a plan to the `TaskBoard`. The framework then monitors the board and automatically notifies agents when their tasks become unblocked, enabling parallel execution and reducing coordinator overhead.
+
+## Requirements
+
+-   **Python Version**: Python 3.11 is the recommended and tested version for this project. Using newer versions of Python may result in dependency conflicts when installing the required packages. For a stable and tested environment, please use Python 3.11.
+
+## Getting Started
+
+### Installation
+
+1.  **Clone the repository:**
+    ```bash
+    git clone https://github.com/your-username/autobyteus.git
+    cd autobyteus
+    ```
+
+2.  **For users:**
+    To install Autobyteus and its core dependencies:
+    ```bash
+    pip install .
+    ```
+
+3.  **For developers:**
+    To install Autobyteus with all development and example dependencies (including the TUI):
+    ```bash
+    pip install -r requirements-dev.txt
+    ```
+
+4.  **Set up Environment Variables:**
+    Create a `.env` file in the root of the project and add your LLM provider API keys:
+    ```
+    # .env
+    OPENAI_API_KEY="sk-..."
+    KIMI_API_KEY="your-kimi-api-key"
+    # etc.
+    ```
+
+### Running the Examples
+
+The best way to experience Autobyteus is to run one of the included examples. The event-driven software engineering team is a great showcase of the framework's capabilities.
+
+```bash
+# Run the event-driven software engineering team example
+python autobyteus/examples/agent_team/event_driven/run_software_engineering_team.py --llm-model gpt-4o
+
+# Run the hierarchical debate team example
+python autobyteus/examples/agent_team/manual_notification/run_debate_team.py --llm-model gpt-4-turbo
+```
+You can see all available models and their identifiers by running an example with the `--help-models` flag.
+
+### Building the Library
+
+To build Autobyteus as a distributable package, follow these steps:
+
+1.  Ensure you have the latest version of `setuptools` and `wheel` installed:
+    ```
+    pip install --upgrade setuptools wheel
+    ```
+
+2.  Build the distribution packages:
+    ```
+    python setup.py sdist bdist_wheel
+    ```
+
+This will create a `dist` directory containing the built `sdist` and `wheel` distributions.
+
+## Contributing
+
+(Add guidelines for contributing to the project)
+
+## License
+
+This project is licensed under the MIT License.

{autobyteus-1.1.5 → autobyteus-1.1.7}/autobyteus/agent/context/agent_config.py

@@ -37,6 +37,7 @@ class AgentConfig:
                  system_prompt: Optional[str] = None,
                  tools: Optional[List['BaseTool']] = None,
                  auto_execute_tools: bool = True,
+                 use_xml_tool_format: bool = False,
                  input_processors: Optional[List['BaseAgentUserInputMessageProcessor']] = None,
                  llm_response_processors: Optional[List['BaseLLMResponseProcessor']] = None,
                  system_prompt_processors: Optional[List['BaseSystemPromptProcessor']] = None,
@@ -57,6 +58,8 @@ class AgentConfig:
                            llm_instance's config will be used as the base.
             tools: An optional list of pre-initialized tool instances (subclasses of BaseTool).
             auto_execute_tools: If True, the agent will execute tools without approval.
+            use_xml_tool_format: If True, forces the agent to use XML format for tool
+                                 definitions and parsing, overriding provider defaults.
             input_processors: A list of input processor instances.
             llm_response_processors: A list of LLM response processor instances.
             system_prompt_processors: A list of system prompt processor instances.
@@ -74,6 +77,7 @@ class AgentConfig:
         self.tools = tools or []
         self.workspace = workspace
         self.auto_execute_tools = auto_execute_tools
+        self.use_xml_tool_format = use_xml_tool_format
         self.input_processors = input_processors or []
         self.llm_response_processors = llm_response_processors if llm_response_processors is not None else list(self.DEFAULT_LLM_RESPONSE_PROCESSORS)
         self.system_prompt_processors = system_prompt_processors if system_prompt_processors is not None else list(self.DEFAULT_SYSTEM_PROMPT_PROCESSORS)
@@ -81,7 +85,7 @@ class AgentConfig:
         self.phase_hooks = phase_hooks or []
         self.initial_custom_data = initial_custom_data
 
-        logger.debug(f"AgentConfig created for name '{self.name}', role '{self.role}'.")
+        logger.debug(f"AgentConfig created for name '{self.name}', role '{self.role}'. XML tool format override: {self.use_xml_tool_format}")
 
     def copy(self) -> 'AgentConfig':
         """
@@ -98,6 +102,7 @@ class AgentConfig:
             system_prompt=self.system_prompt,
             tools=self.tools.copy(),  # Shallow copy the list, but reference the original tool instances
             auto_execute_tools=self.auto_execute_tools,
+            use_xml_tool_format=self.use_xml_tool_format,
             input_processors=self.input_processors.copy(), # Shallow copy the list
             llm_response_processors=self.llm_response_processors.copy(), # Shallow copy the list
             system_prompt_processors=self.system_prompt_processors.copy(), # Shallow copy the list

{autobyteus-1.1.5 → autobyteus-1.1.7}/autobyteus/agent/context/agent_runtime_state.py

@@ -16,6 +16,7 @@ from autobyteus.agent.tool_invocation import ToolInvocation
 if TYPE_CHECKING:
     from autobyteus.agent.phases import AgentPhaseManager 
     from autobyteus.tools.base_tool import BaseTool 
+    from autobyteus.agent.tool_invocation import ToolInvocationTurn
 
 logger = logging.getLogger(__name__)
 
@@ -48,6 +49,9 @@ class AgentRuntimeState:
         self.pending_tool_approvals: Dict[str, ToolInvocation] = {}
         self.custom_data: Dict[str, Any] = custom_data or {}
         
+        # NEW: State for multi-tool call invocation turns, with a very explicit name.
+        self.active_multi_tool_call_turn: Optional['ToolInvocationTurn'] = None
+        
         self.processed_system_prompt: Optional[str] = None
         # self.final_llm_config_for_creation removed
         
@@ -83,7 +87,9 @@ class AgentRuntimeState:
         tools_status = f"{len(self.tool_instances)} Initialized" if self.tool_instances is not None else "Not Initialized"
         input_queues_status = "Initialized" if self.input_event_queues else "Not Initialized"
         # REMOVED output_queues_status from repr
+        active_turn_status = "Active" if self.active_multi_tool_call_turn else "Inactive"
         return (f"AgentRuntimeState(agent_id='{self.agent_id}', current_phase='{phase_repr}', "
                 f"llm_status='{llm_status}', tools_status='{tools_status}', "
                 f"input_queues_status='{input_queues_status}', "
-                f"pending_approvals={len(self.pending_tool_approvals)}, history_len={len(self.conversation_history)})")
+                f"pending_approvals={len(self.pending_tool_approvals)}, history_len={len(self.conversation_history)}, "
+                f"multi_tool_call_turn='{active_turn_status}')")

{autobyteus-1.1.5 → autobyteus-1.1.7}/autobyteus/agent/handlers/llm_user_message_ready_event_handler.py

@@ -1,7 +1,7 @@
 # file: autobyteus/autobyteus/agent/handlers/llm_user_message_ready_event_handler.py
 import logging
 import traceback
-from typing import TYPE_CHECKING, cast, Optional
+from typing import TYPE_CHECKING, cast, Optional, List
 
 from autobyteus.agent.handlers.base_event_handler import AgentEventHandler
 from autobyteus.agent.events import LLMUserMessageReadyEvent, LLMCompleteResponseReceivedEvent 
@@ -53,6 +53,9 @@ class LLMUserMessageReadyEventHandler(AgentEventHandler):
         complete_response_text = ""
         complete_reasoning_text = ""
         token_usage: Optional[TokenUsage] = None
+        complete_image_urls: List[str] = []
+        complete_audio_urls: List[str] = []
+        complete_video_urls: List[str] = []
         
         notifier: Optional['AgentExternalEventNotifier'] = None
         if context.phase_manager:
@@ -72,9 +75,19 @@ class LLMUserMessageReadyEventHandler(AgentEventHandler):
                 if chunk_response.reasoning:
                     complete_reasoning_text += chunk_response.reasoning
 
-                if chunk_response.is_complete and chunk_response.usage:
-                    token_usage = chunk_response.usage
-                    logger.debug(f"Agent '{agent_id}' received final chunk with token usage: {token_usage}")
+                if chunk_response.is_complete:
+                    if chunk_response.usage:
+                        token_usage = chunk_response.usage
+                        logger.debug(f"Agent '{agent_id}' received final chunk with token usage: {token_usage}")
+                    if chunk_response.image_urls:
+                        complete_image_urls.extend(chunk_response.image_urls)
+                        logger.debug(f"Agent '{agent_id}' received final chunk with {len(chunk_response.image_urls)} image URLs.")
+                    if chunk_response.audio_urls:
+                        complete_audio_urls.extend(chunk_response.audio_urls)
+                        logger.debug(f"Agent '{agent_id}' received final chunk with {len(chunk_response.audio_urls)} audio URLs.")
+                    if chunk_response.video_urls:
+                        complete_video_urls.extend(chunk_response.video_urls)
+                        logger.debug(f"Agent '{agent_id}' received final chunk with {len(chunk_response.video_urls)} video URLs.")
 
                 if notifier:
                     try:
@@ -121,20 +134,30 @@ class LLMUserMessageReadyEventHandler(AgentEventHandler):
             logger.info(f"Agent '{agent_id}' enqueued LLMCompleteResponseReceivedEvent with error details from LLMUserMessageReadyEventHandler.")
             return 
 
-        # Add message to history with reasoning
+        # Add message to history with reasoning and multimodal data
         history_entry = {"role": "assistant", "content": complete_response_text}
         if complete_reasoning_text:
             history_entry["reasoning"] = complete_reasoning_text
+        if complete_image_urls:
+            history_entry["image_urls"] = complete_image_urls
+        if complete_audio_urls:
+            history_entry["audio_urls"] = complete_audio_urls
+        if complete_video_urls:
+            history_entry["video_urls"] = complete_video_urls
         context.state.add_message_to_history(history_entry)
         
-        # Create complete response with reasoning
+        # Create complete response with reasoning and multimodal data
         complete_response_obj = CompleteResponse(
             content=complete_response_text,
             reasoning=complete_reasoning_text,
-            usage=token_usage
+            usage=token_usage,
+            image_urls=complete_image_urls,
+            audio_urls=complete_audio_urls,
+            video_urls=complete_video_urls
         )
         llm_complete_event = LLMCompleteResponseReceivedEvent(
             complete_response=complete_response_obj
         )
         await context.input_event_queues.enqueue_internal_system_event(llm_complete_event)
         logger.info(f"Agent '{agent_id}' enqueued LLMCompleteResponseReceivedEvent from LLMUserMessageReadyEventHandler.")
+

autobyteus-1.1.7/autobyteus/agent/handlers/tool_result_event_handler.py

@@ -0,0 +1,141 @@
+# file: autobyteus/autobyteus/agent/handlers/tool_result_event_handler.py
+import logging
+import json 
+from typing import TYPE_CHECKING, Optional, List
+
+from autobyteus.agent.handlers.base_event_handler import AgentEventHandler 
+from autobyteus.agent.events import ToolResultEvent, LLMUserMessageReadyEvent 
+from autobyteus.llm.user_message import LLMUserMessage 
+from autobyteus.agent.tool_execution_result_processor import BaseToolExecutionResultProcessor
+
+if TYPE_CHECKING:
+    from autobyteus.agent.context import AgentContext 
+    from autobyteus.agent.events.notifiers import AgentExternalEventNotifier 
+
+logger = logging.getLogger(__name__)
+
+class ToolResultEventHandler(AgentEventHandler):
+    """
+    Handles ToolResultEvents. It immediately processes and notifies for each
+    individual tool result. If a multi-tool call turn is active, it accumulates
+    these processed results until the turn is complete, then sends a single
+    aggregated message to the LLM.
+    """
+    def __init__(self):
+        logger.info("ToolResultEventHandler initialized.")
+
+    async def _dispatch_aggregated_results_to_llm(self,
+                                                  processed_events: List[ToolResultEvent],
+                                                  context: 'AgentContext'):
+        """
+        Aggregates a list of PRE-PROCESSED tool results into a single message and
+        dispatches it to the LLM.
+        """
+        agent_id = context.agent_id
+        
+        # --- Aggregate results into a single message ---
+        aggregated_content_parts = []
+        for p_event in processed_events:
+            tool_invocation_id = p_event.tool_invocation_id if p_event.tool_invocation_id else 'N/A'
+            content_part: str
+            if p_event.error:
+                content_part = (
+                    f"Tool: {p_event.tool_name} (ID: {tool_invocation_id})\n"
+                    f"Status: Error\n"
+                    f"Details: {p_event.error}"
+                )
+            else:
+                try:
+                    result_str = json.dumps(p_event.result, indent=2) if not isinstance(p_event.result, str) else p_event.result
+                except TypeError: # pragma: no cover
+                    result_str = str(p_event.result)
+                content_part = (
+                    f"Tool: {p_event.tool_name} (ID: {tool_invocation_id})\n"
+                    f"Status: Success\n"
+                    f"Result:\n{result_str}" 
+                )
+            aggregated_content_parts.append(content_part)
+
+        final_content_for_llm = (
+            "The following tool executions have completed. Please analyze their results and decide the next course of action.\n\n"
+            + "\n\n---\n\n".join(aggregated_content_parts)
+        )
+        
+        logger.debug(f"Agent '{agent_id}' preparing aggregated message for LLM:\n---\n{final_content_for_llm}\n---")
+        llm_user_message = LLMUserMessage(content=final_content_for_llm)
+        
+        next_event = LLMUserMessageReadyEvent(llm_user_message=llm_user_message) 
+        await context.input_event_queues.enqueue_internal_system_event(next_event)
+        
+        logger.info(f"Agent '{agent_id}' enqueued LLMUserMessageReadyEvent with aggregated results from {len(processed_events)} tool(s).")
+
+
+    async def handle(self,
+                     event: ToolResultEvent,
+                     context: 'AgentContext') -> None:
+        if not isinstance(event, ToolResultEvent): 
+            logger.warning(f"ToolResultEventHandler received non-ToolResultEvent: {type(event)}. Skipping.")
+            return
+
+        agent_id = context.agent_id
+        notifier: Optional['AgentExternalEventNotifier'] = context.phase_manager.notifier if context.phase_manager else None
+
+        # --- Step 1: Immediately process the incoming event ---
+        processed_event = event
+        processor_instances = context.config.tool_execution_result_processors
+        if processor_instances:
+            for processor_instance in processor_instances:
+                if not isinstance(processor_instance, BaseToolExecutionResultProcessor):
+                    logger.error(f"Agent '{agent_id}': Invalid tool result processor type: {type(processor_instance)}. Skipping.")
+                    continue
+                try:
+                    processed_event = await processor_instance.process(processed_event, context)
+                except Exception as e:
+                    logger.error(f"Agent '{agent_id}': Error applying tool result processor '{processor_instance.get_name()}': {e}", exc_info=True)
+        
+        # --- Step 2: Immediately notify the result of this single tool call ---
+        tool_invocation_id = processed_event.tool_invocation_id if processed_event.tool_invocation_id else 'N/A'
+        if notifier:
+            log_message = ""
+            if processed_event.error:
+                log_message = f"[TOOL_RESULT_ERROR_PROCESSED] Agent_ID: {agent_id}, Tool: {processed_event.tool_name}, Invocation_ID: {tool_invocation_id}, Error: {processed_event.error}"
+            else:
+                log_message = f"[TOOL_RESULT_SUCCESS_PROCESSED] Agent_ID: {agent_id}, Tool: {processed_event.tool_name}, Invocation_ID: {tool_invocation_id}, Result: {str(processed_event.result)}"
+            
+            try:
+                log_data = {
+                    "log_entry": log_message,
+                    "tool_invocation_id": tool_invocation_id,
+                    "tool_name": processed_event.tool_name,
+                }
+                notifier.notify_agent_data_tool_log(log_data)
+                logger.debug(f"Agent '{agent_id}': Notified individual tool result for '{processed_event.tool_name}'.")
+            except Exception as e_notify: 
+                logger.error(f"Agent '{agent_id}': Error notifying tool result log: {e_notify}", exc_info=True)
+
+        # --- Step 3: Manage the multi-tool call turn state ---
+        active_turn = context.state.active_multi_tool_call_turn
+
+        # Case 1: Not a multi-tool call turn, dispatch to LLM immediately.
+        if not active_turn:
+            logger.info(f"Agent '{agent_id}' handling single ToolResultEvent from tool: '{processed_event.tool_name}'.")
+            await self._dispatch_aggregated_results_to_llm([processed_event], context)
+            return
+
+        # Case 2: Multi-tool call turn is active, accumulate results.
+        active_turn.results.append(processed_event)
+        num_results = len(active_turn.results)
+        num_expected = len(active_turn.invocations)
+        logger.info(f"Agent '{agent_id}' handling ToolResultEvent for multi-tool call turn. "
+                    f"Collected {num_results}/{num_expected} results.")
+
+        # If not all results are in, just wait for the next ToolResultEvent.
+        if not active_turn.is_complete():
+            return
+            
+        # If all results are in, dispatch them to the LLM and clean up the turn state.
+        logger.info(f"Agent '{agent_id}': All tool results for the turn collected. Aggregating for LLM.")
+        await self._dispatch_aggregated_results_to_llm(active_turn.results, context)
+        
+        context.state.active_multi_tool_call_turn = None
+        logger.info(f"Agent '{agent_id}': Multi-tool call turn state has been cleared.")