agentic-testing 0.1.0__tar.gz

agentic_testing-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Anant Gupta
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agentic_testing-0.1.0/PKG-INFO

@@ -0,0 +1,144 @@
+Metadata-Version: 2.4
+Name: agentic-testing
+Version: 0.1.0
+Summary: Multi-agent orchestration and testing framework with LangGraph, CrewAI, and FastAPI.
+Author-email: Anant Gupta <anantguptadbl@gmail.com>, Ritesh Jena <riteshjena@gmail.com>
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastapi>=0.110.0
+Requires-Dist: uvicorn>=0.29.0
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: langchain>=0.1.0
+Requires-Dist: langgraph>=1.0.7
+Requires-Dist: crewai
+Requires-Dist: pytest
+Requires-Dist: pytest-asyncio
+Requires-Dist: langserve
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: httpx; extra == "test"
+Requires-Dist: pytest-asyncio; extra == "test"
+Provides-Extra: langgraph
+Requires-Dist: langgraph>=1.0.7; extra == "langgraph"
+Requires-Dist: langchain>=0.1.0; extra == "langgraph"
+Requires-Dist: fastapi>=0.110.0; extra == "langgraph"
+Requires-Dist: uvicorn>=0.29.0; extra == "langgraph"
+Requires-Dist: httpx>=0.27.0; extra == "langgraph"
+Provides-Extra: crewai
+Requires-Dist: crewai; extra == "crewai"
+Requires-Dist: fastapi>=0.110.0; extra == "crewai"
+Requires-Dist: uvicorn>=0.29.0; extra == "crewai"
+Requires-Dist: httpx>=0.27.0; extra == "crewai"
+Dynamic: license-file
+
+
+# Agent Testing Library
+
+This repository provides a comprehensive framework for building, testing, and orchestrating agentic workflows in Python. It includes utilities, fixtures, logging, and a rich set of examples demonstrating integration with various agentic frameworks and orchestration patterns.
+
+## Project Structure
+
+- **agent_test/**: Core testing library for agents
+	- **src/agent_utils/**: Utilities for agent orchestration and metadata
+		- `remoterunnable_utils.py`, `models/agent_info.py`, `models/api_mock_type.py`, `models/global_metadata.py`
+	- **common/**: Logging and shared utilities
+		- `agent_test_logger.py`
+	- **fixture/**: Test fixtures and API mocks
+		- `fixture_class.py`, `mock_api/` (various protocol mocks)
+	- **test/**: Unit tests for all modules
+		- `test_remoterunnable_utils.py`, `test_agent_info.py`, `test_api_mock_type.py`, `test_global_metadata.py`, `test_agent_test_logger.py`, `test_fixture_class.py`
+
+- **examples/**: Practical agentic workflow examples
+	- **common/**: Shared example code
+		- `api1/api_code.py`, `worker1/main.py`, `worker2/main.py`, `worker3/main.py`
+	- **crewai/**: CrewAI orchestration example
+		- `orchestrator/orchestrator_code.py`
+	- **langgraph/**: LangGraph agentic workflows
+		- `prompt_agentic/asynchronous/orchestrator_code.py`, `prompt_agentic/asynchronous/test_orchestrator.py`
+		- `prompt_agentic/synchronous/orchestrator_code.py`, `prompt_agentic/synchronous/test_orchestrator.py`
+		- `simple_graph/orchestrator_code.py`, `simple_graph/test_orchestrator.py`
+
+
+## Examples Section
+
+The `examples/` directory demonstrates how to build and orchestrate agentic workflows using different frameworks and patterns. Below are direct code snippets from key example files:
+
+### common/api1/api_code.py
+```python
+def api_call(input_data):
+	# Simulate API logic
+	return {"result": f"Processed {input_data}"}
+```
+
+### common/worker1/main.py
+```python
+from common.api1.api_code import api_call
+
+def worker1_task(data):
+	response = api_call(data)
+	print("Worker1 received:", response)
+```
+
+### crewai/orchestrator/orchestrator_code.py
+```python
+def orchestrate_agents(agent_list, task):
+	results = []
+	for agent in agent_list:
+		result = agent.run(task)
+		results.append(result)
+	return results
+```
+
+### langgraph/prompt_agentic/asynchronous/orchestrator_code.py
+```python
+import asyncio
+
+async def async_orchestrate(agents, task):
+	tasks = [agent.run_async(task) for agent in agents]
+	return await asyncio.gather(*tasks)
+```
+
+### langgraph/prompt_agentic/synchronous/orchestrator_code.py
+```python
+def sync_orchestrate(agents, task):
+	return [agent.run(task) for agent in agents]
+```
+
+### langgraph/simple_graph/orchestrator_code.py
+```python
+def simple_graph_orchestrate(nodes, input_data):
+	output = input_data
+	for node in nodes:
+		output = node.process(output)
+	return output
+```
+
+## Testing
+
+Unit tests are provided in the `agent_test/test/` directory. Run all tests with:
+
+```bash
+pytest agent_test/test
+```
+
+## License
+
+See LICENSE for details.
+
+## Future Work
+
+Agentic Framework Alternatives (besides langgraph):
+
+CrewAI (multi-agent orchestration)
+Autogen (Microsoft’s multi-agent framework)
+LangChain (chains, agents, tools)
+Haystack (for RAG and agent pipelines)
+OpenAI Function Calling (tool-using agents)
+Semantic Kernel (Microsoft’s orchestration)
+Custom agent frameworks (your own classes)
+HuggingFace Transformers Agents
+LlamaIndex (for agentic workflows)
+DSPy (for programmatic LLM pipelines)
+PromptChainer (open-source agentic framework)
+Direct function/callback-based agents

agentic_testing-0.1.0/README.md

@@ -0,0 +1,110 @@
+
+# Agent Testing Library
+
+This repository provides a comprehensive framework for building, testing, and orchestrating agentic workflows in Python. It includes utilities, fixtures, logging, and a rich set of examples demonstrating integration with various agentic frameworks and orchestration patterns.
+
+## Project Structure
+
+- **agent_test/**: Core testing library for agents
+	- **src/agent_utils/**: Utilities for agent orchestration and metadata
+		- `remoterunnable_utils.py`, `models/agent_info.py`, `models/api_mock_type.py`, `models/global_metadata.py`
+	- **common/**: Logging and shared utilities
+		- `agent_test_logger.py`
+	- **fixture/**: Test fixtures and API mocks
+		- `fixture_class.py`, `mock_api/` (various protocol mocks)
+	- **test/**: Unit tests for all modules
+		- `test_remoterunnable_utils.py`, `test_agent_info.py`, `test_api_mock_type.py`, `test_global_metadata.py`, `test_agent_test_logger.py`, `test_fixture_class.py`
+
+- **examples/**: Practical agentic workflow examples
+	- **common/**: Shared example code
+		- `api1/api_code.py`, `worker1/main.py`, `worker2/main.py`, `worker3/main.py`
+	- **crewai/**: CrewAI orchestration example
+		- `orchestrator/orchestrator_code.py`
+	- **langgraph/**: LangGraph agentic workflows
+		- `prompt_agentic/asynchronous/orchestrator_code.py`, `prompt_agentic/asynchronous/test_orchestrator.py`
+		- `prompt_agentic/synchronous/orchestrator_code.py`, `prompt_agentic/synchronous/test_orchestrator.py`
+		- `simple_graph/orchestrator_code.py`, `simple_graph/test_orchestrator.py`
+
+
+## Examples Section
+
+The `examples/` directory demonstrates how to build and orchestrate agentic workflows using different frameworks and patterns. Below are direct code snippets from key example files:
+
+### common/api1/api_code.py
+```python
+def api_call(input_data):
+	# Simulate API logic
+	return {"result": f"Processed {input_data}"}
+```
+
+### common/worker1/main.py
+```python
+from common.api1.api_code import api_call
+
+def worker1_task(data):
+	response = api_call(data)
+	print("Worker1 received:", response)
+```
+
+### crewai/orchestrator/orchestrator_code.py
+```python
+def orchestrate_agents(agent_list, task):
+	results = []
+	for agent in agent_list:
+		result = agent.run(task)
+		results.append(result)
+	return results
+```
+
+### langgraph/prompt_agentic/asynchronous/orchestrator_code.py
+```python
+import asyncio
+
+async def async_orchestrate(agents, task):
+	tasks = [agent.run_async(task) for agent in agents]
+	return await asyncio.gather(*tasks)
+```
+
+### langgraph/prompt_agentic/synchronous/orchestrator_code.py
+```python
+def sync_orchestrate(agents, task):
+	return [agent.run(task) for agent in agents]
+```
+
+### langgraph/simple_graph/orchestrator_code.py
+```python
+def simple_graph_orchestrate(nodes, input_data):
+	output = input_data
+	for node in nodes:
+		output = node.process(output)
+	return output
+```
+
+## Testing
+
+Unit tests are provided in the `agent_test/test/` directory. Run all tests with:
+
+```bash
+pytest agent_test/test
+```
+
+## License
+
+See LICENSE for details.
+
+## Future Work
+
+Agentic Framework Alternatives (besides langgraph):
+
+CrewAI (multi-agent orchestration)
+Autogen (Microsoft’s multi-agent framework)
+LangChain (chains, agents, tools)
+Haystack (for RAG and agent pipelines)
+OpenAI Function Calling (tool-using agents)
+Semantic Kernel (Microsoft’s orchestration)
+Custom agent frameworks (your own classes)
+HuggingFace Transformers Agents
+LlamaIndex (for agentic workflows)
+DSPy (for programmatic LLM pipelines)
+PromptChainer (open-source agentic framework)
+Direct function/callback-based agents

agentic_testing-0.1.0/agent_test/__init__.py

@@ -0,0 +1,3 @@
+# agent_test package for agentic flow testing utilities
+
+__version__ = "0.1.0"

agentic_testing-0.1.0/agent_test/src/agent_utils/models/agent_info.py

@@ -0,0 +1,12 @@
+from pydantic import BaseModel
+
+class AgentInfo(BaseModel):
+    agent_name: str
+    agent_path: str
+    agent_type: str
+    agent_module_path: str = None
+
+    @property
+    def module_path(self):
+        """Getter for agent_module_path."""
+        return self.agent_path

agentic_testing-0.1.0/agent_test/src/agent_utils/models/api_mock_type.py

@@ -0,0 +1,15 @@
+from enum import Enum, auto
+
+class APIMockType(Enum):
+    REQUESTS = auto()
+    HTTPX = auto()
+    URLLIB = auto()
+    AIOHTTP = auto()
+    GRPC = auto()
+    WEBSOCKETS = auto()
+    CUSTOM = auto()
+    SDK = auto()
+    DB = auto()
+    MQ = auto()
+    GRAPHQL = auto()
+    SOAP = auto()

agentic_testing-0.1.0/agent_test/src/agent_utils/models/global_metadata.py

@@ -0,0 +1,55 @@
+
+from abc import abstractmethod
+
+from agent_test.src.agent_utils.models.api_mock_type import APIMockType
+
+
+class GlobalMetadata:
+    _api_patcher_registry = {}
+
+    @classmethod
+    def identify_patcher_type(cls, api_type: APIMockType = APIMockType.REQUESTS):
+        """
+        Identify the patcher class based on the payload using the factory module.
+        """
+        if not cls._api_patcher_registry:
+            cls._api_patcher_registry = cls._build_api_patcher_registry()
+        # print(f"GlobalMetadata: identify_patcher_type called with api_type={api_type} and the registry={cls._api_patcher_registry} ")
+        return cls._api_patcher_registry.get(api_type, None)
+    
+    def _build_api_patcher_registry():
+        import importlib
+        import pkgutil
+        from agent_test.src.fixture.mock_api import base_mock
+        from agent_test.src.fixture.mock_api.base_mock import BaseAPIMock
+        import agent_test.src.fixture.mock_api
+
+        # Dynamically import all modules in the mock_api package
+        package = agent_test.src.fixture.mock_api
+        for _, module_name, is_pkg in pkgutil.iter_modules(package.__path__, package.__name__ + "."):
+            if not is_pkg and not module_name.endswith("base_mock"):
+                try:
+                    importlib.import_module(module_name)
+                except Exception as e:
+                    print(f"GlobalMetadata: Failed to import {module_name}: {e}")
+
+        # print("GlobalMetadata: Building API patcher registry...")
+        # print("GlobalMetadata: Found subclasses of BaseAPIMock:", BaseAPIMock.__subclasses__())
+        registry = {}
+        for cls in BaseAPIMock.__subclasses__():
+            # print(f"GlobalMetadata: Registering API patcher class {cls} ")
+            instance = cls()
+            api_type = instance.get_api_type()
+            # print(f"GlobalMetadata: Identified api_type={api_type} for class {cls} ")
+            if api_type:
+                registry[api_type] = cls
+        return registry
+
+
+    def get_api_patcher_class(api_type: str):
+        """
+        Factory function to select the correct API patcher class based on payload.
+        Default is RequestsAPIMock.
+        """
+        from agent_test.fixture.mock_api.base_mock import BaseAPIMock
+        return BaseAPIMock._api_patcher_registry.get(api_type)

agentic_testing-0.1.0/agent_test/src/agent_utils/remoterunnable_utils.py

@@ -0,0 +1,47 @@
+import inspect
+import pkgutil
+import importlib
+import inspect
+from langserve import RemoteRunnable
+from langchain_core.runnables import Runnable, RunnableLambda
+from agent_test.src.agent_utils.models.agent_info import AgentInfo
+
+# List of supported runnable types
+RUNNABLE_TYPES = [RemoteRunnable, Runnable, RunnableLambda]
+
+def find_all_remoterunnables(package_name):
+    """
+    Scans the given package and returns a dict of agent_name to RemoteRunnableInfo
+    for all RemoteRunnable objects found in the package and its submodules.
+    """
+    remoterunnables = {}
+    try:
+        package = importlib.import_module(package_name)
+    except Exception as e:
+        print(f"find_all_remoterunnables: Could not import package '{package_name}': {e}")
+        return remoterunnables
+    for _, modname, _ in pkgutil.walk_packages(package.__path__, package.__name__ + "."):
+        try:
+            module = importlib.import_module(modname)
+            for name, obj in inspect.getmembers(module):
+                if isinstance(obj, tuple(RUNNABLE_TYPES)):
+                    info = AgentInfo(
+                        agent_name=name,
+                        agent_path=f"{modname}.{name}",
+                        agent_type=type(obj).__name__,
+                        agent_module_path=modname
+                    )
+                    remoterunnables[name] = info
+        except Exception:
+            pass
+    print(f"find_all_remoterunnables: Found remoterunnables: {remoterunnables}")
+    return remoterunnables
+
+def find_async_nodes_in_graph(graph):
+    """Returns a list of node names that are async in the given LangGraph graph object."""
+    async_nodes = []
+    for node_name, node_func in getattr(graph, 'nodes', {}).items():
+        if inspect.iscoroutinefunction(node_func):
+            async_nodes.append(node_name)
+    print(f"find_async_nodes_in_graph: Found async nodes: {async_nodes}")
+    return async_nodes

agentic_testing-0.1.0/agent_test/src/common/agent_test_logger.py

@@ -0,0 +1,22 @@
+import logging
+
+class AgentTestLogger:
+    _logger = None
+
+    @staticmethod
+    def get_logger(name: str = "agent_test_logger"):
+        if AgentTestLogger._logger is None:
+            logger = logging.getLogger(name)
+            logger.setLevel(logging.DEBUG)
+            if not logger.handlers:
+                handler = logging.StreamHandler()
+                formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+                handler.setFormatter(formatter)
+                logger.addHandler(handler)
+            AgentTestLogger._logger = logger
+        return AgentTestLogger._logger
+    
+    @staticmethod
+    def set_logger(logger):
+        """Set a custom logger instance."""
+        AgentTestLogger._logger = logger

agentic_testing-0.1.0/agent_test/src/fixture/fixture_class.py

@@ -0,0 +1,229 @@
+# FixtureLibrary: Chainable test fixture builder for agent orchestration
+
+from agent_test.src.agent_utils.models.api_mock_type import APIMockType
+from agent_test.src.agent_utils.models.global_metadata import GlobalMetadata
+from agent_test.src.common.agent_test_logger import AgentTestLogger
+from unittest.mock import AsyncMock, Mock, patch
+from agent_test.src.agent_utils.remoterunnable_utils import find_all_remoterunnables
+from agent_test.src.agent_utils.models.agent_info import AgentInfo
+
+logger = AgentTestLogger.get_logger()
+
+class FixtureLibrary:
+
+    def __init__(self, root_path: str = None):
+        if root_path is None:
+            # Use current package path if available, else fallback to 'orchestrator'
+            root_path = __package__ if __package__ else "orchestrator"
+        self._input_state = None
+        self._api_mocks = []
+        self._agent_invocations = []
+        self._agent_responses = []
+        self._patchers = []
+        self.results = []
+        self._root_path = root_path
+        # Load all agent info dict at initialization
+        self.agent_info_dict = find_all_remoterunnables(self._root_path)
+        logger.debug(f"__init__: Initialized FixtureLibrary with members: "
+                 f"_input_state={self._input_state}, "
+                 f"_api_mocks={self._api_mocks}, "
+                 f"_agent_invocations={self._agent_invocations}, "
+                 f"_agent_responses={self._agent_responses}, "
+                 f"_patchers={self._patchers}, "
+                 f"agent_info_dict keys={list(self.agent_info_dict.keys())}")
+
+    def when_input_state(self, state):
+        logger.debug(f"when_input_state: called with state={state}")
+        self._input_state = state
+        logger.debug(f"when_input_state: _input_state set to {self._input_state}")
+        return self
+
+    def mock_api_call(self, api_path, payload, return_value, api_type:APIMockType=APIMockType.REQUESTS):
+        logger.debug(f"mock_api_call: called with api_path={api_path}, payload={payload}, return_value={return_value}")
+        patcher_class = GlobalMetadata.identify_patcher_type(api_type)
+        patcher = patcher_class().create_patcher(api_path, payload, return_value)
+        self._patchers.append((patcher, None, None))
+        self._api_mocks.append((api_path, payload, return_value))
+        logger.debug(f"mock_api_call: _patchers updated: {self._patchers}")
+        logger.debug(f"mock_api_call: _api_mocks updated: {self._api_mocks}")
+        return self
+
+    def expect_agent_invocation(self, agent_name, state, agent_type="invoke", ntimes=1):
+        logger.debug(
+            f"expect_agent_invocation: called with agent_name={agent_name}, state={state}"
+        )
+        if (
+            self.was_agent_method_called(
+                agent_name, agent_type, ntimes, input_args=state
+            )
+            == False
+        ):
+            raise AssertionError(
+                f"Expected agent '{agent_name}' to be invoked with state {state}, but it was not."
+            )
+        return self
+        # self._agent_invocations.append((agent_name, state))
+        # print(f"_agent_invocations updated: {self._agent_invocations}")
+
+    def mock_agent_response(self, agent_name, response_state):
+        logger.debug(
+            f"mock_agent_response: called with agent_name={agent_name}, response_state={response_state}"
+        )
+        # Use agent_info_dict to infer the patch path
+        agent_info = self._get_agent_info(agent_name)
+        module_path = agent_info.agent_path
+        for method in ["invoke", "ainvoke", "batch"]:
+            patcher = self._create_agent_patcher(module_path, method, response_state)
+            self._patchers.append((patcher, agent_name, method))
+        self._agent_responses.append((agent_name, response_state))
+        # logger.debug(f"mock_agent_response: _patchers updated: {self._patchers}")
+        # logger.debug(f"mock_agent_response: _agent_responses updated: {self._agent_responses}")
+        return self
+
+    def _get_agent_info(self, agent_name):
+        agent_info: AgentInfo = self.agent_info_dict.get(agent_name)
+        if agent_info is None:
+            raise ValueError(f"Agent '{agent_name}' not found in agent_info_dict.")
+        # logger.debug(f"_get_agent_info: Found agent_info: {agent_info}  for agent_name: {agent_name}")
+        return agent_info
+
+    def _create_agent_patcher(self, module_path, method, response_state):
+        patch_path = f"{module_path}.{method}"
+        if method in ["ainvoke"]:
+            # Patch async methods to return a coroutine resolved to response_state
+            async_mock = AsyncMock()
+            async_mock.return_value = response_state
+            return patch(patch_path, new=async_mock)
+        else:
+            return patch(patch_path, return_value=response_state)
+
+    def __enter__(self):
+        # logger.debug("__enter__: called. Starting all patchers.")
+        self._started_patches = self._start_all_patchers()
+        # logger.debug(f"__enter__: _started_patches: {self._started_patches}")
+        return self
+
+    def _start_all_patchers(self):
+        return [patcher.start() for patcher, _, _ in self._patchers]
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        # logger.debug("__exit__: called. Stopping all patchers.")
+        self._stop_all_patchers()
+        # logger.debug("__exit__: All patchers stopped.")
+
+    def _stop_all_patchers(self):
+        for patcher, _, _ in self._patchers:
+            patcher.stop()
+
+    def run(self, test_func):
+        # logger.debug(f"run: called with test_func={test_func}")
+        with self:
+            result = test_func(self._input_state)
+        # logger.debug(f"run: result: {result}")
+        return result
+
+    def invoke_function(self, func):
+        # logger.debug(f"invoke_function: called with func={func}")
+        # logger.debug(f"invoke_function: Using _input_state: {self._input_state}")
+        # logger.debug(f"invoke_function: Type of input_state: {type(self._input_state)}")
+        # logger.debug(f"invoke_function: Type of func: {type(func)}")
+        with self:
+            result = func(self._input_state)
+        # logger.debug(f"invoke_function: result: {result}")
+        self.results.append(result)
+        return self
+
+    def invoke_graph(self, graph):
+        # logger.debug(f"invoke_graph: called with graph={graph}")
+        # logger.debug(f"invoke_graph: Using _input_state: {self._input_state}")
+        # logger.debug(f"invoke_graph: Type of input_state: {type(self._input_state)}")
+        # logger.debug(f"invoke_graph: Type of graph: {type(graph)}")
+        with self:
+            result = graph.invoke(self._input_state)
+        # logger.debug(f"invoke_graph: result: {result}")
+        self.results.append(result)
+        return self
+
+    async def ainvoke_graph(self, graph):
+        # logger.debug(f"ainvoke_graph: called with graph={graph}")
+        # logger.debug(f"ainvoke_graph: Using _input_state: {self._input_state}")
+        # logger.debug(f"ainvoke_graph: Type of input_state: {type(self._input_state)}")
+        # logger.debug(f"ainvoke_graph: Type of graph: {type(graph)}")
+        with self:
+            result = await graph.ainvoke(self._input_state)
+        # logger.debug(f"ainvoke_graph: result: {result}")
+        self.results.append(result)
+        return self
+
+    def cleanup(self):
+        self._stop_all_patchers()
+
+    def _get_patch_index_by_agent(self, agent_name, method=None):
+        """
+        Returns the index of the patch for a given agent and method (e.g., 'invoke', 'ainvoke', 'batch').
+        If method is None, returns all indices for the agent.
+        """
+        indices = []
+        for i, patch_info in enumerate(self._patchers):
+            # patch_info is (patcher, agent_name, method) for agent patches, or just patcher for others
+            if isinstance(patch_info, tuple) and len(patch_info) == 3:
+                _, patch_agent_name, patch_method = patch_info
+                if patch_agent_name == agent_name:
+                    if method is None or patch_method == method:
+                        indices.append(i)
+        return indices if method is None else (indices[0] if indices else None)
+
+    def _was_patch_called(self, patch_index):
+        """
+        Returns True if the patch at patch_index was called during the test run.
+        """
+        if not hasattr(self, '_started_patches'):
+            raise RuntimeError("Patches have not been started. Use within a context manager.")
+        mock_obj = self._started_patches[patch_index]
+        return getattr(mock_obj, 'called', False)
+
+    def was_agent_method_called(self, agent_name, method, expected_count=1, input_args=None):
+        """
+        Asserts the patch for the given agent and method was called expected_count times.
+        If input_args is provided, only counts calls with matching args.
+        Returns True if assertion passes, else raises AssertionError.
+        """
+        # logger.debug(f"was_agent_method_called: called with agent_name={agent_name}, method={method}, expected_count={expected_count}, input_args={input_args}")
+        # logger.debug(f"was_agent_method_called: The self._started_patches are: {self._started_patches}")
+        idx = self._get_patch_index_by_agent(agent_name, method)
+        if idx is None:
+            raise ValueError(f"No patch found for agent '{agent_name}' and method '{method}'")
+        if not hasattr(self, '_started_patches'):
+            raise RuntimeError("Patches have not been started. Use within a context manager.")
+        mock_obj = self._started_patches[idx]
+        # logger.debug(f"was_agent_method_called: The mock_obj for agent '{agent_name}' and method '{method}' is: {mock_obj}")
+        # If input_args is None, count all calls
+        if input_args is None:
+            call_count = mock_obj.call_count
+        else:
+            # Filter calls by input_args
+            # logger.debug(f"was_agent_method_called: Counting calls with specific input_args: {input_args} and agent_name: {agent_name}, method: {method}")
+            call_count = 0
+            for call in getattr(mock_obj, 'call_args_list', []):
+                args, kwargs = call
+                # logger.debug(f"was_agent_method_called: The current call has args: {args}, kwargs: {kwargs}")
+                print(f"[DEBUG] Checking call args for agent '{agent_name}' method '{method}': {args} against input_args: {input_args}")
+                if(args and len(args) == 1 and args[0] == input_args):
+                    call_count += 1
+        assert call_count == expected_count, (
+            f"Expected {expected_count} calls to agent '{agent_name}' method '{method}', but got {call_count} (input_args={input_args})"
+        )
+        return True
+
+    def was_api_patch_called(self, api_path):
+        """
+        Returns True if the patch for the given API path was called.
+        """
+        for i, patcher in enumerate(self._patchers):
+            target = getattr(patcher, 'attribute', None) or getattr(patcher, 'target', None)
+            if target == api_path:
+                return self._was_patch_called(i)
+        raise ValueError(f"No patch found for API path '{api_path}'")
+
+
+    # Optionally, add more helpers for assertions or reporting as needed