agentverify 0.1.0__tar.gz → 0.2.0__tar.gz

{agentverify-0.1.0 → agentverify-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentverify
-Version: 0.1.0
+Version: 0.2.0
 Summary: Deterministic testing for AI agents. Assert tool calls, not vibes.
 Author: simukappu
 License: MIT
@@ -18,6 +18,7 @@ Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Software Development :: Testing
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
@@ -110,7 +111,7 @@ def test_output():
     assert_final_output(result, contains="Tokyo")
 ```
 
-> **Using a real agent framework?** You don't need to build dicts by hand — write a small converter function instead. See [Building from Your Framework](#building-from-your-framework) below.
+> **Using a real agent framework?** You don't need to build dicts by hand — use a built-in adapter instead. See [Framework Integration](#framework-integration) below.
 
 ```
 $ pytest test_agent.py -v
@@ -120,11 +121,95 @@ test_agent.py::test_safety PASSED
 test_agent.py::test_output PASSED
 ```
 
+## Real-World Example — Testing the Strands Weather Forecaster
+
+Here's how you'd test the [Strands Weather Forecaster](https://strandsagents.com/docs/examples/python/weather_forecaster/) — an official Strands Agents sample that uses the `http_request` tool to fetch weather data from the National Weather Service API.
+
+```python
+import pytest
+from agentverify import (
+    assert_tool_calls, assert_cost, assert_no_tool_call,
+    assert_final_output, assert_all, ToolCall, ANY, OrderMode,
+)
+
+# Import the weather agent from the Strands sample
+# (see https://strandsagents.com/docs/examples/python/weather_forecaster/)
+from weather_agent import weather_agent
+
+
+@pytest.mark.agentverify
+def test_weather_agent_tool_sequence(cassette):
+    """The agent should call http_request twice: location lookup, then forecast."""
+    with cassette("weather_seattle.yaml", provider="bedrock") as rec:
+        weather_agent("What's the weather in Seattle?")
+
+    result = rec.to_execution_result()
+    assert_tool_calls(
+        result,
+        expected=[
+            ToolCall("http_request", {"method": "GET", "url": ANY}),
+            ToolCall("http_request", {"method": "GET", "url": ANY}),
+        ],
+        order=OrderMode.IN_ORDER,
+        partial_args=True,
+    )
+
+
+@pytest.mark.agentverify
+def test_weather_agent_safety(cassette):
+    """The agent should only read data — no writes, no dangerous tools."""
+    with cassette("weather_seattle.yaml", provider="bedrock") as rec:
+        weather_agent("What's the weather in Seattle?")
+
+    result = rec.to_execution_result()
+    assert_no_tool_call(
+        result,
+        forbidden_tools=["write_file", "execute_command", "delete_file"],
+    )
+
+
+@pytest.mark.agentverify
+def test_weather_agent_all(cassette):
+    """Run all assertions at once — collect all failures."""
+    with cassette("weather_seattle.yaml", provider="bedrock") as rec:
+        weather_agent("What's the weather in Seattle?")
+
+    result = rec.to_execution_result()
+    assert_all(
+        result,
+        lambda r: assert_tool_calls(r, expected=[
+            ToolCall("http_request", {"method": "GET", "url": ANY}),
+        ], order=OrderMode.IN_ORDER, partial_args=True),
+        lambda r: assert_cost(r, max_tokens=15000),
+        lambda r: assert_no_tool_call(r, forbidden_tools=["write_file"]),
+        lambda r: assert_final_output(r, contains="Seattle"),
+    )
+```
+
+Or skip cassettes entirely and use the built-in adapter with a live agent:
+
+```python
+from agentverify.frameworks.strands import from_strands
+def test_weather_agent_live():
+    """Test with a live LLM call (requires API credentials)."""
+    result = weather_agent("What's the weather in Seattle?")
+    er = from_strands(result)
+
+    assert_tool_calls(er, expected=[
+        ToolCall("http_request", {"method": "GET", "url": ANY}),
+        ToolCall("http_request", {"method": "GET", "url": ANY}),
+    ], order=OrderMode.IN_ORDER, partial_args=True)
+
+    assert_cost(er, max_tokens=15000)
+    assert_no_tool_call(er, forbidden_tools=["write_file", "execute_command"])
+    assert_final_output(er, contains="Seattle")
+```
+
 ## 3 Steps to Test Your Agent
 
 ### Step 1: Build an ExecutionResult
 
-Build an `ExecutionResult` from your agent's output. You can construct one from a dict, or use a framework-specific converter (see [examples/](examples/) for Strands Agents and LangChain converters).
+Build an `ExecutionResult` from your agent's output. You can construct one from a dict, use a [built-in adapter](#framework-integration), or write a small converter for your framework.
 
 ```python
 from agentverify import ExecutionResult
@@ -153,7 +238,16 @@ You can also use `ExecutionResult.from_json(json_string)` to parse from a JSON s
 
 #### Building from Your Framework
 
-Every agent framework has its own output format. To use agentverify, you need a small converter function (~20–50 lines) that maps your framework's output to the dict schema above. Here's the general pattern:
+For frameworks with built-in adapter support (Strands Agents, LangChain, LangGraph, OpenAI Agents SDK), use the adapter directly:
+
+```python
+from agentverify.frameworks.strands import from_strands
+
+result = agent("Analyze the files")
+execution_result = from_strands(result)
+```
+
+For other frameworks, write a small converter function (~20–50 lines) that maps your framework's output to the dict schema above. See [`examples/strands-file-organizer/converter.py`](examples/strands-file-organizer/converter.py) and [`examples/langchain-issue-triage/converter.py`](examples/langchain-issue-triage/converter.py) for reference implementations. Here's the general pattern:
 
 ```python
 from agentverify import ExecutionResult, ToolCall, TokenUsage
@@ -180,8 +274,6 @@ def my_framework_to_execution_result(agent_output) -> ExecutionResult:
     )
 ```
 
-See [`examples/strands-file-organizer/converter.py`](examples/strands-file-organizer/converter.py) and [`examples/langchain-issue-triage/converter.py`](examples/langchain-issue-triage/converter.py) for complete, production-ready converters. Built-in framework adapters are planned for a future release (see [Roadmap](#roadmap)).
-
 ### Step 2: Assert
 
 ```python
@@ -208,7 +300,7 @@ assert_final_output(result, contains="Tokyo")
 
 Record real LLM API calls once. Replay them in CI forever — zero cost, deterministic.
 
-**⚠️ Important: Cassette replay uses sequential matching** — responses are returned in recorded order without verifying request content. If your agent's prompts, tools, or model change, the cassette will still replay but may return stale/incorrect responses silently. **Delete and re-record cassettes after significant agent changes.** Request content matching is planned for a future release (see [Roadmap](#roadmap)).
+Cassette replay verifies request content by default — if your model or tools change, replay raises `CassetteRequestMismatchError` instead of silently returning stale responses. Re-record cassettes with `--cassette-mode=record` after significant agent changes. See [Request Matching](#request-matching-stale-cassette-detection) for details.
 
 The `cassette` fixture is a pytest fixture provided by the agentverify plugin. It creates an `LLMCassetteRecorder` that intercepts LLM SDK calls (not HTTP — it patches the SDK's chat completion method directly). Use `@pytest.mark.agentverify` to mark your test, and call your agent code inside the `with cassette(...)` block. After the block exits, call `rec.to_execution_result()` to build the result for assertions.
 
@@ -261,10 +353,61 @@ pytest --cassette-mode=record
 pytest
 ```
 
+### Request Matching (Stale Cassette Detection)
+
+Cassette replay verifies request content by default. Each replay request is compared against the recorded request:
+- **Model name**: must match exactly (empty model in either side skips the check)
+- **Tool names**: the sorted list of tool names must match (empty tools in either side skips the check)
+
+A `CassetteRequestMismatchError` is raised on mismatch with a clear message indicating which field differs and suggesting re-recording.
+
+To disable matching (e.g., during migration):
+
+```bash
+# CLI: disable for all tests
+pytest --no-cassette-match-requests
+```
+
+```python
+# Per-test: disable in the cassette factory call
+with cassette("my_test.yaml", provider="openai", match_requests=False) as rec:
+    run_my_agent("What's the weather?")
+```
+
+### Cassette Sanitization
+
+Cassette files are sanitized by default when recording. API keys, tokens, and other sensitive data are automatically redacted before saving to disk.
+
+Built-in patterns cover:
+- OpenAI API keys (`sk-...`)
+- Anthropic API keys (`sk-ant-...`)
+- AWS access keys (`AKIA...`)
+- Bearer tokens (`Bearer ...`)
+
+Add custom patterns:
+
+```python
+from agentverify import SanitizePattern
+
+custom_patterns = [
+    SanitizePattern(name="internal_token", pattern=r"tok-[a-f0-9]{32}", replacement="tok-***REDACTED***"),
+]
+
+with cassette("test.yaml", provider="openai", sanitize=custom_patterns) as rec:
+    run_my_agent("Do something")
+```
+
+Disable sanitization (not recommended):
+
+```python
+with cassette("test.yaml", provider="openai", sanitize=False) as rec:
+    run_my_agent("Do something")
+```
+
 **Other limitations:**
 
 - `total_cost_usd` is not populated from cassettes. Use `assert_cost(max_tokens=...)` for cassette-based budget checks, or set `total_cost_usd` manually in your `ExecutionResult`.
-- Be mindful not to include sensitive data (API keys, PII, confidential prompts) in cassette files checked into version control.
+- Cassette sanitization covers common API key patterns by default. Add custom `SanitizePattern` objects for application-specific secrets (internal tokens, database credentials, etc.).
 
 ## Assertion Modes
 
@@ -327,14 +470,59 @@ assert_final_output(result, matches=r"\d+°C")
 
 ## Framework Integration
 
-agentverify is framework-agnostic. Build an `ExecutionResult` from any agent framework's output using a converter function. The [`examples/`](examples/) directory includes ready-to-use converters:
+### Built-in Adapters
+
+agentverify ships with built-in adapters for popular agent frameworks. No converter function needed — just import and call:
+
+```python
+# Strands Agents
+from agentverify.frameworks.strands import from_strands
+
+result = agent("Analyze the files")
+execution_result = from_strands(result)
+
+# LangChain
+from agentverify.frameworks.langchain import from_langchain
+
+result = agent_executor.invoke({"input": "Triage the issues"})
+execution_result = from_langchain(result)
+
+# LangGraph
+from agentverify.frameworks.langgraph import from_langgraph
+
+result = agent.invoke({"messages": [{"role": "user", "content": "Hello"}]})
+execution_result = from_langgraph(result)
+
+# OpenAI Agents SDK
+from agentverify.frameworks.openai_agents import from_openai_agents
+
+result = await Runner.run(agent, "What's the weather?")
+execution_result = from_openai_agents(result)
+```
+
+| Framework | Adapter | Input |
+|---|---|---|
+| [Strands Agents](https://github.com/strands-agents/sdk-python) | `from agentverify.frameworks.strands import from_strands` | `AgentResult` |
+| [LangChain](https://github.com/langchain-ai/langchain) | `from agentverify.frameworks.langchain import from_langchain` | `AgentExecutor.invoke()` dict |
+| [LangGraph](https://github.com/langchain-ai/langgraph) | `from agentverify.frameworks.langgraph import from_langgraph` | `create_react_agent` result dict |
+| [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | `from agentverify.frameworks.openai_agents import from_openai_agents` | `RunResult` |
+
+For LangChain, pass the conversation history `messages` list as a second argument to capture token usage:
+
+```python
+execution_result = from_langchain(result, messages=memory.chat_memory.messages)
+```
+
+### Custom Converters
+
+For other frameworks, agentverify is framework-agnostic. Build an `ExecutionResult` from any agent framework's output using a converter function. The [`examples/`](examples/) directory includes reference converters:
 
 | Framework | Converter | Description |
 |---|---|---|
 | [Strands Agents](https://github.com/strands-agents/sdk-python) | [`strands-file-organizer/converter.py`](examples/strands-file-organizer/converter.py) | Converts `AgentResult` → `ExecutionResult` |
 | [LangChain](https://github.com/langchain-ai/langchain) | [`langchain-issue-triage/converter.py`](examples/langchain-issue-triage/converter.py) | Converts `AgentExecutor` output → `ExecutionResult` |
 
-These converters are small (~50 lines) and easy to adapt for your own framework. Built-in framework adapters are planned for a future release (see [Roadmap](#roadmap)).
+These converters are small (~50 lines) and easy to adapt for your own framework.
 
 ## Supported LLM Providers
 
@@ -420,6 +608,7 @@ The [`examples/`](examples/) directory contains end-to-end examples with real ag
 
 | Example | Framework | Description |
 |---|---|---|
+| [`strands-weather-forecaster`](examples/strands-weather-forecaster/) | Strands Agents + Bedrock | Fetches weather via HTTP, verifies tool sequence and safety. Matches the [Real-World Example](#real-world-example--testing-the-strands-weather-forecaster) in this README |
 | [`strands-file-organizer`](examples/strands-file-organizer/) | Strands Agents + Bedrock | Scans a directory via Filesystem MCP, suggests organization. Read-only safety verified |
 | [`langchain-issue-triage`](examples/langchain-issue-triage/) | LangChain + OpenAI | Triages GitHub issues via GitHub MCP. Label and priority suggestions |
 | [`mcp-server`](examples/mcp-server/) | — | Mock GitHub MCP server for token-free testing |
@@ -458,11 +647,13 @@ See each example's README for agent execution instructions and recording mode de
 
 ## Roadmap
 
-- Agent framework adapters — extract `ExecutionResult` directly from Strands Agents, LangChain, and others without writing a converter
+- ~~Agent framework adapters — extract `ExecutionResult` directly from Strands Agents, LangChain, and others without writing a converter~~ ✅ Shipped
+- Framework adapters for Google ADK and CrewAI — pending async support and stable tool-call APIs from these frameworks
+- Responses API cassette adapter — record/replay for OpenAI Agents SDK (Responses API) with end-to-end example
 - Tool mocking/stubbing — test agent routing logic without calling real tools
 - Async support — first-class `asyncio` testing for async agents and tools
-- Cassette request matching — verify request content during replay to detect stale cassettes
-- Cassette sanitization — automatic masking of API keys and sensitive data in recorded cassettes
+- ~~Cassette request matching — verify request content during replay to detect stale cassettes~~ ✅ Shipped
+- ~~Cassette sanitization — automatic masking of API keys and sensitive data in recorded cassettes~~ ✅ Shipped
 - Cost estimation from tokens — auto-calculate `total_cost_usd` from token usage and model pricing
 - YAML/JSON test case definitions — declarative test cases for non-Python CI pipelines
 - CLI test runner — run agent tests without pytest

{agentverify-0.1.0 → agentverify-0.2.0}/README.md

@@ -63,7 +63,7 @@ def test_output():
     assert_final_output(result, contains="Tokyo")
 ```
 
-> **Using a real agent framework?** You don't need to build dicts by hand — write a small converter function instead. See [Building from Your Framework](#building-from-your-framework) below.
+> **Using a real agent framework?** You don't need to build dicts by hand — use a built-in adapter instead. See [Framework Integration](#framework-integration) below.
 
 ```
 $ pytest test_agent.py -v
@@ -73,11 +73,95 @@ test_agent.py::test_safety PASSED
 test_agent.py::test_output PASSED
 ```
 
+## Real-World Example — Testing the Strands Weather Forecaster
+
+Here's how you'd test the [Strands Weather Forecaster](https://strandsagents.com/docs/examples/python/weather_forecaster/) — an official Strands Agents sample that uses the `http_request` tool to fetch weather data from the National Weather Service API.
+
+```python
+import pytest
+from agentverify import (
+    assert_tool_calls, assert_cost, assert_no_tool_call,
+    assert_final_output, assert_all, ToolCall, ANY, OrderMode,
+)
+
+# Import the weather agent from the Strands sample
+# (see https://strandsagents.com/docs/examples/python/weather_forecaster/)
+from weather_agent import weather_agent
+
+
+@pytest.mark.agentverify
+def test_weather_agent_tool_sequence(cassette):
+    """The agent should call http_request twice: location lookup, then forecast."""
+    with cassette("weather_seattle.yaml", provider="bedrock") as rec:
+        weather_agent("What's the weather in Seattle?")
+
+    result = rec.to_execution_result()
+    assert_tool_calls(
+        result,
+        expected=[
+            ToolCall("http_request", {"method": "GET", "url": ANY}),
+            ToolCall("http_request", {"method": "GET", "url": ANY}),
+        ],
+        order=OrderMode.IN_ORDER,
+        partial_args=True,
+    )
+
+
+@pytest.mark.agentverify
+def test_weather_agent_safety(cassette):
+    """The agent should only read data — no writes, no dangerous tools."""
+    with cassette("weather_seattle.yaml", provider="bedrock") as rec:
+        weather_agent("What's the weather in Seattle?")
+
+    result = rec.to_execution_result()
+    assert_no_tool_call(
+        result,
+        forbidden_tools=["write_file", "execute_command", "delete_file"],
+    )
+
+
+@pytest.mark.agentverify
+def test_weather_agent_all(cassette):
+    """Run all assertions at once — collect all failures."""
+    with cassette("weather_seattle.yaml", provider="bedrock") as rec:
+        weather_agent("What's the weather in Seattle?")
+
+    result = rec.to_execution_result()
+    assert_all(
+        result,
+        lambda r: assert_tool_calls(r, expected=[
+            ToolCall("http_request", {"method": "GET", "url": ANY}),
+        ], order=OrderMode.IN_ORDER, partial_args=True),
+        lambda r: assert_cost(r, max_tokens=15000),
+        lambda r: assert_no_tool_call(r, forbidden_tools=["write_file"]),
+        lambda r: assert_final_output(r, contains="Seattle"),
+    )
+```
+
+Or skip cassettes entirely and use the built-in adapter with a live agent:
+
+```python
+from agentverify.frameworks.strands import from_strands
+def test_weather_agent_live():
+    """Test with a live LLM call (requires API credentials)."""
+    result = weather_agent("What's the weather in Seattle?")
+    er = from_strands(result)
+
+    assert_tool_calls(er, expected=[
+        ToolCall("http_request", {"method": "GET", "url": ANY}),
+        ToolCall("http_request", {"method": "GET", "url": ANY}),
+    ], order=OrderMode.IN_ORDER, partial_args=True)
+
+    assert_cost(er, max_tokens=15000)
+    assert_no_tool_call(er, forbidden_tools=["write_file", "execute_command"])
+    assert_final_output(er, contains="Seattle")
+```
+
 ## 3 Steps to Test Your Agent
 
 ### Step 1: Build an ExecutionResult
 
-Build an `ExecutionResult` from your agent's output. You can construct one from a dict, or use a framework-specific converter (see [examples/](examples/) for Strands Agents and LangChain converters).
+Build an `ExecutionResult` from your agent's output. You can construct one from a dict, use a [built-in adapter](#framework-integration), or write a small converter for your framework.
 
 ```python
 from agentverify import ExecutionResult
@@ -106,7 +190,16 @@ You can also use `ExecutionResult.from_json(json_string)` to parse from a JSON s
 
 #### Building from Your Framework
 
-Every agent framework has its own output format. To use agentverify, you need a small converter function (~20–50 lines) that maps your framework's output to the dict schema above. Here's the general pattern:
+For frameworks with built-in adapter support (Strands Agents, LangChain, LangGraph, OpenAI Agents SDK), use the adapter directly:
+
+```python
+from agentverify.frameworks.strands import from_strands
+
+result = agent("Analyze the files")
+execution_result = from_strands(result)
+```
+
+For other frameworks, write a small converter function (~20–50 lines) that maps your framework's output to the dict schema above. See [`examples/strands-file-organizer/converter.py`](examples/strands-file-organizer/converter.py) and [`examples/langchain-issue-triage/converter.py`](examples/langchain-issue-triage/converter.py) for reference implementations. Here's the general pattern:
 
 ```python
 from agentverify import ExecutionResult, ToolCall, TokenUsage
@@ -133,8 +226,6 @@ def my_framework_to_execution_result(agent_output) -> ExecutionResult:
     )
 ```
 
-See [`examples/strands-file-organizer/converter.py`](examples/strands-file-organizer/converter.py) and [`examples/langchain-issue-triage/converter.py`](examples/langchain-issue-triage/converter.py) for complete, production-ready converters. Built-in framework adapters are planned for a future release (see [Roadmap](#roadmap)).
-
 ### Step 2: Assert
 
 ```python
@@ -161,7 +252,7 @@ assert_final_output(result, contains="Tokyo")
 
 Record real LLM API calls once. Replay them in CI forever — zero cost, deterministic.
 
-**⚠️ Important: Cassette replay uses sequential matching** — responses are returned in recorded order without verifying request content. If your agent's prompts, tools, or model change, the cassette will still replay but may return stale/incorrect responses silently. **Delete and re-record cassettes after significant agent changes.** Request content matching is planned for a future release (see [Roadmap](#roadmap)).
+Cassette replay verifies request content by default — if your model or tools change, replay raises `CassetteRequestMismatchError` instead of silently returning stale responses. Re-record cassettes with `--cassette-mode=record` after significant agent changes. See [Request Matching](#request-matching-stale-cassette-detection) for details.
 
 The `cassette` fixture is a pytest fixture provided by the agentverify plugin. It creates an `LLMCassetteRecorder` that intercepts LLM SDK calls (not HTTP — it patches the SDK's chat completion method directly). Use `@pytest.mark.agentverify` to mark your test, and call your agent code inside the `with cassette(...)` block. After the block exits, call `rec.to_execution_result()` to build the result for assertions.
 
@@ -214,10 +305,61 @@ pytest --cassette-mode=record
 pytest
 ```
 
+### Request Matching (Stale Cassette Detection)
+
+Cassette replay verifies request content by default. Each replay request is compared against the recorded request:
+- **Model name**: must match exactly (empty model in either side skips the check)
+- **Tool names**: the sorted list of tool names must match (empty tools in either side skips the check)
+
+A `CassetteRequestMismatchError` is raised on mismatch with a clear message indicating which field differs and suggesting re-recording.
+
+To disable matching (e.g., during migration):
+
+```bash
+# CLI: disable for all tests
+pytest --no-cassette-match-requests
+```
+
+```python
+# Per-test: disable in the cassette factory call
+with cassette("my_test.yaml", provider="openai", match_requests=False) as rec:
+    run_my_agent("What's the weather?")
+```
+
+### Cassette Sanitization
+
+Cassette files are sanitized by default when recording. API keys, tokens, and other sensitive data are automatically redacted before saving to disk.
+
+Built-in patterns cover:
+- OpenAI API keys (`sk-...`)
+- Anthropic API keys (`sk-ant-...`)
+- AWS access keys (`AKIA...`)
+- Bearer tokens (`Bearer ...`)
+
+Add custom patterns:
+
+```python
+from agentverify import SanitizePattern
+
+custom_patterns = [
+    SanitizePattern(name="internal_token", pattern=r"tok-[a-f0-9]{32}", replacement="tok-***REDACTED***"),
+]
+
+with cassette("test.yaml", provider="openai", sanitize=custom_patterns) as rec:
+    run_my_agent("Do something")
+```
+
+Disable sanitization (not recommended):
+
+```python
+with cassette("test.yaml", provider="openai", sanitize=False) as rec:
+    run_my_agent("Do something")
+```
+
 **Other limitations:**
 
 - `total_cost_usd` is not populated from cassettes. Use `assert_cost(max_tokens=...)` for cassette-based budget checks, or set `total_cost_usd` manually in your `ExecutionResult`.
-- Be mindful not to include sensitive data (API keys, PII, confidential prompts) in cassette files checked into version control.
+- Cassette sanitization covers common API key patterns by default. Add custom `SanitizePattern` objects for application-specific secrets (internal tokens, database credentials, etc.).
 
 ## Assertion Modes
 
@@ -280,14 +422,59 @@ assert_final_output(result, matches=r"\d+°C")
 
 ## Framework Integration
 
-agentverify is framework-agnostic. Build an `ExecutionResult` from any agent framework's output using a converter function. The [`examples/`](examples/) directory includes ready-to-use converters:
+### Built-in Adapters
+
+agentverify ships with built-in adapters for popular agent frameworks. No converter function needed — just import and call:
+
+```python
+# Strands Agents
+from agentverify.frameworks.strands import from_strands
+
+result = agent("Analyze the files")
+execution_result = from_strands(result)
+
+# LangChain
+from agentverify.frameworks.langchain import from_langchain
+
+result = agent_executor.invoke({"input": "Triage the issues"})
+execution_result = from_langchain(result)
+
+# LangGraph
+from agentverify.frameworks.langgraph import from_langgraph
+
+result = agent.invoke({"messages": [{"role": "user", "content": "Hello"}]})
+execution_result = from_langgraph(result)
+
+# OpenAI Agents SDK
+from agentverify.frameworks.openai_agents import from_openai_agents
+
+result = await Runner.run(agent, "What's the weather?")
+execution_result = from_openai_agents(result)
+```
+
+| Framework | Adapter | Input |
+|---|---|---|
+| [Strands Agents](https://github.com/strands-agents/sdk-python) | `from agentverify.frameworks.strands import from_strands` | `AgentResult` |
+| [LangChain](https://github.com/langchain-ai/langchain) | `from agentverify.frameworks.langchain import from_langchain` | `AgentExecutor.invoke()` dict |
+| [LangGraph](https://github.com/langchain-ai/langgraph) | `from agentverify.frameworks.langgraph import from_langgraph` | `create_react_agent` result dict |
+| [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | `from agentverify.frameworks.openai_agents import from_openai_agents` | `RunResult` |
+
+For LangChain, pass the conversation history `messages` list as a second argument to capture token usage:
+
+```python
+execution_result = from_langchain(result, messages=memory.chat_memory.messages)
+```
+
+### Custom Converters
+
+For other frameworks, agentverify is framework-agnostic. Build an `ExecutionResult` from any agent framework's output using a converter function. The [`examples/`](examples/) directory includes reference converters:
 
 | Framework | Converter | Description |
 |---|---|---|
 | [Strands Agents](https://github.com/strands-agents/sdk-python) | [`strands-file-organizer/converter.py`](examples/strands-file-organizer/converter.py) | Converts `AgentResult` → `ExecutionResult` |
 | [LangChain](https://github.com/langchain-ai/langchain) | [`langchain-issue-triage/converter.py`](examples/langchain-issue-triage/converter.py) | Converts `AgentExecutor` output → `ExecutionResult` |
 
-These converters are small (~50 lines) and easy to adapt for your own framework. Built-in framework adapters are planned for a future release (see [Roadmap](#roadmap)).
+These converters are small (~50 lines) and easy to adapt for your own framework.
 
 ## Supported LLM Providers
 
@@ -373,6 +560,7 @@ The [`examples/`](examples/) directory contains end-to-end examples with real ag
 
 | Example | Framework | Description |
 |---|---|---|
+| [`strands-weather-forecaster`](examples/strands-weather-forecaster/) | Strands Agents + Bedrock | Fetches weather via HTTP, verifies tool sequence and safety. Matches the [Real-World Example](#real-world-example--testing-the-strands-weather-forecaster) in this README |
 | [`strands-file-organizer`](examples/strands-file-organizer/) | Strands Agents + Bedrock | Scans a directory via Filesystem MCP, suggests organization. Read-only safety verified |
 | [`langchain-issue-triage`](examples/langchain-issue-triage/) | LangChain + OpenAI | Triages GitHub issues via GitHub MCP. Label and priority suggestions |
 | [`mcp-server`](examples/mcp-server/) | — | Mock GitHub MCP server for token-free testing |
@@ -411,11 +599,13 @@ See each example's README for agent execution instructions and recording mode de
 
 ## Roadmap
 
-- Agent framework adapters — extract `ExecutionResult` directly from Strands Agents, LangChain, and others without writing a converter
+- ~~Agent framework adapters — extract `ExecutionResult` directly from Strands Agents, LangChain, and others without writing a converter~~ ✅ Shipped
+- Framework adapters for Google ADK and CrewAI — pending async support and stable tool-call APIs from these frameworks
+- Responses API cassette adapter — record/replay for OpenAI Agents SDK (Responses API) with end-to-end example
 - Tool mocking/stubbing — test agent routing logic without calling real tools
 - Async support — first-class `asyncio` testing for async agents and tools
-- Cassette request matching — verify request content during replay to detect stale cassettes
-- Cassette sanitization — automatic masking of API keys and sensitive data in recorded cassettes
+- ~~Cassette request matching — verify request content during replay to detect stale cassettes~~ ✅ Shipped
+- ~~Cassette sanitization — automatic masking of API keys and sensitive data in recorded cassettes~~ ✅ Shipped
 - Cost estimation from tokens — auto-calculate `total_cost_usd` from token usage and model pricing
 - YAML/JSON test case definitions — declarative test cases for non-Python CI pipelines
 - CLI test runner — run agent tests without pytest

{agentverify-0.1.0 → agentverify-0.2.0}/agentverify/__init__.py

@@ -9,6 +9,13 @@ Public API re-exports for convenient access::
         assert_final_output, assert_all,
         LLMCassetteRecorder, CassetteMode,
     )
+
+Framework adapters (no converter needed)::
+
+    from agentverify.frameworks.strands import from_strands
+    from agentverify.frameworks.langchain import from_langchain
+    from agentverify.frameworks.langgraph import from_langgraph
+    from agentverify.frameworks.openai_agents import from_openai_agents
 """
 
 from agentverify.assertions import (
@@ -19,6 +26,8 @@ from agentverify.assertions import (
     assert_tool_calls,
 )
 from agentverify.cassette.recorder import CassetteMode, LLMCassetteRecorder
+from agentverify.cassette.sanitize import DEFAULT_PATTERNS, SanitizePattern
+from agentverify.errors import CassetteRequestMismatchError
 from agentverify.matchers import ANY, OrderMode
 from agentverify.models import ExecutionResult, TokenUsage, ToolCall
 
@@ -39,4 +48,7 @@ __all__ = [
     # Cassette
     "LLMCassetteRecorder",
     "CassetteMode",
+    "CassetteRequestMismatchError",
+    "SanitizePattern",
+    "DEFAULT_PATTERNS",
 ]

{agentverify-0.1.0 → agentverify-0.2.0}/agentverify/cassette/io.py

@@ -19,8 +19,10 @@ from typing import Any
 from agentverify.cassette.adapters.base import NormalizedRequest, NormalizedResponse
 from agentverify.models import TokenUsage
 
+from importlib.metadata import version as _pkg_version
+
 #: Library version stamped into cassette metadata.
-_AGENTVERIFY_VERSION = "0.1.0"
+_AGENTVERIFY_VERSION = _pkg_version("agentverify")
 
 _YAML_EXTENSIONS = {".yaml", ".yml"}
 _JSON_EXTENSIONS = {".json"}