verifiers 0.1.10.dev5__tar.gz → 0.1.11.dev1__tar.gz

{verifiers-0.1.10.dev5 → verifiers-0.1.11.dev1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: verifiers
-Version: 0.1.10.dev5
+Version: 0.1.11.dev1
 Summary: Verifiers: Environments for LLM Reinforcement Learning
 Project-URL: Homepage, https://github.com/primeintellect-ai/verifiers
 Project-URL: Documentation, https://github.com/primeintellect-ai/verifiers
@@ -22,6 +22,7 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: <3.14,>=3.10
+Requires-Dist: anthropic>=0.78.0
 Requires-Dist: datasets>=3.0.0
 Requires-Dist: gepa
 Requires-Dist: jinja2>=3.1.6
@@ -105,7 +106,7 @@ Verifiers: Environments for LLM Reinforcement Learning
 
 - [01/08/26] v0.1.9 is released, featuring a number of new experimental environment class types, monitor rubrics for automatic metric collection, improved workspace setup flow, improved error handling, bug fixes, and a documentation overhaul.
 - [11/19/25] v0.1.8 is released, featuring a major refactor of the rollout system to use trajectory-based tracking for token-in token-out training across turns, as well as support for truncated or branching rollouts.
-- [11/07/25] Verifiers v0.1.7 is released! This includes an improved quickstart configuration for training with [prime-rl], a new included "nano" trainer (`vf.RLTrainer`, replacing `vf.GRPOTrainer`), and a number of bug fixes and improvements to the documentation.
+- [11/07/25] Verifiers v0.1.7 is released! This includes an improved quickstart configuration for training with [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl), a new included "nano" trainer (`vf.RLTrainer`, replacing `vf.GRPOTrainer`), and a number of bug fixes and improvements to the documentation.
 - [10/27/25] A new iteration of the Prime Intellect [Environments Program](https://docs.google.com/spreadsheets/d/13UDfRDjgIZXsMI2s9-Lmn8KSMMsgk2_zsfju6cx_pNU/edit?gid=0#gid=0) is live!  
 
 
@@ -228,17 +229,17 @@ prime eval run primeintellect/math-python
 
 ## Documentation
 
-**[Environments](environments.md)** — Create datasets, rubrics, and custom multi-turn interaction protocols.
+**[Environments](docs/environments.md)** — Create datasets, rubrics, and custom multi-turn interaction protocols.
 
-**[Evaluation](evaluation.md)** - Evaluate models using your environments.
+**[Evaluation](docs/evaluation.md)** - Evaluate models using your environments.
 
-**[Training](training.md)** — Train models in your environments with reinforcement learning.
+**[Training](docs/training.md)** — Train models in your environments with reinforcement learning.
 
-**[Development](development.md)** — Contributing to verifiers
+**[Development](docs/development.md)** — Contributing to verifiers
 
-**[API Reference](reference.md)** — Understanding the API and data structures
+**[API Reference](docs/reference.md)** — Understanding the API and data structures
 
-**[FAQs](faqs.md)** - Other frequently asked questions.
+**[FAQs](docs/faqs.md)** - Other frequently asked questions.
 
 
 ## Citation

{verifiers-0.1.10.dev5 → verifiers-0.1.11.dev1}/README.md

@@ -36,7 +36,7 @@ Verifiers: Environments for LLM Reinforcement Learning
 
 - [01/08/26] v0.1.9 is released, featuring a number of new experimental environment class types, monitor rubrics for automatic metric collection, improved workspace setup flow, improved error handling, bug fixes, and a documentation overhaul.
 - [11/19/25] v0.1.8 is released, featuring a major refactor of the rollout system to use trajectory-based tracking for token-in token-out training across turns, as well as support for truncated or branching rollouts.
-- [11/07/25] Verifiers v0.1.7 is released! This includes an improved quickstart configuration for training with [prime-rl], a new included "nano" trainer (`vf.RLTrainer`, replacing `vf.GRPOTrainer`), and a number of bug fixes and improvements to the documentation.
+- [11/07/25] Verifiers v0.1.7 is released! This includes an improved quickstart configuration for training with [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl), a new included "nano" trainer (`vf.RLTrainer`, replacing `vf.GRPOTrainer`), and a number of bug fixes and improvements to the documentation.
 - [10/27/25] A new iteration of the Prime Intellect [Environments Program](https://docs.google.com/spreadsheets/d/13UDfRDjgIZXsMI2s9-Lmn8KSMMsgk2_zsfju6cx_pNU/edit?gid=0#gid=0) is live!  
 
 
@@ -159,17 +159,17 @@ prime eval run primeintellect/math-python
 
 ## Documentation
 
-**[Environments](environments.md)** — Create datasets, rubrics, and custom multi-turn interaction protocols.
+**[Environments](docs/environments.md)** — Create datasets, rubrics, and custom multi-turn interaction protocols.
 
-**[Evaluation](evaluation.md)** - Evaluate models using your environments.
+**[Evaluation](docs/evaluation.md)** - Evaluate models using your environments.
 
-**[Training](training.md)** — Train models in your environments with reinforcement learning.
+**[Training](docs/training.md)** — Train models in your environments with reinforcement learning.
 
-**[Development](development.md)** — Contributing to verifiers
+**[Development](docs/development.md)** — Contributing to verifiers
 
-**[API Reference](reference.md)** — Understanding the API and data structures
+**[API Reference](docs/reference.md)** — Understanding the API and data structures
 
-**[FAQs](faqs.md)** - Other frequently asked questions.
+**[FAQs](docs/faqs.md)** - Other frequently asked questions.
 
 
 ## Citation

{verifiers-0.1.10.dev5 → verifiers-0.1.11.dev1}/pyproject.toml

@@ -28,6 +28,7 @@ classifiers = [
 ]
 
 dependencies = [
+    "anthropic>=0.78.0",
     "datasets>=3.0.0",
     "jinja2>=3.1.6",
     "numpy",

{verifiers-0.1.10.dev5 → verifiers-0.1.11.dev1}/tests/README.md

@@ -63,9 +63,11 @@ uv run pytest -m unit
 
 The test suite includes comprehensive support for testing async Environment classes:
 
-### AsyncOpenAI Client Mocking
-- `mock_openai_client` fixture provides a fully mocked AsyncOpenAI client
-- Supports both chat completions and regular completions
+### MockClient (conftest.py)
+- `MockClient(Client)` implements the `get_response()` interface returning `vf.Response` objects
+- `mock_client` fixture provides an instance for tests
+- Supports prompt-to-response mappings via `add_response()`
+- Tracks calls via `call_count` and `last_call_kwargs`
 - No actual API calls are made during testing
 
 ### Test Datasets
@@ -76,22 +78,17 @@ The test suite includes comprehensive support for testing async Environment clas
 ### Async Test Examples
 ```python
 @pytest.mark.asyncio
-async def test_my_async_function(mock_openai_client):
-    env = SingleTurnEnv(client=mock_openai_client, model="test", ...)
+async def test_my_async_function(mock_client):
+    env = SingleTurnEnv(client=mock_client, model="test", ...)
     result = await env.rollout(...)
-    assert result[0] == expected_completion
-
-# MultiTurnEnv testing
-@pytest.mark.asyncio  
-async def test_multiturn_conversation(mock_multiturn_env):
-    # Configure sequential responses
-    responses = ["response1", "response2", "final DONE"]
-    mock_multiturn_env.client.chat.completions.create.side_effect = [
-        create_mock_response(resp) for resp in responses
-    ]
-    
-    completion, state = await mock_multiturn_env.rollout(...)
-    assert len(completion) > 1  # Multiple turns
+    assert mock_client.call_count == 1
+
+# Custom response mapping
+@pytest.mark.asyncio
+async def test_with_custom_response(mock_client):
+    mock_client.set_default_response("DONE")
+    env = SimpleMultiTurnEnv(client=mock_client, model="test", ...)
+    completion, state = await env.rollout(...)
 ```
 
 ### Environment Testing
@@ -103,7 +100,6 @@ async def test_multiturn_conversation(mock_multiturn_env):
   - Completion detection logic
   - State management across turns
 - Tests cover both chat and completion message formats
-- Mocked responses simulate real OpenAI API behavior
 - Error handling and edge cases are tested
 - No real LLM requests are made
 
@@ -112,5 +108,5 @@ async def test_multiturn_conversation(mock_multiturn_env):
 1. Create test files following the `test_*.py` naming convention
 2. Use the fixtures from `conftest.py` for common instances
 3. Add appropriate test markers (`@pytest.mark.asyncio` for async tests)
-4. Use `mock_openai_client` for Environment testing
+4. Use `mock_client` for Environment testing
 5. Follow the existing test structure and naming conventions

{verifiers-0.1.10.dev5 → verifiers-0.1.11.dev1}/tests/conftest.py

@@ -1,12 +1,11 @@
 """Pytest configuration and fixtures for verifiers tests."""
 
+import logging
 from pathlib import Path
-from typing import Callable
-from unittest.mock import AsyncMock, MagicMock
+from typing import Any, Callable
 
 import pytest
 from datasets import Dataset
-from openai.types.chat import ChatCompletionToolParam
 
 from verifiers import (
     MaybeThinkParser,
@@ -23,13 +22,18 @@ from verifiers import (
     XMLParser,
     stop,
 )
+from verifiers.clients.client import Client
 from verifiers.types import (
     GenerateMetadata,
     Info,
+    Response,
+    ResponseMessage,
     RolloutInput,
     RolloutOutput,
     RolloutTiming,
     SamplingArgs,
+    Tool,
+    ToolCall,
     TrajectoryStep,
 )
 from verifiers.utils.save_utils import state_to_output
@@ -82,135 +86,156 @@ def think_parser_with_extractor():
 # Async test fixtures for Environment testing
 
 
-class MockAsyncOpenAI:
-    """Mock AsyncOpenAI client that maps conversation inputs to outputs."""
+class MockClient(Client):
+    """Mocked vf.Client with get_response() to return provider-agnostic vf.Response objects"""
 
     def __init__(self):
-        self.chat_completions = {}  # Maps conversation history to responses
-        self.text_completions = {}  # Maps prompts to responses
-        self.default_chat_response = "This is a test response"
-        self.default_text_response = "This is a test completion"
-        self.base_url = "http://localhost/v1/"  # For testing URL parsing
-
-        # Create mock structure
-        self.chat = MagicMock()
-        self.completions = MagicMock()
-        self.chat.completions = MagicMock()
-
-        # Set up async methods
-        self.chat.completions.create = AsyncMock(
-            side_effect=self._handle_chat_completion
-        )
-        self.completions.create = AsyncMock(side_effect=self._handle_text_completion)
+        self.logger = logging.getLogger(f"{__name__}.MockClient")
+        self._client = None
 
-    def add_chat_response(
-        self, messages, response, finish_reason="stop", tool_calls=None
-    ):
+        self._responses: dict[tuple, dict] = {}
+        self.default_response = "This is a test response"
+
+        # Call tracking
+        self.call_count = 0
+        self.last_call_kwargs: dict[str, Any] = {}
+
+    def add_response(self, messages, response, finish_reason="stop", tool_calls=None):
         """Add a mapped response for specific messages."""
-        # Convert messages to a hashable key
-        key = self._messages_to_key(messages)
-        self.chat_completions[key] = {
+        key = self._messages_to_key(self._normalize_input(messages))
+        self._responses[key] = {
             "content": response,
             "finish_reason": finish_reason,
             "tool_calls": tool_calls,
         }
 
-    def add_text_response(self, prompt, response, finish_reason="stop"):
-        """Add a mapped response for specific prompt."""
-        self.text_completions[prompt] = {
-            "text": response,
-            "finish_reason": finish_reason,
+    def set_default_response(self, response):
+        """Set default response when no mapping found."""
+        self.default_response = response
+
+    async def get_response(
+        self,
+        prompt,
+        model,
+        sampling_args,
+        tools=None,
+        **kwargs,
+    ) -> Response:
+        """Return a Response based on the prompt-to-response mapping."""
+        self.call_count += 1
+        self.last_call_kwargs = {
+            "prompt": prompt,
+            "model": model,
+            "sampling_args": sampling_args,
+            "tools": tools,
+            **kwargs,
         }
 
-    def set_default_responses(self, chat_response=None, text_response=None):
-        """Set default responses when no mapping found."""
-        if chat_response:
-            self.default_chat_response = chat_response
-        if text_response:
-            self.default_text_response = text_response
+        return self._make_response(prompt)
 
-    async def _handle_chat_completion(self, messages, **kwargs):
-        """Handle chat completion requests."""
-        key = self._messages_to_key(messages)
+    def setup_client(self, config):
+        return None
 
-        if key in self.chat_completions:
-            response_data = self.chat_completions[key]
-        else:
-            response_data = {
-                "content": self.default_chat_response,
-                "finish_reason": "stop",
-                "tool_calls": None,
-            }
+    async def to_native_tool(self, tool):
+        pass
 
-        # Create mock response that mimics ChatCompletion
-        from openai.types.chat.chat_completion import ChatCompletion, Choice
-        from openai.types.chat.chat_completion_message import ChatCompletionMessage
-
-        # Create a proper mock that will pass isinstance checks
-        mock_response = MagicMock(spec=ChatCompletion)
-        mock_choice = MagicMock(spec=Choice)
-        mock_message = MagicMock(spec=ChatCompletionMessage)
-
-        # Set the attributes
-        mock_message.content = response_data["content"]
-        mock_message.role = "assistant"
-        mock_message.tool_calls = response_data.get("tool_calls", None)
-        mock_choice.message = mock_message
-        mock_choice.finish_reason = response_data["finish_reason"]
-        mock_choice.index = 0
-
-        mock_response.choices = [mock_choice]
-        mock_response.id = "test-id"
-        mock_response.model = "test-model"
-        mock_response.object = "chat.completion"
-
-        return mock_response
-
-    async def _handle_text_completion(self, prompt, **kwargs):
-        """Handle text completion requests."""
-        if prompt in self.text_completions:
-            response_data = self.text_completions[prompt]
-        else:
-            response_data = {
-                "text": self.default_text_response,
-                "finish_reason": "stop",
-            }
+    async def to_native_prompt(self, messages):
+        return [], {}
+
+    async def get_native_response(
+        self, prompt, model, sampling_args, tools=None, **kwargs
+    ):
+        pass
 
-        # Create mock response that mimics Completion
-        from openai.types.completion import Completion
-        from openai.types.completion_choice import CompletionChoice
+    async def raise_from_native_response(self, response):
+        pass
 
-        # Create a proper mock that will pass isinstance checks
-        mock_response = MagicMock(spec=Completion)
-        mock_choice = MagicMock(spec=CompletionChoice)
+    async def from_native_response(self, response):
+        pass
 
-        # Set the attributes
-        mock_choice.text = response_data["text"]
-        mock_choice.finish_reason = response_data["finish_reason"]
-        mock_choice.index = 0
+    async def close(self) -> None:
+        pass
 
-        mock_response.choices = [mock_choice]
-        mock_response.id = "test-id"
-        mock_response.model = "test-model"
-        mock_response.object = "text_completion"
+    # -- Internal helpers --
 
-        return mock_response
+    @staticmethod
+    def _normalize_input(messages):
+        """Normalize prompt to list-of-dicts form for keying."""
+        if isinstance(messages, str):
+            return [{"role": "text", "content": messages}]
+        return messages
 
     def _messages_to_key(self, messages):
         """Convert messages list to a hashable key."""
-        # Create a simplified representation for hashing
         key_parts = []
         for msg in messages:
-            role = msg["role"]
-            content = msg["content"]
+            if isinstance(msg, dict):
+                role = msg.get("role", "")
+                content = msg.get("content", "")
+            else:
+                role = getattr(msg, "role", "")
+                content = getattr(msg, "content", "")
             key_parts.append(f"{role}:{content}")
         return tuple(key_parts)
 
+    def _convert_tool_calls(self, raw_tool_calls) -> list[ToolCall] | None:
+        """Convert OAI-style tool call objects to vf.ToolCall."""
+        if not raw_tool_calls:
+            return None
+        result: list[ToolCall] = []
+        for tc in raw_tool_calls:
+            if hasattr(tc, "function"):
+                result.append(
+                    ToolCall(
+                        id=tc.id,
+                        name=tc.function.name,
+                        arguments=tc.function.arguments,
+                    )
+                )
+            elif isinstance(tc, dict):
+                func = tc.get("function", {})
+                result.append(
+                    ToolCall(
+                        id=tc.get("id", ""),
+                        name=func.get("name", ""),
+                        arguments=func.get("arguments", ""),
+                    )
+                )
+        return result or None
+
+    def _make_response(self, prompt) -> Response:
+        key = self._messages_to_key(self._normalize_input(prompt))
+        if key in self._responses:
+            data = self._responses[key]
+        else:
+            data = {
+                "content": self.default_response,
+                "finish_reason": "stop",
+                "tool_calls": None,
+            }
+
+        tool_calls = self._convert_tool_calls(data.get("tool_calls"))
+
+        return Response(
+            id="test-id",
+            created=0,
+            model="test-model",
+            usage=None,
+            message=ResponseMessage(
+                content=data["content"],
+                reasoning_content=None,
+                finish_reason=data["finish_reason"],
+                is_truncated=data["finish_reason"] == "length",
+                tokens=None,
+                tool_calls=tool_calls,
+            ),
+        )
+
 
 @pytest.fixture
-def mock_openai_client():
-    """Return a mocked AsyncOpenAI client with input-output mapping."""
-    return MockAsyncOpenAI()
+def mock_client():
+    """Return a MockClient with input-output mapping."""
+    return MockClient()
 
 
 @pytest.fixture
@@ -240,10 +265,10 @@ def sample_chat_dataset():
 
 
 @pytest.fixture
-def mock_singleturn_env(mock_openai_client, sample_dataset):
+def mock_singleturn_env(mock_client, sample_dataset):
     """Return a SingleTurnEnv with mocked client and dataset."""
     return SingleTurnEnv(
-        client=mock_openai_client,
+        client=mock_client,
         model="test-model",
         dataset=sample_dataset,
         system_prompt="You are a helpful assistant.",
@@ -253,7 +278,7 @@ def mock_singleturn_env(mock_openai_client, sample_dataset):
 
 
 @pytest.fixture
-def mock_singleturn_env_completion(mock_openai_client):
+def mock_singleturn_env_completion(mock_client):
     """Return a SingleTurnEnv for completion format testing."""
     completion_dataset = Dataset.from_dict(
         {
@@ -262,7 +287,7 @@ def mock_singleturn_env_completion(mock_openai_client):
         }
     )
     return SingleTurnEnv(
-        client=mock_openai_client,
+        client=mock_client,
         model="test-model",
         dataset=completion_dataset,
         message_type="completion",
@@ -335,10 +360,10 @@ class SimpleMultiTurnEnv(MultiTurnEnv):
 
 
 @pytest.fixture
-def mock_multiturn_env(mock_openai_client, sample_chat_dataset):
+def mock_multiturn_env(mock_client, sample_chat_dataset):
     """Return a MultiTurnEnv for basic testing."""
     return SimpleMultiTurnEnv(
-        client=mock_openai_client,
+        client=mock_client,
         model="test-model",
         dataset=sample_chat_dataset,
         max_turns=3,
@@ -349,10 +374,10 @@ def mock_multiturn_env(mock_openai_client, sample_chat_dataset):
 
 
 @pytest.fixture
-def mock_multiturn_env_max_turns(mock_openai_client, sample_chat_dataset):
+def mock_multiturn_env_max_turns(mock_client, sample_chat_dataset):
     """Return a MultiTurnEnv that tests max_turns limiting."""
     return SimpleMultiTurnEnv(
-        client=mock_openai_client,
+        client=mock_client,
         model="test-model",
         dataset=sample_chat_dataset,
         max_turns=2,
@@ -377,9 +402,9 @@ class BasicToolEnv(ToolEnv):
 
 
 @pytest.fixture
-def mock_tool_env(mock_openai_client, sample_chat_dataset):
+def mock_tool_env(mock_client, sample_chat_dataset):
     return BasicToolEnv(
-        client=mock_openai_client,
+        client=mock_client,
         model="test-model",
         dataset=sample_chat_dataset,
         parser=Parser(),
@@ -413,9 +438,9 @@ class ExampleStatefulToolEnv(StatefulToolEnv):
 
 
 @pytest.fixture
-def mock_stateful_tool_env(mock_openai_client, sample_chat_dataset):
+def mock_stateful_tool_env(mock_client, sample_chat_dataset):
     return ExampleStatefulToolEnv(
-        client=mock_openai_client,
+        client=mock_client,
         model="test-model",
         dataset=sample_chat_dataset,
         parser=Parser(),
@@ -461,7 +486,7 @@ def make_state() -> Callable[..., State]:
         is_completed: bool = True,
         is_truncated: bool = False,
         stop_condition: str | None = "max_turns_reached",
-        oai_tools: list[ChatCompletionToolParam] | None = None,
+        tool_defs: list[Tool] | None = None,
         trajectory: list[TrajectoryStep] = [],
         timing=RolloutTiming(
             generation_ms=0.0,
@@ -483,7 +508,7 @@ def make_state() -> Callable[..., State]:
             is_completed=is_completed,
             is_truncated=is_truncated,
             stop_condition=stop_condition,
-            oai_tools=oai_tools,
+            tool_defs=tool_defs,
             trajectory=trajectory,
             timing=timing,
             error=None,
@@ -529,11 +554,14 @@ def make_metadata() -> Callable[..., GenerateMetadata]:
         time_ms: float = 0.0,
         avg_reward: float = 0.0,
         avg_metrics: dict[str, float] = {},
+        pass_at_k: dict[str, float] = {},
+        pass_all_k: dict[str, float] = {},
+        pass_threshold: float = 0.5,
         usage: dict[str, float] | None = None,
         version_info: dict | None = None,
         state_columns: list[str] = ["foo"],
         path_to_save: Path = Path("test.jsonl"),
-        tools: list[ChatCompletionToolParam] | None = None,
+        tools: list[Tool] | None = None,
     ) -> GenerateMetadata:
         if version_info is None:
             version_info = {
@@ -554,6 +582,9 @@ def make_metadata() -> Callable[..., GenerateMetadata]:
             time_ms=time_ms,
             avg_reward=avg_reward,
             avg_metrics=avg_metrics,
+            pass_at_k=pass_at_k,
+            pass_all_k=pass_all_k,
+            pass_threshold=pass_threshold,
             usage=usage,
             version_info=version_info,
             state_columns=state_columns,

verifiers-0.1.11.dev1/tests/test_build_script.py

@@ -0,0 +1,29 @@
+from pathlib import Path
+
+from verifiers.scripts import build
+
+
+def test_resolve_env_push_target_defaults_to_environments_dir(tmp_path: Path):
+    base_dir = tmp_path / "workspace" / "environments"
+    env_name, env_path = build._resolve_env_push_target("my-env", str(base_dir))
+
+    assert env_name == "my-env"
+    assert env_path == (base_dir / "my_env").resolve()
+
+
+def test_resolve_env_push_target_appends_env_id_to_custom_base_path(tmp_path: Path):
+    base_dir = tmp_path / "workspace" / "custom_envs"
+    env_name, env_path = build._resolve_env_push_target("env-name", str(base_dir))
+
+    assert env_name == "env-name"
+    assert env_path == (base_dir / "env_name").resolve()
+
+
+def test_resolve_env_push_target_uses_explicit_environment_path_when_env_id_missing(
+    tmp_path: Path,
+):
+    explicit_env_path = tmp_path / "workspace" / "environments" / "already_normalized"
+    env_name, env_path = build._resolve_env_push_target(None, str(explicit_env_path))
+
+    assert env_name == "already-normalized"
+    assert env_path == explicit_env_path.resolve()

{verifiers-0.1.10.dev5 → verifiers-0.1.11.dev1}/tests/test_cli_agent_env.py

@@ -159,6 +159,49 @@ class TestCliAgentEnv:
         response = await env.env_response(messages, state)
         assert response == []
 
+    @pytest.mark.asyncio
+    async def test_non_streaming_intercept_tools_use_oai_schema(
+        self, sample_dataset, mock_client
+    ):
+        """OpenAI-formatted intercepted tools should work for non-streaming requests."""
+        env = vf.CliAgentEnv(
+            run_command="python agent.py",
+            dataset=sample_dataset,
+            rubric=vf.Rubric(),
+        )
+        state = await env.init_state(
+            input=sample_dataset[0],
+            client=mock_client,
+            model="test-model",
+        )
+        request_id = "req-test"
+        state["current_request_id"] = request_id
+        env._interception_server.intercepts[request_id] = {
+            "stream": False,
+            "tools": [
+                {
+                    "type": "function",
+                    "function": {
+                        "name": "echo",
+                        "description": "echo tool",
+                        "parameters": {},
+                    },
+                }
+            ],
+        }
+
+        response = await env.get_model_response(
+            state=state,
+            prompt=sample_dataset[0]["prompt"],
+            client=mock_client,
+            model="test-model",
+        )
+
+        assert isinstance(response, vf.Response)
+        kwargs = mock_client.last_call_kwargs
+        assert kwargs["tools"] is not None
+        assert kwargs["tools"][0].name == "echo"
+
 
 class TestHarborEnv:
     """Tests for HarborEnv."""