verifiers 0.2.2.dev12__tar.gz → 0.2.2.dev14__tar.gz

{verifiers-0.2.2.dev12 → verifiers-0.2.2.dev14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: verifiers
-Version: 0.2.2.dev12
+Version: 0.2.2.dev14
 Summary: Verifiers: Environments for LLM Reinforcement Learning
 Project-URL: Homepage, https://github.com/primeintellect-ai/verifiers
 Project-URL: Documentation, https://github.com/primeintellect-ai/verifiers
@@ -97,7 +97,6 @@ uv tool install prime
 - The [docs](docs/) contain short, human-written guides and overviews about the architecture.
 - The [AGENTS.md](AGENTS.md) and [skills](skills/) are for coding agents and go into more details.
 
-
 ## Citation
 
 Originally created by Will Brown ([@willccbb](https://github.com/willccbb)).

{verifiers-0.2.2.dev12 → verifiers-0.2.2.dev14}/README.md

@@ -30,7 +30,6 @@ uv tool install prime
 - The [docs](docs/) contain short, human-written guides and overviews about the architecture.
 - The [AGENTS.md](AGENTS.md) and [skills](skills/) are for coding agents and go into more details.
 
-
 ## Citation
 
 Originally created by Will Brown ([@willccbb](https://github.com/willccbb)).

{verifiers-0.2.2.dev12 → verifiers-0.2.2.dev14}/verifiers/envs/experimental/README.md

@@ -2,10 +2,7 @@
 
 Newer and more experimental environment classes that may have some sharper edges + change more frequently.
 
-For new taskset/harness environments, use the `verifiers.v1` `Taskset` /
-`Harness` format (`vf.Env`, `vf.Taskset`, and `vf.Harness`). The legacy
-experimental `ComposableEnv` stack remains in this subtree for existing
-experiments, but new environments should not be built on that format.
+For new taskset/harness environments, use the `verifiers.v1` `Taskset` / `Harness` format (`vf.Env`, `vf.Taskset`, and `vf.Harness`). The legacy experimental `ComposableEnv` stack remains in this subtree for existing experiments, but new environments should not be built on that format.
 
 ## SandboxMixin
 

{verifiers-0.2.2.dev12 → verifiers-0.2.2.dev14}/verifiers/envs/experimental/composable/README.md

@@ -1,8 +1,6 @@
 # Legacy Composable Task / Agent Architecture
 
-This is the legacy experimental taskset/harness stack. New environments should
-use the `verifiers.v1` `Taskset` / `Harness` format (`vf.Env`, `vf.Taskset`,
-and `vf.Harness`) instead.
+This is the legacy experimental taskset/harness stack. New environments should use the `verifiers.v1` `Taskset` / `Harness` format (`vf.Env`, `vf.Taskset`, and `vf.Harness`) instead.
 
 This stack separates **what to solve** (the task) from **how to solve it** (the agent) by reusing the battle-tested `CliAgentEnv` and delegating task-specific behavior to a `TaskSet`.
 
@@ -113,21 +111,16 @@ class MySWETaskSet(SandboxTaskSet):
 
 ## Scoring
 
-Each taskset provides its own rubric via `get_rubric()`. The rubric owns **all
-scoring logic** — running tests, reading files, computing rewards. It is NOT
-just a state reader.
+Each taskset provides its own rubric via `get_rubric()`. The rubric owns **all scoring logic** — running tests, reading files, computing rewards. It is NOT just a state reader.
 
-Use `keep_sandbox_for_scoring=True` on the environment so the sandbox stays
-alive after the rollout for the rubric to use.
+Use `keep_sandbox_for_scoring=True` on the environment so the sandbox stays alive after the rollout for the rubric to use.
 
 ```
 Rollout completes → sandbox kept alive → rubric.score_rollout() runs tests
     → rubric.cleanup() deletes sandbox
 ```
 
-This enables **retrying scoring independently of the rollout**: if scoring
-fails (transient sandbox error), the rubric can retry without re-running the
-agent.
+This enables **retrying scoring independently of the rollout**: if scoring fails (transient sandbox error), the rubric can retry without re-running the agent.
 
 ### Writing a rubric
 
@@ -155,8 +148,7 @@ class MySWERubric(vf.Rubric):
 
 ## State context
 
-All `SandboxTaskSet` methods (`setup`, `validate_instance`) and rubric reward
-functions receive `state` which contains:
+All `SandboxTaskSet` methods (`setup`, `validate_instance`) and rubric reward functions receive `state` which contains:
 
 - `state["sandbox_client"]` — the async sandbox client
 - `state["sandbox_id"]` — current sandbox ID

{verifiers-0.2.2.dev12 → verifiers-0.2.2.dev14}/verifiers/envs/experimental/composable/tasksets/swe/README.md

@@ -5,9 +5,7 @@
 - Prime images: ✅ means task images are available in our registry; ❌ means
   they are not known to be available there yet.
 - Validation: ✅ means the taskset has a linked `prime-data` PR and was
-  validated with
-  [`SandboxDebugEnv`](../../../../../../docs/v0/environments.md#integrations-and-experimental-environments),
-  — not yet complete.
+  validated with [`SandboxDebugEnv`](../../../../../../docs/v0/environments.md#integrations-and-experimental-environments), — not yet complete.
 
 ## Progress
 
@@ -300,12 +298,8 @@
 1. Add or port the taskset under this directory and register its backend in
    [`make_swe_taskset(...)`](swe_tasksets.py).
 2. Mirror task images that will run at scale into the Prime image registry so
-   sandbox startup uses quick pulls and large sweeps avoid upstream registry
-   rate limits.
+   sandbox startup uses quick pulls and large sweeps avoid upstream registry rate limits.
 3. Prefer the upstream dataset shape and evaluation lifecycle, then publish a
-   filtered Prime dataset through `prime-data` when validation identifies rows
-   to exclude.
+   filtered Prime dataset through `prime-data` when validation identifies rows to exclude.
 4. Validate with
-   [`SandboxDebugEnv`](../../sandbox_debug_env.py): no-op runs should fail real tasks,
-   gold-patch runs should pass, and repeated passes should separate task
-   quality issues from sandbox or infrastructure failures.
+   [`SandboxDebugEnv`](../../sandbox_debug_env.py): no-op runs should fail real tasks, gold-patch runs should pass, and repeated passes should separate task quality issues from sandbox or infrastructure failures.

{verifiers-0.2.2.dev12 → verifiers-0.2.2.dev14}/verifiers/envs/integrations/README.md

@@ -3,7 +3,7 @@
 Integrations with third-party environment libraries, which may require additional dependencies.
 
 | Environment | Extra | Install Command |
-|-------------|-------|-----------------|
+| ------------- | ------- | ----------------- |
 | `TextArenaEnv` | `ta` | `uv add 'verifiers[ta]'` |
 | `ReasoningGymEnv` | `rg` | `uv add 'verifiers[rg]'` |
 | `BrowserEnv` | `browser` | `uv add 'verifiers[browser]'` |
@@ -57,7 +57,7 @@ env = BrowserEnv(
 ### DOM Mode Options
 
 | Parameter | Default | Description |
-|-----------|---------|-------------|
+| ----------- | --------- | ------------- |
 | `stagehand_model` | `"openai/gpt-4o-mini"` | Model Stagehand uses for page understanding |
 | `model_api_key` | `MODEL_API_KEY` env | API key for Stagehand's model |
 | `proxy_model_to_stagehand` | `False` | Route LLM calls through verifiers client |
@@ -80,7 +80,7 @@ CUA mode automatically deploys the CUA server to Browserbase sandboxes using a p
 #### Execution Modes
 
 | Mode | Parameter | Startup | Use Case |
-|------|-----------|---------|----------|
+| ------ | ----------- | --------- | ---------- |
 | **Pre-built Image** (default) | `use_prebuilt_image=True` | ~5-10s | Production, fastest startup |
 | **Binary Upload** | `use_prebuilt_image=False` | ~30-60s | Custom server modifications |
 | **Manual Server** | `use_sandbox=False` | Manual | Local development/debugging |
@@ -137,9 +137,7 @@ Locally, export these in your shell. On the [Environments Hub](https://app.prime
 
 Drop-in adapter for [OpenEnv](https://github.com/meta-pytorch/OpenEnv) environments. Always runs in Prime Sandboxes and uses OpenEnv's schema to choose between simulation (step/reset) and MCP tool-calling.
 
-The Verifiers adapter uses OpenEnv's public async clients. The bundled OpenEnv
-project under `proj/` declares its own server dependencies for the sandbox
-image.
+The Verifiers adapter uses OpenEnv's public async clients. The bundled OpenEnv project under `proj/` declares its own server dependencies for the sandbox image.
 
 ### Quick Start
 
@@ -197,5 +195,4 @@ Define a prompt renderer that converts each OpenEnv observation into a non-empty
 - `environments/openenv_echo/proj`: verbatim copy of OpenEnv `envs/echo_env` (MCP contract).
 - `environments/openenv_textarena/proj`: verbatim copy of OpenEnv `envs/textarena_env` (gym contract, default `Wordle-v0`).
 
-Include both `proj/` and `.build.json` in your environment package so installs
-from the Environments Hub work without extra setup.
+Include both `proj/` and `.build.json` in your environment package so installs from the Environments Hub work without extra setup.

{verifiers-0.2.2.dev12 → verifiers-0.2.2.dev14}/verifiers/v1/agent.py

@@ -147,13 +147,15 @@ class Agent:
             harness_config = harness_config_type("bash")(id="bash")
         self.config = config
         self.harness = load_harness(harness_config)
-        if client is None:
+        self._owns_client = client is None
+        if self._owns_client:
             client = resolve_client(config.client or EvalClientConfig())
         self.ctx = ModelContext(
             model=config.model,
             client=client,
             sampling=config.sampling if config.sampling is not None else Sampling(),
         )
+        self._closed = False
         self.runtime_config: RuntimeConfig = self.harness.config.runtime
         self.interception = interception
         self.limits = RolloutLimits(
@@ -173,6 +175,8 @@ class Agent:
     async def __aenter__(self) -> "Agent":
         if self._entered:
             raise RuntimeError("Agent is already entered; enter it once and share it")
+        if self._closed:
+            raise RuntimeError("Agent is closed; create a new agent")
         self._entered = True
         if self.interception is None:
             # Sized to the runtime policy (remote needs the tunnel); runs the
@@ -186,14 +190,22 @@ class Agent:
                 # A failed __aenter__ gets no __aexit__ from `async with`: unwind
                 # here, or the agent stays "already entered" forever.
                 self._entered, self._server = False, None
+                if self._owns_client:
+                    self._closed = True
+                    await self.ctx.client.close()
                 raise
         return self
 
     async def __aexit__(self, *exc) -> None:
         self._entered = False
         server, self._server = self._server, None
-        if server is not None:
-            await server.__aexit__(*exc)
+        try:
+            if server is not None:
+                await server.__aexit__(*exc)
+        finally:
+            if self._owns_client:
+                self._closed = True
+                await self.ctx.client.close()
 
     def _interception_for(
         self, run_is_local: bool, task: Task, shared_tools: Mapping
@@ -230,6 +242,8 @@ class Agent:
         `on_trace` observes the trace the moment it's minted, before any I/O.
         Retries whole while the trace ends with a retryable error (`config.retries`)
         — never into a borrowed box; the final trace keeps earlier attempts' errors."""
+        if self._closed:
+            raise RuntimeError("Agent is closed; create a new agent")
         retry = self.config.retries
         history: list = []
         for attempt in range(retry.max_retries + 1):