verifiers 0.1.15.dev168__tar.gz → 0.1.15.dev170__tar.gz

{verifiers-0.1.15.dev168 → verifiers-0.1.15.dev170}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: verifiers
-Version: 0.1.15.dev168
+Version: 0.1.15.dev170
 Summary: Verifiers: Environments for LLM Reinforcement Learning
 Project-URL: Homepage, https://github.com/primeintellect-ai/verifiers
 Project-URL: Documentation, https://github.com/primeintellect-ai/verifiers
@@ -22,8 +22,10 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: <3.14,>=3.10
+Requires-Dist: aiohttp>=3.9.0
 Requires-Dist: aiolimiter>=1.2.1
 Requires-Dist: anthropic>=0.78.0
+Requires-Dist: certifi
 Requires-Dist: datasets<4.7.0,>=3.0.0
 Requires-Dist: gepa
 Requires-Dist: httpx>=0.27.0
@@ -35,10 +37,12 @@ Requires-Dist: nest-asyncio>=1.6.0
 Requires-Dist: numpy
 Requires-Dist: openai-agents>=0.0.7
 Requires-Dist: openai>=1.108.1
+Requires-Dist: pillow
 Requires-Dist: prime-pydantic-config[toml]
 Requires-Dist: prime-sandboxes>=0.2.25
 Requires-Dist: prime-tunnel>=0.1.6
 Requires-Dist: pydantic>=2.11.9
+Requires-Dist: pymupdf
 Requires-Dist: pyzmq>=27.1.0
 Requires-Dist: regex<2026.4.4
 Requires-Dist: requests

{verifiers-0.1.15.dev168 → verifiers-0.1.15.dev170}/pyproject.toml

@@ -53,6 +53,10 @@ dependencies = [
     "setproctitle>=1.3.0",
     "regex<2026.4.4",
     "httpx>=0.27.0",
+    "aiohttp>=3.9.0",
+    "pymupdf",
+    "pillow",
+    "certifi",
     "prime-pydantic-config[toml]",
     "uvloop>=0.21.0; sys_platform != 'win32' and sys_platform != 'cygwin' and platform_python_implementation != 'PyPy'",
 ]

verifiers-0.1.15.dev170/verifiers/envs/experimental/composable/tasksets/search/README.md

@@ -0,0 +1,37 @@
+# Search Tasksets
+
+Composable search/research tasksets for agents that solve live information-seeking tasks in a sandbox.
+
+The search family is intentionally backend-oriented, mirroring the SWE taskset pattern while keeping the task contract research-centric: each task expects a single final answer rather than a code patch. Agents may use web/search tools, browser helpers, or other sandbox resources provided by the paired environment.
+
+## Backends
+
+| Backend | Source | Default dataset | Status |
+|---|---|---|---|
+| `openseeker` | [PolarSeeker/OpenSeeker](https://github.com/PolarSeeker/OpenSeeker) | [`PolarSeeker/OpenSeeker-v1-Data`](https://huggingface.co/datasets/PolarSeeker/OpenSeeker-v1-Data) | Binary semantic answer judge |
+| `quest` | [OSU-NLP-Group/QUEST](https://github.com/OSU-NLP-Group/QUEST) | [`osunlp/QUEST-RL-Data`](https://huggingface.co/datasets/osunlp/QUEST-RL-Data) | Objective tasks supported |
+
+## Usage
+
+```python
+from verifiers.envs.experimental.composable.tasksets.search import make_search_taskset
+
+taskset = make_search_taskset(backend="openseeker")
+taskset = make_search_taskset(backend="quest", category="objective")
+```
+
+`make_search_taskset()` dispatches by backend name. Unknown backends raise `ValueError` with the available backend list.
+
+## Output Contract
+
+Search tasksets should define their own output contract. The `quest` and `openseeker` backends expect the agent to write one final researched response to `/task/answer.txt`, including supporting URLs/citations when available. Scratch reasoning, tool traces, and logs should not be written as the final answer.
+
+## Error Handling
+
+Search tasksets should use the framework error taxonomy for infrastructure failures:
+
+- `vf.SandboxError` for sandbox setup, command, or lifecycle failures.
+- `vf.ModelError` for judge/model provider failures.
+- `vf.InfraError` for dataset, evaluator, or external runtime failures.
+
+Incorrect answers should not set `state["error"]`; they should score normally, often as `0.0`.

verifiers-0.1.15.dev170/verifiers/envs/experimental/composable/tasksets/search/__init__.py

@@ -0,0 +1,9 @@
+"""Composable search/research tasksets."""
+
+from .search_tasksets import (
+    make_openseeker_taskset,
+    make_quest_taskset,
+    make_search_taskset,
+)
+
+__all__ = ["make_openseeker_taskset", "make_quest_taskset", "make_search_taskset"]

verifiers-0.1.15.dev170/verifiers/envs/experimental/composable/tasksets/search/openseeker/README.md

@@ -0,0 +1,33 @@
+# OpenSeeker Search Taskset
+
+Composable search taskset for [`PolarSeeker/OpenSeeker-v1-Data`](https://huggingface.co/datasets/PolarSeeker/OpenSeeker-v1-Data), the OpenSeeker v1 release associated with arXiv `2603.15594`.
+
+OpenSeeker v1 data contains synthesized deep-search QA pairs plus trajectories generated with `search` and `visit` tools. The public OpenSeeker evaluator scores only the final answer: it sends the question, gold answer, and model response to an LLM judge and expects `A` for correct or `B` for incorrect. This backend preserves that binary semantic answer-judge contract.
+
+## Usage
+
+```python
+from verifiers.envs.experimental.composable.tasksets.search import make_search_taskset
+
+taskset = make_search_taskset(backend="openseeker")
+```
+
+## Arguments
+
+| Argument | Default | Description |
+|---|---:|---|
+| `dataset_name` | `PolarSeeker/OpenSeeker-v1-Data` | Hugging Face dataset name. |
+| `split` | `train` | Dataset split. |
+| `trajectory_correctness` | `Correct` | Keep rows with this trajectory label. Use `None` or `all` for all rows. |
+| `min_tool_calls` | `None` | Optional lower bound for `number of tool calls`. |
+| `max_tool_calls` | `None` | Optional upper bound for `number of tool calls`. |
+| `include_trajectory` | `False` | Include the large source trajectory in task metadata. |
+| `answer_file` | `/task/answer.txt` | Final answer path in the sandbox. |
+| `judge_model` | `openai/gpt-5.4-mini` | OpenAI-compatible model used for binary answer judging. |
+| `judge_base_url` | `https://api.pinference.ai/api/v1` | Judge API base URL. |
+| `judge_api_key_var` | `PRIME_API_KEY` | Env var containing the judge API key. |
+| `judge_sampling_args` | `None` | Extra sampling args for judge calls. |
+
+## Output Contract
+
+Agents should write one final answer to `/task/answer.txt`. The answer should directly satisfy the question and may include supporting URLs/citations. The judge ignores citation verification and evaluates whether the final response semantically contains the gold answer without contradictions.

verifiers-0.1.15.dev170/verifiers/envs/experimental/composable/tasksets/search/openseeker/__init__.py

@@ -0,0 +1,5 @@
+"""OpenSeeker search taskset."""
+
+from .taskset import OpenSeekerRubric, OpenSeekerTaskSet
+
+__all__ = ["OpenSeekerRubric", "OpenSeekerTaskSet"]