verifiers 0.1.15.dev438__tar.gz → 0.1.15.dev439__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/.gitignore +1 -0
- verifiers-0.1.15.dev439/PKG-INFO +126 -0
- verifiers-0.1.15.dev439/README.md +45 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/README.md +1 -1
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/browser_env/README.md +1 -1
- verifiers-0.1.15.dev438/PKG-INFO +0 -304
- verifiers-0.1.15.dev438/README.md +0 -223
- verifiers-0.1.15.dev438/tests/README.md +0 -112
- verifiers-0.1.15.dev438/verifiers/AGENTS.md +0 -153
- verifiers-0.1.15.dev438/verifiers/v1/ARCHITECTURE.md +0 -250
- verifiers-0.1.15.dev438/verifiers/v1/GUIDE.md +0 -927
- verifiers-0.1.15.dev438/verifiers/v1/README.md +0 -272
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/LICENSE +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/pyproject.toml +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/AGENTS.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/conftest.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_browser_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_build_script.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_cli_agent_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_client_auth_errors.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_client_config.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_client_multimodal_types.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_composable_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_context_token_metrics.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_decorator_ranks.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_endpoint_registry.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_env_group.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_env_server.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_environment.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_environment_extra.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_envs.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_error_chain.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_eval_display.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_eval_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_gepa_cli.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_gepa_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_gym_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_harbor_env_mcp.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_imports.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_init_script.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_install_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_interception_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_logging.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_math_rubric.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_maybe_think_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_message_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_message_utils_multimodal.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_multiturn_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_nemorl_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_openai_chat_completions_token_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_openai_responses_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_opencode_rlm_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_openenv_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_path_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_per_turn_timing.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_pricing_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_prime_plugin.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_renderer_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_renderer_e2e.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_rlm_composable_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_rubric.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_rubric_group.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_sandbox_debug_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_sandbox_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_sandbox_mixin.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_save_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_setup_script.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_singleturn_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_stateful_tool_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_think_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_tool_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_tool_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_trajectory_processing.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_tui_info_formatting.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_types.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_wordle_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/test_xml_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/conftest.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/counter_tool_v1.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/echo_agentic_v1.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/echo_multi_v0.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/echo_tool_v1.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/echo_user_sim_v1.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/echo_v0.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/echo_v1.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/pyproject.toml +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/fixtures/tool_response_image_v1.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_color_codeword_tasks.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_configs.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_e2e.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_envs.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_graph.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_judges.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_legacy.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_scoring.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/tests/v1/test_trace.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/commands/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/commands/build.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/commands/eval.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/commands/gepa.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/commands/init.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/commands/install.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/commands/setup.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/plugins/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/plugins/prime.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/cli/tui.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/anthropic_messages_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/nemorl_chat_completions_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/openai_chat_completions_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/openai_chat_completions_token_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/openai_completions_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/openai_responses_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/clients/renderer_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/decorators.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/AGENTS.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/env_group.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/environment.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/README.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/cli_agent_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/README.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/_filter.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/composable_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/harnesses/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/harnesses/mini_swe_agent.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/harnesses/opencode.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/harnesses/prompt.txt +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/harnesses/rlm.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/sandbox_debug_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/swe_debug_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/task.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/cp/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/cp/cp_task.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/cp/test_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/harbor/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/harbor/cli_gym.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/harbor/harbor.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/harbor/terminal_lego.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/math/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/math/math_task.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/README.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/openseeker/README.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/openseeker/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/openseeker/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/README.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/api_tools/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/api_tools/tool_pdf.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/eval_toolkit.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/evaluator.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/llm_client/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/llm_client/base_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/prompts/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/prompts/cache_prompts.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/utils/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/utils/cache_filesys.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/utils/load_eval_script.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/utils/misc.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/utils/tool_visit.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/utils/url_tools.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/obj_task_eval/verification_tree.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/open_ended.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/quest/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/redsearcher/README.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/redsearcher/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/redsearcher/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/search/search_tasksets.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/multi_swe/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/multi_swe/extract_fix_patch.sh +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/multi_swe/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/openswe/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/openswe/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/r2e_gym/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/r2e_gym/log_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/r2e_gym/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/scale_swe/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/scale_swe/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/shared/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/shared/test_patch.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_bench/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_bench/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_lego/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_lego/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_rebench_v2/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_rebench_v2/log_parsers.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_rebench_v2/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_smith/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_smith/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/composable/tasksets/swe/swe_tasksets.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/gym_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/harbor_env/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/harbor_env/env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/harbor_env/mcp.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/mcp_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/opencode_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/opencode_qa_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/opencode_rlm_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/sandbox_mixin.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/utils/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/utils/file_locks.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/experimental/utils/git_checkout_cache.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/README.md +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/browser_env/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/browser_env/browser_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/browser_env/modes/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/browser_env/modes/base.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/browser_env/modes/cua_mode.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/browser_env/modes/dom_mode.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/openenv_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/reasoninggym_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/integrations/textarena_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/multiturn_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/python_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/sandbox_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/singleturn_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/stateful_tool_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/envs/tool_env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/errors.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/gepa/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/gepa/adapter.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/gepa/config.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/gepa/display.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/gepa/gepa_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/parsers/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/parsers/maybe_think_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/parsers/parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/parsers/think_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/parsers/xml_parser.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/rubrics/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/rubrics/experimental/hybrid_math_rubric.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/rubrics/judge_rubric.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/rubrics/math_rubric.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/rubrics/rubric.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/rubrics/rubric_group.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/build.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/eval.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/gepa.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/init.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/install.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/setup.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/scripts/tui.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/client/env_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/client/zmq_env_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/server/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/server/env_router.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/server/env_server.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/server/env_worker.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/server/zmq_env_server.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/serve/types.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/types.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/async_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/client_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/config_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/data_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/display_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/env_config_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/env_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/error_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/eval_display.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/eval_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/heartbeat.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/import_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/install_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/interception_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/logging_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/message_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/metric_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/path_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/pricing_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/process_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/response_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/save_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/serve_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/thread_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/threaded_sandbox_client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/tool_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/usage_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/utils/version_utils.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/dashboard/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/dashboard/base.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/dashboard/eval.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/dashboard/replay.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/dashboard/validate.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/debug.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/eval/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/eval/main.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/eval/resume.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/eval/runner.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/init.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/output.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/replay.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/resolve.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/serve.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/cli/validate.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/clients/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/clients/client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/clients/config.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/clients/eval.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/clients/train.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/configs/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/configs/debug.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/configs/eval.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/configs/init.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/configs/replay.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/configs/serve.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/configs/validate.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/decorators.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/dialects/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/dialects/anthropic.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/dialects/base.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/dialects/chat.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/dialects/responses.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/env.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/episode.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/errors.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/graph.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/codex/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/codex/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/default/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/default/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/default/program.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/kimi_code/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/kimi_code/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/mini_swe_agent/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/mini_swe_agent/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/mini_swe_agent/program.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/null/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/null/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/null/program.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/rlm/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/rlm/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/terminus_2/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/terminus_2/harness.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/harnesses/terminus_2/program.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/interception/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/interception/pool.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/interception/server.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/judge.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/judges/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/judges/reference.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/judges/reference.txt +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/judges/rubric.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/judges/rubric.txt +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/legacy.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/loaders.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/mcp/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/mcp/launch.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/mcp/server.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/mcp/toolset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/mcp/user.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/push.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/retries.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/rollout.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/runtimes/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/runtimes/base.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/runtimes/docker.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/runtimes/limiters.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/runtimes/modal.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/runtimes/prime.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/runtimes/subprocess.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/scoring.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/serve/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/serve/client.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/serve/pool.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/serve/server.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/serve/types.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/state.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/task.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/harbor/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/harbor/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/lean/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/lean/scoring.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/lean/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/textarena/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/tasksets/textarena/taskset.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/trace.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/types.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/utils/__init__.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/utils/aio.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/utils/format.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/utils/install.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/utils/logging.py +0 -0
- {verifiers-0.1.15.dev438 → verifiers-0.1.15.dev439}/verifiers/v1/utils/memory.py +0 -0
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: verifiers
|
|
3
|
+
Version: 0.1.15.dev439
|
|
4
|
+
Summary: Verifiers: Environments for LLM Reinforcement Learning
|
|
5
|
+
Project-URL: Homepage, https://github.com/primeintellect-ai/verifiers
|
|
6
|
+
Project-URL: Documentation, https://github.com/primeintellect-ai/verifiers
|
|
7
|
+
Project-URL: Repository, https://github.com/primeintellect-ai/verifiers.git
|
|
8
|
+
Project-URL: Issues, https://github.com/primeintellect-ai/verifiers/issues
|
|
9
|
+
Author-email: William Brown <williambrown97@gmail.com>
|
|
10
|
+
License-Expression: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Keywords: agentic-rl,agents,environments,eval,grpo,harness,llm,multi-turn,reinforcement-learning,rl,rlvr,tool-use,train,verifiers
|
|
13
|
+
Classifier: Development Status :: 4 - Beta
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Intended Audience :: Science/Research
|
|
16
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
17
|
+
Classifier: Programming Language :: Python :: 3
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
21
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
23
|
+
Requires-Python: <3.14,>=3.11
|
|
24
|
+
Requires-Dist: aiohttp>=3.9.0
|
|
25
|
+
Requires-Dist: aiolimiter>=1.2.1
|
|
26
|
+
Requires-Dist: anthropic>=0.78.0
|
|
27
|
+
Requires-Dist: datasets<4.7.0,>=3.0.0
|
|
28
|
+
Requires-Dist: gepa
|
|
29
|
+
Requires-Dist: httpx>=0.27.0
|
|
30
|
+
Requires-Dist: loguru>=0.7.0
|
|
31
|
+
Requires-Dist: math-verify>=0.8.0
|
|
32
|
+
Requires-Dist: mcp>=1.14.1
|
|
33
|
+
Requires-Dist: msgpack>=1.1.2
|
|
34
|
+
Requires-Dist: numpy
|
|
35
|
+
Requires-Dist: openai-agents>=0.0.7
|
|
36
|
+
Requires-Dist: openai>=1.108.1
|
|
37
|
+
Requires-Dist: prime-pydantic-config[toml]>=0.3.0.dev86
|
|
38
|
+
Requires-Dist: prime-sandboxes>=0.2.27
|
|
39
|
+
Requires-Dist: prime-tunnel>=0.1.8
|
|
40
|
+
Requires-Dist: pydantic>=2.11.9
|
|
41
|
+
Requires-Dist: pyzmq>=27.1.0
|
|
42
|
+
Requires-Dist: renderers>=0.1.8.dev40
|
|
43
|
+
Requires-Dist: requests
|
|
44
|
+
Requires-Dist: rich
|
|
45
|
+
Requires-Dist: setproctitle>=1.3.0
|
|
46
|
+
Requires-Dist: tenacity>=8.5.0
|
|
47
|
+
Requires-Dist: tomli-w>=1.0.0
|
|
48
|
+
Requires-Dist: typing-extensions; python_version < '3.12'
|
|
49
|
+
Requires-Dist: uvloop>=0.21.0; sys_platform != 'win32' and sys_platform != 'cygwin' and platform_python_implementation != 'PyPy'
|
|
50
|
+
Provides-Extra: browser
|
|
51
|
+
Requires-Dist: aiohttp>=3.9.0; extra == 'browser'
|
|
52
|
+
Requires-Dist: python-dotenv>=1.0.0; extra == 'browser'
|
|
53
|
+
Requires-Dist: stagehand>=3.0.0; extra == 'browser'
|
|
54
|
+
Provides-Extra: harbor
|
|
55
|
+
Requires-Dist: harbor==0.14.0; (python_version >= '3.12') and extra == 'harbor'
|
|
56
|
+
Provides-Extra: modal
|
|
57
|
+
Requires-Dist: modal>=1.4.0; extra == 'modal'
|
|
58
|
+
Provides-Extra: notebook
|
|
59
|
+
Requires-Dist: nest-asyncio>=1.6.0; extra == 'notebook'
|
|
60
|
+
Provides-Extra: quest
|
|
61
|
+
Requires-Dist: certifi; extra == 'quest'
|
|
62
|
+
Requires-Dist: pillow; extra == 'quest'
|
|
63
|
+
Requires-Dist: pymupdf; extra == 'quest'
|
|
64
|
+
Provides-Extra: rg
|
|
65
|
+
Requires-Dist: reasoning-gym; extra == 'rg'
|
|
66
|
+
Provides-Extra: rl
|
|
67
|
+
Requires-Dist: accelerate>=1.4.0; extra == 'rl'
|
|
68
|
+
Requires-Dist: deepspeed>=0.17.6; extra == 'rl'
|
|
69
|
+
Requires-Dist: flash-attn>=2.8.3; extra == 'rl'
|
|
70
|
+
Requires-Dist: liger-kernel>=0.5.10; extra == 'rl'
|
|
71
|
+
Requires-Dist: peft; extra == 'rl'
|
|
72
|
+
Requires-Dist: requests; extra == 'rl'
|
|
73
|
+
Requires-Dist: torch<2.9.0,>=2.8.0; extra == 'rl'
|
|
74
|
+
Requires-Dist: transformers>=4.56.2; extra == 'rl'
|
|
75
|
+
Requires-Dist: vllm<0.11.0,>=0.10.0; extra == 'rl'
|
|
76
|
+
Requires-Dist: wandb; extra == 'rl'
|
|
77
|
+
Provides-Extra: ta
|
|
78
|
+
Requires-Dist: nltk>=3.9.2; extra == 'ta'
|
|
79
|
+
Requires-Dist: textarena==0.7.4; extra == 'ta'
|
|
80
|
+
Description-Content-Type: text/markdown
|
|
81
|
+
|
|
82
|
+
<p align="center">
|
|
83
|
+
<picture>
|
|
84
|
+
<source media="(prefers-color-scheme: light)" srcset="https://github.com/user-attachments/assets/40c36e38-c5bd-4c5a-9cb3-f7b902cd155d">
|
|
85
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8">
|
|
86
|
+
<img alt="Prime Intellect" src="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8" width="312" style="max-width: 100%;">
|
|
87
|
+
</picture>
|
|
88
|
+
</p>
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
# Overview
|
|
93
|
+
|
|
94
|
+
verifiers is our library for creating environments to train and evaluate LLMs.
|
|
95
|
+
|
|
96
|
+
verifiers is tightly integrated with the [Environments Hub](https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars), as well as our training framework [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) and our [Hosted Training](https://app.primeintellect.ai/dashboard/training) platform.
|
|
97
|
+
|
|
98
|
+
## Installation
|
|
99
|
+
|
|
100
|
+
We recommend installing the [Prime CLI](https://github.com/PrimeIntellect-ai/prime) to interact with the environments.
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
# install uv
|
|
104
|
+
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
105
|
+
# install the prime CLI
|
|
106
|
+
uv tool install prime
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Further reading
|
|
110
|
+
|
|
111
|
+
- The [docs](docs/) contain short, human-written guides and overviews about the architecture.
|
|
112
|
+
- The [AGENTS.md](AGENTS.md) and [skills](skills/) are for coding agents and go into more details.
|
|
113
|
+
|
|
114
|
+
|
|
115
|
+
## Citation
|
|
116
|
+
|
|
117
|
+
Originally created by Will Brown ([@willccbb](https://github.com/willccbb)).
|
|
118
|
+
|
|
119
|
+
```bibtex
|
|
120
|
+
@misc{brown_verifiers_2025,
|
|
121
|
+
author = {William Brown},
|
|
122
|
+
title = {{Verifiers}: Environments for LLM Reinforcement Learning},
|
|
123
|
+
howpublished = {\url{https://github.com/PrimeIntellect-ai/verifiers}},
|
|
124
|
+
year = {2025}
|
|
125
|
+
}
|
|
126
|
+
```
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<picture>
|
|
3
|
+
<source media="(prefers-color-scheme: light)" srcset="https://github.com/user-attachments/assets/40c36e38-c5bd-4c5a-9cb3-f7b902cd155d">
|
|
4
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8">
|
|
5
|
+
<img alt="Prime Intellect" src="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8" width="312" style="max-width: 100%;">
|
|
6
|
+
</picture>
|
|
7
|
+
</p>
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Overview
|
|
12
|
+
|
|
13
|
+
verifiers is our library for creating environments to train and evaluate LLMs.
|
|
14
|
+
|
|
15
|
+
verifiers is tightly integrated with the [Environments Hub](https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars), as well as our training framework [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) and our [Hosted Training](https://app.primeintellect.ai/dashboard/training) platform.
|
|
16
|
+
|
|
17
|
+
## Installation
|
|
18
|
+
|
|
19
|
+
We recommend installing the [Prime CLI](https://github.com/PrimeIntellect-ai/prime) to interact with the environments.
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
# install uv
|
|
23
|
+
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
24
|
+
# install the prime CLI
|
|
25
|
+
uv tool install prime
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Further reading
|
|
29
|
+
|
|
30
|
+
- The [docs](docs/) contain short, human-written guides and overviews about the architecture.
|
|
31
|
+
- The [AGENTS.md](AGENTS.md) and [skills](skills/) are for coding agents and go into more details.
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
## Citation
|
|
35
|
+
|
|
36
|
+
Originally created by Will Brown ([@willccbb](https://github.com/willccbb)).
|
|
37
|
+
|
|
38
|
+
```bibtex
|
|
39
|
+
@misc{brown_verifiers_2025,
|
|
40
|
+
author = {William Brown},
|
|
41
|
+
title = {{Verifiers}: Environments for LLM Reinforcement Learning},
|
|
42
|
+
howpublished = {\url{https://github.com/PrimeIntellect-ai/verifiers}},
|
|
43
|
+
year = {2025}
|
|
44
|
+
}
|
|
45
|
+
```
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
they are not known to be available there yet.
|
|
7
7
|
- Validation: ✅ means the taskset has a linked `prime-data` PR and was
|
|
8
8
|
validated with
|
|
9
|
-
[`SandboxDebugEnv`](../../../../../../docs/environments.md#integrations-and-experimental-environments),
|
|
9
|
+
[`SandboxDebugEnv`](../../../../../../docs/v0/environments.md#integrations-and-experimental-environments),
|
|
10
10
|
— not yet complete.
|
|
11
11
|
|
|
12
12
|
## Progress
|
|
@@ -23,7 +23,7 @@ Or with pip/uv pip:
|
|
|
23
23
|
uv pip install -e ".[browser]"
|
|
24
24
|
```
|
|
25
25
|
|
|
26
|
-
When you publish an environment that uses `BrowserEnv`, list `verifiers[browser]` in that package’s `pyproject.toml` `dependencies` so installs from the [Environments Hub](https://app.primeintellect.ai/dashboard/environments) pull the extra. Validate required variables early in `load_environment()` with `vf.ensure_keys([...])` (see [Required API Keys](../../../../docs/environments.md#required-api-keys) in the Verifiers environments guide).
|
|
26
|
+
When you publish an environment that uses `BrowserEnv`, list `verifiers[browser]` in that package’s `pyproject.toml` `dependencies` so installs from the [Environments Hub](https://app.primeintellect.ai/dashboard/environments) pull the extra. Validate required variables early in `load_environment()` with `vf.ensure_keys([...])` (see [Required API Keys](../../../../docs/v0/environments.md#required-api-keys) in the Verifiers environments guide).
|
|
27
27
|
|
|
28
28
|
## Required credentials
|
|
29
29
|
|
verifiers-0.1.15.dev438/PKG-INFO
DELETED
|
@@ -1,304 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: verifiers
|
|
3
|
-
Version: 0.1.15.dev438
|
|
4
|
-
Summary: Verifiers: Environments for LLM Reinforcement Learning
|
|
5
|
-
Project-URL: Homepage, https://github.com/primeintellect-ai/verifiers
|
|
6
|
-
Project-URL: Documentation, https://github.com/primeintellect-ai/verifiers
|
|
7
|
-
Project-URL: Repository, https://github.com/primeintellect-ai/verifiers.git
|
|
8
|
-
Project-URL: Issues, https://github.com/primeintellect-ai/verifiers/issues
|
|
9
|
-
Author-email: William Brown <williambrown97@gmail.com>
|
|
10
|
-
License-Expression: MIT
|
|
11
|
-
License-File: LICENSE
|
|
12
|
-
Keywords: agentic-rl,agents,environments,eval,grpo,harness,llm,multi-turn,reinforcement-learning,rl,rlvr,tool-use,train,verifiers
|
|
13
|
-
Classifier: Development Status :: 4 - Beta
|
|
14
|
-
Classifier: Intended Audience :: Developers
|
|
15
|
-
Classifier: Intended Audience :: Science/Research
|
|
16
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
17
|
-
Classifier: Programming Language :: Python :: 3
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
21
|
-
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
-
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
23
|
-
Requires-Python: <3.14,>=3.11
|
|
24
|
-
Requires-Dist: aiohttp>=3.9.0
|
|
25
|
-
Requires-Dist: aiolimiter>=1.2.1
|
|
26
|
-
Requires-Dist: anthropic>=0.78.0
|
|
27
|
-
Requires-Dist: datasets<4.7.0,>=3.0.0
|
|
28
|
-
Requires-Dist: gepa
|
|
29
|
-
Requires-Dist: httpx>=0.27.0
|
|
30
|
-
Requires-Dist: loguru>=0.7.0
|
|
31
|
-
Requires-Dist: math-verify>=0.8.0
|
|
32
|
-
Requires-Dist: mcp>=1.14.1
|
|
33
|
-
Requires-Dist: msgpack>=1.1.2
|
|
34
|
-
Requires-Dist: numpy
|
|
35
|
-
Requires-Dist: openai-agents>=0.0.7
|
|
36
|
-
Requires-Dist: openai>=1.108.1
|
|
37
|
-
Requires-Dist: prime-pydantic-config[toml]>=0.3.0.dev86
|
|
38
|
-
Requires-Dist: prime-sandboxes>=0.2.27
|
|
39
|
-
Requires-Dist: prime-tunnel>=0.1.8
|
|
40
|
-
Requires-Dist: pydantic>=2.11.9
|
|
41
|
-
Requires-Dist: pyzmq>=27.1.0
|
|
42
|
-
Requires-Dist: renderers>=0.1.8.dev40
|
|
43
|
-
Requires-Dist: requests
|
|
44
|
-
Requires-Dist: rich
|
|
45
|
-
Requires-Dist: setproctitle>=1.3.0
|
|
46
|
-
Requires-Dist: tenacity>=8.5.0
|
|
47
|
-
Requires-Dist: tomli-w>=1.0.0
|
|
48
|
-
Requires-Dist: typing-extensions; python_version < '3.12'
|
|
49
|
-
Requires-Dist: uvloop>=0.21.0; sys_platform != 'win32' and sys_platform != 'cygwin' and platform_python_implementation != 'PyPy'
|
|
50
|
-
Provides-Extra: browser
|
|
51
|
-
Requires-Dist: aiohttp>=3.9.0; extra == 'browser'
|
|
52
|
-
Requires-Dist: python-dotenv>=1.0.0; extra == 'browser'
|
|
53
|
-
Requires-Dist: stagehand>=3.0.0; extra == 'browser'
|
|
54
|
-
Provides-Extra: harbor
|
|
55
|
-
Requires-Dist: harbor==0.14.0; (python_version >= '3.12') and extra == 'harbor'
|
|
56
|
-
Provides-Extra: modal
|
|
57
|
-
Requires-Dist: modal>=1.4.0; extra == 'modal'
|
|
58
|
-
Provides-Extra: notebook
|
|
59
|
-
Requires-Dist: nest-asyncio>=1.6.0; extra == 'notebook'
|
|
60
|
-
Provides-Extra: quest
|
|
61
|
-
Requires-Dist: certifi; extra == 'quest'
|
|
62
|
-
Requires-Dist: pillow; extra == 'quest'
|
|
63
|
-
Requires-Dist: pymupdf; extra == 'quest'
|
|
64
|
-
Provides-Extra: rg
|
|
65
|
-
Requires-Dist: reasoning-gym; extra == 'rg'
|
|
66
|
-
Provides-Extra: rl
|
|
67
|
-
Requires-Dist: accelerate>=1.4.0; extra == 'rl'
|
|
68
|
-
Requires-Dist: deepspeed>=0.17.6; extra == 'rl'
|
|
69
|
-
Requires-Dist: flash-attn>=2.8.3; extra == 'rl'
|
|
70
|
-
Requires-Dist: liger-kernel>=0.5.10; extra == 'rl'
|
|
71
|
-
Requires-Dist: peft; extra == 'rl'
|
|
72
|
-
Requires-Dist: requests; extra == 'rl'
|
|
73
|
-
Requires-Dist: torch<2.9.0,>=2.8.0; extra == 'rl'
|
|
74
|
-
Requires-Dist: transformers>=4.56.2; extra == 'rl'
|
|
75
|
-
Requires-Dist: vllm<0.11.0,>=0.10.0; extra == 'rl'
|
|
76
|
-
Requires-Dist: wandb; extra == 'rl'
|
|
77
|
-
Provides-Extra: ta
|
|
78
|
-
Requires-Dist: nltk>=3.9.2; extra == 'ta'
|
|
79
|
-
Requires-Dist: textarena==0.7.4; extra == 'ta'
|
|
80
|
-
Description-Content-Type: text/markdown
|
|
81
|
-
|
|
82
|
-
<p align="center">
|
|
83
|
-
<picture>
|
|
84
|
-
<source media="(prefers-color-scheme: light)" srcset="https://github.com/user-attachments/assets/40c36e38-c5bd-4c5a-9cb3-f7b902cd155d">
|
|
85
|
-
<source media="(prefers-color-scheme: dark)" srcset="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8">
|
|
86
|
-
<img alt="Prime Intellect" src="https://github.com/user-attachments/assets/6414bc9b-126b-41ca-9307-9e982430cde8" width="312" style="max-width: 100%;">
|
|
87
|
-
</picture>
|
|
88
|
-
</p>
|
|
89
|
-
|
|
90
|
-
---
|
|
91
|
-
|
|
92
|
-
<h3 align="center">
|
|
93
|
-
Verifiers: Environments for LLM Reinforcement Learning
|
|
94
|
-
</h3>
|
|
95
|
-
|
|
96
|
-
<p align="center">
|
|
97
|
-
<a href="https://docs.primeintellect.ai/verifiers">Documentation</a> •
|
|
98
|
-
<a href="https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars">Environments Hub</a> •
|
|
99
|
-
<a href="https://github.com/PrimeIntellect-ai/prime-rl">PRIME-RL</a>
|
|
100
|
-
</p>
|
|
101
|
-
|
|
102
|
-
---
|
|
103
|
-
|
|
104
|
-
<p align="center">
|
|
105
|
-
<a href="https://github.com/PrimeIntellect-ai/verifiers/actions/workflows/style.yml">
|
|
106
|
-
<img src="https://github.com/PrimeIntellect-ai/verifiers/actions/workflows/style.yml/badge.svg" alt="Style" />
|
|
107
|
-
</a>
|
|
108
|
-
<a href="https://github.com/PrimeIntellect-ai/verifiers/actions/workflows/test.yml">
|
|
109
|
-
<img src="https://github.com/PrimeIntellect-ai/verifiers/actions/workflows/test.yml/badge.svg" alt="Test" />
|
|
110
|
-
</a>
|
|
111
|
-
<a href="https://github.com/PrimeIntellect-ai/verifiers/actions/workflows/publish-envs.yml">
|
|
112
|
-
<img src="https://github.com/PrimeIntellect-ai/verifiers/actions/workflows/publish-envs.yml/badge.svg" alt="Envs" />
|
|
113
|
-
</a>
|
|
114
|
-
</p>
|
|
115
|
-
|
|
116
|
-
## News & Updates
|
|
117
|
-
|
|
118
|
-
- [05/07/26] v0.1.14 is released, featuring the v1 Taskset/Harness API, shared eval and training config shape, model-family starter configs, OpenAI Responses and renderer-backed clients, per-turn timing, GEPA prompt artifacts, Lean guard markers, and release/infrastructure hardening.
|
|
119
|
-
- [04/28/26] v0.1.13.dev8 is released, featuring per-rollout wall-clock timeouts for `MultiTurnEnv`, CLI timeout config, sandbox timeout propagation, and smaller `CliAgentEnv` and RLM fixes.
|
|
120
|
-
- [04/17/26] v0.1.12 is released, featuring upstreamed opencode and RLM harnesses/tasksets, major `RLMEnv` improvements (context dropping, prompt builder, hardened transport), multi-worker env server support, expanded `vf-tui` capabilities, and richer eval configuration.
|
|
121
|
-
- [03/12/26] v0.1.11 is released, featuring a unified client stack, major `RLMEnv` and env server reliability improvements, a substantially refined eval TUI, new pass@k and ablation sweep support, and bundled opencode environments.
|
|
122
|
-
- [02/10/26] v0.1.10 is released, featuring OpenEnv and BrowserEnv integrations, resumed evals, improved rollout and token tracking, safer sandbox lifecycle behavior, refreshed workspace setup, and opencode harbor improvements.
|
|
123
|
-
- [01/08/26] v0.1.9 is released, featuring a number of new experimental environment class types, monitor rubrics for automatic metric collection, improved workspace setup flow, improved error handling, bug fixes, and a documentation overhaul.
|
|
124
|
-
- [11/19/25] v0.1.8 is released, featuring a major refactor of the rollout system to use trajectory-based tracking for token-in token-out training across turns, as well as support for truncated or branching rollouts.
|
|
125
|
-
- [11/07/25] Verifiers v0.1.7 is released! This includes an improved quickstart configuration for training with [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl), a new included "nano" trainer (`vf.RLTrainer`, replacing `vf.GRPOTrainer`), and a number of bug fixes and improvements to the documentation.
|
|
126
|
-
- [10/27/25] A new iteration of the Prime Intellect [Environments Program](https://docs.google.com/spreadsheets/d/13UDfRDjgIZXsMI2s9-Lmn8KSMMsgk2_zsfju6cx_pNU/edit?gid=0#gid=0) is live!
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
# Overview
|
|
130
|
-
|
|
131
|
-
Verifiers is our library for creating environments to train and evaluate LLMs.
|
|
132
|
-
|
|
133
|
-
Environments contain everything required to run and evaluate a model on a particular task:
|
|
134
|
-
- A *dataset* of task inputs
|
|
135
|
-
- A *harness* for the model (tools, sandboxes, context management, etc.)
|
|
136
|
-
- A reward function or *rubric* to score the model's performance
|
|
137
|
-
|
|
138
|
-
Environments can be used for training models with reinforcement learning (RL), evaluating capabilities, generating synthetic data, experimenting with agent harnesses, and more.
|
|
139
|
-
|
|
140
|
-
Verifiers is tightly integrated with the [Environments Hub](https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars), as well as our training framework [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl) and our [Hosted Training](https://app.primeintellect.ai/dashboard/training) platform.
|
|
141
|
-
|
|
142
|
-
## Getting Started
|
|
143
|
-
|
|
144
|
-
Ensure you have `uv` installed, as well as the `prime` [CLI](https://docs.primeintellect.ai/cli-reference/introduction) tool:
|
|
145
|
-
```bash
|
|
146
|
-
# install uv
|
|
147
|
-
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
148
|
-
# install the prime CLI
|
|
149
|
-
uv tool install prime
|
|
150
|
-
# log in to the Prime Intellect platform
|
|
151
|
-
prime login
|
|
152
|
-
```
|
|
153
|
-
To set up a new workspace for developing environments, do:
|
|
154
|
-
```bash
|
|
155
|
-
# ~/dev/my-lab
|
|
156
|
-
prime lab setup
|
|
157
|
-
```
|
|
158
|
-
|
|
159
|
-
This sets up a Python project if needed (with `uv init`), installs `verifiers` (with `uv add verifiers`), creates the recommended workspace structure, and downloads useful starter files:
|
|
160
|
-
```
|
|
161
|
-
configs/
|
|
162
|
-
├── endpoints.toml # OpenAI-compatible API endpoint configuration
|
|
163
|
-
├── rl/ # Example configs for Hosted Training
|
|
164
|
-
├── eval/ # Example multi-environment eval configs
|
|
165
|
-
└── gepa/ # Example configs for prompt optimization
|
|
166
|
-
.prime/
|
|
167
|
-
└── skills/ # Bundled workflow skills for create/browse/review/eval/GEPA/train/brainstorm
|
|
168
|
-
environments/
|
|
169
|
-
└── AGENTS.md # Documentation for AI coding agents
|
|
170
|
-
AGENTS.md # Top-level documentation for AI coding agents
|
|
171
|
-
CLAUDE.md # Claude-specific pointer to AGENTS.md
|
|
172
|
-
```
|
|
173
|
-
|
|
174
|
-
Alternatively, add `verifiers` to an existing project:
|
|
175
|
-
```bash
|
|
176
|
-
uv add verifiers && prime lab setup --skip-install
|
|
177
|
-
```
|
|
178
|
-
|
|
179
|
-
Environments built with Verifiers are self-contained Python modules. To initialize a fresh environment template, do:
|
|
180
|
-
```bash
|
|
181
|
-
prime env init my-env # creates a new template in ./environments/my_env
|
|
182
|
-
```
|
|
183
|
-
Add an explicit harness loader when the environment owns harness behavior:
|
|
184
|
-
```bash
|
|
185
|
-
prime env init my-env --with-harness
|
|
186
|
-
```
|
|
187
|
-
For OpenEnv integration, use:
|
|
188
|
-
```bash
|
|
189
|
-
prime env init my-openenv --openenv
|
|
190
|
-
```
|
|
191
|
-
Then copy your OpenEnv project into `environments/my_openenv/proj/` and build the image with:
|
|
192
|
-
```bash
|
|
193
|
-
uv run vf-build my-openenv
|
|
194
|
-
```
|
|
195
|
-
|
|
196
|
-
This will create a new module called `my_env` with a basic environment template.
|
|
197
|
-
```
|
|
198
|
-
environments/my_env/
|
|
199
|
-
├── my_env.py # Main implementation
|
|
200
|
-
├── pyproject.toml # Dependencies and metadata
|
|
201
|
-
└── README.md # Documentation
|
|
202
|
-
```
|
|
203
|
-
|
|
204
|
-
Environment modules should expose a `load_environment` function which returns an
|
|
205
|
-
environment object. For simple legacy environments, this can still be a direct
|
|
206
|
-
constructor:
|
|
207
|
-
```python
|
|
208
|
-
# my_env.py
|
|
209
|
-
import verifiers as vf
|
|
210
|
-
|
|
211
|
-
def load_environment(dataset_name: str = 'gsm8k') -> vf.Environment:
|
|
212
|
-
dataset = vf.load_example_dataset(dataset_name) # 'question'
|
|
213
|
-
async def correct_answer(completion, answer) -> float:
|
|
214
|
-
completion_ans = completion[-1]['content']
|
|
215
|
-
return 1.0 if completion_ans == answer else 0.0
|
|
216
|
-
rubric = vf.Rubric(funcs=[correct_answer])
|
|
217
|
-
env = vf.SingleTurnEnv(dataset=dataset, rubric=rubric)
|
|
218
|
-
return env
|
|
219
|
-
```
|
|
220
|
-
|
|
221
|
-
The same environment package is the unit used by evals and `prime-rl`. The
|
|
222
|
-
trainer owns model, endpoint, sampling, and rollout count; v1-specific options
|
|
223
|
-
stay on the taskset or harness config that owns them:
|
|
224
|
-
|
|
225
|
-
```toml
|
|
226
|
-
# configs/rl/my-v1-env.toml
|
|
227
|
-
model = "Qwen/Qwen3-30B-A3B-Instruct-2507"
|
|
228
|
-
max_steps = 100
|
|
229
|
-
batch_size = 256
|
|
230
|
-
rollouts_per_example = 8
|
|
231
|
-
|
|
232
|
-
[sampling]
|
|
233
|
-
max_tokens = 4096
|
|
234
|
-
|
|
235
|
-
[[env]]
|
|
236
|
-
id = "my-env"
|
|
237
|
-
|
|
238
|
-
[env.harness]
|
|
239
|
-
max_turns = 1
|
|
240
|
-
|
|
241
|
-
[env.taskset]
|
|
242
|
-
system_prompt = "Reverse text exactly."
|
|
243
|
-
|
|
244
|
-
[env.taskset.scoring.contains_answer]
|
|
245
|
-
weight = 1.0
|
|
246
|
-
```
|
|
247
|
-
|
|
248
|
-
```bash
|
|
249
|
-
prime env install my-env
|
|
250
|
-
```
|
|
251
|
-
|
|
252
|
-
For self-managed training launch commands, use the `prime-rl` documentation.
|
|
253
|
-
|
|
254
|
-
To run a local evaluation with any OpenAI-compatible model, do:
|
|
255
|
-
```bash
|
|
256
|
-
prime eval run my-env -m openai/gpt-5-nano # run and save eval results locally
|
|
257
|
-
```
|
|
258
|
-
Evaluations use [Prime Inference](https://docs.primeintellect.ai/inference/overview) by default; configure your own API endpoints in `./configs/endpoints.toml`.
|
|
259
|
-
|
|
260
|
-
View local evaluation results in the terminal UI:
|
|
261
|
-
```bash
|
|
262
|
-
prime eval view
|
|
263
|
-
```
|
|
264
|
-
|
|
265
|
-
To publish the environment to the [Environments Hub](https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars), do:
|
|
266
|
-
```bash
|
|
267
|
-
prime env push --path ./environments/my_env
|
|
268
|
-
```
|
|
269
|
-
|
|
270
|
-
To run an evaluation directly from the Environments Hub, do:
|
|
271
|
-
```bash
|
|
272
|
-
prime eval run primeintellect/math-python
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
## Documentation
|
|
276
|
-
|
|
277
|
-
**[Environments](docs/environments.md)** — Create datasets, rubrics, and custom multi-turn interaction protocols.
|
|
278
|
-
|
|
279
|
-
**[Evaluation](docs/evaluation.md)** - Evaluate models using your environments.
|
|
280
|
-
|
|
281
|
-
**[Training](docs/training.md)** — Train models in your environments with reinforcement learning.
|
|
282
|
-
|
|
283
|
-
**[Development](docs/development.md)** — Contributing to verifiers
|
|
284
|
-
|
|
285
|
-
**[API Reference](docs/reference.md)** — Understanding the API and data structures
|
|
286
|
-
|
|
287
|
-
**[FAQs](docs/faqs.md)** - Other frequently asked questions.
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
## Citation
|
|
291
|
-
|
|
292
|
-
Originally created by Will Brown ([@willccbb](https://github.com/willccbb)).
|
|
293
|
-
|
|
294
|
-
If you use this code in your research, please cite:
|
|
295
|
-
|
|
296
|
-
```bibtex
|
|
297
|
-
@misc{brown_verifiers_2025,
|
|
298
|
-
author = {William Brown},
|
|
299
|
-
title = {{Verifiers}: Environments for LLM Reinforcement Learning},
|
|
300
|
-
howpublished = {\url{https://github.com/PrimeIntellect-ai/verifiers}},
|
|
301
|
-
note = {Commit abcdefg • accessed DD Mon YYYY},
|
|
302
|
-
year = {2025}
|
|
303
|
-
}
|
|
304
|
-
```
|