hud-python 0.2.1__py3-none-any.whl → 0.2.3__py3-none-any.whl

{hud_python-0.2.1.dist-info → hud_python-0.2.3.dist-info}/METADATA

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: hud-python
-Version: 0.2.1
+Version: 0.2.3
 Summary: SDK for the HUD evaluation platform.
 Project-URL: Homepage, https://github.com/hud-evals/hud-sdk
 Project-URL: Bug Tracker, https://github.com/hud-evals/hud-sdk/issues
 Project-URL: Documentation, https://hud.so
-Author-email: Human Union Data SDK <founders@hud.so>
+Author-email: HUD SDK <founders@hud.so>
 License: MIT License
         
         Copyright (c) 2025 Human Union Data, Inc
@@ -37,8 +37,14 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Requires-Python: <3.14,>=3.10
 Requires-Dist: aiodocker>=0.24.0
+Requires-Dist: anthropic
 Requires-Dist: httpx<1,>=0.23.0
 Requires-Dist: inspect-ai>=0.3.80
+Requires-Dist: ipykernel
+Requires-Dist: langchain
+Requires-Dist: langchain-openai
+Requires-Dist: numpy
+Requires-Dist: openai
 Requires-Dist: pillow>=11.1.0
 Requires-Dist: pydantic-settings<3,>=2
 Requires-Dist: pydantic<3,>=2
@@ -53,8 +59,11 @@ Requires-Dist: jupyter-client; extra == 'dev'
 Requires-Dist: jupyter-core; extra == 'dev'
 Requires-Dist: openai; extra == 'dev'
 Requires-Dist: pyright==1.1.364; extra == 'dev'
+Requires-Dist: pytest-asyncio; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: pytest-mock; extra == 'dev'
 Requires-Dist: pytest<9,>=8.1.1; extra == 'dev'
-Requires-Dist: ruff==0.9.8; extra == 'dev'
+Requires-Dist: ruff==0.11.8; extra == 'dev'
 Description-Content-Type: text/markdown
 
 # HUD
@@ -88,17 +97,17 @@ pip install hud-python
 
 ### Simple Browser Example with Claude Computer Use
 
-> This example uses the `@job("test-run")` decorator, so the results of this run will appear under the job named "test-run" on the your [HUD Jobs page](https://app.hud.so/jobs).
+> This example uses the `@register_job("test-run")` decorator, so the results of this run will appear under the job named "test-run" on the your [HUD Jobs page](https://app.hud.so/jobs).
 
 Make sure your have defined your `ANTRHOPIC_API_KEY` in environment variables to run Claude.
 
 ```python
 import asyncio
-from hud import gym, job
+from hud import gym, register_job
 from hud.task import Task
 from hud.agent import ClaudeAgent
 
-@job("test-run")
+@register_job("test-run")
 async def main():
     task = Task(
         prompt="Insert the text 'capybara' into the search bar",
@@ -117,10 +126,9 @@ async def main():
     obs, _ = await env.reset() # Gets first observation
     for i in range(5):
         actions, done = await agent.predict(obs)
-        if done:
-            break
-        
+
         obs, reward, terminated, info = await env.step(actions)
+        if done or terminated: break
     
     # Evaluate and close
     result = await env.evaluate()
@@ -132,22 +140,37 @@ if __name__ == "__main__":
 
 ```
 
+Alternatively, run a full evaluation set via the ```run_job``` command:
+
+```python
+from hud import load_taskset, run_job, ClaudeAgent
+
+# load
+taskset = load_taskset("GAIA")
+
+# evaluate
+job = await run_job(ClaudeAgent, taskset, "test-gaia-job")
+
+# get results OR view them in app.hud.so
+print(await job.get_analytics())
+```
+
 ## Documentation Sections
 
 Explore the core concepts and features of the SDK:
 
-*   **[Tasks and TaskSets](/concepts/task)**: Define goals, context, setup, and evaluation criteria for agent scenarios. This includes both interactive and **question-answering (QA)** style tasks.
-*   **[Environments](/concepts/environment)**: Understand the browser and OS runtimes where agents interact.
-*   **[Agents](/concepts/agent)**: Learn about the agent architecture (Claude, Operator) and how they process observations and predict actions.
-*   **[Adapters](/concepts/adapter)**: See how actions and observations are translated between agents and environments.
-*   **[Jobs](/concepts/job)**: Group related runs for analysis and viewing on the HUD platform.
-*   **[Trajectories](/concepts/trajectory)**: Understand the recorded data from each agent run.
+*   **[Tasks and TaskSets](https://documentation.hud.so/concepts/task)**: Define goals, context, setup, and evaluation criteria for agent scenarios. This includes both interactive and **question-answering (QA)** style tasks.
+*   **[Environments](https://documentation.hud.so/concepts/environment)**: Understand the browser and OS runtimes where agents interact.
+*   **[Agents](https://documentation.hud.so/concepts/agent)**: Learn about the agent architecture (Claude, Operator) and how they process observations and predict actions.
+*   **[Adapters](https://documentation.hud.so/concepts/adapter)**: See how actions and observations are translated between agents and environments.
+*   **[Jobs](https://documentation.hud.so/concepts/job)**: Group related runs for analysis and viewing on the HUD platform.
+*   **[Trajectories](https://documentation.hud.so/concepts/trajectory)**: Understand the recorded data from each agent run.
 *   **Advanced Topics**:
-    *   **[CLA Action Details](/advanced/cla-details)**: Explore the standardized action format.
-    *   **[Custom Environments](/advanced/custom-environments)**: Build your own Docker-based local or remote environments.
-    *   **[Advanced Environment Control](/advanced/environment-control)**: Use `invoke`, `execute`, and `_setup` for finer control.
+    *   **[CLA Action Details](https://documentation.hud.so/advanced/cla-details)**: Explore the standardized action format.
+    *   **[Custom Environments](https://documentation.hud.so/advanced/custom-environments)**: Build your own Docker-based local or remote environments.
+    *   **[Advanced Environment Control](https://documentation.hud.so/advanced/environment-control)**: Use `invoke`, `execute`, and `_setup` for finer control.
 
-*   **[Full API Reference](/api-reference/gym)**: Detailed specifications for all modules and classes.
+*   **[Full API Reference](https://documentation.hud.so/api-reference/gym)**: Detailed specifications for all modules and classes.
 
 ## [Examples](examples/)
 
@@ -160,7 +183,7 @@ We recommend you first take a look at the example notebooks showing how to use t
 
 ## Documentation
 
-For comprehensive guides, examples, and API reference, visit [our docs](https://docs.hud.so/introduction)
+For comprehensive guides, examples, and API reference, visit [our docs](https://documentation.hud.so/introduction)
 
 ## License
 
@@ -172,7 +195,7 @@ If you use this SDK in your research, please cite it as follows:
 
 ```bibtex
 @software{hud2025agentevalplatform,
-  author = {HUD and Jay Ram and Lorenss Martinsons and Parth Patel and Max Muoto and Oskars Putans and Govind Pimpale and Mayank Singamreddy and Nguyen Nhat Minh},
+  author = {HUD and Jay Ram and Lorenss Martinsons and Parth Patel and Oskars Putans and Govind Pimpale and Mayank Singamreddy and Nguyen Nhat Minh},
   title = {{HUD: An Evaluation Platform for Agents}},
   date = {2025-04},
   url = {https://github.com/hud-evals/hud-sdk},

hud_python-0.2.3.dist-info/RECORD

@@ -0,0 +1,62 @@
+hud/__init__.py,sha256=t9GVlRu8L4-nFL7sz3M1d9KgJc1_WnM3vnyBGVHYA30,529
+hud/exceptions.py,sha256=DNfaMmFGGAZsLwSVl8HILyQE5Eg5ygPLhBa7HLYcxU8,5167
+hud/gym.py,sha256=hM1oH9DFFmxHLO9lIU4O-zkvRDCPZ5v-4BnfzNSE75E,3603
+hud/job.py,sha256=RxXaFJQvcYbPCMIsuQRdrEyi_q3WMgdcNXypLmZqlbU,23351
+hud/settings.py,sha256=1lOg2MieyBlmGmXiaiMk0mQvMj4N1-jcNeHCFLK5ZJA,1339
+hud/task.py,sha256=2E4_BTqbfkertrPDkBM4M1CPYQ8Db-Iulv22f6dInWg,5112
+hud/taskset.py,sha256=MgY9PyxKt4vO8775Qory04NA4o-BYTjirxBntslGPmk,3253
+hud/trajectory.py,sha256=OrcRbxK_ejFp1VhJCjZnM1WCmCXxEOK4CxNjCngcsjo,3721
+hud/types.py,sha256=_6eqLCOyxceMX_Oo8YwJQARHLb0N9zsEtc8ZCRJbf9I,2473
+hud/adapters/__init__.py,sha256=zz24KdC_e9TJPgWo6y57_8SzevEE5ak4Cm6tXzMxwRk,266
+hud/adapters/claude/__init__.py,sha256=i7QEF-29FLb9qxp1eYtXs-adIk_tG54tL-9g6d3xodk,100
+hud/adapters/claude/adapter.py,sha256=xN8IHQH-xrIXxfWxoPV0VBHFvmsJZwq8PC4Ri_FU1Ew,5867
+hud/adapters/common/__init__.py,sha256=BjdZWJVs_AKtpFrt-tNsdQRjnz7D97DFEQirJ-r0mp8,118
+hud/adapters/common/adapter.py,sha256=fxdz1S6uwtHczLoDzmbuT0icQ49ywGMTiuVv8oJL4g4,5822
+hud/adapters/common/types.py,sha256=k7Ic9YhgSWg3zZAtrFaOXINg06qbD1pYNzKs1obMFQE,5018
+hud/adapters/common/tests/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+hud/adapters/common/tests/test_adapter.py,sha256=rTD36LjvytHqMIyOLDyrn0RLIkd20s6f6dwoBEarJaw,8744
+hud/adapters/operator/__init__.py,sha256=31vTRs268_TOLd-TeQRKau5bDYy78wxCNpJFhD5_l8U,104
+hud/adapters/operator/adapter.py,sha256=P07FsvpS11BN4IoWNJUwUPkTW6QmJoboWhxCO7exCWs,2989
+hud/agent/__init__.py,sha256=kolJUkXlqbpULqvV4ZHaeY7ABaDKvyOOcnB5aqGtIgw,321
+hud/agent/base.py,sha256=AfysqGBhmOt1a24AXxrpq1TVI5UHxxv1oJbUd1-hKKo,3696
+hud/agent/claude.py,sha256=Gz5ydfdQZFPM1e1D0qfprkNzJooAXzlXYXnrug0AIFk,7428
+hud/agent/langchain.py,sha256=giAmw2w84nugToKm-5FMlpJSB7fJh4pa_e7hERTX1Qs,8720
+hud/agent/operator.py,sha256=DPDN8oAXjiS44brljxYsYYIQtZdodgjYTDUeDqAiFFo,8156
+hud/env/__init__.py,sha256=wVEesXMXM5hcNXQHt0-PN4-9RnE69DEnQENS7uJSv_Y,266
+hud/env/client.py,sha256=brhfLkWGSuvxl3vqGMCQT-vXfj8rUbJMhE3zJg9WMDA,869
+hud/env/docker_client.py,sha256=9ltQyz30u_iitbMK2kfzs1yhs9aeC4V65Y7A1lXl23A,10200
+hud/env/environment.py,sha256=eaWssjNDH1MnTjUM8NE6XdRqGz8Zi1eWakE4HTJO0ns,14799
+hud/env/local_docker_client.py,sha256=OqBnlQEV4t1TlhaU-maL3noBB8YNWWQnj-iWXPyrWYc,7751
+hud/env/remote_client.py,sha256=080Yi2na4t5bXMgtRaRQNCmuY7dK898FyjQG6vv-0WE,6156
+hud/env/remote_docker_client.py,sha256=5akBq7g-8I0DL0nsQHW9sUAbau6PyNyDNb77ygc_-r4,7183
+hud/evaluators/__init__.py,sha256=V5nktEAw3EDn2Y537pjia5Y1IjdLBIPrDjTs6YTCdX4,153
+hud/evaluators/base.py,sha256=ALO9Rj-R_9HtHIHYp84bsQQD12De0XnCTwad78_T5-k,771
+hud/evaluators/inspect.py,sha256=ZvrTXLpgibyvQ5aNXAMP4quyXISrRQHg9besDcuCx7U,692
+hud/evaluators/judge.py,sha256=N3gEQGwVin9Ir80wWw6VtaL0xrlzitbmItaLm0he5gY,5962
+hud/evaluators/match.py,sha256=8YVQD942myX72Jkme2JFIVlmKhFXEa3CgGTjLC8O5n4,4701
+hud/evaluators/remote.py,sha256=kmD_XIU20KvX0NKgaEEKTTKHp0KVRa_3jUEgONh2nkY,2054
+hud/evaluators/tests/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+hud/evaluators/tests/test_inspect.py,sha256=8dMjgQfXOJGcS8gP6TzoBbQiG_NYuRL6IobMG7euJdU,376
+hud/evaluators/tests/test_judge.py,sha256=c1GaAeq_WpBVgBlx-gQncHrOPokzKNxlbgiC8W8hxYI,7829
+hud/evaluators/tests/test_match.py,sha256=C04GoluyT9i41YZ65xEjN7tKHQbENbrpNhNtUd4ivmA,3919
+hud/evaluators/tests/test_remote.py,sha256=YdJpyyuRLkYP0e3jTUkD3zobS2WHQPePn8yBZtYOIN4,3243
+hud/server/__init__.py,sha256=IPxPCqtPLguryN-nBq78Sakypw2bRiE2iHv3SXG8YRk,139
+hud/server/requests.py,sha256=U0WEeY8AzsLMSICebxQyF0FEbXpKieh-3Qnisd0C6a4,7881
+hud/server/tests/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+hud/server/tests/test_requests.py,sha256=63YCbykcib5MxKxm-OgHJPLX3QC7hmgIwnWaYukVM6s,9077
+hud/utils/__init__.py,sha256=oSl_gGoS272X2VFnBYX8hLxcP2xgGoBYQXAuLhtQgw8,260
+hud/utils/common.py,sha256=nAhGgcOo132p5Qg48u6WV_NVN-oiag-4To--C1SYSDU,3052
+hud/utils/config.py,sha256=kGVuR7oKXjmJfeUAnbt53GesCJf_HPvvV1R6s9xjvq4,3549
+hud/utils/progress.py,sha256=suikwFM8sdSfkV10nAOEaInDhG4XKgOSvFePg4jSj1A,5927
+hud/utils/telemetry.py,sha256=hrVIx2rUjSGyy9IVxTZ_3Jii83PiHjyFRd5ls2whimM,1863
+hud/utils/tests/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+hud/utils/tests/test_common.py,sha256=gbYpQKBNdbCcEH0v1UZpxLt_NW2T5sETMIJKvy8S8pY,1658
+hud/utils/tests/test_config.py,sha256=dPlXYWuMrxX-NOYbf0vdJ27TJpfacKG8eiKOSGOcfDU,4079
+hud/utils/tests/test_progress.py,sha256=QunwDgi_heQXhDgmC25zgjr-sFUu5FdJ_1aYigMKeIc,6351
+hud/utils/tests/test_telemetry.py,sha256=t0An1RTBaE0dZVEpF4uwuq5k1R-PXFR5k4u71h60tx8,1224
+hud/utils/tests/test_version.py,sha256=62vk6xiDrLNDuTqAMv_u6um7-hgrPwpUxWp59krP6H8,159
+hud/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+hud_python-0.2.3.dist-info/METADATA,sha256=glE63rOG2gk-GdExhlD7a0lrwU3HvsLBpI42aCW1cBs,8108
+hud_python-0.2.3.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+hud_python-0.2.3.dist-info/licenses/LICENSE,sha256=yIzBheVUf86FC1bztAcr7RYWWNxyd3B-UJQ3uddg1HA,1078
+hud_python-0.2.3.dist-info/RECORD,,

hud_python-0.2.1.dist-info/RECORD

@@ -1,44 +0,0 @@
-hud/__init__.py,sha256=HFL1iwPhLZd7z--2QADzipur68XlekwGrOzU2vWL-Vw,464
-hud/gym.py,sha256=cKjIuJS7A0vJx4K7fctpUjIEv8TkW5x6aB_PRrODrDY,3651
-hud/job.py,sha256=E4RN1CkppRQVy46RWCUDjNIyhMa7lNlFfCgpky2vKFk,5463
-hud/settings.py,sha256=rv8TiZx4wmBzIoEEkOzoywC0nt8UZXlHxIa_LW4tWAg,1346
-hud/task.py,sha256=aNbHMlO7r1cm5DcO0QLU1SZ7EawOFw9W6DZwTNy72-4,5383
-hud/taskset.py,sha256=xDPBXeDm4AlSOwl-MM98lN0x6PmGV8t9jv7sNyS_u0c,2426
-hud/trajectory.py,sha256=PA-sE2iyt2BctO2Dex-2ZaRmS95AkEXTicZjHCVCYqE,3749
-hud/types.py,sha256=fJZnzK3j3mq7G0gO5TbqRaN92qT4xAb4jUNOXIX8ZZ0,2395
-hud/adapters/__init__.py,sha256=0RNQgrzBCkhNBq1Q7JRESN1WfUVLs_99fR5g1re3APs,207
-hud/adapters/claude/__init__.py,sha256=i7QEF-29FLb9qxp1eYtXs-adIk_tG54tL-9g6d3xodk,100
-hud/adapters/claude/adapter.py,sha256=x0qQglWsg7n8DJ_NacsymlUQBnkpqNVguUlkQRpYX-A,5955
-hud/adapters/common/__init__.py,sha256=BjdZWJVs_AKtpFrt-tNsdQRjnz7D97DFEQirJ-r0mp8,118
-hud/adapters/common/adapter.py,sha256=ls-gXtg1N_SQc211rkDb3LL511HNZv6etm1nx2ZtrkQ,5808
-hud/adapters/common/types.py,sha256=APxGEmoePwjF7OYXAKqBTVT73PJTFV0eBmbURbaT5xk,5091
-hud/adapters/operator/__init__.py,sha256=31vTRs268_TOLd-TeQRKau5bDYy78wxCNpJFhD5_l8U,104
-hud/adapters/operator/adapter.py,sha256=svHgjCdUeMyfgfGzRO3ItGWTKGkm3tmldO2zfjX_sGI,3301
-hud/agent/__init__.py,sha256=cI3bqfmG2_Lwzn2RjrxV0X9qIxCRDiffwd1UaWToct4,238
-hud/agent/base.py,sha256=RThJ_h4A3oU23zyvvKtxY2a_YM03Vd1XYDXdY3bAf8g,3881
-hud/agent/claude.py,sha256=tbDKAzGCLJPnUnHc8eV-zZmj3ZG6QQx0ukWKoO4Ekec,7445
-hud/agent/operator.py,sha256=44t19TzcCrS1N3-rnD25ZLXx5s4Io8On27LomALuugs,8185
-hud/env/__init__.py,sha256=BHFY_N0kEI142pjWtMyqUb3BGnoiekY8evRCIbSbO2w,271
-hud/env/client.py,sha256=SPR6ct6NFxmIrgIi3K8tEC-vnqOmCbCBtuT81PaVjuY,869
-hud/env/docker_client.py,sha256=56_u3Ri4NulGcBumAg-7-KilmFmBKthOwEIM5bOLOZc,10418
-hud/env/environment.py,sha256=Xyq4KQO9aWYPwZ0uESAetB5EEZgmlEnZVc7sA0DLz2c,13706
-hud/env/local_docker_client.py,sha256=TCD9z1qjafxjwAWLatAL8d587_ioMDHjs8T5cBgusr8,7789
-hud/env/remote_client.py,sha256=iJiwueuf98xOx0_Y2ltu_63BwKIKNvohhim73Goq74E,5804
-hud/env/remote_docker_client.py,sha256=FwaO7NyygDt9oe3pDD7PwUS21pxzc465mwcXk-Cx-60,6838
-hud/evaluators/__init__.py,sha256=XophB666xPnurhQ_ygfW44h0Jh0BQGCgUzCXEOG2Q1g,158
-hud/evaluators/base.py,sha256=CNbrvFWQfl1YuBxJKzuG4_TBAdAf0TOQA3hl7eGsbaA,782
-hud/evaluators/inspect.py,sha256=eTu9E2eBFe4jd4sPtXL0_vGIEY8aiEmTgmN__v77jvI,735
-hud/evaluators/judge.py,sha256=0T9DHFRR38oH2X1d87t58SBSAhbKWRH5PlljzCa3rkg,6449
-hud/evaluators/match.py,sha256=iFJ_qqaHJQ19TICdQPjHgTubBhq7izCIET5qU_104Fk,4719
-hud/evaluators/remote.py,sha256=NVUJJvrpGQj2eL-aFxzTEnAWW7iuSI9eDWtar54dc6E,2174
-hud/server/__init__.py,sha256=cxDKTwMdGzhj7bYajtejN8XCt7K8Xq3eKB2No0qBpoY,169
-hud/server/requests.py,sha256=s8LZZYWT1wl7lPu2vwRaYPZs9_gjKwSg3LZLvS5-s6E,9085
-hud/utils/__init__.py,sha256=LnoI2tQUnd-mQ4eg-gpJJgmHBBIhggJ6c9ap7MBgrfs,260
-hud/utils/common.py,sha256=XJZ-hKJkeaNmelG2QD5ybi9FpZQS1ErA40fAYzUSHVE,2742
-hud/utils/config.py,sha256=ePi3GDo8mDUnOZ5G5HyMprqGRvxrxCMfixGNuTOA8rQ,3266
-hud/utils/telemetry.py,sha256=md7AuKxtDqsONMeeTOHen1XpmNds8CbXROX_PnkDxFc,1993
-hud/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-hud_python-0.2.1.dist-info/METADATA,sha256=f2lyqGmu9L7_zgCOqrhZ6ZX1JUU6Z0e92bRTfmojSqQ,7219
-hud_python-0.2.1.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
-hud_python-0.2.1.dist-info/licenses/LICENSE,sha256=yIzBheVUf86FC1bztAcr7RYWWNxyd3B-UJQ3uddg1HA,1078
-hud_python-0.2.1.dist-info/RECORD,,