hud-python 0.2.0__tar.gz → 0.2.2__tar.gz

{hud_python-0.2.0 → hud_python-0.2.2}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: hud-python
-Version: 0.2.0
+Version: 0.2.2
 Summary: SDK for the HUD evaluation platform.
-Project-URL: Homepage, https://github.com/Human-Data/hud-sdk
-Project-URL: Bug Tracker, https://github.com/Human-Data/hud-sdk/issues
+Project-URL: Homepage, https://github.com/hud-evals/hud-sdk
+Project-URL: Bug Tracker, https://github.com/hud-evals/hud-sdk/issues
 Project-URL: Documentation, https://hud.so
-Author-email: Human Union Data SDK <founders@hud.so>
+Author-email: HUD SDK <founders@hud.so>
 License: MIT License
         
         Copyright (c) 2025 Human Union Data, Inc
@@ -37,8 +37,14 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Requires-Python: <3.14,>=3.10
 Requires-Dist: aiodocker>=0.24.0
+Requires-Dist: anthropic
 Requires-Dist: httpx<1,>=0.23.0
 Requires-Dist: inspect-ai>=0.3.80
+Requires-Dist: ipykernel
+Requires-Dist: langchain
+Requires-Dist: langchain-openai
+Requires-Dist: numpy
+Requires-Dist: openai
 Requires-Dist: pillow>=11.1.0
 Requires-Dist: pydantic-settings<3,>=2
 Requires-Dist: pydantic<3,>=2
@@ -57,7 +63,7 @@ Requires-Dist: pytest<9,>=8.1.1; extra == 'dev'
 Requires-Dist: ruff==0.9.8; extra == 'dev'
 Description-Content-Type: text/markdown
 
-# HUD SDK - Human-Agent Interaction Toolkit
+# HUD
 
 A Python SDK for creating, evaluating, and benchmarking agent interactions with web browsers and OS environments.
 
@@ -86,21 +92,20 @@ export HUD_API_KEY=your_api_key_here
 pip install hud-python
 ```
 
-### Simple Browser Example with Operator
+### Simple Browser Example with Claude Computer Use
 
 > This example uses the `@job("test-run")` decorator, so the results of this run will appear under the job named "test-run" on the your [HUD Jobs page](https://app.hud.so/jobs).
 
+Make sure your have defined your `ANTRHOPIC_API_KEY` in environment variables to run Claude.
+
 ```python
-import os
 import asyncio
 from hud import gym, job
 from hud.task import Task
-from hud.utils import stream
-from hud.agent import OperatorAgent
+from hud.agent import ClaudeAgent
 
 @job("test-run")
 async def main():
-    # Define a simple task
     task = Task(
         prompt="Insert the text 'capybara' into the search bar",
         gym="hud-browser",
@@ -108,26 +113,19 @@ async def main():
         evaluate=("contains_text", "capybara")
     )
     
-    # Create environment
+    # Create environment using the gym module
     env = await gym.make(task)
     
-    # Get URLs and display live view (optional)
-    # urls = await env.get_urls()
-    # stream(urls["live_url"])
-    
     # Initialize Operator agent (API key is loaded automatically)
-    agent = OperatorAgent()
+    agent = ClaudeAgent()
     
-    # Agent loop
-    obs, _ = env.reset()
+    # Agent loop with predict and step functions
+    obs, _ = await env.reset() # Gets first observation
     for i in range(5):
         actions, done = await agent.predict(obs)
-        if done:
-            break
-        
+
         obs, reward, terminated, info = await env.step(actions)
-        if terminated:
-            break
+        if done or terminated: break
     
     # Evaluate and close
     result = await env.evaluate()
@@ -139,35 +137,50 @@ if __name__ == "__main__":
 
 ```
 
+Alternatively, run a full evaluation set via the ```run_job``` command:
+
+```python
+from hud import load_taskset, run_job, ClaudeAgent
+
+# load
+taskset = load_taskset("GAIA")
+
+# evaluate
+job = await run_job(ClaudeAgent, taskset, "test-gaia-job")
+
+# get results OR view them in app.hud.so
+print(await job.get_analytics())
+```
+
 ## Documentation Sections
 
 Explore the core concepts and features of the SDK:
 
-*   **[Tasks and TaskSets](/concepts/task)**: Define goals, context, setup, and evaluation criteria for agent scenarios.
-*   **[Environments](/concepts/environment)**: Understand the browser and OS runtimes where agents interact.
-*   **[Agents](/concepts/agent)**: Learn about the agent architecture (Claude, Operator) and how they process observations and predict actions.
-*   **[Adapters](/concepts/adapter)**: See how actions and observations are translated between agents and environments.
-*   **[Jobs](/concepts/job)**: Group related runs for analysis and viewing on the HUD platform.
-*   **[Trajectories](/concepts/trajectory)**: Understand the recorded data from each agent run.
+*   **[Tasks and TaskSets](https://documentation.hud.so/concepts/task)**: Define goals, context, setup, and evaluation criteria for agent scenarios. This includes both interactive and **question-answering (QA)** style tasks.
+*   **[Environments](https://documentation.hud.so/concepts/environment)**: Understand the browser and OS runtimes where agents interact.
+*   **[Agents](https://documentation.hud.so/concepts/agent)**: Learn about the agent architecture (Claude, Operator) and how they process observations and predict actions.
+*   **[Adapters](https://documentation.hud.so/concepts/adapter)**: See how actions and observations are translated between agents and environments.
+*   **[Jobs](https://documentation.hud.so/concepts/job)**: Group related runs for analysis and viewing on the HUD platform.
+*   **[Trajectories](https://documentation.hud.so/concepts/trajectory)**: Understand the recorded data from each agent run.
 *   **Advanced Topics**:
-    *   **[Custom Environments](/advanced/custom-environments)**: Build your own Docker-based local or remote environments.
-    *   **[Advanced Environment Control](/advanced/environment-control)**: Use `invoke`, `execute`, and `_setup` for finer control.
-    *   **[CLA Action Details](/advanced/cla-details)**: Dive deeper into the standardized action format.
+    *   **[CLA Action Details](https://documentation.hud.so/advanced/cla-details)**: Explore the standardized action format.
+    *   **[Custom Environments](https://documentation.hud.so/advanced/custom-environments)**: Build your own Docker-based local or remote environments.
+    *   **[Advanced Environment Control](https://documentation.hud.so/advanced/environment-control)**: Use `invoke`, `execute`, and `_setup` for finer control.
 
-*   **[Full API Reference](/api-reference/gym)**: Detailed specifications for all modules and classes.
+*   **[Full API Reference](https://documentation.hud.so/api-reference/gym)**: Detailed specifications for all modules and classes.
 
 ## [Examples](examples/)
 
-We provide several example notebooks showing how to use the HUD SDK:
+We recommend you first take a look at the example notebooks showing how to use the HUD SDK:
 
 1. [Browser Basics](examples/browser_use.ipynb) - Simple browser interaction with live view
 2. [Task Design](examples/tasks.ipynb) - Creating and customizing tasks
-3. [OSWorld](examples/osworld.ipynb) - Working with OS environments
+3. [OSWorld](examples/osworld.ipynb) - Running the OSWorld benchmark
 4. [Local Development](examples/local.ipynb) - Setting up local custom environments
 
 ## Documentation
 
-For comprehensive guides, examples, and API reference, visit [our docs](https://docs.hud.so/introduction)
+For comprehensive guides, examples, and API reference, visit [our docs](https://documentation.hud.so/introduction)
 
 ## License
 
@@ -180,9 +193,9 @@ If you use this SDK in your research, please cite it as follows:
 ```bibtex
 @software{hud2025agentevalplatform,
   author = {HUD and Jay Ram and Lorenss Martinsons and Parth Patel and Max Muoto and Oskars Putans and Govind Pimpale and Mayank Singamreddy and Nguyen Nhat Minh},
-  title = {{HUD: An Evaluation Platform for Computer Use Agents}},
-  date = {2025-03},
-  url = {https://github.com/Human-Data/hud-sdk},
+  title = {{HUD: An Evaluation Platform for Agents}},
+  date = {2025-04},
+  url = {https://github.com/hud-evals/hud-sdk},
   langid = {en}
 }
 ```

hud_python-0.2.2/README.md

@@ -0,0 +1,136 @@
+# HUD
+
+A Python SDK for creating, evaluating, and benchmarking agent interactions with web browsers and OS environments.
+
+> **Early Release Notice**: This SDK is currently in early release status. The API is evolving and may change in future releases as we gather feedback and improve functionality.
+
+[![PyPI version](https://img.shields.io/pypi/v/hud-python)](https://pypi.org/project/hud-python/)
+
+[📚 Documentation](https://documentation.hud.so) | [🏠 Homepage](https://hud.so)
+
+## API Key Setup
+
+Before getting started, you'll need to obtain an API key:
+
+1. Visit [app.hud.so](https://app.hud.so) to create a free account and generate your API key
+2. Set it in your environment or .env file:
+
+```bash
+export HUD_API_KEY=your_api_key_here
+```
+
+## Quick Start
+
+### Installation
+
+```bash
+pip install hud-python
+```
+
+### Simple Browser Example with Claude Computer Use
+
+> This example uses the `@job("test-run")` decorator, so the results of this run will appear under the job named "test-run" on the your [HUD Jobs page](https://app.hud.so/jobs).
+
+Make sure your have defined your `ANTRHOPIC_API_KEY` in environment variables to run Claude.
+
+```python
+import asyncio
+from hud import gym, job
+from hud.task import Task
+from hud.agent import ClaudeAgent
+
+@job("test-run")
+async def main():
+    task = Task(
+        prompt="Insert the text 'capybara' into the search bar",
+        gym="hud-browser",
+        setup=("goto", "google.com"),
+        evaluate=("contains_text", "capybara")
+    )
+    
+    # Create environment using the gym module
+    env = await gym.make(task)
+    
+    # Initialize Operator agent (API key is loaded automatically)
+    agent = ClaudeAgent()
+    
+    # Agent loop with predict and step functions
+    obs, _ = await env.reset() # Gets first observation
+    for i in range(5):
+        actions, done = await agent.predict(obs)
+
+        obs, reward, terminated, info = await env.step(actions)
+        if done or terminated: break
+    
+    # Evaluate and close
+    result = await env.evaluate()
+    print(f"Evaluation result: {result}")
+    await env.close()
+
+if __name__ == "__main__":
+    asyncio.run(main())
+
+```
+
+Alternatively, run a full evaluation set via the ```run_job``` command:
+
+```python
+from hud import load_taskset, run_job, ClaudeAgent
+
+# load
+taskset = load_taskset("GAIA")
+
+# evaluate
+job = await run_job(ClaudeAgent, taskset, "test-gaia-job")
+
+# get results OR view them in app.hud.so
+print(await job.get_analytics())
+```
+
+## Documentation Sections
+
+Explore the core concepts and features of the SDK:
+
+*   **[Tasks and TaskSets](https://documentation.hud.so/concepts/task)**: Define goals, context, setup, and evaluation criteria for agent scenarios. This includes both interactive and **question-answering (QA)** style tasks.
+*   **[Environments](https://documentation.hud.so/concepts/environment)**: Understand the browser and OS runtimes where agents interact.
+*   **[Agents](https://documentation.hud.so/concepts/agent)**: Learn about the agent architecture (Claude, Operator) and how they process observations and predict actions.
+*   **[Adapters](https://documentation.hud.so/concepts/adapter)**: See how actions and observations are translated between agents and environments.
+*   **[Jobs](https://documentation.hud.so/concepts/job)**: Group related runs for analysis and viewing on the HUD platform.
+*   **[Trajectories](https://documentation.hud.so/concepts/trajectory)**: Understand the recorded data from each agent run.
+*   **Advanced Topics**:
+    *   **[CLA Action Details](https://documentation.hud.so/advanced/cla-details)**: Explore the standardized action format.
+    *   **[Custom Environments](https://documentation.hud.so/advanced/custom-environments)**: Build your own Docker-based local or remote environments.
+    *   **[Advanced Environment Control](https://documentation.hud.so/advanced/environment-control)**: Use `invoke`, `execute`, and `_setup` for finer control.
+
+*   **[Full API Reference](https://documentation.hud.so/api-reference/gym)**: Detailed specifications for all modules and classes.
+
+## [Examples](examples/)
+
+We recommend you first take a look at the example notebooks showing how to use the HUD SDK:
+
+1. [Browser Basics](examples/browser_use.ipynb) - Simple browser interaction with live view
+2. [Task Design](examples/tasks.ipynb) - Creating and customizing tasks
+3. [OSWorld](examples/osworld.ipynb) - Running the OSWorld benchmark
+4. [Local Development](examples/local.ipynb) - Setting up local custom environments
+
+## Documentation
+
+For comprehensive guides, examples, and API reference, visit [our docs](https://documentation.hud.so/introduction)
+
+## License
+
+[MIT License](LICENSE)
+
+## Citation
+
+If you use this SDK in your research, please cite it as follows:
+
+```bibtex
+@software{hud2025agentevalplatform,
+  author = {HUD and Jay Ram and Lorenss Martinsons and Parth Patel and Max Muoto and Oskars Putans and Govind Pimpale and Mayank Singamreddy and Nguyen Nhat Minh},
+  title = {{HUD: An Evaluation Platform for Agents}},
+  date = {2025-04},
+  url = {https://github.com/hud-evals/hud-sdk},
+  langid = {en}
+}
+```

{hud_python-0.2.0 → hud_python-0.2.2}/docs/advanced/cla-details.mdx

@@ -69,6 +69,11 @@ Here are some key CLA types grouped by category:
 *   **`ScreenshotFetch`**: Requests a screenshot (used internally, typically not sent by agents directly).
 *   **`PositionFetch`**: Requests the current cursor position (used internally).
 
+### Response Actions
+
+*   **`ResponseAction`**: Used to submit a final text answer.
+    *   `text: str`: The final textual response from the agent.
+
 ### Custom Actions
 
 *   **`CustomAction`**: Allows defining arbitrary actions specific to a custom environment controller.

{hud_python-0.2.0 → hud_python-0.2.2}/docs/concepts/environment.mdx

@@ -48,6 +48,20 @@ env_os = await gym.make("OSWorld-Ubuntu")
 
 Environments created this way won't have a default `Task` associated unless you explicitly reset them with one later using `env.reset()`. The `gym.make()` function also automatically links the environment to an active [Job](/concepts/job) if one was defined using the `@job` decorator.
 
+## Available Environment Types
+
+The HUD SDK provides several standard environment types, specified via the `gym` attribute in a [Task](/concepts/task) or directly in `hud.gym.make()`:
+
+*   **`"hud-browser"`**: Provides a remote Chromium browser instance managed via Playwright. Ideal for web navigation, form interaction, and testing web applications.
+    *   [See `hud-browser` Details](../environments/hud-browser.mdx)
+*   **`"hud-ubuntu"`**: Provides a remote Ubuntu desktop environment accessed via VNC. Suitable for tasks involving GUI applications, file system interaction, or running Linux software.
+    *   [See `hud-ubuntu` Details](../environments/hud-ubuntu.mdx)
+*   **`"qa"`**: A non-interactive environment for question-answering tasks where the agent provides a direct textual response.
+    *   [See `qa` Environment Details](../environments/qa.mdx)
+*   **`CustomGym`**: Allows defining and running your own [Custom Environments](../advanced/custom-environments.mdx) using Docker, either locally or remotely. This provides maximum flexibility for specific testing needs.
+
+The `gym` attribute in a Task tells `hud.gym.make()` which environment to instantiate.
+
 ## Interaction Loop
 
 The standard interaction flow involves the [Agent](/concepts/agent) and the Environment:
@@ -67,13 +81,14 @@ obs, _ = await env.reset()
 for _ in range(10):
     # 2. Agent predicts action(s)
     actions, done = await agent.predict(obs)
-    if done: break
 
     # 3. Execute action(s) in environment
     obs, reward, terminated, info = await env.step(actions)
-    if terminated: break
+    if done or terminated: break
 ```
 
+*   **Note on QA Tasks:** For [Question-Answering Tasks](/concepts/task#defining-question-answering-qa-tasks), the agent might only need one `predict` call. The agent should output a `ResponseAction`, which the environment stores. The subsequent `env.evaluate()` call then checks this stored response. The environment itself remains largely passive for QA.
+
 ## Key Methods
 
 *   **`env.step(actions: list[CLA] | None = None)`**: Executes actions (or gets initial state). Returns `(Observation, reward, terminated, info)`.

{hud_python-0.2.0 → hud_python-0.2.2}/docs/concepts/task.mdx

@@ -30,11 +30,10 @@ task = Task(
     prompt="Log in to example.com with username 'test'",
     gym="hud-browser", # Request a browser environment
     setup=[ # Actions run by gym.make(task)
-        ("goto", "https://example.com/login"),
-        {"function": "wait_for_element", "args": ["#username"]}
+        ("goto", "https://example.com/login")
     ],
     evaluate={ # Logic run by env.evaluate()
-        "function": "check_login_status", 
+        "function": "page_contains", 
         "args": ["test"]
     }
 )
@@ -60,7 +59,10 @@ Both `setup` and `evaluate` accept configurations defining function calls within
 *   **Purpose:** Determines task success after the agent finishes.
 *   **Execution:** Triggered by `await env.evaluate()`.
 *   **Result:** The return value of `env.evaluate()`, often a reward score (e.g., `1.0` or `0.0`). This is stored in the `reward` field of the [Trajectory](/concepts/trajectory) if linked to a [Job](/concepts/job).
-*   **Examples:** `("contains_text", "Success!")`, `("file_exists", "/path/to/output.txt")`. Check specific environment controller docs for available functions.
+*   **Examples:**
+    *   Interactive: `("contains_text", "Success!")`, `("file_exists", "/path/to/output.txt")`. These typically call functions *within* the active environment controller.
+    *   QA: `("response_includes", "Paris")`. These functions often check the text stored in `env.final_response` (which comes from the agent's `ResponseAction`).
+*   **Note:** Check specific environment or evaluation service documentation for available functions.
 
 ## TaskSet
 
@@ -79,11 +81,13 @@ Load predefined sets from the HUD platform:
 ```python
 from hud import load_taskset
 
-taskset = await load_taskset("OSWorld-Ubuntu-Links")
+taskset = await load_taskset("OSWorld-Ubuntu")
 print(f"Number of tasks: {len(taskset)}") # TaskSet acts like a list
 first_task = taskset[0]
 ```
 
+Currently supported TaskSets available via `load_taskset` include OSWorld, GAIA, and WebVoyager subsets.
+
 ### Creating a TaskSet Manually
 
 ```python
@@ -99,4 +103,31 @@ my_taskset = TaskSet(tasks=[task1, task2], description="My set")
 *   [Environment](/concepts/environment): Where Tasks are executed and evaluated.
 *   [Agent](/concepts/agent): Aims to complete the Task `prompt`.
 *   [Job](/concepts/job): Groups runs of different Tasks.
-*   [Trajectory](/concepts/trajectory): Records the execution of a Task.
+*   [Trajectory](/concepts/trajectory): Records the execution of a Task.
+
+### Defining Question-Answering (QA) Tasks
+
+While HUD excels at interactive tasks, you can also define tasks that are primarily question-answering. The key differences are:
+
+*   **`gym`:** You might still use an existing environment type like `"hud-browser"` if you want the QA to happen *within* that context (e.g., asking the agent to answer based on a webpage). For pure QA without environment interaction, a future specific `"qa"` gym type might be introduced, but currently, you'd use an existing type.
+*   **`prompt`:** Contains the question for the agent.
+*   **`setup`:** Often minimal or unnecessary for pure QA.
+*   **`evaluate`:** Defines how to check the agent's final text answer. This typically involves calling a specific evaluation function that compares the agent's final submitted response (see `ResponseAction` in [CLA Details](/advanced/cla-details)) against expected criteria. The `env.final_response` attribute holds the text submitted by the agent via `ResponseAction`.
+*   **`target`:** (Recommended) Store the ground truth answer in the `metadata` or potentially a dedicated `target` field for clarity during evaluation function design.
+
+```python
+from hud.task import Task
+
+qa_task = Task(
+    prompt="What is the powerhouse of the cell?",
+    gym="hud-browser", # Or potentially a future "qa" type
+    # No complex setup needed for pure QA
+    setup=(),
+    # Evaluation checks the agent's final submitted text response
+    evaluate=("response_includes", "mitochondria"), # Assumes a function checking env.final_response
+)
+```
+
+The [Agent](/concepts/agent) handling such a task should recognize it doesn't need complex interaction and output a `ResponseAction` containing the final answer. The `env.evaluate()` call then triggers the specified check (like `response_includes`) against the stored response.
+
+### <a name="configuration-styles"></a>Configuration Styles (`setup` and `evaluate`)

{hud_python-0.2.0 → hud_python-0.2.2}/docs/docs.json

@@ -14,6 +14,7 @@
         "group": "Getting Started",
         "pages": [
           "quickstart",
+          "running-your-agent",
           "installation"
         ]
       },
@@ -28,12 +29,20 @@
           "concepts/trajectory"
         ]
       },
+      {
+        "group": "Environments",
+        "pages": [
+          "environments/hud-browser",
+          "environments/hud-ubuntu",
+          "environments/qa"
+        ]
+      },
       {
         "group": "Advanced Topics",
         "pages": [
+          "advanced/cla-details",
           "advanced/custom-environments",
-          "advanced/environment-control",
-          "advanced/cla-details"
+          "advanced/environment-control"
         ]
       },
       {
@@ -59,13 +68,13 @@
     "links": [
       {
         "label": "GitHub",
-        "href": "https://github.com/Human-Data/hud-sdk"
+        "href": "https://github.com/hud-evals/hud-sdk"
       }
     ]
   },
   "footer": {
     "socials": {
-      "github": "https://github.com/Human-Data/hud-sdk",
+      "github": "https://github.com/hud-evals/hud-sdk",
       "website": "https://hud.so"
     }
   }

hud_python-0.2.2/docs/environments/hud-browser.mdx

@@ -0,0 +1,67 @@
+# HUD Browser Environment
+
+## Introduction
+
+The `hud-browser` environment provides a remote Chromium browser instance, managed by Playwright, for agents to interact with websites. It's ideal for tasks involving web navigation, form filling, information retrieval, and testing web applications.
+
+## Setup
+
+Setup actions for the `hud-browser` are defined in the `setup` attribute of a [Task](../concepts/task.mdx) and executed by `hud.gym.make()`. They typically involve browser controller functions.
+
+*   **`goto(url: str)`**: Navigates the browser to the specified `url`. Automatically prepends `http://` if no scheme is provided. Waits for `domcontentloaded` (up to 10s timeout) and adds a 1s wait for rendering.
+    ```python
+    # Example Task Setup:
+    setup=[("goto", "https://google.com")]
+    ```
+*   **Other common setup functions coming soon:** `wait_for_element`, `click`, `type`, `set_cookies` etc. 
+
+Refer to [Task Setup Configuration](../concepts/task.mdx#setup-configuration) for how to define these.
+
+## Step Interaction
+
+Agents interact with the browser environment by sending a list of [CLA Actions](../advanced/cla-details.mdx) to `env.step()`. An [Adapter](../concepts/adapter.mdx) typically handles the conversion from the agent model's output to the CLA format.
+
+Common CLAs used with `hud-browser`:
+*   [`ClickAction`](../advanced/cla-details.mdx#mouse-actions)
+*   [`MoveAction`](../advanced/cla-details.mdx#mouse-actions)
+*   [`TypeAction`](../advanced/cla-details.mdx#keyboard-actions)
+*   [`PressAction`](../advanced/cla-details.mdx#keyboard-actions)
+*   [`ScrollAction`](../advanced/cla-details.mdx#mouse-actions)
+*   [`DragAction`](../advanced/cla-details.mdx#mouse-actions)
+*   [`ResponseAction`](../advanced/cla-details.mdx#response-actions) (to submit a final text answer)
+
+*See [CLA Action Details](../advanced/cla-details.mdx) for the full specification.*
+
+## Evaluate
+
+The `evaluate` attribute of a [Task](../concepts/task.mdx) defines how success is measured using `env.evaluate()`. This calls functions within the browser controller.
+
+Built-in evaluation functions for `hud-browser`:
+
+*   **`url_match(expected_url: str)`**: Checks if the current browser URL exactly matches `expected_url`. Returns `1.0` for a match, `0.0` otherwise.
+    ```python
+    # Example Task Evaluation:
+    evaluate=("url_match", "https://google.com/search?q=expected")
+    ```
+*   **`page_contains(texts: list[str])`** (alias `contains_text`): Checks if *all* strings in `texts` are present in `page.content()`. Returns `1.0` if all texts are found, `0.0` otherwise.
+    ```python
+    # Example Task Evaluation:
+    evaluate=("page_contains", ["Search Results", "About 1,000,000 results"])
+    ```
+*   **`sheet_contains(texts: list[str])`**: Custom function for Google Sheets. Returns `1.0` if any text is found, `0.0` otherwise.
+    ```python
+    # Example Task Evaluation:
+    evaluate=("sheet_contains", ["Expected value in cell A1"])
+    ```
+*   **`cookie_exists(cookie_names: list[str])`**: Checks if all cookies in `cookie_names` exist in `context.cookies()`. Returns `1.0` if all exist, `0.0` otherwise.
+    ```python
+    # Example Task Evaluation:
+    evaluate=("cookie_exists", ["session_id", "user_pref"])
+    ```
+*   **`cookie_match(name_value_pairs: list[str])`**: Checks if cookies exist *and* match expected values. `name_value_pairs` format: `[name1, value1, name2, value2, ...]`. Returns `1.0` if all match, `0.0` otherwise.
+    ```python
+    # Example Task Evaluation:
+    evaluate=("cookie_match", ["user_id", "12345", "theme", "dark"])
+    ```
+
+Refer to [Task Evaluation Configuration](../concepts/task.mdx#evaluation-configuration) for more details. 

hud_python-0.2.2/docs/environments/hud-ubuntu.mdx

@@ -0,0 +1,55 @@
+# HUD Ubuntu Environment
+
+## Introduction
+
+The `hud-ubuntu` environment provides a remote Ubuntu OS instance with a graphical desktop, accessed via a VNC connection displayed in the browser. It's suitable for tasks requiring interaction with GUI applications, the file system, or running specific software within a Linux desktop environment.
+
+## Setup
+
+The environment setup simply launches the Ubuntu desktop session within the VNC viewer. 
+
+*Specific pre-launch setup functions (e.g., pre-installing packages, setting environment variables) are planned for future releases.*
+
+Currently, any necessary setup (like installing software or creating files) must be performed by the agent *after* the environment starts, using standard interaction actions.
+
+Refer to [Task Setup Configuration](../concepts/task.mdx#setup-configuration) for the general concept of how setup steps *could* be defined in a Task, although they are not currently implemented for this specific environment.
+
+## Step Interaction
+
+Agents interact with the Ubuntu desktop environment by sending a list of [CLA Actions](../advanced/cla-details.mdx) to `env.step()`. An [Adapter](../concepts/adapter.mdx) typically handles the conversion from the agent model's output to the CLA format.
+
+Available CLA actions for interacting with the graphical desktop:
+
+**Keyboard Actions:**
+*   [`TypeAction`](../advanced/cla-details.mdx#keyboard-actions): Simulates typing text into the focused application or window element.
+    ```python
+    # Example: Typing into a text editor
+    TypeAction(text="Hello, Ubuntu!")
+    ```
+*   [`PressAction`](../advanced/cla-details.mdx#keyboard-actions): For sending hotkeys (e.g., `Ctrl+C`, `Alt+F4`).
+*   [`KeyDownAction` / `KeyUpAction`](../advanced/cla-details.mdx#keyboard-actions): For holding/releasing modifier keys (e.g., holding Shift while clicking).
+
+**Mouse Actions:**
+*   [`ClickAction`](../advanced/cla-details.mdx#mouse-actions): To click on GUI elements (buttons, icons, menus, etc.).
+*   [`MoveAction`](../advanced/cla-details.mdx#mouse-actions): To move the mouse cursor to specific coordinates or elements.
+*   [`ScrollAction`](../advanced/cla-details.mdx#mouse-actions): To scroll within windows or applications.
+*   [`DragAction`](../advanced/cla-details.mdx#mouse-actions): To perform drag-and-drop operations.
+
+**Control & Response Actions:**
+*   [`WaitAction`](../advanced/cla-details.mdx#control-actions): To introduce pauses if needed.
+*   [`ResponseAction`](../advanced/cla-details.mdx#response-actions): Used by the agent to submit its final answer or result text.
+
+**Other Actions (less common for direct agent use):**
+*   [`ScreenshotFetch`](../advanced/cla-details.mdx#fetch-actions-get-information): Usually handled internally by the environment/agent loop.
+*   [`PositionFetch`](../advanced/cla-details.mdx#fetch-actions-get-information): Usually handled internally.
+*   [`CustomAction`](../advanced/cla-details.mdx#custom-actions): For potential future custom environment extensions.
+
+*See [CLA Action Details](../advanced/cla-details.mdx) for the full specification of each action and its parameters.*
+
+## Evaluate
+
+*Specific evaluation functions for `hud-ubuntu` (e.g., checking file content, application state, process status) are planned for future releases.*
+
+Currently, evaluation often relies on the agent submitting a final answer via `ResponseAction`, which can then be checked using generic QA evaluators defined in the [Task](../concepts/task.mdx) (like `response_includes`, `response_matches`). Alternatively, evaluation might involve visually inspecting the final state via the VNC connection or checking logs if the agent was tasked with producing specific output.
+
+Refer to [Task Evaluation Configuration](../concepts/task.mdx#evaluation-configuration) for the general concept of defining evaluation steps.