cua-computer 0.4.4__tar.gz → 0.4.6__tar.gz

{cua_computer-0.4.4 → cua_computer-0.4.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: cua-computer
-Version: 0.4.4
+Version: 0.4.6
 Summary: Computer-Use Interface (CUI) framework powering Cua
 Author-Email: TryCua <gh@trycua.com>
 Requires-Python: >=3.11
@@ -26,8 +26,8 @@ Description-Content-Type: text/markdown
 <h1>
   <div class="image-wrapper" style="display: inline-block;">
     <picture>
-      <source media="(prefers-color-scheme: dark)" alt="logo" height="150" srcset="../../img/logo_white.png" style="display: block; margin: auto;">
-      <source media="(prefers-color-scheme: light)" alt="logo" height="150" srcset="../../img/logo_black.png" style="display: block; margin: auto;">
+      <source media="(prefers-color-scheme: dark)" alt="logo" height="150" srcset="https://raw.githubusercontent.com/trycua/cua/main/img/logo_white.png" style="display: block; margin: auto;">
+      <source media="(prefers-color-scheme: light)" alt="logo" height="150" srcset="https://raw.githubusercontent.com/trycua/cua/main/img/logo_black.png" style="display: block; margin: auto;">
       <img alt="Shows my svg">
     </picture>
   </div>
@@ -44,7 +44,7 @@ Description-Content-Type: text/markdown
 ### Get started with Computer
 
 <div align="center">
-    <img src="../../img/computer.png"/>
+    <img src="https://raw.githubusercontent.com/trycua/cua/main/img/computer.png"/>
 </div>
 
 ```python
@@ -87,82 +87,11 @@ The `cua-computer` PyPi package pulls automatically the latest executable versio
 
 Refer to this notebook for a step-by-step guide on how to use the Computer-Use Interface (CUI):
 
-- [Computer-Use Interface (CUI)](../../notebooks/computer_nb.ipynb)
+- [Computer-Use Interface (CUI)](https://github.com/trycua/cua/blob/main/notebooks/computer_nb.ipynb)
 
-## Using the Gradio Computer UI
-
-The computer module includes a Gradio UI for creating and sharing demonstration data. We make it easy for people to build community datasets for better computer use models with an upload to Huggingface feature.
-
-```bash
-# Install with UI support
-pip install "cua-computer[ui]"
-```
-
-> **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI.
-
-### Building and Sharing Demonstrations with Huggingface
-
-Follow these steps to contribute your own demonstrations:
-
-#### 1. Set up Huggingface Access
-
-Set your HF_TOKEN in a .env file or in your environment variables:
-
-```bash
-# In .env file
-HF_TOKEN=your_huggingface_token
-```
-
-#### 2. Launch the Computer UI
-
-```python
-# launch_ui.py
-from computer.ui.gradio.app import create_gradio_ui
-from dotenv import load_dotenv
-load_dotenv('.env')
-
-app = create_gradio_ui()
-app.launch(share=False)
-```
-
-For examples, see [Computer UI Examples](../../examples/computer_ui_examples.py)
-
-#### 3. Record Your Tasks
-
-<details open>
-<summary>View demonstration video</summary>
-<video src="https://github.com/user-attachments/assets/de3c3477-62fe-413c-998d-4063e48de176" controls width="600"></video>
-</details>
-
-Record yourself performing various computer tasks using the UI.
-
-#### 4. Save Your Demonstrations
-
-<details open>
-<summary>View demonstration video</summary>
-<video src="https://github.com/user-attachments/assets/5ad1df37-026a-457f-8b49-922ae805faef" controls width="600"></video>
-</details>
-
-Save each task by picking a descriptive name and adding relevant tags (e.g., "office", "web-browsing", "coding").
-
-#### 5. Record Additional Demonstrations
-
-Repeat steps 3 and 4 until you have a good amount of demonstrations covering different tasks and scenarios.
-
-#### 6. Upload to Huggingface
-
-<details open>
-<summary>View demonstration video</summary>
-<video src="https://github.com/user-attachments/assets/c586d460-3877-4b5f-a736-3248886d2134" controls width="600"></video>
-</details>
-
-Upload your dataset to Huggingface by:
-- Naming it as `{your_username}/{dataset_name}`
-- Choosing public or private visibility
-- Optionally selecting specific tags to upload only tasks with certain tags
-
-#### Examples and Resources
-
-- Example Dataset: [ddupont/test-dataset](https://huggingface.co/datasets/ddupont/test-dataset)
-- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua)
+## Docs
 
+- [Computers](https://trycua.com/docs/computer-sdk/computers)
+- [Commands](https://trycua.com/docs/computer-sdk/commands)
+- [Computer UI](https://trycua.com/docs/computer-sdk/computer-ui)
+- [Sandboxed Python](https://trycua.com/docs/computer-sdk/sandboxed-python)

cua_computer-0.4.6/README.md

@@ -0,0 +1,73 @@
+<div align="center">
+<h1>
+  <div class="image-wrapper" style="display: inline-block;">
+    <picture>
+      <source media="(prefers-color-scheme: dark)" alt="logo" height="150" srcset="https://raw.githubusercontent.com/trycua/cua/main/img/logo_white.png" style="display: block; margin: auto;">
+      <source media="(prefers-color-scheme: light)" alt="logo" height="150" srcset="https://raw.githubusercontent.com/trycua/cua/main/img/logo_black.png" style="display: block; margin: auto;">
+      <img alt="Shows my svg">
+    </picture>
+  </div>
+
+  [![Python](https://img.shields.io/badge/Python-333333?logo=python&logoColor=white&labelColor=333333)](#)
+  [![macOS](https://img.shields.io/badge/macOS-000000?logo=apple&logoColor=F0F0F0)](#)
+  [![Discord](https://img.shields.io/badge/Discord-%235865F2.svg?&logo=discord&logoColor=white)](https://discord.com/invite/mVnXXpdE85)
+  [![PyPI](https://img.shields.io/pypi/v/cua-computer?color=333333)](https://pypi.org/project/cua-computer/)
+</h1>
+</div>
+
+**cua-computer** is a Computer-Use Interface (CUI) framework powering Cua for interacting with local macOS and Linux sandboxes, PyAutoGUI-compatible, and pluggable with any AI agent systems (Cua, Langchain, CrewAI, AutoGen). Computer relies on [Lume](https://github.com/trycua/lume) for creating and managing sandbox environments.
+
+### Get started with Computer
+
+<div align="center">
+    <img src="https://raw.githubusercontent.com/trycua/cua/main/img/computer.png"/>
+</div>
+
+```python
+from computer import Computer
+
+computer = Computer(os_type="macos", display="1024x768", memory="8GB", cpu="4")
+try:
+    await computer.run()
+    
+    screenshot = await computer.interface.screenshot()
+    with open("screenshot.png", "wb") as f:
+        f.write(screenshot)
+    
+    await computer.interface.move_cursor(100, 100)
+    await computer.interface.left_click()
+    await computer.interface.right_click(300, 300)
+    await computer.interface.double_click(400, 400)
+
+    await computer.interface.type("Hello, World!")
+    await computer.interface.press_key("enter")
+
+    await computer.interface.set_clipboard("Test clipboard")
+    content = await computer.interface.copy_to_clipboard()
+    print(f"Clipboard content: {content}")
+finally:
+    await computer.stop()
+```
+
+## Install
+
+To install the Computer-Use Interface (CUI):
+
+```bash
+pip install "cua-computer[all]"
+```
+
+The `cua-computer` PyPi package pulls automatically the latest executable version of Lume through [pylume](https://github.com/trycua/pylume).
+
+## Run
+
+Refer to this notebook for a step-by-step guide on how to use the Computer-Use Interface (CUI):
+
+- [Computer-Use Interface (CUI)](https://github.com/trycua/cua/blob/main/notebooks/computer_nb.ipynb)
+
+## Docs
+
+- [Computers](https://trycua.com/docs/computer-sdk/computers)
+- [Commands](https://trycua.com/docs/computer-sdk/commands)
+- [Computer UI](https://trycua.com/docs/computer-sdk/computer-ui)
+- [Sandboxed Python](https://trycua.com/docs/computer-sdk/sandboxed-python)

{cua_computer-0.4.4 → cua_computer-0.4.6}/computer/computer.py

@@ -154,8 +154,8 @@ class Computer:
         self.interface_logger = Logger("computer.interface", verbosity)
 
         if not use_host_computer_server:
-            if ":" not in image or len(image.split(":")) != 2:
-                raise ValueError("Image must be in the format <image_name>:<tag>")
+            if ":" not in image:
+                image = f"{image}:latest"
 
             if not name:
                 # Normalize the name to be used for the VM

cua_computer-0.4.6/computer/diorama_computer.py

@@ -0,0 +1,243 @@
+import asyncio
+from .interface.models import KeyType, Key
+
+class DioramaComputer:
+    """
+    A Computer-compatible proxy for Diorama that sends commands over the ComputerInterface.
+    """
+    def __init__(self, computer, apps):
+        """
+        Initialize the DioramaComputer with a computer instance and list of apps.
+        
+        Args:
+            computer: The computer instance to proxy commands through
+            apps: List of applications available in the diorama environment
+        """
+        self.computer = computer
+        self.apps = apps
+        self.interface = DioramaComputerInterface(computer, apps)
+        self._initialized = False
+
+    async def __aenter__(self):
+        """
+        Async context manager entry point.
+        
+        Returns:
+            self: The DioramaComputer instance
+        """
+        self._initialized = True
+        return self
+
+    async def run(self):
+        """
+        Initialize and run the DioramaComputer if not already initialized.
+        
+        Returns:
+            self: The DioramaComputer instance
+        """
+        if not self._initialized:
+            await self.__aenter__()
+        return self
+
+class DioramaComputerInterface:
+    """
+    Diorama Interface proxy that sends diorama_cmds via the Computer's interface.
+    """
+    def __init__(self, computer, apps):
+        """
+        Initialize the DioramaComputerInterface.
+        
+        Args:
+            computer: The computer instance to send commands through
+            apps: List of applications available in the diorama environment
+        """
+        self.computer = computer
+        self.apps = apps
+        self._scene_size = None
+
+    async def _send_cmd(self, action, arguments=None):
+        """
+        Send a command to the diorama interface through the computer.
+        
+        Args:
+            action (str): The action/command to execute
+            arguments (dict, optional): Additional arguments for the command
+            
+        Returns:
+            The result from the diorama command execution
+            
+        Raises:
+            RuntimeError: If the computer interface is not initialized or command fails
+        """
+        arguments = arguments or {}
+        arguments = {"app_list": self.apps, **arguments}
+        # Use the computer's interface (must be initialized)
+        iface = getattr(self.computer, "_interface", None)
+        if iface is None:
+            raise RuntimeError("Computer interface not initialized. Call run() first.")
+        result = await iface.diorama_cmd(action, arguments)
+        if not result.get("success"):
+            raise RuntimeError(f"Diorama command failed: {result.get('error')}\n{result.get('trace')}")
+        return result.get("result")
+
+    async def screenshot(self, as_bytes=True):
+        """
+        Take a screenshot of the diorama scene.
+        
+        Args:
+            as_bytes (bool): If True, return image as bytes; if False, return PIL Image object
+            
+        Returns:
+            bytes or PIL.Image: Screenshot data in the requested format
+        """
+        from PIL import Image
+        import base64
+        result = await self._send_cmd("screenshot")
+        # assume result is a b64 string of an image
+        img_bytes = base64.b64decode(result)
+        import io
+        img = Image.open(io.BytesIO(img_bytes))
+        self._scene_size = img.size
+        return img_bytes if as_bytes else img
+
+    async def get_screen_size(self):
+        """
+        Get the dimensions of the diorama scene.
+        
+        Returns:
+            dict: Dictionary containing 'width' and 'height' keys with pixel dimensions
+        """
+        if not self._scene_size:
+            await self.screenshot(as_bytes=False)
+        return {"width": self._scene_size[0], "height": self._scene_size[1]}
+
+    async def move_cursor(self, x, y):
+        """
+        Move the cursor to the specified coordinates.
+        
+        Args:
+            x (int): X coordinate to move cursor to
+            y (int): Y coordinate to move cursor to
+        """
+        await self._send_cmd("move_cursor", {"x": x, "y": y})
+
+    async def left_click(self, x=None, y=None):
+        """
+        Perform a left mouse click at the specified coordinates or current cursor position.
+        
+        Args:
+            x (int, optional): X coordinate to click at. If None, clicks at current cursor position
+            y (int, optional): Y coordinate to click at. If None, clicks at current cursor position
+        """
+        await self._send_cmd("left_click", {"x": x, "y": y})
+
+    async def right_click(self, x=None, y=None):
+        """
+        Perform a right mouse click at the specified coordinates or current cursor position.
+        
+        Args:
+            x (int, optional): X coordinate to click at. If None, clicks at current cursor position
+            y (int, optional): Y coordinate to click at. If None, clicks at current cursor position
+        """
+        await self._send_cmd("right_click", {"x": x, "y": y})
+
+    async def double_click(self, x=None, y=None):
+        """
+        Perform a double mouse click at the specified coordinates or current cursor position.
+        
+        Args:
+            x (int, optional): X coordinate to double-click at. If None, clicks at current cursor position
+            y (int, optional): Y coordinate to double-click at. If None, clicks at current cursor position
+        """
+        await self._send_cmd("double_click", {"x": x, "y": y})
+
+    async def scroll_up(self, clicks=1):
+        """
+        Scroll up by the specified number of clicks.
+        
+        Args:
+            clicks (int): Number of scroll clicks to perform upward. Defaults to 1
+        """
+        await self._send_cmd("scroll_up", {"clicks": clicks})
+
+    async def scroll_down(self, clicks=1):
+        """
+        Scroll down by the specified number of clicks.
+        
+        Args:
+            clicks (int): Number of scroll clicks to perform downward. Defaults to 1
+        """
+        await self._send_cmd("scroll_down", {"clicks": clicks})
+
+    async def drag_to(self, x, y, duration=0.5):
+        """
+        Drag from the current cursor position to the specified coordinates.
+        
+        Args:
+            x (int): X coordinate to drag to
+            y (int): Y coordinate to drag to
+            duration (float): Duration of the drag operation in seconds. Defaults to 0.5
+        """
+        await self._send_cmd("drag_to", {"x": x, "y": y, "duration": duration})
+
+    async def get_cursor_position(self):
+        """
+        Get the current cursor position.
+        
+        Returns:
+            dict: Dictionary containing the current cursor coordinates
+        """
+        return await self._send_cmd("get_cursor_position")
+
+    async def type_text(self, text):
+        """
+        Type the specified text at the current cursor position.
+        
+        Args:
+            text (str): The text to type
+        """
+        await self._send_cmd("type_text", {"text": text})
+
+    async def press_key(self, key):
+        """
+        Press a single key.
+        
+        Args:
+            key: The key to press
+        """
+        await self._send_cmd("press_key", {"key": key})
+
+    async def hotkey(self, *keys):
+        """
+        Press multiple keys simultaneously as a hotkey combination.
+        
+        Args:
+            *keys: Variable number of keys to press together. Can be Key enum instances or strings
+            
+        Raises:
+            ValueError: If any key is not a Key enum or string type
+        """
+        actual_keys = []
+        for key in keys:
+            if isinstance(key, Key):
+                actual_keys.append(key.value)
+            elif isinstance(key, str):
+                # Try to convert to enum if it matches a known key
+                key_or_enum = Key.from_string(key)
+                actual_keys.append(key_or_enum.value if isinstance(key_or_enum, Key) else key_or_enum)
+            else:
+                raise ValueError(f"Invalid key type: {type(key)}. Must be Key enum or string.")
+        await self._send_cmd("hotkey", {"keys": actual_keys})
+
+    async def to_screen_coordinates(self, x, y):
+        """
+        Convert coordinates to screen coordinates.
+        
+        Args:
+            x (int): X coordinate to convert
+            y (int): Y coordinate to convert
+            
+        Returns:
+            dict: Dictionary containing the converted screen coordinates
+        """
+        return await self._send_cmd("to_screen_coordinates", {"x": x, "y": y})

{cua_computer-0.4.4 → cua_computer-0.4.6}/computer/providers/winsandbox/provider.py

@@ -5,6 +5,7 @@ import asyncio
 import logging
 import time
 from typing import Dict, Any, Optional, List
+from pathlib import Path
 
 from ..base import BaseVMProvider, VMProviderType
 
@@ -242,8 +243,15 @@ class WinSandboxProvider(BaseVMProvider):
             
             networking = run_opts.get("networking", self.networking)
             
-            # Create folder mappers if shared directories are specified
+            # Create folder mappers; always map a persistent venv directory on host for caching packages
             folder_mappers = []
+            # Ensure host side persistent venv directory exists (Path.home()/wsb_venv)
+            host_wsb_env = Path.home() / ".cua" / "wsb_cache"
+            try:
+                host_wsb_env.mkdir(parents=True, exist_ok=True)
+            except Exception:
+                # If cannot create, continue without persistent mapping
+                host_wsb_env = None
             shared_directories = run_opts.get("shared_directories", [])
             for shared_dir in shared_directories:
                 if isinstance(shared_dir, dict):
@@ -255,6 +263,15 @@ class WinSandboxProvider(BaseVMProvider):
                     
                 if host_path and os.path.exists(host_path):
                     folder_mappers.append(winsandbox.FolderMapper(host_path))
+
+            # Add mapping for the persistent venv directory (read/write) so it appears in Sandbox Desktop
+            if host_wsb_env is not None and host_wsb_env.exists():
+                try:
+                    folder_mappers.append(
+                        winsandbox.FolderMapper(str(host_wsb_env), read_only=False)
+                    )
+                except Exception as e:
+                    self.logger.warning(f"Failed to map host winsandbox_venv: {e}")
             
             self.logger.info(f"Creating Windows Sandbox: {name}")
             self.logger.info(f"Memory: {memory_mb}MB, Networking: {networking}")
@@ -290,8 +307,10 @@ class WinSandboxProvider(BaseVMProvider):
             
             self.logger.info(f"Windows Sandbox {name} created successfully")
             
+            venv_exists = (host_wsb_env / "venv" / "Lib" / "site-packages" / "computer_server").exists() if host_wsb_env else False
+
             # Setup the computer server in the sandbox
-            await self._setup_computer_server(sandbox, name)
+            await self._setup_computer_server(sandbox, name, wait_for_venv=(not venv_exists))
             
             return {
                 "success": True,
@@ -423,7 +442,7 @@ class WinSandboxProvider(BaseVMProvider):
             if total_attempts % 10 == 0:
                 self.logger.info(f"Still waiting for Windows Sandbox {name} IP after {total_attempts} attempts...")
     
-    async def _setup_computer_server(self, sandbox, name: str, visible: bool = False):
+    async def _setup_computer_server(self, sandbox, name: str, visible: bool = False, wait_for_venv: bool = True):
         """Setup the computer server in the Windows Sandbox using RPyC.
         
         Args:
@@ -471,10 +490,12 @@ class WinSandboxProvider(BaseVMProvider):
                 creationflags=creation_flags,
                 shell=False
             )
-            
-            # # Sleep for 30 seconds
-            # await asyncio.sleep(30)
 
+            if wait_for_venv:
+                print("Waiting for venv to be created for the first time setup of Windows Sandbox...")
+                print("This may take a minute...")
+                await asyncio.sleep(120)
+            
             ip = await self.get_ip(name)
             self.logger.info(f"Sandbox IP: {ip}")
             self.logger.info(f"Setup script started in background in sandbox {name} with PID: {process.pid}")

{cua_computer-0.4.4 → cua_computer-0.4.6}/computer/providers/winsandbox/setup_script.ps1

@@ -79,23 +79,48 @@ try {
     $pythonVersion = & $pythonExe --version 2>&1
     Write-Host "Python version: $pythonVersion"
 
-    # Step 2: Install cua-computer-server directly
-    Write-Host "Step 2: Installing cua-computer-server..."
+    # Step 2: Create a dedicated virtual environment in mapped Desktop folder (persistent)
+    Write-Host "Step 2: Creating virtual environment (if needed)..."
+    $cachePath = "C:\Users\WDAGUtilityAccount\Desktop\wsb_cache"
+    $venvPath = "C:\Users\WDAGUtilityAccount\Desktop\wsb_cache\venv"
+    if (!(Test-Path $venvPath)) {
+        Write-Host "Creating venv at: $venvPath"
+        & $pythonExe -m venv $venvPath
+    } else {
+        Write-Host "Venv already exists at: $venvPath"
+    }
+    # Hide the folder to keep Desktop clean
+    try {
+        $item = Get-Item $cachePath -ErrorAction SilentlyContinue
+        if ($item) {
+            if (-not ($item.Attributes -band [IO.FileAttributes]::Hidden)) {
+                $item.Attributes = $item.Attributes -bor [IO.FileAttributes]::Hidden
+            }
+        }
+    } catch { }
+    $venvPython = Join-Path $venvPath "Scripts\python.exe"
+    if (!(Test-Path $venvPython)) {
+        throw "Virtual environment Python not found at $venvPython"
+    }
+    Write-Host "Using venv Python: $venvPython"
+
+    # Step 3: Install cua-computer-server into the venv
+    Write-Host "Step 3: Installing cua-computer-server..."
     
     Write-Host "Upgrading pip..."
-    & $pythonExe -m pip install --upgrade pip --quiet
+    & $venvPython -m pip install --upgrade pip --quiet
     
     Write-Host "Installing cua-computer-server..."
-    & $pythonExe -m pip install cua-computer-server --quiet
+    & $venvPython -m pip install cua-computer-server
     
     Write-Host "cua-computer-server installation completed."
 
-    # Step 3: Start computer server in background
-    Write-Host "Step 3: Starting computer server in background..."
-    Write-Host "Starting computer server with: $pythonExe"
+    # Step 4: Start computer server in background using the venv Python
+    Write-Host "Step 4: Starting computer server in background..."
+    Write-Host "Starting computer server with: $venvPython"
     
     # Start the computer server in the background
-    $serverProcess = Start-Process -FilePath $pythonExe -ArgumentList "-m", "computer_server.main" -WindowStyle Hidden -PassThru
+    $serverProcess = Start-Process -FilePath $venvPython -ArgumentList "-m", "computer_server.main" -WindowStyle Hidden -PassThru
     Write-Host "Computer server started in background with PID: $($serverProcess.Id)"
     
     # Give it a moment to start

{cua_computer-0.4.4 → cua_computer-0.4.6}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "pdm.backend"
 
 [project]
 name = "cua-computer"
-version = "0.4.4"
+version = "0.4.6"
 description = "Computer-Use Interface (CUI) framework powering Cua"
 readme = "README.md"
 authors = [
@@ -57,7 +57,7 @@ target-version = [
 
 [tool.ruff]
 line-length = 100
-target-version = "0.4.4"
+target-version = "0.4.6"
 select = [
     "E",
     "F",
@@ -71,7 +71,7 @@ docstring-code-format = true
 
 [tool.mypy]
 strict = true
-python_version = "0.4.4"
+python_version = "0.4.6"
 ignore_missing_imports = true
 disallow_untyped_defs = true
 check_untyped_defs = true

cua_computer-0.4.4/README.md

@@ -1,144 +0,0 @@
-<div align="center">
-<h1>
-  <div class="image-wrapper" style="display: inline-block;">
-    <picture>
-      <source media="(prefers-color-scheme: dark)" alt="logo" height="150" srcset="../../img/logo_white.png" style="display: block; margin: auto;">
-      <source media="(prefers-color-scheme: light)" alt="logo" height="150" srcset="../../img/logo_black.png" style="display: block; margin: auto;">
-      <img alt="Shows my svg">
-    </picture>
-  </div>
-
-  [![Python](https://img.shields.io/badge/Python-333333?logo=python&logoColor=white&labelColor=333333)](#)
-  [![macOS](https://img.shields.io/badge/macOS-000000?logo=apple&logoColor=F0F0F0)](#)
-  [![Discord](https://img.shields.io/badge/Discord-%235865F2.svg?&logo=discord&logoColor=white)](https://discord.com/invite/mVnXXpdE85)
-  [![PyPI](https://img.shields.io/pypi/v/cua-computer?color=333333)](https://pypi.org/project/cua-computer/)
-</h1>
-</div>
-
-**cua-computer** is a Computer-Use Interface (CUI) framework powering Cua for interacting with local macOS and Linux sandboxes, PyAutoGUI-compatible, and pluggable with any AI agent systems (Cua, Langchain, CrewAI, AutoGen). Computer relies on [Lume](https://github.com/trycua/lume) for creating and managing sandbox environments.
-
-### Get started with Computer
-
-<div align="center">
-    <img src="../../img/computer.png"/>
-</div>
-
-```python
-from computer import Computer
-
-computer = Computer(os_type="macos", display="1024x768", memory="8GB", cpu="4")
-try:
-    await computer.run()
-    
-    screenshot = await computer.interface.screenshot()
-    with open("screenshot.png", "wb") as f:
-        f.write(screenshot)
-    
-    await computer.interface.move_cursor(100, 100)
-    await computer.interface.left_click()
-    await computer.interface.right_click(300, 300)
-    await computer.interface.double_click(400, 400)
-
-    await computer.interface.type("Hello, World!")
-    await computer.interface.press_key("enter")
-
-    await computer.interface.set_clipboard("Test clipboard")
-    content = await computer.interface.copy_to_clipboard()
-    print(f"Clipboard content: {content}")
-finally:
-    await computer.stop()
-```
-
-## Install
-
-To install the Computer-Use Interface (CUI):
-
-```bash
-pip install "cua-computer[all]"
-```
-
-The `cua-computer` PyPi package pulls automatically the latest executable version of Lume through [pylume](https://github.com/trycua/pylume).
-
-## Run
-
-Refer to this notebook for a step-by-step guide on how to use the Computer-Use Interface (CUI):
-
-- [Computer-Use Interface (CUI)](../../notebooks/computer_nb.ipynb)
-
-## Using the Gradio Computer UI
-
-The computer module includes a Gradio UI for creating and sharing demonstration data. We make it easy for people to build community datasets for better computer use models with an upload to Huggingface feature.
-
-```bash
-# Install with UI support
-pip install "cua-computer[ui]"
-```
-
-> **Note:** For precise control of the computer, we recommend using VNC or Screen Sharing instead of the Computer Gradio UI.
-
-### Building and Sharing Demonstrations with Huggingface
-
-Follow these steps to contribute your own demonstrations:
-
-#### 1. Set up Huggingface Access
-
-Set your HF_TOKEN in a .env file or in your environment variables:
-
-```bash
-# In .env file
-HF_TOKEN=your_huggingface_token
-```
-
-#### 2. Launch the Computer UI
-
-```python
-# launch_ui.py
-from computer.ui.gradio.app import create_gradio_ui
-from dotenv import load_dotenv
-load_dotenv('.env')
-
-app = create_gradio_ui()
-app.launch(share=False)
-```
-
-For examples, see [Computer UI Examples](../../examples/computer_ui_examples.py)
-
-#### 3. Record Your Tasks
-
-<details open>
-<summary>View demonstration video</summary>
-<video src="https://github.com/user-attachments/assets/de3c3477-62fe-413c-998d-4063e48de176" controls width="600"></video>
-</details>
-
-Record yourself performing various computer tasks using the UI.
-
-#### 4. Save Your Demonstrations
-
-<details open>
-<summary>View demonstration video</summary>
-<video src="https://github.com/user-attachments/assets/5ad1df37-026a-457f-8b49-922ae805faef" controls width="600"></video>
-</details>
-
-Save each task by picking a descriptive name and adding relevant tags (e.g., "office", "web-browsing", "coding").
-
-#### 5. Record Additional Demonstrations
-
-Repeat steps 3 and 4 until you have a good amount of demonstrations covering different tasks and scenarios.
-
-#### 6. Upload to Huggingface
-
-<details open>
-<summary>View demonstration video</summary>
-<video src="https://github.com/user-attachments/assets/c586d460-3877-4b5f-a736-3248886d2134" controls width="600"></video>
-</details>
-
-Upload your dataset to Huggingface by:
-- Naming it as `{your_username}/{dataset_name}`
-- Choosing public or private visibility
-- Optionally selecting specific tags to upload only tasks with certain tags
-
-#### Examples and Resources
-
-- Example Dataset: [ddupont/test-dataset](https://huggingface.co/datasets/ddupont/test-dataset)
-- Find Community Datasets: 🔍 [Browse CUA Datasets on Huggingface](https://huggingface.co/datasets?other=cua)
-

cua_computer-0.4.4/computer/diorama_computer.py

@@ -1,104 +0,0 @@
-import asyncio
-from .interface.models import KeyType, Key
-
-class DioramaComputer:
-    """
-    A Computer-compatible proxy for Diorama that sends commands over the ComputerInterface.
-    """
-    def __init__(self, computer, apps):
-        self.computer = computer
-        self.apps = apps
-        self.interface = DioramaComputerInterface(computer, apps)
-        self._initialized = False
-
-    async def __aenter__(self):
-        self._initialized = True
-        return self
-
-    async def run(self):
-        if not self._initialized:
-            await self.__aenter__()
-        return self
-
-class DioramaComputerInterface:
-    """
-    Diorama Interface proxy that sends diorama_cmds via the Computer's interface.
-    """
-    def __init__(self, computer, apps):
-        self.computer = computer
-        self.apps = apps
-        self._scene_size = None
-
-    async def _send_cmd(self, action, arguments=None):
-        arguments = arguments or {}
-        arguments = {"app_list": self.apps, **arguments}
-        # Use the computer's interface (must be initialized)
-        iface = getattr(self.computer, "_interface", None)
-        if iface is None:
-            raise RuntimeError("Computer interface not initialized. Call run() first.")
-        result = await iface.diorama_cmd(action, arguments)
-        if not result.get("success"):
-            raise RuntimeError(f"Diorama command failed: {result.get('error')}\n{result.get('trace')}")
-        return result.get("result")
-
-    async def screenshot(self, as_bytes=True):
-        from PIL import Image
-        import base64
-        result = await self._send_cmd("screenshot")
-        # assume result is a b64 string of an image
-        img_bytes = base64.b64decode(result)
-        import io
-        img = Image.open(io.BytesIO(img_bytes))
-        self._scene_size = img.size
-        return img_bytes if as_bytes else img
-
-    async def get_screen_size(self):
-        if not self._scene_size:
-            await self.screenshot(as_bytes=False)
-        return {"width": self._scene_size[0], "height": self._scene_size[1]}
-
-    async def move_cursor(self, x, y):
-        await self._send_cmd("move_cursor", {"x": x, "y": y})
-
-    async def left_click(self, x=None, y=None):
-        await self._send_cmd("left_click", {"x": x, "y": y})
-
-    async def right_click(self, x=None, y=None):
-        await self._send_cmd("right_click", {"x": x, "y": y})
-
-    async def double_click(self, x=None, y=None):
-        await self._send_cmd("double_click", {"x": x, "y": y})
-
-    async def scroll_up(self, clicks=1):
-        await self._send_cmd("scroll_up", {"clicks": clicks})
-
-    async def scroll_down(self, clicks=1):
-        await self._send_cmd("scroll_down", {"clicks": clicks})
-
-    async def drag_to(self, x, y, duration=0.5):
-        await self._send_cmd("drag_to", {"x": x, "y": y, "duration": duration})
-
-    async def get_cursor_position(self):
-        return await self._send_cmd("get_cursor_position")
-
-    async def type_text(self, text):
-        await self._send_cmd("type_text", {"text": text})
-
-    async def press_key(self, key):
-        await self._send_cmd("press_key", {"key": key})
-
-    async def hotkey(self, *keys):
-        actual_keys = []
-        for key in keys:
-            if isinstance(key, Key):
-                actual_keys.append(key.value)
-            elif isinstance(key, str):
-                # Try to convert to enum if it matches a known key
-                key_or_enum = Key.from_string(key)
-                actual_keys.append(key_or_enum.value if isinstance(key_or_enum, Key) else key_or_enum)
-            else:
-                raise ValueError(f"Invalid key type: {type(key)}. Must be Key enum or string.")
-        await self._send_cmd("hotkey", {"keys": actual_keys})
-
-    async def to_screen_coordinates(self, x, y):
-        return await self._send_cmd("to_screen_coordinates", {"x": x, "y": y})