daytona-use-computer 0.0.48__py3-none-any.whl

daytona_use_computer-0.0.48.dist-info/METADATA

@@ -0,0 +1,197 @@
+Metadata-Version: 2.5
+Name: daytona-use-computer
+Version: 0.0.48
+Summary: Python SDK for use.computer macOS sandboxes
+Project-URL: Homepage, https://use.computer
+Project-URL: Documentation, https://api.use.computer/docs
+Project-URL: Repository, https://github.com/daytona/use-computer-sdk
+Author: use.computer
+Keywords: automation,computer-use,macos,sandbox,vnc
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: httpx>=0.27
+Provides-Extra: agents
+Requires-Dist: anthropic>=0.86; extra == 'agents'
+Requires-Dist: google-genai>=1; extra == 'agents'
+Requires-Dist: litellm>=1; extra == 'agents'
+Requires-Dist: openai>=1; extra == 'agents'
+Requires-Dist: pillow>=10; extra == 'agents'
+Requires-Dist: tinker-cookbook>=0.1.0; (python_full_version >= '3.11') and extra == 'agents'
+Requires-Dist: tinker>=0.14.0; (python_full_version >= '3.11') and extra == 'agents'
+Description-Content-Type: text/markdown
+
+# use-computer Python SDK (`daytona-use-computer`)
+
+use.computer gives you macOS sandboxes: VMs on dedicated Apple M4 Mac minis that you reserve for 24 hours or more, up to 2 VMs at a time per Mac.
+
+```bash
+pip install daytona-use-computer
+export USE_COMPUTER_API_KEY=uc_live_...
+```
+
+Optional agent integrations are installed explicitly:
+
+```bash
+pip install "daytona-use-computer[agents]"        # computer-use agents and provider SDKs
+```
+
+Base installs only the SDK client and `httpx`. The `agents` extra installs the agent runtime plus model-provider dependencies (Anthropic, OpenAI, Gemini, LiteLLM).
+
+## Quickstart
+
+Flow: sign up → $100 starter credit → reserve a Mac mini (dashboard, or `client.reserve(hours=24)` in the SDK) → `create()` a macOS sandbox → drive it (mouse, keyboard, screenshot, exec, files, recording, UI tree, VNC) → `delete()`. Reservations cost $1.91/hour per Mac; the starter credit pays for them.
+
+```python
+from use_computer import Computer
+
+client = Computer()
+
+# 1. Reserve one M4 Mac Mini for 24 hours
+reservation = client.reserve(hours=24, mac_model="m4")
+
+# 2. Launch a macOS sandbox on the reserved Mac
+with client.create(reservation_id=reservation.id) as mac:
+    # 3. Drive the macOS sandbox
+    mac.exec("open -a Safari")
+    mac.keyboard.type("hello from use.computer")
+    mac.mouse.click(500, 500)
+    png = mac.screenshot.take_full_screen()
+    print("Sandbox:", mac.sandbox_id)
+    # Open this sandbox's viewer from the use.computer dashboard.
+```
+
+Never print or share `vnc_url`: it contains your account API key. Open the viewer from the dashboard instead.
+
+The optional `mac_model` picks the Mac: `"m4"` (the default; 10 vCPU and 16 GiB RAM) or `"m4-pro"` (12 or more vCPU and 24 GiB RAM). `reservation.mac_model` reports it.
+
+Each `create()` can set `cpu`, `memory_gib`, and `disk_gib`. Omitted fields use the Mac's warm VM size, which is 5 vCPU and 8 GiB RAM today and starts right away. Any other size boots a new VM, which can take up to 5 minutes, so `create()` waits up to 10 minutes when you set any of them. A Mac runs at most 2 sandboxes at a time, and their sizes must fit in the Mac. The sandbox reports `cpu`, `memory_gib`, `disk_gib`, and `boot` (`"warm"` or `"cold"`):
+
+```python
+with client.create(reservation_id=reservation.id, cpu=8, memory_gib=12) as mac:
+    print(mac.cpu, mac.memory_gib, mac.boot)
+```
+
+If the size does not fit, `create()` raises `SandboxResourcesError`. Its `code` is `invalid_resources`, `resources_exceed_mac`, `sandbox_limit_reached`, or `insufficient_capacity`, and its `message` explains the refusal. `AsyncComputer` accepts the same options. Check `Computer().platforms(reservation_id=...)["macos"]["capacity"]` for the selected reservation's `max` and `used` counts.
+
+The older `vm_layout` reservation option still works but is deprecated; the SDK sends it only when you pass it. `"split"` (the server default) allows two sandboxes, and `"whole"` allows one sandbox at a time.
+
+When `ephemeral` is omitted, the server default is the reservation lifetime. Explicit `ephemeral=True` opts into destruction after 2 idle minutes; call `sandbox.start_keepalive(interval=30)` during long model-think periods. Explicit `ephemeral=False` disables idle destruction. Context managers still delete their sandbox on exit.
+
+This default applies to the source release documented here. Published Python 0.0.46
+still defaults to `ephemeral=True`; pass `ephemeral=False` explicitly on that
+version. The next tagged release includes the omitted-field default above.
+
+## Computer actions
+
+Coordinates use screenshot pixels. `mouse.click(x, y, button="middle")` selects the
+middle button; `click_count=3` sends one native triple-click sequence.
+`mouse.down()` / `mouse.up()` hold and release the left button at the current
+cursor, with optional `button`, `x`, and `y`. `keyboard.hold("shift", 0.5)` holds
+keys for seconds. `mouse.drag(..., path=[{"x": 10, "y": 20}, ...])` preserves all
+path points. `mouse.scroll(x, y, scroll_x=40, scroll_y=-80)` sends both pixel
+deltas (positive right/down); the direction/amount form remains available.
+The async methods have the same arguments.
+
+Provider mappings are recorded in
+[`action_manifest.json`](use_computer/agents/action_manifest.json).
+Anthropic tool pixels are scaled to sandbox pixels; Gemini uses normalized
+0..999 coordinates. Control and Command remain distinct. Invalid actions and
+gateway errors are not reported as executed.
+
+Agent logs separate action execution from `settle+screenshot`: the latter
+includes a deliberate two-second settle, screenshot requests, and any recovery.
+These are not model inference timings or raw screenshot latency.
+
+`exec(command, timeout=120)` uses the gateway's non-login `/bin/zsh -c`;
+`exec_ax` uses `/bin/sh` through the desktop server. The timeout is seconds
+and is sent to the gateway and used as the HTTP timeout. For a 200-second
+command, use `timeout=300`. Results retain separate stdout, stderr, and
+`return_code`; a nonzero process exit is not an HTTP transport failure.
+
+## Application automation permissions
+
+macOS asks for consent when a process sends Apple events to another application
+through AppleScript or `osascript`. That consent dialog cannot be answered inside
+the sandbox. The gateway approves every installed application when creating a
+sandbox or restoring a snapshot. After installing another app, call
+`sandbox.permissions.allow_automation()` to approve it before scripting it:
+
+```python
+# Inside an async macOS sandbox session:
+result = await sandbox.exec("brew install --cask firefox")
+if result.return_code != 0:
+    raise RuntimeError(result.stderr)
+approval = await sandbox.permissions.allow_automation(
+    bundle_ids=["org.mozilla.firefox"]
+)
+print(approval.success, approval.targets, approval.changed)
+
+# Alternatively, approve all currently installed applications:
+approval = await sandbox.permissions.allow_automation(installed=True)
+```
+
+The synchronous `MacOSSandbox` exposes the same method without `await`.
+Approvals cover Apple events sent through `osascript`, SSH sessions, `exec`, and
+Terminal.
+
+`allow_automation(bundle_ids=None, installed=False)` sends
+`POST /v1/sandboxes/{sandboxID}/permissions/automation`. Supply specific
+`bundle_ids`, `installed=True` for all installed applications, or both in one
+call. Omitted bundle IDs and the default `installed=False` are not sent.
+The 200 response is decoded as the exported `AutomationApproval` dataclass:
+`success: bool`, `targets: int` (applications considered), and `changed: int`
+(approvals added or flipped from denied).
+
+The gateway validates bundle IDs against
+`^[A-Za-z0-9][A-Za-z0-9._-]{0,254}$`. Malformed IDs or an empty request receive
+HTTP 400 with `{"error": ...}`. The SDK does not validate them locally; it raises
+the standard `httpx.HTTPStatusError`, whose `response` retains the status and
+error body.
+
+## Runtime Snapshots
+
+macOS snapshots let you seed a VM once, snapshot it, then create new sandboxes from that snapshot version. macOS snapshots preserve disk state, so installed apps, accounts, and seeded files are available immediately:
+
+```python
+from use_computer import Computer
+
+client = Computer()
+
+# Configure the macOS desktop by hand over VNC, then snapshot it.
+with client.create(type="macos") as mac:
+    print("Configure sandbox in the dashboard viewer:", mac.sandbox_id)
+    input("Press Enter once the desktop is ready to snapshot...")
+    snapshot = mac.snapshot("chrome-seeded-macos")
+
+# New sandboxes boot from that saved disk state, no setup needed.
+with client.create(type="macos", snapshot=snapshot.version) as seeded:
+    print("Seeded sandbox ready:", seeded.sandbox_id)
+```
+
+Use `client.snapshots()` to list saved snapshot versions.
+
+## Examples
+
+| File | What it shows |
+| --- | --- |
+| [`examples/_1_hello_macos.py`](../examples/python/_1_hello_macos.py) | reserve → create → exec → keyboard → screenshot |
+| [`examples/_2_recording.py`](../examples/python/_2_recording.py) | start / stop / download a screen recording |
+| [`examples/_3_file_transfer.py`](../examples/python/_3_file_transfer.py) | upload bytes, download a file back |
+| [`examples/_4_keepalive.py`](../examples/python/_4_keepalive.py) | heartbeat for ephemeral sessions idle > 2 min |
+| [`examples/_5_snapshots.py`](../examples/python/_5_snapshots.py) | snapshot seeded macOS state |
+
+## HTTP API
+
+Every SDK method wraps `https://api.use.computer/v1/...` with `Authorization: Bearer uc_live_...`. Swagger: [api.use.computer/docs](https://api.use.computer/docs). OpenAPI spec: [api.use.computer/openapi.yaml](https://api.use.computer/openapi.yaml).
+
+Gateway URLs, whether passed explicitly or via `USE_COMPUTER_BASE_URL`, must use HTTPS except for loopback HTTP. The SDK does not load `.env` files by default; load one explicitly in your application if needed. Authenticated requests, including keepalive and agent reference screenshots, do not follow redirects.
+
+Lifecycle calls (`reserve`, `create`, and `snapshot`, including restores via `create(snapshot=...)`) send a fresh UUID v4 `Idempotency-Key` per call. Idempotent HTTP methods and keyed lifecycle mutations retry transient failures using the same key and body; unkeyed mutations are not retried. A new lifecycle method invocation is a new intent with a new key.

daytona_use_computer-0.0.48.dist-info/RECORD

@@ -0,0 +1,58 @@
+use_computer/__init__.py,sha256=MJDAjH5gBRCK4XAw-Y6hukzPJ39efmF2VAP1zAaGy_U,1355
+use_computer/py.typed,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
+use_computer/setup_files.py,sha256=oEKvx0hr8PSFMBhQZ58QtPg5gaK5zuoCj1MwTfSRJ-E,1552
+use_computer/agents/__init__.py,sha256=X1J76U0qfdoh-JudGHhrxdnMCRJJGrvVXwek2iDkcfI,1968
+use_computer/agents/action_manifest.json,sha256=Ztvb3IWP4DsbJxsXsQgus-t6XG_VlBaOIZwM9ESbnKk,3492
+use_computer/agents/core.py,sha256=nqYxLpemx75oG1ZzsXNRwFgicvjye6hBTDq-GwmjOz8,1613
+use_computer/agents/types.py,sha256=EbaD5UbtTTTXD4jVHcTz6--Ct8nUgOWpIxTaAgyOTXM,6099
+use_computer/agents/base/__init__.py,sha256=E20b25cDAjlPEwJG4sO58G1RzO4oTPGWgABVLWki78M,939
+use_computer/agents/base/actions.py,sha256=gvoYkXMMRWxHX8sGtOfqOpUepaJ7BSisERslEBO2Oh0,11320
+use_computer/agents/base/agent.py,sha256=-DYpjCOC-D9XE9im3HfNnilswBuRk-VrO0wT6Q2ylaU,10565
+use_computer/agents/base/artifacts.py,sha256=b9f3Anul1VbAkhhWc4ygJ30YE2_FT5oEvBelylv3Jfs,3504
+use_computer/agents/base/desktop.py,sha256=dyZ_YGOPXeiwsz-L4kqoPWaALYkdGVyOUzuBxaV38rE,3678
+use_computer/agents/base/final_state.py,sha256=Fl4ppFO5t5L6kKBwJJMD7igXVX3b29iAub3jT_-xU0Q,5252
+use_computer/agents/base/prompts.py,sha256=rajDoTQY2Cd8F2vLYH6Pi4jH_dJlb85Yc5hl5uhJ8XE,1413
+use_computer/agents/base/recordings.py,sha256=qq4QdBAFmyk0qvzjtbSTxEnx6kTHnN7LfgqQ8sxQWA4,6352
+use_computer/agents/base/sandbox.py,sha256=ORol50XgG0RA8dnDE5ElDBAr6kxUg2hNNZ0TEKKATpE,184
+use_computer/agents/base/screenshots.py,sha256=KUmhmcyDhqrAdnqPzlrB0VoPlaiFMPugqhBOM2U2HLA,1902
+use_computer/agents/base/task_setup.py,sha256=eCJxoYgiaMIvo3c8oCGEOr5nPLHbkDLZHhWunqrkuW0,9060
+use_computer/agents/base/trajectory.py,sha256=o9KRnAu4MdQFQyIUwovmT2OUAMjp6YHZ6OYGLZ6Zvog,849
+use_computer/agents/debug/README.md,sha256=POF_k8QxWf6-1-0QXbOq6rnHfIL-SBauWXv8WGWeF08,454
+use_computer/agents/debug/__init__.py,sha256=9mHCYf-5WXw0Qf3FRceUtoY7bOI_eausfS1PxxCNWlE,157
+use_computer/agents/debug/actions.py,sha256=eFpL-aF5hXROY6Md0LJ_LWYWIwWUlGiuMNE1pXrltIU,3368
+use_computer/agents/debug/agent.py,sha256=px76iqvPFUt2o-OXN4Rkn3fdZnJk5mlbj3d4RZkv81c,7357
+use_computer/agents/debug/models.py,sha256=i1uUnnBciOJR-pN1UEoE0XAsYXGiiSR16TREJUyF1CM,2615
+use_computer/agents/prompts/anthropic.txt,sha256=BXWzYsYG0MZ439tZONID9JXP-2OyFg1Yfu_yOUzRREg,521
+use_computer/agents/prompts/gemini.txt,sha256=aZGjx2b_s_eGeYiGZyxI7Q6tZlzxzBpB1pv8YdRU_lU,356
+use_computer/agents/prompts/openai.txt,sha256=BXWzYsYG0MZ439tZONID9JXP-2OyFg1Yfu_yOUzRREg,521
+use_computer/agents/prompts/osworld_pyautogui.txt,sha256=R7tCB8GXq1uwSB7VfzmjLupzZTypgKj4r9oYqZQ2t2U,1896
+use_computer/agents/prompts/pyautogui.txt,sha256=-WEM_I9IqRKriIe-HQke71gGDm_rrvTUAjoxPKiXUZs,2558
+use_computer/agents/providers/__init__.py,sha256=9Rc7haJ7X__mklzPkQYWxRagyBfmJv0lG_yh2nNQCKI,75
+use_computer/agents/providers/anthropic.py,sha256=WU2Xz9kcQu2Qo9hh8UWGF060pk626A0Yi7tq7g4Csnw,12066
+use_computer/agents/providers/gemini.py,sha256=rFHnPXkBz8XaX-o3cdRfMTRBo9NHaVsl8GKZyBTJGpQ,15818
+use_computer/agents/providers/generic.py,sha256=wcTRjUzJxuO8BAOioo6WexXW-kbMScAHojRI98DWx5U,22102
+use_computer/agents/providers/openai.py,sha256=sEWXXY-XEQ53M7NQtK_HtFZ1chXhUskxNEFBq_3PHg0,12371
+use_computer/agents/providers/osworld.py,sha256=0BTuCDBeoN-jk-aLEaXLT4TZT2kbsFLeFyfetr49Qao,7709
+use_computer/agents/providers/tinker_backend.py,sha256=Xz7WW3JBB4sIoBJ8WPM5e6_IYcQDp85_kuaRc3E-IRU,6422
+use_computer/automation/__init__.py,sha256=ObAF_vOnDN7zQOSWRI5RkeSaM3Szq980dIZtKSoLKpI,746
+use_computer/automation/ax_transpile.py,sha256=p0vWzDrIcCjteeRIqrBnk7c4wdRI5BgzcBYQmDVHL10,18607
+use_computer/automation/osworld.py,sha256=WIH-zrJtUnJ9QYqTEEdR8Wg4iQ2EAFBhq0dXTl8zxno,17785
+use_computer/automation/parsers.py,sha256=apQQxkTrJbySRYwRAsvYuidu7_f7JqAfvzW4MOx0L08,9368
+use_computer/core/__init__.py,sha256=OAfgVoYbRunhI4nhO5RTlT8LHkFMhHFdVDyWhNyKV-Q,1109
+use_computer/core/client.py,sha256=NHBdNJZRXHjzF9fqwAwTLJX4LL2r8P-_OwjeunU165o,14396
+use_computer/core/errors.py,sha256=mq9NDbx-9BZKH_NIQFfz31mPx3NvAz6AxbeHSMr1esc,3701
+use_computer/core/models.py,sha256=eDTIRno27mcxKc2n8vwN0KYva8FQTLN3bmfkuUzNZ-E,3170
+use_computer/core/retry.py,sha256=oX_V5GURBEkMo4MosoB9AzItktcOj8l6LUIJuc0-DYE,4594
+use_computer/core/sandbox.py,sha256=T-j4dK4IlPCG7PnzRTr_Q6e99L9dspFuCqSnLtht0go,17357
+use_computer/macos/__init__.py,sha256=lMsFdDRTsmoVz6jKtlijVsKaA2H9eqTqkKrNx4t8jpQ,183
+use_computer/macos/keyboard.py,sha256=XjGAojVqSRA8gbviTRMfYrHjDLMNs8bzaYaErgSD0RI,2796
+use_computer/macos/mouse.py,sha256=M8rboCduyJsozKgJMjxO-xubviLeuMMol2tH4ITx1pg,6896
+use_computer/resources/__init__.py,sha256=vV5Fe9sSwbJLEaU1O1RKkKNZ36OdHAV-L1woTRikiew,630
+use_computer/resources/accessibility.py,sha256=G50lwUcyEi0FeuacEAxLWykY-OmqanasYiwKGTwN5PM,1543
+use_computer/resources/display.py,sha256=F6F1kBhVS3N7zhThsqaro8yrtIOttiITJCdxSBE7jeg,1137
+use_computer/resources/permissions.py,sha256=uH7QHaSax98HFj1mfZMSMCcRmRk9NPjMVWvPs3pE8l4,2404
+use_computer/resources/recording.py,sha256=6B8qg8346DADTZzI_EEehQFcy5Bbr6UGQNlT-KEoE7Q,3864
+use_computer/resources/screenshot.py,sha256=8AhbSOA647KqPNB6YhOjLunOiM9c9aYL14hwxnPPin0,2229
+daytona_use_computer-0.0.48.dist-info/METADATA,sha256=Nf37s8GUeyMFFCMgE1HXvxsAZZHeQMNwjWmXSjpMQcI,10616
+daytona_use_computer-0.0.48.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
+daytona_use_computer-0.0.48.dist-info/RECORD,,

daytona_use_computer-0.0.48.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.4
+Root-Is-Purelib: true
+Tag: py3-none-any

use_computer/__init__.py

@@ -0,0 +1,58 @@
+from use_computer.automation.osworld import AsyncOSWorldClient, OSWorldClient
+from use_computer.automation.parsers import Action, parse_pyautogui, parse_xdotool
+from use_computer.core.client import AsyncComputer, Computer, Reservation
+from use_computer.core.errors import (
+    BillingRequiredError,
+    PlatformNotSupportedError,
+    ReservationRequiredError,
+    SandboxResourcesError,
+    UseComputerError,
+)
+from use_computer.core.models import (
+    AccessibilityTree,
+    ActionResult,
+    ActResult,
+    AutomationApproval,
+    CursorPosition,
+    DisplayInfo,
+    ExecResult,
+    RecordingInfo,
+)
+from use_computer.core.sandbox import (
+    AsyncMacOSSandbox,
+    AsyncSandbox,
+    MacOSSandbox,
+    Sandbox,
+    SandboxType,
+    Snapshot,
+)
+
+__all__ = [
+    "Action",
+    "AccessibilityTree",
+    "ActionResult",
+    "ActResult",
+    "AsyncComputer",
+    "AsyncMacOSSandbox",
+    "AsyncOSWorldClient",
+    "AsyncSandbox",
+    "AutomationApproval",
+    "BillingRequiredError",
+    "Computer",
+    "CursorPosition",
+    "DisplayInfo",
+    "ExecResult",
+    "MacOSSandbox",
+    "OSWorldClient",
+    "PlatformNotSupportedError",
+    "RecordingInfo",
+    "Reservation",
+    "ReservationRequiredError",
+    "Sandbox",
+    "SandboxResourcesError",
+    "SandboxType",
+    "Snapshot",
+    "UseComputerError",
+    "parse_pyautogui",
+    "parse_xdotool",
+]

use_computer/agents/__init__.py

@@ -0,0 +1,52 @@
+"""Optional computer-use agents.
+
+Install provider-backed agents with `daytona-use-computer[agents]`.
+"""
+
+from __future__ import annotations
+
+from importlib import import_module
+from typing import Any
+
+from use_computer.agents.core import AgentRunOptions, AgentRunResult, AgentUsage
+
+_LAZY_EXPORTS = {
+    "AnthropicComputerAgent": (
+        "use_computer.agents.providers.anthropic",
+        "AnthropicComputerAgent",
+    ),
+    "AnthropicCUAAgent": ("use_computer.agents.providers.anthropic", "AnthropicCUAAgent"),
+    "DebugComputerAgent": ("use_computer.agents.debug.agent", "DebugComputerAgent"),
+    "DebugCUAAgent": ("use_computer.agents.debug.agent", "DebugCUAAgent"),
+    "GeminiComputerAgent": ("use_computer.agents.providers.gemini", "GeminiComputerAgent"),
+    "GeminiCUAAgent": ("use_computer.agents.providers.gemini", "GeminiCUAAgent"),
+    "GenericComputerAgent": ("use_computer.agents.providers.generic", "GenericComputerAgent"),
+    "GenericCUAAgent": ("use_computer.agents.providers.generic", "GenericCUAAgent"),
+    "OpenAIComputerAgent": ("use_computer.agents.providers.openai", "OpenAIComputerAgent"),
+    "OpenAICUAAgent": ("use_computer.agents.providers.openai", "OpenAICUAAgent"),
+    "OSWorldAgent": ("use_computer.agents.providers.osworld", "OSWorldAgent"),
+    "OSWorldComputerAgent": ("use_computer.agents.providers.osworld", "OSWorldComputerAgent"),
+}
+
+__all__ = [
+    "AgentRunOptions",
+    "AgentRunResult",
+    "AgentUsage",
+    *_LAZY_EXPORTS,
+]
+
+
+def __getattr__(name: str) -> Any:
+    target = _LAZY_EXPORTS.get(name)
+    if target is None:
+        raise AttributeError(name)
+    module_name, attr = target
+    try:
+        module = import_module(module_name)
+    except ModuleNotFoundError as exc:
+        raise ModuleNotFoundError(
+            f"{name} requires optional agent dependencies. Install `daytona-use-computer[agents]`."
+        ) from exc
+    value = getattr(module, attr)
+    globals()[name] = value
+    return value

use_computer/agents/action_manifest.json

@@ -0,0 +1,40 @@
+{
+  "schema_version": 1,
+  "route_prefix": "/v1/sandboxes/{id}",
+  "status": "implemented; real production GUI proof pending",
+  "coordinates": {"gateway": "screenshot pixels", "anthropic": "1280x800 tool pixels scaled independently by screenshot_width/1280 and screenshot_height/800", "openai": "screenshot pixels", "gemini": "floor(coord/1000*screen_dimension), input 0..999"},
+  "anthropic": {
+    "key": {"routes": ["POST /keyboard/press", "POST /keyboard/hotkey"], "emulation": "normalize aliases; plus-separated combo uses hotkey"},
+    "hold_key": {"routes": ["POST /keyboard/hold"], "payload": "keys=text, duration=seconds"},
+    "type": {"routes": ["POST /keyboard/type"]},
+    "mouse_move": {"routes": ["POST /mouse/move"]},
+    "left_mouse_down": {"routes": ["POST /mouse/down"]},
+    "left_mouse_up": {"routes": ["POST /mouse/up"]},
+    "left_click": {"routes": ["POST /mouse/click"], "payload": "button=left"},
+    "left_click_drag": {"routes": ["GET /mouse/position", "POST /mouse/drag"], "emulation": "current cursor is start when start_coordinate omitted"},
+    "right_click": {"routes": ["POST /mouse/click"], "payload": "button=right"},
+    "middle_click": {"routes": ["POST /mouse/click"], "payload": "button=middle"},
+    "double_click": {"routes": ["POST /mouse/click"], "payload": "double=true"},
+    "triple_click": {"routes": ["POST /mouse/click"], "payload": "click_count=3"},
+    "scroll": {"routes": ["POST /mouse/scroll"], "payload": "direction=scroll_direction, amount=scroll_amount"},
+    "cursor_position": {"routes": ["GET /mouse/position"], "emulation": "Anthropic provider divides gateway coordinates by tool-to-screen scale before reporting"}
+  },
+  "openai": {
+    "click": {"routes": ["POST /mouse/click"], "buttons": {"left":"left", "right":"right", "wheel":"middle"}},
+    "double_click": {"routes": ["POST /mouse/click"], "payload": "double=true"},
+    "drag": {"routes": ["POST /mouse/drag"], "payload": "path preserves every {x,y} point in order"},
+    "keypress": {"routes": ["POST /keyboard/press", "POST /keyboard/hotkey"], "emulation": "join keys with + and normalize aliases"},
+    "scroll": {"routes": ["POST /mouse/scroll"], "payload": "scroll_x and scroll_y retained; positive right/down"},
+    "type": {"routes": ["POST /keyboard/type"]},
+    "move": {"routes": ["POST /mouse/move"]}
+  },
+  "gemini": {
+    "click_at": {"routes": ["POST /mouse/click"]},
+    "hover_at": {"routes": ["POST /mouse/move"]},
+    "type_text_at": {"routes": ["POST /mouse/click", "POST /keyboard/hotkey", "POST /keyboard/press", "POST /keyboard/type"], "emulation": "click focus; command+a and Delete unless clear_before_typing=false; type; optional Return"},
+    "key_combination": {"routes": ["POST /keyboard/press", "POST /keyboard/hotkey"]},
+    "scroll_at": {"routes": ["POST /mouse/scroll"], "payload": "magnitude default800 denormalized by affected screen dimension; direction signed into scroll_x/scroll_y", "reference": "https://github.com/google/computer-use-preview/blob/main/agent.py"},
+    "drag_and_drop": {"routes": ["POST /mouse/drag"], "payload": "from_x/from_y/to_x/to_y"}
+  },
+  "notes": ["Coordinate-free clicks and scroll first GET /mouse/position.", "Actions except cursor_position are followed by a two-second settle and screenshot; timing label settle+screenshot includes both.", "Control remains Control, never rewritten to Command.", "Unknown actions return Invalid action, never executed; gateway HTTP errors are reported as failed."]
+}

use_computer/agents/base/__init__.py

@@ -0,0 +1,31 @@
+"""Public base-agent surface for CUA agents."""
+
+from use_computer.agents.base.actions import _annotate_click, execute_action
+from use_computer.agents.base.agent import BaseCUAAgent
+from use_computer.agents.base.desktop import (
+    DesktopProfile,
+    OpenAIComputerEnvironment,
+    desktop_profile,
+)
+from use_computer.agents.base.prompts import build_system_prompt, load_prompt
+from use_computer.agents.base.sandbox import ComputerSandbox
+from use_computer.agents.base.screenshots import (
+    scale_screenshot_for_model,
+    screenshot_cap_for_model,
+)
+from use_computer.agents.base.trajectory import write_trajectory
+
+__all__ = [
+    "BaseCUAAgent",
+    "ComputerSandbox",
+    "DesktopProfile",
+    "OpenAIComputerEnvironment",
+    "_annotate_click",
+    "build_system_prompt",
+    "desktop_profile",
+    "execute_action",
+    "load_prompt",
+    "scale_screenshot_for_model",
+    "screenshot_cap_for_model",
+    "write_trajectory",
+]