use-computer-cli 0.1.0__py3-none-any.whl

use_computer/skill/SKILL.md

@@ -0,0 +1,118 @@
+---
+name: use-computer
+description: Execute input on a screen — move, click, double-click, right-click, drag, scroll, type text, press key combinations, capture a screenshot. Use whenever you need to act on a GUI, locally or over VNC. Pair it with ui-locator, which tells you where to click.
+x-skill-id: use-computer
+x-skill-version: "1"
+---
+
+# use-computer
+
+You act on a screen through the `use-computer` CLI. It moves a real pointer and types real
+keystrokes. It does not decide *what* to click — you do, usually with coordinates from
+`ui-locator`.
+
+## Contract
+
+- **stdout** is one JSON object per run. Parse it.
+- **stderr** is diagnostics. Read it only when debugging.
+- **exit codes**: `0` success, `1` failure, `2` bad usage.
+
+Always pass `--use <profile>` unless a default profile is configured.
+
+## Coordinates: the thing that goes wrong
+
+A screenshot on a HiDPI display is larger than the space the OS clicks in. Coordinates from
+`ui-locator` are in **screenshot** pixels — the space it looked at. That is the default here,
+so pass them through unchanged. Pass `--space actuation` only if you already converted them
+yourself, which you should not do.
+
+If a run fails saying the scale is unknown, take a screenshot first (`use-computer screenshot`)
+and read `screen` from the result; do not compute a factor and retry with different numbers.
+
+## Acting
+
+```bash
+use-computer click --x 120 --y 340 --use staging
+use-computer double-click --x 120 --y 340 --use staging
+use-computer right-click --x 120 --y 340 --use staging
+use-computer move --x 120 --y 340 --use staging
+use-computer drag --from-x 10 --from-y 20 --to-x 300 --to-y 400 --use staging
+use-computer scroll --amount 3 --direction down --use staging
+use-computer type --text "hello world" --use staging
+use-computer key ctrl+s --use staging
+use-computer screenshot --out shot.png --use staging
+```
+
+`type` sends literal text. `ctrl+a` given to `type` types seven characters — use `key` for
+shortcuts.
+
+## Key syntax
+
+Modifiers `ctrl`, `alt`, `shift`, `cmd` (aliases: `control`, `option`, `super`, `win`, `meta`)
+joined to a key with `+`: `ctrl+shift+t`, `cmd+space`, `alt+f4`, `enter`. Named keys: `enter`,
+`tab`, `esc`, `space`, `backspace`, `delete`, `insert`, `home`, `end`, `pageup`, `pagedown`,
+`up`, `down`, `left`, `right`, `f1`–`f24`. One spelling works on every backend.
+
+## Batch — prefer this
+
+Opening a VNC connection costs more than the action does. Send the whole plan in one run; it
+executes over one connection.
+
+```bash
+echo '[
+  {"action":"click","x":120,"y":340,"verify":true},
+  {"action":"type","text":"hello","delay":0.2},
+  {"action":"key","combo":"enter"}
+]' | use-computer - --use staging
+```
+
+`batch` is the default command, so `use-computer -` and `use-computer actions.json` work. A
+batch stops at the first failure and reports `failed_index`, so you can resume from a known
+point. `--continue-on-error` runs the rest anyway.
+
+## Verify — how you know it worked
+
+A click that lands on nothing looks exactly like a click that worked. With `--verify` (or
+`"verify": true` on one action) each action reports:
+
+```json
+"change": {"changed": true, "magnitude": 0.18, "threshold": 0.002, "bbox": [40,120,600,400]}
+```
+
+- `changed: false` after a click → the coordinate was probably stale. **Ask ui-locator again.
+  Do not click the same pixel twice.**
+- `changed: true` with a tiny `magnitude` in a corner → a clock or a caret, not a response.
+
+Verification costs two screenshots per action, so use it on the actions whose effect you need
+to confirm, not on every one.
+
+## Before you act on something risky
+
+`--dry-run` resolves and logs everything — profile, scaled coordinates, normalised keys —
+without performing any of it. Results come back with `"performed": false`. Rehearse a batch you
+are unsure about.
+
+## When it refuses
+
+- **`BackendNotAvailableError`** — the extra is not installed. The message names it.
+- **Local backend not enabled** — the `local` backend controls the user's own machine and needs
+  an explicit opt-in. Tell the user to set `allow-local = true` in the profile; do not work
+  around it.
+- **Permission denied** — macOS Accessibility or Screen Recording. The message names which. Only
+  the user can grant it.
+- **Coordinate space error** — see above. Take a screenshot; do not guess a factor.
+
+## Configuration
+
+`use-computer config show` prints every resolved value, the layer it came from, and the exact
+environment variable that would override it. Run it first when a profile behaves unexpectedly.
+
+If there is no config at all, you can create one without a human:
+
+```bash
+use-computer config init --backend vnc --host 10.0.0.5 --profile staging
+```
+
+It refuses to overwrite an existing config, and it will not enable the local backend for you —
+that opt-in is the user's to give. After writing, it opens the backend and reports the screen and
+its scale; a failed probe means the config is on disk but wrong.

use_computer/skill/__init__.py

@@ -0,0 +1,185 @@
+"""The agent skill, and the commands that install it.
+
+The skill ships as package data *inside* the package. A copy at the repository root would not
+be in the wheel, and a skill that is not in the wheel does not exist for anyone who installed
+from PyPI.
+
+Nothing here ever writes to or deletes a directory that does not carry this skill's frontmatter
+marker: removing a directory someone else owns is unrecoverable, and the marker is the proof of
+ownership.
+"""
+
+from __future__ import annotations
+
+from enum import Enum
+from importlib import resources
+from pathlib import Path
+
+from pydantic import BaseModel, ConfigDict
+
+from use_computer.errors import ConfigError
+
+#: Frontmatter key/value that marks a directory as owned by this skill.
+MARKER_KEY = "x-skill-id"
+MARKER_VALUE = "use-computer"
+
+#: Directory name the skill is installed under.
+SKILL_NAME = "use-computer"
+SKILL_FILE = "SKILL.md"
+
+#: The neutral layout, used when a project shows no preference.
+NEUTRAL_DIR = Path(".agents") / "skills"
+CLAUDE_DIR = Path(".claude") / "skills"
+
+
+class Scope(str, Enum):
+    USER = "user"
+    PROJECT = "project"
+    AGENTS = "agents"
+    CLAUDE = "claude"
+
+
+class SkillState(BaseModel):
+    """Where the skill is, and whether it matches the bundled copy."""
+
+    model_config = ConfigDict(frozen=True)
+
+    scope: Scope
+    path: Path
+    installed: bool
+    owned: bool
+    up_to_date: bool
+
+    @property
+    def status(self) -> str:
+        if not self.installed:
+            return "missing"
+        if not self.owned:
+            return "foreign"
+        return "up-to-date" if self.up_to_date else "outdated"
+
+
+def bundled_text() -> str:
+    """The bundled SKILL.md, read as package data -- the package may be zipped."""
+    return resources.files(__package__).joinpath(SKILL_FILE).read_text(encoding="utf-8")
+
+
+def skills_dir(scope: Scope, *, root: Path | None = None, override: Path | None = None) -> Path:
+    """The skills directory for a scope.
+
+    The ``project`` scope follows the layout the target project already uses, and defaults to
+    the neutral ``.agents/skills`` when neither exists.
+    """
+    if override is not None:
+        return override
+    base = root or Path.cwd()
+    if scope is Scope.CLAUDE:
+        return base / CLAUDE_DIR
+    if scope is Scope.AGENTS:
+        return base / NEUTRAL_DIR
+    if scope is Scope.PROJECT:
+        if (base / CLAUDE_DIR).is_dir():
+            return base / CLAUDE_DIR
+        if (base / NEUTRAL_DIR).is_dir():
+            return base / NEUTRAL_DIR
+        return base / NEUTRAL_DIR
+    home = Path.home()
+    if (home / CLAUDE_DIR).is_dir():
+        return home / CLAUDE_DIR
+    return home / NEUTRAL_DIR
+
+
+def target_path(scope: Scope, *, root: Path | None = None, override: Path | None = None) -> Path:
+    return skills_dir(scope, root=root, override=override) / SKILL_NAME / SKILL_FILE
+
+
+def carries_marker(path: Path) -> bool:
+    """Whether a SKILL.md declares this skill's frontmatter marker."""
+    if not path.is_file():
+        return False
+    try:
+        text = path.read_text(encoding="utf-8")
+    except OSError:
+        return False
+    return _marker_in_frontmatter(text)
+
+
+def _marker_in_frontmatter(text: str) -> bool:
+    lines = text.splitlines()
+    if not lines or lines[0].strip() != "---":
+        return False
+    for line in lines[1:]:
+        if line.strip() == "---":
+            return False
+        key, _, value = line.partition(":")
+        if key.strip() == MARKER_KEY and value.strip().strip("\"'") == MARKER_VALUE:
+            return True
+    return False
+
+
+def status(scope: Scope, *, root: Path | None = None, override: Path | None = None) -> SkillState:
+    path = target_path(scope, root=root, override=override)
+    installed = path.is_file()
+    owned = carries_marker(path)
+    up_to_date = False
+    if installed and owned:
+        up_to_date = path.read_text(encoding="utf-8") == bundled_text()
+    return SkillState(
+        scope=scope, path=path, installed=installed, owned=owned, up_to_date=up_to_date
+    )
+
+
+def install(
+    scope: Scope,
+    *,
+    root: Path | None = None,
+    override: Path | None = None,
+    force: bool = False,
+) -> SkillState:
+    """Write the bundled skill into the scope's skills directory.
+
+    Raises:
+        ConfigError: when a file is already there and ``force`` was not given, or when the
+            existing file belongs to another skill.
+    """
+    path = target_path(scope, root=root, override=override)
+    if path.exists():
+        if not carries_marker(path):
+            raise ConfigError(
+                f"{path} exists and does not carry the {MARKER_KEY}: {MARKER_VALUE} marker, so "
+                "it belongs to something else. Refusing to touch it."
+            )
+        if not force:
+            raise ConfigError(f"{path} already exists. Pass --force to overwrite it.")
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(bundled_text(), encoding="utf-8")
+    return status(scope, root=root, override=override)
+
+
+def update(
+    scope: Scope, *, root: Path | None = None, override: Path | None = None
+) -> SkillState:
+    """Refresh an installed skill. Installing over our own copy is what update means."""
+    path = target_path(scope, root=root, override=override)
+    if path.exists() and not carries_marker(path):
+        raise ConfigError(
+            f"{path} does not carry the {MARKER_KEY}: {MARKER_VALUE} marker. Refusing to touch it."
+        )
+    return install(scope, root=root, override=override, force=True)
+
+
+def remove(scope: Scope, *, root: Path | None = None, override: Path | None = None) -> SkillState:
+    """Delete an installed skill, and only ever one that carries the marker."""
+    path = target_path(scope, root=root, override=override)
+    if not path.exists():
+        return status(scope, root=root, override=override)
+    if not carries_marker(path):
+        raise ConfigError(
+            f"{path} does not carry the {MARKER_KEY}: {MARKER_VALUE} marker, so it belongs to "
+            "something else. Refusing to remove it."
+        )
+    path.unlink()
+    parent = path.parent
+    if parent.is_dir() and not any(parent.iterdir()):
+        parent.rmdir()
+    return status(scope, root=root, override=override)

use_computer_cli-0.1.0.dist-info/METADATA

@@ -0,0 +1,150 @@
+Metadata-Version: 2.5
+Name: use-computer-cli
+Version: 0.1.0
+Summary: Execute input on a screen for computer-use agents: move, click, drag, scroll, type, key, screenshot.
+Project-URL: Homepage, https://github.com/applica-software-guru/use-computer
+Author: Bruno Fortunato
+License: MIT License
+        
+        Copyright (c) 2026 Bruno Fortunato
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: agent,automation,computer-use,screenshot,vnc
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: pillow>=10
+Requires-Dist: pydantic-settings>=2.2
+Requires-Dist: pydantic>=2.7
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: rich>=13
+Requires-Dist: tomli>=2.0; python_version < '3.11'
+Requires-Dist: typer>=0.16
+Provides-Extra: dev
+Requires-Dist: mypy>=1.11; extra == 'dev'
+Requires-Dist: pytest-mock>=3.12; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Provides-Extra: local
+Requires-Dist: mss>=10; extra == 'local'
+Requires-Dist: pynput>=1.8; extra == 'local'
+Provides-Extra: vnc
+Requires-Dist: vncdotool>=1.3; extra == 'vnc'
+Description-Content-Type: text/markdown
+
+# use-computer
+
+Executes input on a screen for computer-use agents: move, click, double-click, right-click,
+drag, scroll, type text, press key combinations, and capture a screenshot.
+
+`use-computer` is the **acting** half of a pair. [ui-locator](https://github.com/applica-software-guru/ui-locator)
+answers *where* the Invia button is and returns pixel coordinates; `use-computer` performs the
+click there. Both are driven by another AI agent through a CLI that emits JSON on stdout and
+diagnostics on stderr, with a Python API underneath.
+
+## Install
+
+```bash
+pip install use-computer-cli            # no backend
+pip install "use-computer-cli[local]"   # drive this machine's display (pynput + mss)
+pip install "use-computer-cli[vnc]"     # drive a remote framebuffer over RFB (vncdotool)
+```
+
+Backends are optional extras, imported lazily, so the package installs without them.
+
+The distribution is `use-computer-cli` because `use-computer` is taken on PyPI by an unrelated
+project. The command it installs is `use-computer`, and the package it imports is `use_computer`.
+
+## Use
+
+```bash
+use-computer click --x 120 --y 340 --use staging
+use-computer type --text "hello" --use staging
+use-computer key ctrl+s --use staging
+use-computer screenshot --use laptop --out shot.png
+
+# a batch runs over one connection -- the default command, so `batch` may be omitted
+echo '[{"action":"click","x":120,"y":340},{"action":"key","combo":"enter"}]' \
+  | use-computer - --use staging --verify
+```
+
+stdout is one JSON object per run; every diagnostic goes to stderr. Exit codes: `0` success,
+`1` failure, `2` bad usage.
+
+## Three problems it solves
+
+- **Coordinate spaces.** A screenshot on a HiDPI display is larger than the space the OS clicks
+  in. Every coordinate carries its space, `use-computer` scales between them, and it refuses to
+  guess when the ratio is unknown.
+- **Setup cost.** Opening a VNC connection dominates a single action, so one run performs a
+  batch of actions over one connection.
+- **Blind actuation.** A click that lands on nothing looks exactly like a click that worked, so
+  `--verify` compares the screen before and after and reports whether it changed.
+
+## Configure
+
+```bash
+use-computer config init                                  # asks, then proves it works
+use-computer config init --backend vnc --host 10.0.0.5    # doesn't ask
+use-computer config init --backend local --allow-local
+```
+
+`config init` writes the file below, then opens the backend it just configured and reports the
+screen geometry and scale — so a coordinate space whose ratio cannot be derived surfaces at setup
+rather than at the first click that lands in the wrong place.
+
+It writes `.use-computer/config.toml` at the project root (found by walking up, the way git finds
+its own):
+
+```toml
+default-profile = "laptop"
+delay = 0.1
+
+[profiles.laptop]
+backend = "local"
+allow-local = true
+
+[profiles.staging]
+backend = "vnc"
+host = "10.0.0.5"
+port = 5900
+```
+
+Secrets go in `.use-computer/.env`, which is not committed. `use-computer config show` prints
+every resolved value, the layer it came from and the variable that would override it.
+
+## The agent skill
+
+Instructions for the calling agent ship inside the package and are installed from it, so they
+always match the installed version:
+
+```bash
+use-computer skill install --scope project
+```
+
+## Documentation
+
+This package is developed with [SDD](https://github.com/applica-software-guru/sdd). The specs
+it implements live in `product/` and `system/` at the repository root.

use_computer_cli-0.1.0.dist-info/RECORD

@@ -0,0 +1,20 @@
+use_computer/__init__.py,sha256=fUBZ-Ci9StV2qm5L-4CYaxxucVbzoxbLh6kBl2N_NS0,2429
+use_computer/actions.py,sha256=NUb1VGlZoR6wgbHjujUlAMDCtfhuiyP40ROCsNTF8x8,5353
+use_computer/cli.py,sha256=5z446VAYA95bhcF2MSGfeU6lDmW1W8jAavDuPy2AEM4,22792
+use_computer/compare.py,sha256=kgnGc5wP6RzGpuXXq26wa2yW2XK1d6fc7EJc_HnkBlU,4324
+use_computer/config.py,sha256=eTDOuhCCl98QnQHQD10IYXT_gEBXXIiEymjIbqiaOqg,19510
+use_computer/coordinates.py,sha256=qugbalE87_IE_yiZUw__YqUwjh9obH5sUIJHalncQE0,3969
+use_computer/errors.py,sha256=J1lfkrOWPm_h-EDqNF0UaNhbp3VVxdKqtTtKeyDdSAo,1710
+use_computer/keys.py,sha256=Z-xTLBIRAiG2prl0O-TenMUNj1eJpVvUpEgp32kDlPs,6171
+use_computer/runner.py,sha256=U7TWPXEDaYKn5xtk1iAvv8_F5fhyCzJ04MfSZ7ICMqM,10296
+use_computer/backends/__init__.py,sha256=siniU8D02IH2B_lCvpZ5AXzYV7rb4sLO1srjsW6osZg,1102
+use_computer/backends/base.py,sha256=BLBBKGXSwXTSEQkzXQndtdbnv1cXBZEnEyaZoWycqBU,2230
+use_computer/backends/local.py,sha256=AcsEd6pGzwQ2UNPf82YJgZ_bFGbs7ml8Jl24fQjWJNY,8168
+use_computer/backends/vnc.py,sha256=4_7PK5rIBeJ8u3OaJ3jUNy5nLR8HrmYiS4UcWnhSOx8,6093
+use_computer/skill/__init__.py,sha256=a_93vXTA99pNHdGxr_FD6r5rSPCP7MmDLkEBYj5VPUI,6230
+use_computer/skill/SKILL.md,sha256=jLtV10F-0e31y6fYjUBVU8ElF3DsmW3ZaSovhmOsXJ0,4968
+use_computer_cli-0.1.0.dist-info/METADATA,sha256=NcGyHn6s02oM5PXqndpz3JE4qkCrGEooZk0xE20iOiM,6085
+use_computer_cli-0.1.0.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
+use_computer_cli-0.1.0.dist-info/entry_points.txt,sha256=5k_dpdhwgzZSckS5mtXJvLMYC1s06myYEdbjWKeoQjg,55
+use_computer_cli-0.1.0.dist-info/licenses/LICENSE,sha256=GwYgROcUjah7q6rGkOnEClxr39GXUXUZWyqog-iBw9c,1072
+use_computer_cli-0.1.0.dist-info/RECORD,,

use_computer_cli-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.0
+Root-Is-Purelib: true
+Tag: py3-none-any

use_computer_cli-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+use-computer = use_computer.cli:main

use_computer_cli-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Bruno Fortunato
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.