pctr 0.2.2__tar.gz → 0.3.0__tar.gz

pctr-0.3.0/PKG-INFO

@@ -0,0 +1,136 @@
+Metadata-Version: 2.4
+Name: pctr
+Version: 0.3.0
+Summary: Element-based Windows UI Automation CLI for AI agents. Find controls by name/type/id and click, type, hold, drag, and screenshot - no pixel coordinates.
+Author: Space-lab515
+License-Expression: MIT
+Keywords: automation,windows,uia,pywinauto,desktop,agent,computer-use,rpa
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Win32 (MS Windows)
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: System :: Networking
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pywinauto>=0.6.8; platform_system == "Windows"
+Requires-Dist: uiautomation>=2.0; platform_system == "Windows"
+Requires-Dist: pyautogui>=0.9.54; platform_system == "Windows"
+Requires-Dist: pydirectinput>=1.0.4; platform_system == "Windows"
+Requires-Dist: pillow>=9.0; platform_system == "Windows"
+Provides-Extra: vision
+Requires-Dist: ultralytics>=8.2; extra == "vision"
+Dynamic: license-file
+
+# pctr
+
+Element-based Windows UI Automation CLI for AI agents.
+
+Instead of screenshotting and guessing pixel coordinates, `pctr` finds controls
+by their **name / control type / automation id** and acts on them. Window moves,
+DPI changes, and layout shifts don't break your script.
+
+Built on `pywinauto` (UI Automation) with `pyautogui` / `pydirectinput` for raw
+mouse and keyboard input. On top of UIA it also offers **OCR**
+(`Windows.Media.Ocr`) and **YOLO-World** open-vocabulary detection for text and
+objects that aren't exposed as controls.
+
+## Install
+
+```bash
+pip install pctr
+pip install "pctr[vision]"   # adds ultralytics for `pctr look`
+```
+
+Windows only.
+
+## Commands
+
+| Command | What it does |
+|---------|--------------|
+| `pctr windows [--filter RE] [--process PID]` | List top-level windows (`pid=… \| title`). |
+| `pctr tree --title RE [--depth N] [--limit N]` | Dump the UIA control tree of a window. |
+| `pctr find --title RE [--name RE] [--limit N] [--nth N]` | List matching elements. |
+| `pctr click --title RE --name RE [--method auto\|invoke\|mouse] [--dbl]` | Click an element. |
+| `pctr set --title RE --name RE --text S` | Set a field via the UIA ValuePattern (exact, instant). |
+| `pctr type [--title RE --name RE] --text S [--delay 0.03] [--chunk 1]` | Send keystrokes (see notes). |
+| `pctr keys --keys "{ENTER}"` | Send a global key combo (pywinauto syntax). |
+| `pctr hotkey --keys ctrl+s` | Send a modifier combo (prefer this over `keys "^s"`). |
+| `pctr focus --title RE` | Bring a window to the foreground. |
+| `pctr wait --title RE --name RE [--timeout S]` | Wait until an element exists. |
+| `pctr shot [--title RE] --out PATH` | Screenshot the screen or a window. |
+| `pctr size` | Print primary screen size as `WxH`. |
+| `pctr desktop <action>` | Virtual desktops: `list` / `windows` / `where` / `on-current` / `next` / `prev` / `new` / `close` / `move-window`. |
+| `pctr move --x N --y N` / `down` / `up` / `hold --ms N` | Raw mouse control (`--direct` for games). |
+| `pctr drag --start x,y --end x,y [--duration S]` | Drag between two points. |
+| `pctr keydown / keyup --key a` / `keyhold --key w --ms 1500` | Hold keyboard keys (`--direct` for games). |
+| `pctr ocr [--title RE]` | OCR the screen/window; list words with boxes. |
+| `pctr ocrfind --text RE` | OCR then list matches with click centers (single words). |
+| `pctr ocrclick --text RE [--nth N]` | OCR then click a word. |
+| `pctr look --for "a dog" [--title RE] [--conf X]` | YOLO-World detect objects by text prompt. |
+| `pctr lookclick --for "a dog" [--nth N]` | Detect then click the top match. |
+| `pctr skill [--install]` / `pctr setup` | Print/install the agent skill + an AGENTS.md section. |
+
+Common filters: `--title` (regex), `--name` (regex), `--control-type`
+(`Button`, `Edit`, `Pane`, `MenuItem`, …), `--auto-id`, `--nth`, `--process`.
+
+`click`, `set`, `wait` accept **any** selector - `--name` is not required.
+`tree`/`find` require a window selector (`--title` or `--process`).
+
+## The loop
+
+```bash
+pctr tree  --title "Notepad" --limit 60      # see what's clickable
+pctr click --title "Notepad" --name "^File$"
+pctr set   --title "Notepad" --name "Text Editor" --text "hello"
+```
+
+## Virtual desktops
+
+```bash
+pctr desktop list                       # desktops (#index, guid, [current], window count)
+pctr desktop windows --desktop 1        # windows on desktop #1
+pctr desktop where --title "App"        # which desktop a window is on
+pctr desktop on-current --title "App"   # True / False
+pctr desktop next | prev | new | close  # switch (moves the active desktop)
+pctr desktop move-window --title RE --to N
+```
+
+- Switching uses the standard Win+Ctrl hotkeys and moves the **active** desktop
+  (you included). Add `--direct` if a game has focus.
+- **Cross-desktop control:** UIA actions (`click --method invoke`, `set`) reach
+  windows on another desktop **without switching** (accessibility layer, not
+  screen input). Raw mouse/keys only affect the active desktop.
+- **`move-window` is blocked on Windows 11** (third-party `MoveWindowToDesktop`
+  returns `Access denied`); it works on Windows 10.
+
+## Exit codes
+
+- `0` success, `1` no match / window not found, `2` usage/validation error.
+- Errors print one `error: …` line - no tracebacks.
+
+## Notes
+
+- Prefer `set` (ValuePattern) over `type` for text fields - it can't drop or
+  reorder characters. `type`'s element path uses pywinauto `type_keys`, which
+  interprets `{ } + ^ % ~ ( )` as key syntax, so it is **not** literal.
+- Global `type` / `keys` go to the OS-focused window - `pctr focus` first.
+- Prefer `pctr hotkey --keys ctrl+s` over `keys "^s"`; the `^` modifier is
+  fragile through shells.
+- `ocrfind`/`ocrclick` match one **word** at a time (Windows OCR tokenizes words).
+- `look` reads its model from `--model` or the `PCTR_YOLO_MODEL` env var; progress
+  bars go to stderr so stdout stays parseable.
+- Qt apps expose a rich tree including embedded webviews. Electron apps expose
+  only the window frame (use `focus` + global keys, or drive them over CDP).
+- Games ignore synthetic input - pass `--direct` to mouse/key commands and focus
+  the game first.
+
+## License
+
+MIT

pctr-0.3.0/README.md

@@ -0,0 +1,106 @@
+# pctr
+
+Element-based Windows UI Automation CLI for AI agents.
+
+Instead of screenshotting and guessing pixel coordinates, `pctr` finds controls
+by their **name / control type / automation id** and acts on them. Window moves,
+DPI changes, and layout shifts don't break your script.
+
+Built on `pywinauto` (UI Automation) with `pyautogui` / `pydirectinput` for raw
+mouse and keyboard input. On top of UIA it also offers **OCR**
+(`Windows.Media.Ocr`) and **YOLO-World** open-vocabulary detection for text and
+objects that aren't exposed as controls.
+
+## Install
+
+```bash
+pip install pctr
+pip install "pctr[vision]"   # adds ultralytics for `pctr look`
+```
+
+Windows only.
+
+## Commands
+
+| Command | What it does |
+|---------|--------------|
+| `pctr windows [--filter RE] [--process PID]` | List top-level windows (`pid=… \| title`). |
+| `pctr tree --title RE [--depth N] [--limit N]` | Dump the UIA control tree of a window. |
+| `pctr find --title RE [--name RE] [--limit N] [--nth N]` | List matching elements. |
+| `pctr click --title RE --name RE [--method auto\|invoke\|mouse] [--dbl]` | Click an element. |
+| `pctr set --title RE --name RE --text S` | Set a field via the UIA ValuePattern (exact, instant). |
+| `pctr type [--title RE --name RE] --text S [--delay 0.03] [--chunk 1]` | Send keystrokes (see notes). |
+| `pctr keys --keys "{ENTER}"` | Send a global key combo (pywinauto syntax). |
+| `pctr hotkey --keys ctrl+s` | Send a modifier combo (prefer this over `keys "^s"`). |
+| `pctr focus --title RE` | Bring a window to the foreground. |
+| `pctr wait --title RE --name RE [--timeout S]` | Wait until an element exists. |
+| `pctr shot [--title RE] --out PATH` | Screenshot the screen or a window. |
+| `pctr size` | Print primary screen size as `WxH`. |
+| `pctr desktop <action>` | Virtual desktops: `list` / `windows` / `where` / `on-current` / `next` / `prev` / `new` / `close` / `move-window`. |
+| `pctr move --x N --y N` / `down` / `up` / `hold --ms N` | Raw mouse control (`--direct` for games). |
+| `pctr drag --start x,y --end x,y [--duration S]` | Drag between two points. |
+| `pctr keydown / keyup --key a` / `keyhold --key w --ms 1500` | Hold keyboard keys (`--direct` for games). |
+| `pctr ocr [--title RE]` | OCR the screen/window; list words with boxes. |
+| `pctr ocrfind --text RE` | OCR then list matches with click centers (single words). |
+| `pctr ocrclick --text RE [--nth N]` | OCR then click a word. |
+| `pctr look --for "a dog" [--title RE] [--conf X]` | YOLO-World detect objects by text prompt. |
+| `pctr lookclick --for "a dog" [--nth N]` | Detect then click the top match. |
+| `pctr skill [--install]` / `pctr setup` | Print/install the agent skill + an AGENTS.md section. |
+
+Common filters: `--title` (regex), `--name` (regex), `--control-type`
+(`Button`, `Edit`, `Pane`, `MenuItem`, …), `--auto-id`, `--nth`, `--process`.
+
+`click`, `set`, `wait` accept **any** selector - `--name` is not required.
+`tree`/`find` require a window selector (`--title` or `--process`).
+
+## The loop
+
+```bash
+pctr tree  --title "Notepad" --limit 60      # see what's clickable
+pctr click --title "Notepad" --name "^File$"
+pctr set   --title "Notepad" --name "Text Editor" --text "hello"
+```
+
+## Virtual desktops
+
+```bash
+pctr desktop list                       # desktops (#index, guid, [current], window count)
+pctr desktop windows --desktop 1        # windows on desktop #1
+pctr desktop where --title "App"        # which desktop a window is on
+pctr desktop on-current --title "App"   # True / False
+pctr desktop next | prev | new | close  # switch (moves the active desktop)
+pctr desktop move-window --title RE --to N
+```
+
+- Switching uses the standard Win+Ctrl hotkeys and moves the **active** desktop
+  (you included). Add `--direct` if a game has focus.
+- **Cross-desktop control:** UIA actions (`click --method invoke`, `set`) reach
+  windows on another desktop **without switching** (accessibility layer, not
+  screen input). Raw mouse/keys only affect the active desktop.
+- **`move-window` is blocked on Windows 11** (third-party `MoveWindowToDesktop`
+  returns `Access denied`); it works on Windows 10.
+
+## Exit codes
+
+- `0` success, `1` no match / window not found, `2` usage/validation error.
+- Errors print one `error: …` line - no tracebacks.
+
+## Notes
+
+- Prefer `set` (ValuePattern) over `type` for text fields - it can't drop or
+  reorder characters. `type`'s element path uses pywinauto `type_keys`, which
+  interprets `{ } + ^ % ~ ( )` as key syntax, so it is **not** literal.
+- Global `type` / `keys` go to the OS-focused window - `pctr focus` first.
+- Prefer `pctr hotkey --keys ctrl+s` over `keys "^s"`; the `^` modifier is
+  fragile through shells.
+- `ocrfind`/`ocrclick` match one **word** at a time (Windows OCR tokenizes words).
+- `look` reads its model from `--model` or the `PCTR_YOLO_MODEL` env var; progress
+  bars go to stderr so stdout stays parseable.
+- Qt apps expose a rich tree including embedded webviews. Electron apps expose
+  only the window frame (use `focus` + global keys, or drive them over CDP).
+- Games ignore synthetic input - pass `--direct` to mouse/key commands and focus
+  the game first.
+
+## License
+
+MIT

{pctr-0.2.2 → pctr-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pctr"
-version = "0.2.2"
+version = "0.3.0"
 description = "Element-based Windows UI Automation CLI for AI agents. Find controls by name/type/id and click, type, hold, drag, and screenshot - no pixel coordinates."
 readme = "README.md"
 requires-python = ">=3.10"

pctr-0.3.0/src/pctr/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.3.0"

{pctr-0.2.2 → pctr-0.3.0}/src/pctr/cli.py

@@ -2,7 +2,7 @@ import argparse
 import re
 import sys
 import time
-from typing import NoReturn
+from typing import Any, NoReturn, cast
 
 for _stream in (sys.stdout, sys.stderr):
     _reconf = getattr(_stream, "reconfigure", None)
@@ -62,6 +62,49 @@ def _find_window(title_re, process=None, timeout=5.0, index=0):
     )
 
 
+def _enum_top_windows():
+    """Top-level visible windows via Win32 EnumWindows (no UIA - survives a wedged UIA provider)."""
+    import ctypes
+    from ctypes import wintypes
+    user32 = ctypes.windll.user32
+    out = []
+    proc = ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
+
+    def cb(hwnd, lp):
+        n = user32.GetWindowTextLengthW(hwnd)
+        if n and user32.IsWindowVisible(hwnd):
+            buf = ctypes.create_unicode_buffer(n + 1)
+            user32.GetWindowTextW(hwnd, buf, n + 1)
+            pid = wintypes.DWORD()
+            user32.GetWindowThreadProcessId(hwnd, ctypes.byref(pid))
+            out.append((int(hwnd), buf.value, int(pid.value)))
+        return True
+
+    user32.EnumWindows(proc(cb), 0)
+    return out
+
+
+def _find_hwnd(title_re, process=None, timeout=5.0):
+    """Find a top-level window handle by title regex (Win32, no UIA)."""
+    rx = _compile(title_re, "title") if title_re else None
+    deadline = time.time() + timeout
+    last = 0
+    while True:
+        matches = [
+            w for w in _enum_top_windows()
+            if (rx is None or rx.search(w[1])) and (process is None or w[2] == process)
+        ]
+        last = len(matches)
+        if matches:
+            return matches[0][0]
+        if time.time() >= deadline:
+            break
+        time.sleep(0.25)
+    raise SystemExit(
+        "window not found: title_re=%r process=%r (saw %d match(es))" % (title_re, process, last)
+    )
+
+
 def _matches(win, name=None, control_type=None, auto_id=None):
     rx_name = _compile(name, "name") if name is not None else None
     out = []
@@ -118,17 +161,12 @@ def _get_element(args):
 def cmd_windows(args):
     rx = _compile(args.filter, "filter") if args.filter else None
     proc = getattr(args, "process", None)
-    for w in _iter_windows():
-        t = ""
-        try:
-            t = w.window_text()
-        except Exception:
-            pass
-        if rx and rx.search(t) is None:
+    for hwnd, title, pid in _enum_top_windows():
+        if rx and rx.search(title) is None:
             continue
-        if proc is not None and w.process_id() != proc:
+        if proc is not None and pid != proc:
             continue
-        print("pid=%s | %s" % (w.process_id(), t))
+        print("pid=%s | %s" % (pid, title))
 
 
 def _walk(root, max_depth):
@@ -370,6 +408,179 @@ def cmd_size(args):
     print("%dx%d" % (w, h))
 
 
+def _vdm() -> Any:
+    from ctypes import POINTER
+    from ctypes.wintypes import BOOL, HWND
+    import comtypes
+    from comtypes import COMMETHOD, GUID, HRESULT, IUnknown
+
+    class IVirtualDesktopManager(IUnknown):
+        _iid_ = GUID("{A5CD92FF-29BE-454C-8D04-D82879FB3F1B}")
+        _methods_ = [
+            COMMETHOD([], HRESULT, "IsWindowOnCurrentVirtualDesktop",
+                      (["in"], HWND, "w"), (["out"], POINTER(BOOL), "on")),
+            COMMETHOD([], HRESULT, "GetWindowDesktopId",
+                      (["in"], HWND, "w"), (["out"], POINTER(GUID), "did")),
+            COMMETHOD([], HRESULT, "MoveWindowToDesktop",
+                      (["in"], HWND, "w"), (["in"], POINTER(GUID), "did")),
+        ]
+
+    return cast(Any, comtypes.CoCreateInstance(
+        GUID("{AA509086-5CA9-4C25-8F95-589D3C07B48A}"),
+        interface=IVirtualDesktopManager,
+    ))
+
+
+def _guid_norm(g):
+    return str(g).strip("{}").lower()
+
+
+def _vdesk_guids():
+    """Ordered desktop GUIDs from the shell's registry (no undocumented COM)."""
+    import uuid
+    import winreg
+    k = winreg.OpenKey(
+        winreg.HKEY_CURRENT_USER,
+        r"Software\Microsoft\Windows\CurrentVersion\Explorer\VirtualDesktops",
+    )
+    try:
+        raw = winreg.QueryValueEx(k, "VirtualDesktopIDs")[0]
+    finally:
+        winreg.CloseKey(k)
+    return [_guid_norm(uuid.UUID(bytes_le=raw[i:i + 16])) for i in range(0, len(raw) - 15, 16)]
+
+
+def _list_windows_with_desktop():
+    import ctypes
+    from ctypes import wintypes
+    vdm: Any = _vdm()
+    user32 = ctypes.windll.user32
+    out = []
+    proc = ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
+
+    def cb(hwnd, lp):
+        n = user32.GetWindowTextLengthW(hwnd)
+        if n and user32.IsWindowVisible(hwnd):
+            buf = ctypes.create_unicode_buffer(n + 1)
+            user32.GetWindowTextW(hwnd, buf, n + 1)
+            try:
+                d = _guid_norm(vdm.GetWindowDesktopId(wintypes.HWND(hwnd)))
+            except Exception:
+                d = ""
+            out.append((hwnd, buf.value, d))
+        return True
+
+    user32.EnumWindows(proc(cb), 0)
+    return out
+
+
+def _desktop_list(args):
+    import ctypes
+    from ctypes import wintypes
+    guids = _vdesk_guids()
+    vdm: Any = _vdm()
+    fg = ctypes.windll.user32.GetForegroundWindow()
+    try:
+        cur = _guid_norm(vdm.GetWindowDesktopId(wintypes.HWND(fg)))
+    except Exception:
+        cur = ""
+    counts = {}
+    for _hwnd, _title, d in _list_windows_with_desktop():
+        counts[d] = counts.get(d, 0) + 1
+    for i, d in enumerate(guids):
+        print("#%d %s%s  (%d windows)" % (i, d, "  [current]" if d == cur else "", counts.get(d, 0)))
+
+
+def _desktop_windows(args):
+    guids = _vdesk_guids()
+    want = getattr(args, "desktop", None)
+    for hwnd, title, d in _list_windows_with_desktop():
+        i = guids.index(d) if d in guids else -1
+        if want is not None and i != want:
+            continue
+        print("desktop=#%s | hwnd=%s | %s" % (i, hwnd, title))
+
+
+def _desktop_where(args, bool_only):
+    if not getattr(args, "title", None) and getattr(args, "process", None) is None:
+        _fail("%s needs --title (or --process)" % args.action)
+    import ctypes
+    from ctypes import wintypes
+    vdm: Any = _vdm()
+    h = _find_hwnd(args.title, getattr(args, "process", None), getattr(args, "timeout", 5.0))
+    hwnd = wintypes.HWND(int(h))
+    if bool_only:
+        on = vdm.IsWindowOnCurrentVirtualDesktop(hwnd)
+        print("on_current=%s" % ("True" if on else "False"))
+        return
+    guids = _vdesk_guids()
+    d = _guid_norm(vdm.GetWindowDesktopId(hwnd))
+    i = guids.index(d) if d in guids else -1
+    print("desktop=#%d %s" % (i, d))
+
+
+def _desktop_move(args):
+    import ctypes
+    from ctypes import wintypes
+    import comtypes
+    if not getattr(args, "title", None):
+        _fail("move-window needs --title")
+    if getattr(args, "to", None) is None:
+        _fail("move-window needs --to N")
+    guids = _vdesk_guids()
+    n = args.to
+    if n < 0 or n >= len(guids):
+        _fail("desktop #%d out of range (0-%d)" % (n, len(guids) - 1))
+    vdm: Any = _vdm()
+    h = _find_hwnd(args.title, getattr(args, "process", None), getattr(args, "timeout", 5.0))
+    hwnd = wintypes.HWND(int(h))
+    try:
+        vdm.MoveWindowToDesktop(hwnd, comtypes.GUID("{" + guids[n] + "}"))
+    except Exception as e:
+        msg = str(e).lower()
+        if "denied" in msg or "0x80070005" in msg or "access" in msg:
+            _fail(
+                "Windows 11 blocks third-party MoveWindowToDesktop (Access denied). "
+                "read (list/where/windows) and UIA actions still work cross-desktop; "
+                "moving windows between desktops works on Windows 10."
+            )
+        _fail("move-window failed: %s" % e)
+    print("moved %r to desktop #%d" % (args.title, n))
+
+
+def cmd_desktop(args):
+    action = args.action
+    if action == "list":
+        return _desktop_list(args)
+    if action == "windows":
+        return _desktop_windows(args)
+    if action == "on-current":
+        return _desktop_where(args, bool_only=True)
+    if action == "where":
+        return _desktop_where(args, bool_only=False)
+    if action == "move-window":
+        return _desktop_move(args)
+    combos = {
+        "next": ("win", "ctrl", "right"),
+        "prev": ("win", "ctrl", "left"),
+        "new": ("win", "ctrl", "d"),
+        "close": ("win", "ctrl", "f4"),
+    }
+    combo = combos[action]
+    if getattr(args, "direct", False):
+        import pydirectinput
+        pydirectinput.PAUSE = 0.01
+        for k in combo:
+            pydirectinput.keyDown(k)
+        for k in reversed(combo):
+            pydirectinput.keyUp(k)
+    else:
+        import pyautogui
+        pyautogui.FAILSAFE = False
+        pyautogui.hotkey(*combo)
+    print("desktop: %s" % action)
+
+
 def cmd_drag(args):
     m = _mouse_backend(args)
     x1, y1 = _parse_pair(args.start)
@@ -775,6 +986,16 @@ def build_parser():
     sz = sub.add_parser("size", help="Print primary screen size as WxH")
     sz.set_defaults(func=cmd_size)
 
+    dp = sub.add_parser("desktop", help="Virtual desktops: list / windows / where / on-current / next / prev / new / close / move-window")
+    dp.add_argument("action", choices=["list", "windows", "where", "on-current", "next", "prev", "new", "close", "move-window"])
+    dp.add_argument("--direct", action="store_true", help="Use pydirectinput for the switch hotkey")
+    dp.add_argument("--title", default=None, help="Window target (where / on-current / move-window)")
+    dp.add_argument("--process", type=int, default=None)
+    dp.add_argument("--timeout", type=float, default=5.0)
+    dp.add_argument("--desktop", type=int, default=None, help="Filter `windows` to this desktop index")
+    dp.add_argument("--to", type=int, default=None, help="Target desktop index for move-window")
+    dp.set_defaults(func=cmd_desktop)
+
     dr = sub.add_parser("drag", help="Drag from --start to --end (x,y pairs)")
     dr.add_argument("--start", required=True, help="x,y")
     dr.add_argument("--end", required=True, help="x,y")

{pctr-0.2.2 → pctr-0.3.0}/src/pctr/data/AGENTS.md

@@ -19,5 +19,6 @@ pctr look --for "a red error icon"    # fallback: open-vocab object detection
 - Exit codes: `0` ok, `1` no match, `2` usage error (clean `error:` line, no tracebacks).
 - Games ignore synthetic input: add `--direct` to mouse/key commands.
 - Electron apps expose only the frame - use `focus` + global keys, or CDP.
+- Virtual desktops: `pctr desktop list/windows/where/on-current` (read-only), `next/prev/new/close` (switches the active desktop), `move-window --to N`. UIA actions (`invoke`/`set`) reach windows on other desktops without switching; raw mouse/keys do not. `move-window` is blocked on Win11.
 
 Run `pctr -h` for all commands. Full skill: `pctr skill`.

{pctr-0.2.2 → pctr-0.3.0}/src/pctr/data/SKILL.md

@@ -45,6 +45,7 @@ back as `--name`, `--control-type`, `--auto-id`.
 | `wait --title RE --name RE [--timeout S]` | Wait for an element. |
 | `shot [--title RE] --out PATH` | Screenshot the screen or a window. |
 | `size` | Print primary screen size as WxH. |
+| `desktop <action>` | Windows virtual desktops: `list` / `windows` / `where` / `on-current` / `next` / `prev` / `new` / `close` / `move-window`. |
 | `move --x N --y N` / `down` / `up` / `hold --ms N` | Raw mouse control (`--direct` for games). |
 | `drag --start x,y --end x,y [--duration S]` | Drag between points. |
 | `keydown / keyup --key a` / `keyhold --key w --ms 1500` | Hold keyboard keys (`--direct` for games). |
@@ -89,6 +90,24 @@ Errors print a single `error: ...` line (no tracebacks).
 - **Canvas / games / custom UI**: no tree - use `ocr` for text and `look` for objects.
 - **Games** (Minecraft, Trailmakers, ...) ignore synthetic input from `pyautogui`; pass `--direct` to mouse/key commands to use `pydirectinput`, and `pctr focus` the game first.
 
+## Virtual desktops
+
+pctr can see and drive windows across Windows virtual desktops.
+
+```bash
+pctr desktop list                       # desktops (#index, guid, [current], window count)
+pctr desktop windows --desktop 1        # windows living on desktop #1
+pctr desktop where --title "App"        # which desktop a window is on
+pctr desktop on-current --title "App"   # True / False
+pctr desktop next | prev | new | close  # switch (moves the active desktop)
+pctr desktop move-window --title RE --to N
+```
+
+- **Switching** (`next`/`prev`/`new`/`close`) uses the standard Win+Ctrl hotkeys and moves the **active** desktop (you included). Add `--direct` if a game has focus.
+- **Cross-desktop control:** UIA element actions (`click --method invoke`, `set`) reach windows on *another* desktop **without switching**, because they go through the accessibility layer, not screen input. Raw mouse/keys (mouse-fallback click, global `type`/`keys`, `--direct`) only affect the active desktop.
+- `list` / `windows` / `where` / `on-current` are read-only.
+- **`move-window` is blocked on Windows 11** - third-party `MoveWindowToDesktop` returns `Access denied` (works on Windows 10). Reads and UIA actions still work cross-desktop.
+
 ## OCR / YOLO notes
 
 - `ocrfind`/`ocrclick` match **one word at a time** (Windows OCR tokenizes into words) - match a single distinctive word, not a phrase.

pctr-0.3.0/src/pctr.egg-info/PKG-INFO

@@ -0,0 +1,136 @@
+Metadata-Version: 2.4
+Name: pctr
+Version: 0.3.0
+Summary: Element-based Windows UI Automation CLI for AI agents. Find controls by name/type/id and click, type, hold, drag, and screenshot - no pixel coordinates.
+Author: Space-lab515
+License-Expression: MIT
+Keywords: automation,windows,uia,pywinauto,desktop,agent,computer-use,rpa
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Win32 (MS Windows)
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: System :: Networking
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pywinauto>=0.6.8; platform_system == "Windows"
+Requires-Dist: uiautomation>=2.0; platform_system == "Windows"
+Requires-Dist: pyautogui>=0.9.54; platform_system == "Windows"
+Requires-Dist: pydirectinput>=1.0.4; platform_system == "Windows"
+Requires-Dist: pillow>=9.0; platform_system == "Windows"
+Provides-Extra: vision
+Requires-Dist: ultralytics>=8.2; extra == "vision"
+Dynamic: license-file
+
+# pctr
+
+Element-based Windows UI Automation CLI for AI agents.
+
+Instead of screenshotting and guessing pixel coordinates, `pctr` finds controls
+by their **name / control type / automation id** and acts on them. Window moves,
+DPI changes, and layout shifts don't break your script.
+
+Built on `pywinauto` (UI Automation) with `pyautogui` / `pydirectinput` for raw
+mouse and keyboard input. On top of UIA it also offers **OCR**
+(`Windows.Media.Ocr`) and **YOLO-World** open-vocabulary detection for text and
+objects that aren't exposed as controls.
+
+## Install
+
+```bash
+pip install pctr
+pip install "pctr[vision]"   # adds ultralytics for `pctr look`
+```
+
+Windows only.
+
+## Commands
+
+| Command | What it does |
+|---------|--------------|
+| `pctr windows [--filter RE] [--process PID]` | List top-level windows (`pid=… \| title`). |
+| `pctr tree --title RE [--depth N] [--limit N]` | Dump the UIA control tree of a window. |
+| `pctr find --title RE [--name RE] [--limit N] [--nth N]` | List matching elements. |
+| `pctr click --title RE --name RE [--method auto\|invoke\|mouse] [--dbl]` | Click an element. |
+| `pctr set --title RE --name RE --text S` | Set a field via the UIA ValuePattern (exact, instant). |
+| `pctr type [--title RE --name RE] --text S [--delay 0.03] [--chunk 1]` | Send keystrokes (see notes). |
+| `pctr keys --keys "{ENTER}"` | Send a global key combo (pywinauto syntax). |
+| `pctr hotkey --keys ctrl+s` | Send a modifier combo (prefer this over `keys "^s"`). |
+| `pctr focus --title RE` | Bring a window to the foreground. |
+| `pctr wait --title RE --name RE [--timeout S]` | Wait until an element exists. |
+| `pctr shot [--title RE] --out PATH` | Screenshot the screen or a window. |
+| `pctr size` | Print primary screen size as `WxH`. |
+| `pctr desktop <action>` | Virtual desktops: `list` / `windows` / `where` / `on-current` / `next` / `prev` / `new` / `close` / `move-window`. |
+| `pctr move --x N --y N` / `down` / `up` / `hold --ms N` | Raw mouse control (`--direct` for games). |
+| `pctr drag --start x,y --end x,y [--duration S]` | Drag between two points. |
+| `pctr keydown / keyup --key a` / `keyhold --key w --ms 1500` | Hold keyboard keys (`--direct` for games). |
+| `pctr ocr [--title RE]` | OCR the screen/window; list words with boxes. |
+| `pctr ocrfind --text RE` | OCR then list matches with click centers (single words). |
+| `pctr ocrclick --text RE [--nth N]` | OCR then click a word. |
+| `pctr look --for "a dog" [--title RE] [--conf X]` | YOLO-World detect objects by text prompt. |
+| `pctr lookclick --for "a dog" [--nth N]` | Detect then click the top match. |
+| `pctr skill [--install]` / `pctr setup` | Print/install the agent skill + an AGENTS.md section. |
+
+Common filters: `--title` (regex), `--name` (regex), `--control-type`
+(`Button`, `Edit`, `Pane`, `MenuItem`, …), `--auto-id`, `--nth`, `--process`.
+
+`click`, `set`, `wait` accept **any** selector - `--name` is not required.
+`tree`/`find` require a window selector (`--title` or `--process`).
+
+## The loop
+
+```bash
+pctr tree  --title "Notepad" --limit 60      # see what's clickable
+pctr click --title "Notepad" --name "^File$"
+pctr set   --title "Notepad" --name "Text Editor" --text "hello"
+```
+
+## Virtual desktops
+
+```bash
+pctr desktop list                       # desktops (#index, guid, [current], window count)
+pctr desktop windows --desktop 1        # windows on desktop #1
+pctr desktop where --title "App"        # which desktop a window is on
+pctr desktop on-current --title "App"   # True / False
+pctr desktop next | prev | new | close  # switch (moves the active desktop)
+pctr desktop move-window --title RE --to N
+```
+
+- Switching uses the standard Win+Ctrl hotkeys and moves the **active** desktop
+  (you included). Add `--direct` if a game has focus.
+- **Cross-desktop control:** UIA actions (`click --method invoke`, `set`) reach
+  windows on another desktop **without switching** (accessibility layer, not
+  screen input). Raw mouse/keys only affect the active desktop.
+- **`move-window` is blocked on Windows 11** (third-party `MoveWindowToDesktop`
+  returns `Access denied`); it works on Windows 10.
+
+## Exit codes
+
+- `0` success, `1` no match / window not found, `2` usage/validation error.
+- Errors print one `error: …` line - no tracebacks.
+
+## Notes
+
+- Prefer `set` (ValuePattern) over `type` for text fields - it can't drop or
+  reorder characters. `type`'s element path uses pywinauto `type_keys`, which
+  interprets `{ } + ^ % ~ ( )` as key syntax, so it is **not** literal.
+- Global `type` / `keys` go to the OS-focused window - `pctr focus` first.
+- Prefer `pctr hotkey --keys ctrl+s` over `keys "^s"`; the `^` modifier is
+  fragile through shells.
+- `ocrfind`/`ocrclick` match one **word** at a time (Windows OCR tokenizes words).
+- `look` reads its model from `--model` or the `PCTR_YOLO_MODEL` env var; progress
+  bars go to stderr so stdout stays parseable.
+- Qt apps expose a rich tree including embedded webviews. Electron apps expose
+  only the window frame (use `focus` + global keys, or drive them over CDP).
+- Games ignore synthetic input - pass `--direct` to mouse/key commands and focus
+  the game first.
+
+## License
+
+MIT

pctr-0.2.2/PKG-INFO

@@ -1,96 +0,0 @@
-Metadata-Version: 2.4
-Name: pctr
-Version: 0.2.2
-Summary: Element-based Windows UI Automation CLI for AI agents. Find controls by name/type/id and click, type, hold, drag, and screenshot - no pixel coordinates.
-Author: Space-lab515
-License-Expression: MIT
-Keywords: automation,windows,uia,pywinauto,desktop,agent,computer-use,rpa
-Classifier: Development Status :: 4 - Beta
-Classifier: Environment :: Win32 (MS Windows)
-Classifier: Intended Audience :: Developers
-Classifier: Operating System :: Microsoft :: Windows
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Topic :: Software Development :: Testing
-Classifier: Topic :: System :: Networking
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: pywinauto>=0.6.8; platform_system == "Windows"
-Requires-Dist: uiautomation>=2.0; platform_system == "Windows"
-Requires-Dist: pyautogui>=0.9.54; platform_system == "Windows"
-Requires-Dist: pydirectinput>=1.0.4; platform_system == "Windows"
-Requires-Dist: pillow>=9.0; platform_system == "Windows"
-Provides-Extra: vision
-Requires-Dist: ultralytics>=8.2; extra == "vision"
-Dynamic: license-file
-
-# pctr
-
-Element-based Windows UI Automation CLI for AI agents.
-
-Instead of screenshotting and guessing pixel coordinates, `pctr` finds controls
-by their **name / control type / automation id** and acts on them. Window moves,
-DPI changes, and layout shifts don't break your script.
-
-Built on `pywinauto` (UI Automation) with `pyautogui` / `pydirectinput` for raw
-mouse and keyboard input.
-
-## Install
-
-```bash
-pip install pctr
-```
-
-Windows only.
-
-## Commands
-
-| Command | What it does |
-|---------|--------------|
-| `pctr windows [--filter RE]` | List top-level windows (`pid=… \| title`). |
-| `pctr tree --title RE [--limit N]` | Dump the UIA control tree of a window. |
-| `pctr find --title RE --name RE [--control-type T] [--auto-id ID]` | List matching elements. |
-| `pctr click --title RE --name RE [--method auto\|invoke\|mouse] [--dbl]` | Click an element. |
-| `pctr set --title RE --name RE --text S` | Set a field via the UIA ValuePattern. |
-| `pctr type [--title RE --name RE] --text S [--delay 0.03] [--chunk 1]` | Type literal text. |
-| `pctr keys --keys "{ENTER}"` | Send a global key combo. |
-| `pctr hotkey --keys win+shift+s` | Send a modifier combo. |
-| `pctr focus --title RE` | Bring a window to the foreground. |
-| `pctr wait --title RE --name RE [--timeout S]` | Wait until an element exists. |
-| `pctr shot [--title RE] --out PATH` | Screenshot the screen or a window. |
-| `pctr size` | Print primary screen size as `WxH`. |
-| `pctr move --x N --y N` | Move the cursor. |
-| `pctr down / up [--button left]` | Press / release a mouse button. |
-| `pctr hold --ms 1000 [--x N --y N]` | Press, hold, release. |
-| `pctr drag --start x,y --end x,y [--duration S]` | Drag between two points. |
-| `pctr keydown / keyup --key a` | Hold / release a keyboard key. |
-| `pctr keyhold --key w --ms 1500` | Hold a key for a duration. |
-
-Common filters: `--title` (regex), `--name` (regex), `--control-type`
-(`Button`, `Edit`, `Pane`, `MenuItem`, …), `--auto-id`, `--nth`, `--process`.
-
-## The loop
-
-```bash
-pctr tree  --title "Notepad" --limit 60      # see what's clickable
-pctr click --title "Notepad" --name "^File$"
-pctr set   --title "Notepad" --name "Text Editor" --text "hello"
-```
-
-## Notes
-
-- Prefer `set` (ValuePattern) over `type` for text fields - it can't drop or
-  reorder characters.
-- `type` is sent in small chunks (`--chunk 1 --delay 0.03`); raise the delay for
-  Qt/Electron apps that garble fast input.
-- Global `type` / `keys` go to the OS-focused window - `pctr focus` first.
-- Qt apps expose a rich tree including embedded webviews. Electron apps expose
-  only the window frame unless accessibility is enabled (use `focus` + `keys`).
-
-## License
-
-MIT

pctr-0.2.2/README.md

@@ -1,66 +0,0 @@
-# pctr
-
-Element-based Windows UI Automation CLI for AI agents.
-
-Instead of screenshotting and guessing pixel coordinates, `pctr` finds controls
-by their **name / control type / automation id** and acts on them. Window moves,
-DPI changes, and layout shifts don't break your script.
-
-Built on `pywinauto` (UI Automation) with `pyautogui` / `pydirectinput` for raw
-mouse and keyboard input.
-
-## Install
-
-```bash
-pip install pctr
-```
-
-Windows only.
-
-## Commands
-
-| Command | What it does |
-|---------|--------------|
-| `pctr windows [--filter RE]` | List top-level windows (`pid=… \| title`). |
-| `pctr tree --title RE [--limit N]` | Dump the UIA control tree of a window. |
-| `pctr find --title RE --name RE [--control-type T] [--auto-id ID]` | List matching elements. |
-| `pctr click --title RE --name RE [--method auto\|invoke\|mouse] [--dbl]` | Click an element. |
-| `pctr set --title RE --name RE --text S` | Set a field via the UIA ValuePattern. |
-| `pctr type [--title RE --name RE] --text S [--delay 0.03] [--chunk 1]` | Type literal text. |
-| `pctr keys --keys "{ENTER}"` | Send a global key combo. |
-| `pctr hotkey --keys win+shift+s` | Send a modifier combo. |
-| `pctr focus --title RE` | Bring a window to the foreground. |
-| `pctr wait --title RE --name RE [--timeout S]` | Wait until an element exists. |
-| `pctr shot [--title RE] --out PATH` | Screenshot the screen or a window. |
-| `pctr size` | Print primary screen size as `WxH`. |
-| `pctr move --x N --y N` | Move the cursor. |
-| `pctr down / up [--button left]` | Press / release a mouse button. |
-| `pctr hold --ms 1000 [--x N --y N]` | Press, hold, release. |
-| `pctr drag --start x,y --end x,y [--duration S]` | Drag between two points. |
-| `pctr keydown / keyup --key a` | Hold / release a keyboard key. |
-| `pctr keyhold --key w --ms 1500` | Hold a key for a duration. |
-
-Common filters: `--title` (regex), `--name` (regex), `--control-type`
-(`Button`, `Edit`, `Pane`, `MenuItem`, …), `--auto-id`, `--nth`, `--process`.
-
-## The loop
-
-```bash
-pctr tree  --title "Notepad" --limit 60      # see what's clickable
-pctr click --title "Notepad" --name "^File$"
-pctr set   --title "Notepad" --name "Text Editor" --text "hello"
-```
-
-## Notes
-
-- Prefer `set` (ValuePattern) over `type` for text fields - it can't drop or
-  reorder characters.
-- `type` is sent in small chunks (`--chunk 1 --delay 0.03`); raise the delay for
-  Qt/Electron apps that garble fast input.
-- Global `type` / `keys` go to the OS-focused window - `pctr focus` first.
-- Qt apps expose a rich tree including embedded webviews. Electron apps expose
-  only the window frame unless accessibility is enabled (use `focus` + `keys`).
-
-## License
-
-MIT

pctr-0.2.2/src/pctr/__init__.py

@@ -1 +0,0 @@
-__version__ = "0.2.2"

pctr-0.2.2/src/pctr.egg-info/PKG-INFO

@@ -1,96 +0,0 @@
-Metadata-Version: 2.4
-Name: pctr
-Version: 0.2.2
-Summary: Element-based Windows UI Automation CLI for AI agents. Find controls by name/type/id and click, type, hold, drag, and screenshot - no pixel coordinates.
-Author: Space-lab515
-License-Expression: MIT
-Keywords: automation,windows,uia,pywinauto,desktop,agent,computer-use,rpa
-Classifier: Development Status :: 4 - Beta
-Classifier: Environment :: Win32 (MS Windows)
-Classifier: Intended Audience :: Developers
-Classifier: Operating System :: Microsoft :: Windows
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Topic :: Software Development :: Testing
-Classifier: Topic :: System :: Networking
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: pywinauto>=0.6.8; platform_system == "Windows"
-Requires-Dist: uiautomation>=2.0; platform_system == "Windows"
-Requires-Dist: pyautogui>=0.9.54; platform_system == "Windows"
-Requires-Dist: pydirectinput>=1.0.4; platform_system == "Windows"
-Requires-Dist: pillow>=9.0; platform_system == "Windows"
-Provides-Extra: vision
-Requires-Dist: ultralytics>=8.2; extra == "vision"
-Dynamic: license-file
-
-# pctr
-
-Element-based Windows UI Automation CLI for AI agents.
-
-Instead of screenshotting and guessing pixel coordinates, `pctr` finds controls
-by their **name / control type / automation id** and acts on them. Window moves,
-DPI changes, and layout shifts don't break your script.
-
-Built on `pywinauto` (UI Automation) with `pyautogui` / `pydirectinput` for raw
-mouse and keyboard input.
-
-## Install
-
-```bash
-pip install pctr
-```
-
-Windows only.
-
-## Commands
-
-| Command | What it does |
-|---------|--------------|
-| `pctr windows [--filter RE]` | List top-level windows (`pid=… \| title`). |
-| `pctr tree --title RE [--limit N]` | Dump the UIA control tree of a window. |
-| `pctr find --title RE --name RE [--control-type T] [--auto-id ID]` | List matching elements. |
-| `pctr click --title RE --name RE [--method auto\|invoke\|mouse] [--dbl]` | Click an element. |
-| `pctr set --title RE --name RE --text S` | Set a field via the UIA ValuePattern. |
-| `pctr type [--title RE --name RE] --text S [--delay 0.03] [--chunk 1]` | Type literal text. |
-| `pctr keys --keys "{ENTER}"` | Send a global key combo. |
-| `pctr hotkey --keys win+shift+s` | Send a modifier combo. |
-| `pctr focus --title RE` | Bring a window to the foreground. |
-| `pctr wait --title RE --name RE [--timeout S]` | Wait until an element exists. |
-| `pctr shot [--title RE] --out PATH` | Screenshot the screen or a window. |
-| `pctr size` | Print primary screen size as `WxH`. |
-| `pctr move --x N --y N` | Move the cursor. |
-| `pctr down / up [--button left]` | Press / release a mouse button. |
-| `pctr hold --ms 1000 [--x N --y N]` | Press, hold, release. |
-| `pctr drag --start x,y --end x,y [--duration S]` | Drag between two points. |
-| `pctr keydown / keyup --key a` | Hold / release a keyboard key. |
-| `pctr keyhold --key w --ms 1500` | Hold a key for a duration. |
-
-Common filters: `--title` (regex), `--name` (regex), `--control-type`
-(`Button`, `Edit`, `Pane`, `MenuItem`, …), `--auto-id`, `--nth`, `--process`.
-
-## The loop
-
-```bash
-pctr tree  --title "Notepad" --limit 60      # see what's clickable
-pctr click --title "Notepad" --name "^File$"
-pctr set   --title "Notepad" --name "Text Editor" --text "hello"
-```
-
-## Notes
-
-- Prefer `set` (ValuePattern) over `type` for text fields - it can't drop or
-  reorder characters.
-- `type` is sent in small chunks (`--chunk 1 --delay 0.03`); raise the delay for
-  Qt/Electron apps that garble fast input.
-- Global `type` / `keys` go to the OS-focused window - `pctr focus` first.
-- Qt apps expose a rich tree including embedded webviews. Electron apps expose
-  only the window frame unless accessibility is enabled (use `focus` + `keys`).
-
-## License
-
-MIT