@iaforged/context-code 2.3.3 → 2.3.6

package/dist/src/utils/computerControlMcp/server/README.md

@@ -0,0 +1,193 @@
+# Computer Control MCP
+
+### MCP server that provides computer control capabilities, like mouse, keyboard, OCR, etc. using PyAutoGUI, RapidOCR, ONNXRuntime. Similar to 'computer-use' by Anthropic. With Zero External Dependencies.
+
+<div align="center" style="text-align:center;font-family: monospace; display: flex; align-items: center; justify-content: center; width: 100%; gap: 10px">
+    <a href="https://nextjs-boilerplate-ashy-nine-64.vercel.app/demo-computer-control"><img
+            src="https://komarev.com/ghpvc/?username=AB498&label=DEMO&style=for-the-badge&color=CC0000" /></a>
+    <a href="https://discord.gg/ZeeqSBpjU2"><img
+            src="https://img.shields.io/discord/1095854826786668545?style=for-the-badge&color=0000CC" alt="Discord"></a>
+    <a href="https://img.shields.io/badge/License-MIT-yellow.svg"><img
+            src="https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge&color=00CC00" alt="License: MIT"></a>
+    <a href="https://pypi.org/project/computer-control-mcp"><img
+            src="https://img.shields.io/pypi/v/computer-control-mcp?style=for-the-badge" alt="PyPi"></a>
+</div>
+
+---
+
+![MCP Computer Control Demo](https://github.com/AB498/computer-control-mcp/blob/main/demonstration.gif?raw=true)
+
+## Quick Usage (MCP Setup Using `uvx`)
+
+***Note:** Running `uvx computer-control-mcp@latest` for the first time will download python dependencies (around 70MB) which may take some time. Recommended to run this in a terminal before using it as MCP. Subsequent runs will be instant.* 
+
+```json
+{
+  "mcpServers": {
+    "computer-control-mcp": {
+      "command": "uvx",
+      "args": ["computer-control-mcp@latest"]
+    }
+  }
+}
+```
+
+OR install globally with `pip`:
+```bash
+pip install computer-control-mcp
+```
+Then run the server with:
+```bash
+computer-control-mcp # instead of uvx computer-control-mcp, so you can use the latest version, also you can `uv cache clean` to clear the cache and `uvx` again to use latest version.
+```
+
+## Features
+
+- Control mouse movements and clicks
+- Type text at the current cursor position
+- Take screenshots of the entire screen or specific windows with optional saving to downloads directory
+- Extract text from screenshots using OCR (Optical Character Recognition)
+- List and activate windows
+- Press keyboard keys
+- Drag and drop operations
+- Enhanced screenshot capture for GPU-accelerated windows (Windows only)
+
+## Note on GPU-accelerated Windows
+
+Traditional screenshot methods like GDI/PrintWindow fail to capture GPU-accelerated windows, resulting in black screens. This impacts games, media players, Electron apps, browsers with GPU acceleration, streaming software, and CAD tools. Use WGC through `take_screenshot` tool's flag or `ENV` variable
+
+## Configuration
+
+### Custom Screenshot Directory
+
+By default, screenshots are saved to the OS downloads directory. You can customize this by setting the `COMPUTER_CONTROL_MCP_SCREENSHOT_DIR` environment variable:
+
+```json
+{
+  "mcpServers": {
+    "computer-control-mcp": {
+      "command": "uvx",
+      "args": ["computer-control-mcp@latest"],
+      "env": {
+        "COMPUTER_CONTROL_MCP_SCREENSHOT_DIR": "C:\\Users\\YourName\\Pictures\\Screenshots"
+      }
+    }
+  }
+}
+```
+
+Or set it system-wide:
+```bash
+# Windows (PowerShell)
+$env:COMPUTER_CONTROL_MCP_SCREENSHOT_DIR = "C:\Users\YourName\Pictures\Screenshots"
+
+# macOS/Linux
+export COMPUTER_CONTROL_MCP_SCREENSHOT_DIR="/home/yourname/Pictures/Screenshots"
+```
+
+If the specified directory doesn't exist, the server will fall back to the default downloads directory.
+
+### Automatic WGC for Specific Windows
+
+You can configure the system to automatically use Windows Graphics Capture (WGC) for specific windows by setting the `COMPUTER_CONTROL_MCP_WGC_PATTERNS` environment variable. This variable should contain comma-separated patterns that match window titles:
+
+```json
+{
+  "mcpServers": {
+    "computer-control-mcp": {
+      "command": "uvx",
+      "args": ["computer-control-mcp@latest"],
+      "env": {
+        "COMPUTER_CONTROL_MCP_WGC_PATTERNS": "obs, discord, game, steam"
+      }
+    }
+  }
+}
+```
+
+Or set it system-wide:
+```bash
+# Windows (PowerShell)
+$env:COMPUTER_CONTROL_MCP_WGC_PATTERNS = "obs, discord, game, steam"
+
+# macOS/Linux
+export COMPUTER_CONTROL_MCP_WGC_PATTERNS="obs, discord, game, steam"
+```
+
+When this variable is set, any window whose title contains any of the specified patterns will automatically use WGC for screenshot capture, eliminating black screens for GPU-accelerated applications.
+
+## Available Tools
+
+### Mouse Control
+- `click_screen(x: int, y: int)`: Click at specified screen coordinates
+- `move_mouse(x: int, y: int)`: Move mouse cursor to specified coordinates
+- `drag_mouse(from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 0.5)`: Drag mouse from one position to another
+- `mouse_down(button: str = "left")`: Hold down a mouse button ('left', 'right', 'middle')
+- `mouse_up(button: str = "left")`: Release a mouse button ('left', 'right', 'middle')
+
+### Keyboard Control
+- `type_text(text: str)`: Type the specified text at current cursor position
+- `press_key(key: str)`: Press a specified keyboard key
+- `key_down(key: str)`: Hold down a specific keyboard key until released
+- `key_up(key: str)`: Release a specific keyboard key
+- `press_keys(keys: Union[str, List[Union[str, List[str]]]])`: Press keyboard keys (supports single keys, sequences, and combinations)
+
+### Screen and Window Management
+- `take_screenshot(title_pattern: str = None, use_regex: bool = False, threshold: int = 60, scale_percent_for_ocr: int = None, save_to_downloads: bool = False, use_wgc: bool = False)`: Capture screen or window
+- `take_screenshot_with_ocr(title_pattern: str = None, use_regex: bool = False, threshold: int = 10, scale_percent_for_ocr: int = None, save_to_downloads: bool = False)`: Extract adn return text with coordinates using OCR from screen or window
+- `get_screen_size()`: Get current screen resolution
+- `list_windows()`: List all open windows
+- `activate_window(title_pattern: str, use_regex: bool = False, threshold: int = 60)`: Bring specified window to foreground
+- `wait_milliseconds(milliseconds: int)`: Wait for a specified number of milliseconds
+
+## Development
+
+### Setting up the Development Environment
+
+```bash
+# Clone the repository
+git clone https://github.com/AB498/computer-control-mcp.git
+cd computer-control-mcp
+
+# Build/Run:
+
+# 1. Install in development mode | Meaning that your edits to source code will be reflected in the installed package.
+pip install -e .
+
+# Then Start server | This is equivalent to `uvx computer-control-mcp@latest` just the local code is used
+computer-control-mcp
+
+# -- OR --
+
+# 2. Build after `pip install hatch` | This needs version increment in orer to reflect code changes
+hatch build
+
+# Windows
+$latest = Get-ChildItem .\dist\*.whl | Sort-Object LastWriteTime -Descending | Select-Object -First 1
+pip install $latest.FullName --upgrade 
+
+# Non-windows
+pip install dist/*.whl --upgrade
+
+# Run
+computer-control-mcp
+```
+
+### Running Tests
+
+```bash
+python -m pytest
+```
+
+## API Reference
+
+See the [API Reference](docs/api.md) for detailed information about the available functions and classes.
+
+## License
+
+MIT
+
+## For more information or help
+
+- [Email (abcd49800@gmail.com)](mailto:abcd49800@gmail.com)
+- [Discord (CodePlayground)](https://discord.gg/ZeeqSBpjU2)

package/dist/src/utils/computerControlMcp/server/pyproject.toml

@@ -0,0 +1,52 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "computer-control-mcp"
+version = "0.3.10"
+description = "MCP server that provides computer control capabilities, like mouse, keyboard, OCR, etc. using PyAutoGUI, RapidOCR, ONNXRuntime. Similar to 'computer-use' by Anthropic. With Zero External Dependencies."
+readme = "README.md"
+requires-python = ">=3.10"
+license = {text = "MIT"}
+authors = [{name = "AB498", email = "abcd49800@gmail.com"}]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Libraries",
+    "Topic :: Utilities"
+]
+dependencies = [
+    "pyautogui==0.9.54",
+    "mcp[cli]==1.13.0",
+    "pillow==11.3.0",
+    "pygetwindow==0.0.9; sys_platform=='win32'",
+    "pywinctl==0.4.1",
+    "fuzzywuzzy==0.18.0",
+    "rapidocr==3.3.1",
+    "onnxruntime==1.22.0",
+    "rapidocr_onnxruntime==1.2.3",
+    "opencv-python==4.12.0.88",
+    "python-Levenshtein>=0.20.9",
+    "mss>=7.0.0",
+    "windows-capture>=1.0.0; sys_platform=='win32'"
+]
+
+[project.urls]
+Homepage = "https://github.com/AB498/computer-control-mcp"
+Issues = "https://github.com/AB498/computer-control-mcp/issues"
+Documentation = "https://github.com/AB498/computer-control-mcp#readme"
+
+[project.scripts]
+computer-control-mcp = "computer_control_mcp.cli:main"
+computer-control-mcp-server = "computer_control_mcp.server:main"
+
+[tool.hatch.build]
+sources = ["src"]
+packages = ["src/computer_control_mcp"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/computer_control_mcp"]

package/dist/src/utils/computerControlMcp/server/smithery.yaml

@@ -0,0 +1,13 @@
+# Smithery configuration file: https://smithery.ai/docs/config#smitheryyaml
+
+startCommand:
+  type: stdio
+  configSchema:
+    # JSON Schema defining the configuration options for the MCP.
+    type: object
+    description: Empty config
+  commandFunction:
+    # A JS function that produces the CLI command based on the given config to start the MCP on stdio.
+    |-
+    (config) => ({ command: 'python', args: ['src/computer_control_mcp/core.py'] })
+  exampleConfig: {}

package/dist/src/utils/computerControlMcp/server/src/README.md

@@ -0,0 +1,12 @@
+# Computer Control MCP Source Code
+
+This directory contains the source code for the Computer Control MCP package.
+
+## Structure
+
+- `computer_control_mcp/`: Main package directory
+  - `__init__.py`: Package initialization
+  - `__main__.py`: Entry point for running as a module
+  - `core.py`: Core functionality
+  - `cli.py`: Command-line interface
+  - `gui.py`: Graphical user interface for testing

package/dist/src/utils/computerControlMcp/server/src/computer_control_mcp/__init__.py

@@ -0,0 +1,11 @@
+"""
+Computer Control MCP - Python package for computer control via MCP.
+
+This package provides computer control capabilities using PyAutoGUI through a
+Model Context Protocol (MCP) server.
+"""
+
+from computer_control_mcp.core import mcp, main
+
+__version__ = "0.1.2"
+__all__ = ["mcp", "main"]

package/dist/src/utils/computerControlMcp/server/src/computer_control_mcp/__main__.py

@@ -0,0 +1,21 @@
+"""
+Entry point for running the Computer Control MCP as a module.
+
+This module serves as the main entry point for the package.
+When executed directly (e.g., with `python -m computer_control_mcp`),
+it will run the CLI interface.
+
+For CLI functionality, use:
+    computer-control-mcp <command>
+    python -m computer_control_mcp <command>
+"""
+
+from computer_control_mcp.cli import main as cli_main
+
+def main():
+    """Main entry point for the package."""
+    # Run the CLI when the module is executed directly
+    cli_main()
+
+if __name__ == "__main__":
+    main()

package/dist/src/utils/computerControlMcp/server/src/computer_control_mcp/cli.py

@@ -0,0 +1,128 @@
+"""
+Command-line interface for Computer Control MCP.
+
+This module provides a command-line interface for interacting with the Computer Control MCP.
+"""
+
+import argparse
+import sys
+from computer_control_mcp.core import mcp, main as run_server
+
+def parse_args():
+    """Parse command-line arguments."""
+    parser = argparse.ArgumentParser(description="Computer Control MCP CLI")
+
+    subparsers = parser.add_subparsers(dest="command", help="Command to run")
+
+    # Server command
+    server_parser = subparsers.add_parser("server", help="Run the MCP server")
+
+    # Click command
+    click_parser = subparsers.add_parser("click", help="Click at specified coordinates")
+    click_parser.add_argument("x", type=int, help="X coordinate")
+    click_parser.add_argument("y", type=int, help="Y coordinate")
+
+    # Type text command
+    type_parser = subparsers.add_parser("type", help="Type text at current cursor position")
+    type_parser.add_argument("text", help="Text to type")
+
+    # Screenshot command
+    screenshot_parser = subparsers.add_parser("screenshot", help="Take a screenshot")
+    screenshot_parser.add_argument("--mode", choices=["all_windows", "single_window", "whole_screen"],
+                                  default="whole_screen", help="Screenshot mode")
+    screenshot_parser.add_argument("--title", help="Window title pattern (for single_window mode)")
+    screenshot_parser.add_argument("--regex", action="store_true", help="Use regex for title matching")
+    screenshot_parser.add_argument("--output", help="Output file path (if not provided, saves to downloads directory)")
+    screenshot_parser.add_argument("--no-save", action="store_true", help="Don't save images to downloads directory")
+
+    # List windows command
+    subparsers.add_parser("list-windows", help="List all open windows")
+
+    # GUI command
+    subparsers.add_parser("gui", help="Launch the GUI test harness")
+
+    return parser.parse_args()
+
+def main():
+    """Main entry point for the CLI."""
+    args = parse_args()
+
+    if args.command == "server":
+        run_server()
+
+    elif args.command == "click":
+        # Call the tool using the call_tool method
+        import asyncio
+        result = asyncio.run(mcp.call_tool("click_screen", {"x": args.x, "y": args.y}))
+        print(result)
+
+    elif args.command == "type":
+        # Call the tool using the call_tool method
+        import asyncio
+        result = asyncio.run(mcp.call_tool("type_text", {"text": args.text}))
+        print(result)
+
+    elif args.command == "screenshot":
+        if args.mode == "single_window" and not args.title:
+            print("Error: --title is required for single_window mode")
+            sys.exit(1)
+
+        # Call the tool using the call_tool method
+        import asyncio
+        result = asyncio.run(mcp.call_tool("take_screenshot", {
+            "mode": args.mode,
+            "title_pattern": args.title,
+            "use_regex": args.regex,
+            "save_to_downloads": not args.no_save
+        }))
+
+        if args.output:
+            # Save the screenshot to a specific file path provided by user
+            with open(args.output, "wb") as f:
+                f.write(result.image.data)
+            print(f"Screenshot saved to {args.output}")
+        elif hasattr(result, 'file_path'):
+            # If image was saved to downloads, show the path
+            print(f"Screenshot saved to {result.file_path}")
+        else:
+            print("Screenshot taken successfully")
+
+        # If we have multiple results (all_windows mode)
+        if args.mode == "all_windows" and isinstance(result, list):
+            print("\nAll screenshots:")
+            for i, item in enumerate(result):
+                if hasattr(item, 'file_path'):
+                    window_title = item.window_info.title if hasattr(item, 'window_info') else f"Window {i+1}"
+                    print(f"{i+1}. {window_title}: {item.file_path}")
+
+    elif args.command == "list-windows":
+        # Call the tool using the call_tool method
+        import asyncio
+        result = asyncio.run(mcp.call_tool("list_windows", {}))
+
+        # Parse the result
+        windows = []
+        for item in result:
+            if hasattr(item, 'text'):
+                try:
+                    import json
+                    window_info = json.loads(item.text)
+                    windows.append(window_info)
+                except json.JSONDecodeError:
+                    print(f"Failed to parse window info: {item.text}")
+
+        # Display the windows
+        for i, window in enumerate(windows):
+            print(f"{i+1}. {window.get('title')} ({window.get('width')}x{window.get('height')})")
+
+    elif args.command == "gui":
+        from computer_control_mcp.gui import main as run_gui
+        run_gui()
+
+    else:
+        # When no command is specified, run the server by default
+        print("MCP server started!")
+        run_server()
+
+if __name__ == "__main__":
+    main()