screen-vision 0.2.1__tar.gz

screen_vision-0.2.1/PKG-INFO

@@ -0,0 +1,22 @@
+Metadata-Version: 2.4
+Name: screen-vision
+Version: 0.2.1
+Summary: MCP server that gives Claude Code the ability to see your screen
+Project-URL: Repository, https://github.com/avicuna/screen-vision
+Requires-Python: >=3.11
+Requires-Dist: mcp[cli]>=1.0.0
+Requires-Dist: mss>=9.0.0
+Requires-Dist: Pillow>=10.0.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: qrcode[pil]>=7.4.0
+Requires-Dist: websockets>=12.0
+Requires-Dist: httpx>=0.27.0
+Provides-Extra: full
+Requires-Dist: paddleocr>=2.7.0; extra == "full"
+Requires-Dist: pytesseract>=0.3.10; extra == "full"
+Requires-Dist: faster-whisper>=1.0.0; extra == "full"
+Requires-Dist: sounddevice>=0.4.6; extra == "full"
+Requires-Dist: opencv-python>=4.8.0; extra == "full"
+Provides-Extra: test
+Requires-Dist: pytest>=8.0.0; extra == "test"
+Requires-Dist: pytest-asyncio>=0.23.0; extra == "test"

screen_vision-0.2.1/README.md

@@ -0,0 +1,121 @@
+# Screen Vision MCP Server
+
+> Give Claude Code the ability to see your screen
+
+Screen Vision lets Claude capture screenshots, watch your screen in real-time with audio transcription, analyze video files, and read text via OCR. It runs locally as an MCP server — Claude sees what you see, when you ask.
+
+## Quick Start
+
+```bash
+pip install screen-vision
+```
+
+Then add to your Claude Code MCP config (`.mcp.json`):
+```json
+{
+  "mcpServers": {
+    "screen-vision": {
+      "command": "python3",
+      "args": ["-m", "screen_vision"]
+    }
+  }
+}
+```
+
+**Other install methods:**
+```bash
+# From git (with all optional extras)
+pip install "screen-vision[full]"
+
+# Local development
+pip install -e ".[full]"
+```
+
+**Optional system deps** (not required — tools gracefully degrade without them):
+```bash
+brew install tesseract   # Enables OCR (read_screen_text)
+brew install ffmpeg      # Enables video analysis (analyze_video)
+```
+
+## What You Can Say
+
+```
+"Take a screenshot of my screen"          → capture_screen
+"Capture the Chrome window"               → capture_window
+"Watch my screen for 1 minute"            → watch_screen (with audio transcription)
+"Analyze the video at ~/Downloads/demo.mp4" → analyze_video
+"Read the text on my screen"              → read_screen_text
+"What window am I in?"                    → get_active_context (no screenshot)
+"What's on my screen right now?"          → understand_screen (AI analysis)
+"Analyze this photo I AirDropped"         → analyze_image
+```
+
+## Tools (10)
+
+| Tool | What it does | Needs |
+|------|-------------|-------|
+| `capture_screen` | Full screen capture with delay + multi-monitor | — |
+| `capture_region` | Capture a specific rectangular area | — |
+| `capture_window` | Capture a window by title | — |
+| `list_monitors` | List displays with resolutions | — |
+| `get_active_context` | Window/cursor/monitor info (no image) | — |
+| `read_screen_text` | OCR text extraction from screen | tesseract |
+| `understand_screen` | AI-powered screen analysis | GenAI Gateway |
+| `analyze_image` | Analyze a dropped/AirDropped image file | — |
+| `watch_screen` | Watch screen with frame sampling + audio | ffmpeg (audio) |
+| `analyze_video` | Extract keyframes from video files | ffmpeg |
+
+## Security
+
+Screen Vision includes security controls for corporate environments:
+
+- **PII/PCI scanning** — Detects credit card numbers, SSNs, phone numbers, email addresses in OCR text
+- **App deny-list** — Blocks captures of Slack, Teams, Zoom, banking apps, password managers
+- **Call detection** — Blocks captures during active audio calls
+- **Rate limits** — 200 captures/session, 2s minimum interval, 5min max watch duration
+- **Audit logs** — All captures logged to `~/.screen-vision/audit.log`
+
+## Dependencies
+
+**Core** (always installed): `mcp[cli]`, `mss`, `Pillow`, `numpy`, `httpx`
+
+**Optional** (`pip install screen-vision[full]`):
+- `pytesseract` — OCR (needs `brew install tesseract`)
+- `faster-whisper` — Audio transcription
+- `sounddevice` — Audio recording
+- `opencv-python` — Video processing
+- `paddleocr` — Alternative OCR engine
+
+**Python 3.11+ required.**
+
+## Development
+
+```bash
+pip install -e ".[full,test]"
+pytest tests/ -v
+ruff check src/
+```
+
+### Project Structure
+```
+src/screen_vision/
+├── server.py          # MCP server — 13 tools
+├── capture.py         # Screen capture (mss)
+├── context.py         # Window/cursor/monitor info
+├── ocr.py             # OCR (pytesseract + paddleocr)
+├── security.py        # PII/PCI scanner + app deny-list
+├── watcher.py         # Screen watching + audio transcription
+├── video.py           # Video keyframe extraction (ffmpeg)
+├── audio.py           # Audio recording + Whisper transcription
+├── analyze.py         # Image analysis
+├── understanding.py   # AI screen analysis (GenAI Gateway)
+└── config.py          # Configuration
+```
+
+## Author
+
+**Alex Vicuna** — [github.com/avicuna](https://github.com/avicuna)
+
+## Contributing
+
+Issues and PRs welcome: https://github.com/avicuna/screen-vision

screen_vision-0.2.1/pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["setuptools>=68.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "screen-vision"
+version = "0.2.1"
+description = "MCP server that gives Claude Code the ability to see your screen"
+requires-python = ">=3.11"
+dependencies = [
+    "mcp[cli]>=1.0.0",
+    "mss>=9.0.0",
+    "Pillow>=10.0.0",
+    "numpy>=1.24.0",
+    "qrcode[pil]>=7.4.0",
+    "websockets>=12.0",
+    "httpx>=0.27.0",
+]
+
+[project.optional-dependencies]
+full = [
+    "paddleocr>=2.7.0",
+    "pytesseract>=0.3.10",
+    "faster-whisper>=1.0.0",
+    "sounddevice>=0.4.6",
+    "opencv-python>=4.8.0",
+]
+test = [
+    "pytest>=8.0.0",
+    "pytest-asyncio>=0.23.0",
+]
+
+[project.scripts]
+screen-vision-mcp = "screen_vision.server:main"
+
+[project.urls]
+Repository = "https://github.com/avicuna/screen-vision"
+
+[tool.setuptools.packages.find]
+where = ["src"]

screen_vision-0.2.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

screen_vision-0.2.1/src/screen_vision/__init__.py

@@ -0,0 +1 @@
+"""Screen Vision MCP Server — see your screen from Claude Code."""

screen_vision-0.2.1/src/screen_vision/__main__.py

@@ -0,0 +1,4 @@
+"""Allow running as `python -m screen_vision`."""
+from screen_vision.server import main
+
+main()

screen_vision-0.2.1/src/screen_vision/analyze.py

@@ -0,0 +1,149 @@
+"""Image analysis for file-based inputs (AirDrop, file drop)."""
+
+import base64
+import io
+import os
+from dataclasses import dataclass
+from datetime import datetime, UTC
+from typing import Optional
+
+from PIL import Image
+
+from screen_vision.config import get_config
+from screen_vision.ocr import run_ocr
+from screen_vision.security import SecurityScanner
+
+
+@dataclass
+class AnalyzeResult:
+    """Result of image analysis."""
+
+    base64_image: str
+    resolution: str  # e.g., "1920x1080"
+    source: str  # "file"
+    file_name: str
+    ocr_text: str
+    timestamp: str  # ISO 8601 format
+    security_redactions: int  # Count of PII redactions (work mode only)
+    error: Optional[str] = None
+
+
+def analyze_image(file_path: str, prompt: str = "") -> AnalyzeResult:
+    """Analyze an image file with OCR and security scanning.
+
+    Args:
+        file_path: Path to the image file
+        prompt: Optional analysis prompt (unused for now, reserved for future)
+
+    Returns:
+        AnalyzeResult with image data, OCR text, and security info
+
+    Raises:
+        ValueError: If file doesn't exist or exceeds size limits
+    """
+    config = get_config()
+
+    # Validate file exists
+    if not os.path.exists(file_path):
+        return AnalyzeResult(
+            base64_image="",
+            resolution="0x0",
+            source="file",
+            file_name=os.path.basename(file_path),
+            ocr_text="",
+            timestamp=datetime.now(UTC).isoformat(),
+            security_redactions=0,
+            error=f"File not found: {file_path}",
+        )
+
+    # Check file size (max 50MB)
+    max_size_bytes = 50 * 1024 * 1024
+    file_size = os.path.getsize(file_path)
+    if file_size > max_size_bytes:
+        return AnalyzeResult(
+            base64_image="",
+            resolution="0x0",
+            source="file",
+            file_name=os.path.basename(file_path),
+            ocr_text="",
+            timestamp=datetime.now(UTC).isoformat(),
+            security_redactions=0,
+            error=f"File size {file_size} bytes exceeds maximum {max_size_bytes} bytes",
+        )
+
+    try:
+        # Open and convert to RGB
+        image = Image.open(file_path)
+        if image.mode != "RGB":
+            image = image.convert("RGB")
+
+        # Resize if too large (max 2048px on longest side)
+        max_dimension = 2048
+        width, height = image.size
+        if max(width, height) > max_dimension:
+            if width > height:
+                new_width = max_dimension
+                new_height = int(height * (max_dimension / width))
+            else:
+                new_height = max_dimension
+                new_width = int(width * (max_dimension / height))
+            image = image.resize((new_width, new_height), Image.Resampling.LANCZOS)
+
+        # Run OCR
+        ocr_result = run_ocr(image)
+        ocr_text = ocr_result.text
+
+        # Security scan (work mode only)
+        security_redactions = 0
+        if config.is_work_mode:
+            scanner = SecurityScanner(enabled=True)
+            scan_result = scanner.scan_text(ocr_text)
+
+            # Block if PCI or secrets detected
+            if scan_result.should_block:
+                return AnalyzeResult(
+                    base64_image="",
+                    resolution=f"{image.size[0]}x{image.size[1]}",
+                    source="file",
+                    file_name=os.path.basename(file_path),
+                    ocr_text="",
+                    timestamp=datetime.now(UTC).isoformat(),
+                    security_redactions=0,
+                    error=f"Security scan blocked: {len([f for f in scan_result.findings if f.action == 'BLOCK'])} sensitive items detected",
+                )
+
+            # Count PII redactions
+            security_redactions = len(
+                [f for f in scan_result.findings if f.action == "REDACT"]
+            )
+
+        # Encode as base64 JPEG (quality 75, EXIF stripped)
+        buffer = io.BytesIO()
+        image.save(
+            buffer, format="JPEG", quality=config.default_jpeg_quality, exif=b""
+        )
+        buffer.seek(0)
+        base64_image = base64.b64encode(buffer.read()).decode("utf-8")
+
+        return AnalyzeResult(
+            base64_image=base64_image,
+            resolution=f"{image.size[0]}x{image.size[1]}",
+            source="file",
+            file_name=os.path.basename(file_path),
+            ocr_text=ocr_text,
+            timestamp=datetime.now(UTC).isoformat(),
+            security_redactions=security_redactions,
+            error=None,
+        )
+
+    except Exception as e:
+        return AnalyzeResult(
+            base64_image="",
+            resolution="0x0",
+            source="file",
+            file_name=os.path.basename(file_path),
+            ocr_text="",
+            timestamp=datetime.now(UTC).isoformat(),
+            security_redactions=0,
+            error=f"Error processing image: {str(e)}",
+        )

screen_vision-0.2.1/src/screen_vision/audio.py

@@ -0,0 +1,211 @@
+"""Audio recording and transcription for Screen Vision.
+
+Handles microphone recording, call detection, and speech-to-text transcription.
+Gracefully degrades when optional dependencies (sounddevice, faster-whisper) are missing.
+"""
+
+import subprocess
+from dataclasses import dataclass
+from typing import Any
+
+import numpy as np
+
+# Try to import optional audio dependencies
+try:
+    import sounddevice as sd
+    SOUNDDEVICE_AVAILABLE = True
+except ImportError:
+    sd = None
+    SOUNDDEVICE_AVAILABLE = False
+
+try:
+    from faster_whisper import WhisperModel
+    WHISPER_AVAILABLE = True
+except ImportError:
+    WhisperModel = None
+    WHISPER_AVAILABLE = False
+
+
+# Lazy singleton for Whisper model to avoid reloading on every transcribe() call
+_whisper_model = None
+
+def _get_whisper_model():
+    """Get or create the Whisper model singleton."""
+    global _whisper_model
+    if _whisper_model is None:
+        _whisper_model = WhisperModel("tiny", device="cpu", compute_type="int8")
+    return _whisper_model
+
+
+@dataclass
+class TranscriptSegment:
+    """A segment of transcribed speech with timing information."""
+
+    text: str
+    start_time: float  # seconds
+    end_time: float  # seconds
+    nearest_frame_index: int | None = None
+
+
+# Call detection constants
+CALL_APPS = ["zoom.us", "Microsoft Teams", "Slack", "FaceTime", "Google Chrome"]
+
+
+class AudioRecorder:
+    """Records audio from microphone and transcribes with Whisper."""
+
+    def __init__(self, sample_rate: int = 16000):
+        """Initialize audio recorder.
+
+        Args:
+            sample_rate: Audio sample rate in Hz (default 16000 for Whisper)
+        """
+        self.sample_rate = sample_rate
+        self.buffer: np.ndarray | None = None
+        self._recording = False
+
+    def start(self, duration_seconds: float) -> None:
+        """Record microphone audio for specified duration.
+
+        Args:
+            duration_seconds: How long to record in seconds
+
+        Raises:
+            RuntimeError: If sounddevice is not installed
+
+        Stores audio in self.buffer as mono float32 numpy array.
+        This is a blocking call - it waits for the full duration.
+        """
+        if not SOUNDDEVICE_AVAILABLE:
+            raise RuntimeError(
+                "sounddevice not installed. Install with: pip install sounddevice"
+            )
+
+        # Calculate number of frames needed
+        frames = int(duration_seconds * self.sample_rate)
+
+        # Record audio - blocks until complete
+        self._recording = True
+        recording = sd.rec(
+            frames=frames,
+            samplerate=self.sample_rate,
+            channels=1,  # mono
+            dtype='float32'
+        )
+        sd.wait()  # Wait for recording to complete
+        self._recording = False
+
+        # Store as flattened mono array
+        self.buffer = recording.flatten()
+
+    def stop(self) -> None:
+        """Stop recording if active.
+
+        This is safe to call even if not recording.
+        """
+        if self._recording:
+            # In blocking mode, this doesn't do much, but provided for API consistency
+            self._recording = False
+
+    def transcribe(self) -> list[TranscriptSegment]:
+        """Transcribe audio buffer using Whisper.
+
+        Returns:
+            List of TranscriptSegment objects with timestamped text.
+            Empty list if Whisper is not installed or buffer is empty.
+
+        Requires faster-whisper to be installed. Falls back gracefully if missing.
+        """
+        if not WHISPER_AVAILABLE:
+            return []
+
+        if self.buffer is None or len(self.buffer) == 0:
+            return []
+
+        # Get the singleton Whisper model (avoids reloading on every call)
+        model = _get_whisper_model()
+
+        # Transcribe with word-level timestamps
+        segments_iter, _ = model.transcribe(
+            self.buffer,
+            language="en",
+            word_timestamps=False  # Segment-level is sufficient
+        )
+
+        # Convert to our TranscriptSegment format
+        segments = []
+        for segment in segments_iter:
+            segments.append(
+                TranscriptSegment(
+                    text=segment.text,
+                    start_time=segment.start,
+                    end_time=segment.end
+                )
+            )
+
+        return segments
+
+    def clear(self) -> None:
+        """Clear the audio buffer (zero-persistence)."""
+        self.buffer = None
+
+
+def is_call_active() -> bool:
+    """Check if any call application is using the microphone.
+
+    Returns:
+        True if a known call app is using the mic, False otherwise.
+
+    Gracefully returns False if process checking fails.
+    """
+    try:
+        processes = _get_mic_using_processes()
+        return any(p["name"] in CALL_APPS for p in processes)
+    except Exception:
+        # Fail gracefully - assume no call active
+        return False
+
+
+def _get_mic_using_processes() -> list[dict[str, Any]]:
+    """Get processes currently using the microphone.
+
+    Returns:
+        List of dicts with "name" and "pid" keys.
+
+    Raises:
+        subprocess.SubprocessError: If lsof command fails
+    """
+    try:
+        # On macOS, check which processes are accessing audio devices
+        # Using lsof to find processes with audio file descriptors
+        result = subprocess.run(
+            ["lsof", "-c", "zoom", "-c", "Teams", "-c", "Slack", "-c", "FaceTime", "-c", "Chrome"],
+            capture_output=True,
+            text=True,
+            timeout=5
+        )
+
+        processes = []
+        if result.returncode == 0 and result.stdout:
+            # Parse lsof output
+            lines = result.stdout.strip().split('\n')
+            seen_pids = set()
+
+            for line in lines[1:]:  # Skip header
+                parts = line.split()
+                if len(parts) >= 2:
+                    command = parts[0]
+                    pid = parts[1]
+
+                    if pid not in seen_pids:
+                        seen_pids.add(pid)
+                        processes.append({
+                            "name": command,
+                            "pid": int(pid) if pid.isdigit() else 0
+                        })
+
+        return processes
+
+    except (subprocess.SubprocessError, FileNotFoundError, ValueError):
+        # lsof not available or failed - return empty list
+        return []