skill-library-mcp 1.0.0

package/skills/computer-use-agents/SKILL.md

@@ -0,0 +1,315 @@
+---
+name: computer-use-agents
+description: "Build AI agents that interact with computers like humans do - viewing screens, moving cursors, clicking buttons, and typing text. Covers Anthropic's Computer Use, OpenAI's Operator/CUA, and open-source alternatives. Critical focus on sandboxing, security, and handling the unique challenges of vision-based control. Use when: computer use, desktop automation agent, screen control AI, vision-based agent, GUI automation."
+source: vibeship-spawner-skills (Apache 2.0)
+---
+
+# Computer Use Agents
+
+## Patterns
+
+### Perception-Reasoning-Action Loop
+
+The fundamental architecture of computer use agents: observe screen,
+reason about next action, execute action, repeat. This loop integrates
+vision models with action execution through an iterative pipeline.
+
+Key components:
+1. PERCEPTION: Screenshot captures current screen state
+2. REASONING: Vision-language model analyzes and plans
+3. ACTION: Execute mouse/keyboard operations
+4. FEEDBACK: Observe result, continue or correct
+
+Critical insight: Vision agents are completely still during "thinking"
+phase (1-5 seconds), creating a detectable pause pattern.
+
+
+**When to use**: ['Building any computer use agent from scratch', 'Integrating vision models with desktop control', 'Understanding agent behavior patterns']
+
+```python
+from anthropic import Anthropic
+from PIL import Image
+import base64
+import pyautogui
+import time
+
+class ComputerUseAgent:
+    """
+    Perception-Reasoning-Action loop implementation.
+    Based on Anthropic Computer Use patterns.
+    """
+
+    def __init__(self, client: Anthropic, model: str = "claude-sonnet-4-20250514"):
+        self.client = client
+        self.model = model
+        self.max_steps = 50  # Prevent runaway loops
+        self.action_delay = 0.5  # Seconds between actions
+
+    def capture_screenshot(self) -> str:
+        """Capture screen and return base64 encoded image."""
+        screenshot = pyautogui.screenshot()
+        # Resize for token efficiency (1280x800 is good balance)
+        screenshot = screenshot.resize((1280, 800), Image.LANCZOS)
+
+        import io
+        buffer = io.BytesIO()
+        screenshot.save(buffer, format="PNG")
+        return base64.b64encode(buffer.getvalue()).decode()
+
+    def execute_action(self, action: dict) -> dict:
+        """Execute mouse/keyboard action on the computer."""
+        action_type = action.get("type")
+
+        if action_type == "click":
+            x, y = action["x"], action["y"]
+            button = action.get("button", "left")
+            pyautogui.click(x, y, button=button)
+            return {"success": True, "action": f"clicked at ({x}, {y})"}
+
+        elif action_type == "type":
+            text = action["text"]
+            pyautogui.typewrite(text, interval=0.02)
+            return {"success": True, "action": f"typed {len(text)} chars"}
+
+        elif action_type == "key":
+            key = action["key"]
+            pyautogui.press(key)
+            return {"success": True, "action": f"pressed {key}"}
+
+        elif action_type == "scroll":
+            direction = action.get("direction", "down")
+            amount = action.get("amount", 3)
+            scroll = -amount if direction == "down" else amount
+            pyautogui.scroll(scroll)
+            return {"success": True, "action": f"scrolled {dir
+```
+
+### Sandboxed Environment Pattern
+
+Computer use agents MUST run in isolated, sandboxed environments.
+Never give agents direct access to your main system - the security
+risks are too high. Use Docker containers with virtual desktops.
+
+Key isolation requirements:
+1. NETWORK: Restrict to necessary endpoints only
+2. FILESYSTEM: Read-only or scoped to temp directories
+3. CREDENTIALS: No access to host credentials
+4. SYSCALLS: Filter dangerous system calls
+5. RESOURCES: Limit CPU, memory, time
+
+The goal is "blast radius minimization" - if the agent goes wrong,
+damage is contained to the sandbox.
+
+
+**When to use**: ['Deploying any computer use agent', 'Testing agent behavior safely', 'Running untrusted automation tasks']
+
+```python
+# Dockerfile for sandboxed computer use environment
+# Based on Anthropic's reference implementation pattern
+
+FROM ubuntu:22.04
+
+# Install desktop environment
+RUN apt-get update && apt-get install -y \
+    xvfb \
+    x11vnc \
+    fluxbox \
+    xterm \
+    firefox \
+    python3 \
+    python3-pip \
+    supervisor
+
+# Security: Create non-root user
+RUN useradd -m -s /bin/bash agent && \
+    mkdir -p /home/agent/.vnc
+
+# Install Python dependencies
+COPY requirements.txt /tmp/
+RUN pip3 install -r /tmp/requirements.txt
+
+# Security: Drop capabilities
+RUN apt-get install -y --no-install-recommends libcap2-bin && \
+    setcap -r /usr/bin/python3 || true
+
+# Copy agent code
+COPY --chown=agent:agent . /app
+WORKDIR /app
+
+# Supervisor config for virtual display + VNC
+COPY supervisord.conf /etc/supervisor/conf.d/
+
+# Expose VNC port only (not desktop directly)
+EXPOSE 5900
+
+# Run as non-root
+USER agent
+
+CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
+
+---
+
+# docker-compose.yml with security constraints
+version: '3.8'
+
+services:
+  computer-use-agent:
+    build: .
+    ports:
+      - "5900:5900"  # VNC for observation
+      - "8080:8080"  # API for control
+
+    # Security constraints
+    security_opt:
+      - no-new-privileges:true
+      - seccomp:seccomp-profile.json
+
+    # Resource limits
+    deploy:
+      resources:
+        limits:
+          cpus: '2'
+          memory: 4G
+        reservations:
+          cpus: '0.5'
+          memory: 1G
+
+    # Network isolation
+    networks:
+      - agent-network
+
+    # No access to host filesystem
+    volumes:
+      - agent-tmp:/tmp
+
+    # Read-only root filesystem
+    read_only: true
+    tmpfs:
+      - /run
+      - /var/run
+
+    # Environment
+    environment:
+      - DISPLAY=:99
+      - NO_PROXY=localhost
+
+networks:
+  agent-network:
+    driver: bridge
+    internal: true  # No internet by default
+
+volumes:
+  agent-tmp:
+
+---
+
+# Python wrapper with additional runtime sandboxing
+import subprocess
+import os
+from dataclasses im
+```
+
+### Anthropic Computer Use Implementation
+
+Official implementation pattern using Claude's computer use capability.
+Claude 3.5 Sonnet was the first frontier model to offer computer use.
+Claude Opus 4.5 is now the "best model in the world for computer use."
+
+Key capabilities:
+- screenshot: Capture current screen state
+- mouse: Click, move, drag operations
+- keyboard: Type text, press keys
+- bash: Run shell commands
+- text_editor: View and edit files
+
+Tool versions:
+- computer_20251124 (Opus 4.5): Adds zoom action for detailed inspection
+- computer_20250124 (All other models): Standard capabilities
+
+Critical limitation: "Some UI elements (like dropdowns and scrollbars)
+might be tricky for Claude to manipulate" - Anthropic docs
+
+
+**When to use**: ['Building production computer use agents', 'Need highest quality vision understanding', 'Full desktop control (not just browser)']
+
+```python
+from anthropic import Anthropic
+from anthropic.types.beta import (
+    BetaToolComputerUse20241022,
+    BetaToolBash20241022,
+    BetaToolTextEditor20241022,
+)
+import subprocess
+import base64
+from PIL import Image
+import io
+
+class AnthropicComputerUse:
+    """
+    Official Anthropic Computer Use implementation.
+
+    Requires:
+    - Docker container with virtual display
+    - VNC for viewing agent actions
+    - Proper tool implementations
+    """
+
+    def __init__(self):
+        self.client = Anthropic()
+        self.model = "claude-sonnet-4-20250514"  # Best for computer use
+        self.screen_size = (1280, 800)
+
+    def get_tools(self) -> list:
+        """Define computer use tools."""
+        return [
+            BetaToolComputerUse20241022(
+                type="computer_20241022",
+                name="computer",
+                display_width_px=self.screen_size[0],
+                display_height_px=self.screen_size[1],
+            ),
+            BetaToolBash20241022(
+                type="bash_20241022",
+                name="bash",
+            ),
+            BetaToolTextEditor20241022(
+                type="text_editor_20241022",
+                name="str_replace_editor",
+            ),
+        ]
+
+    def execute_tool(self, name: str, input: dict) -> dict:
+        """Execute a tool and return result."""
+
+        if name == "computer":
+            return self._handle_computer_action(input)
+        elif name == "bash":
+            return self._handle_bash(input)
+        elif name == "str_replace_editor":
+            return self._handle_editor(input)
+        else:
+            return {"error": f"Unknown tool: {name}"}
+
+    def _handle_computer_action(self, input: dict) -> dict:
+        """Handle computer control actions."""
+        action = input.get("action")
+
+        if action == "screenshot":
+            # Capture via xdotool/scrot
+            subprocess.run(["scrot", "/tmp/screenshot.png"])
+
+            with open("/tmp/screenshot.png", "rb") as f:
+            
+```
+
+## ⚠️ Sharp Edges
+
+| Issue | Severity | Solution |
+|-------|----------|----------|
+| Issue | critical | ## Defense in depth - no single solution works |
+| Issue | medium | ## Add human-like variance to actions |
+| Issue | high | ## Use keyboard alternatives when possible |
+| Issue | medium | ## Accept the tradeoff |
+| Issue | high | ## Implement context management |
+| Issue | high | ## Monitor and limit costs |
+| Issue | critical | ## ALWAYS use sandboxing |

package/skills/computer-vision-expert/SKILL.md

@@ -0,0 +1,70 @@
+---
+name: computer-vision-expert
+description: SOTA Computer Vision Expert (2026). Specialized in YOLO26, Segment Anything 3 (SAM 3), Vision Language Models, and real-time spatial analysis.
+---
+
+# Computer Vision Expert (SOTA 2026)
+
+**Role**: Advanced Vision Systems Architect & Spatial Intelligence Expert
+
+## Purpose
+To provide expert guidance on designing, implementing, and optimizing state-of-the-art computer vision pipelines. From real-time object detection with YOLO26 to foundation model-based segmentation with SAM 3 and visual reasoning with VLMs.
+
+## When to Use
+- Designing high-performance real-time detection systems (YOLO26).
+- Implementing zero-shot or text-guided segmentation tasks (SAM 3).
+- Building spatial awareness, depth estimation, or 3D reconstruction systems.
+- Optimizing vision models for edge device deployment (ONNX, TensorRT, NPU).
+- Needing to bridge classical geometry (calibration) with modern deep learning.
+
+## Capabilities
+
+### 1. Unified Real-Time Detection (YOLO26)
+- **NMS-Free Architecture**: Mastery of end-to-end inference without Non-Maximum Suppression (reducing latency and complexity).
+- **Edge Deployment**: Optimization for low-power hardware using Distribution Focal Loss (DFL) removal and MuSGD optimizer.
+- **Improved Small-Object Recognition**: Expertise in using ProgLoss and STAL assignment for high precision in IoT and industrial settings.
+
+### 2. Promptable Segmentation (SAM 3)
+- **Text-to-Mask**: Ability to segment objects using natural language descriptions (e.g., "the blue container on the right").
+- **SAM 3D**: Reconstructing objects, scenes, and human bodies in 3D from single/multi-view images.
+- **Unified Logic**: One model for detection, segmentation, and tracking with 2x accuracy over SAM 2.
+
+### 3. Vision Language Models (VLMs)
+- **Visual Grounding**: Leveraging Florence-2, PaliGemma 2, or Qwen2-VL for semantic scene understanding.
+- **Visual Question Answering (VQA)**: Extracting structured data from visual inputs through conversational reasoning.
+
+### 4. Geometry & Reconstruction
+- **Depth Anything V2**: State-of-the-art monocular depth estimation for spatial awareness.
+- **Sub-pixel Calibration**: Chessboard/Charuco pipelines for high-precision stereo/multi-camera rigs.
+- **Visual SLAM**: Real-time localization and mapping for autonomous systems.
+
+## Patterns
+
+### 1. Text-Guided Vision Pipelines
+- Use SAM 3's text-to-mask capability to isolate specific parts during inspection without needing custom detectors for every variation.
+- Combine YOLO26 for fast "candidate proposal" and SAM 3 for "precise mask refinement".
+
+### 2. Deployment-First Design
+- Leverage YOLO26's simplified ONNX/TensorRT exports (NMS-free).
+- Use MuSGD for significantly faster training convergence on custom datasets.
+
+### 3. Progressive 3D Scene Reconstruction
+- Integrate monocular depth maps with geometric homographies to build accurate 2.5D/3D representations of scenes.
+
+## Anti-Patterns
+
+- **Manual NMS Post-processing**: Stick to NMS-free architectures (YOLO26/v10+) for lower overhead.
+- **Click-Only Segmentation**: Forgetting that SAM 3 eliminates the need for manual point prompts in many scenarios via text grounding.
+- **Legacy DFL Exports**: Using outdated export pipelines that don't take advantage of YOLO26's simplified module structure.
+
+## Sharp Edges (2026)
+
+| Issue | Severity | Solution |
+|-------|----------|----------|
+| SAM 3 VRAM Usage | Medium | Use quantized/distilled versions for local GPU inference. |
+| Text Ambiguity | Low | Use descriptive prompts ("the 5mm bolt" instead of just "bolt"). |
+| Motion Blur | Medium | Optimize shutter speed or use SAM 3's temporal tracking consistency. |
+| Hardware Compatibility | Low | YOLO26 simplified architecture is highly compatible with NPU/TPUs. |
+
+## Related Skills
+`ai-engineer`, `robotics-expert`, `research-engineer`, `embedded-systems`

package/skills/concise-planning/SKILL.md

@@ -0,0 +1,62 @@
+---
+name: concise-planning
+description: Use when a user asks for a plan for a coding task, to generate a clear, actionable, and atomic checklist.
+---
+
+# Concise Planning
+
+## Goal
+
+Turn a user request into a **single, actionable plan** with atomic steps.
+
+## Workflow
+
+### 1. Scan Context
+
+- Read `README.md`, docs, and relevant code files.
+- Identify constraints (language, frameworks, tests).
+
+### 2. Minimal Interaction
+
+- Ask **at most 1–2 questions** and only if truly blocking.
+- Make reasonable assumptions for non-blocking unknowns.
+
+### 3. Generate Plan
+
+Use the following structure:
+
+- **Approach**: 1-3 sentences on what and why.
+- **Scope**: Bullet points for "In" and "Out".
+- **Action Items**: A list of 6-10 atomic, ordered tasks (Verb-first).
+- **Validation**: At least one item for testing.
+
+## Plan Template
+
+```markdown
+# Plan
+
+<High-level approach>
+
+## Scope
+
+- In:
+- Out:
+
+## Action Items
+
+[ ] <Step 1: Discovery>
+[ ] <Step 2: Implementation>
+[ ] <Step 3: Implementation>
+[ ] <Step 4: Validation/Testing>
+[ ] <Step 5: Rollout/Commit>
+
+## Open Questions
+
+- <Question 1 (max 3)>
+```
+
+## Checklist Guidelines
+
+- **Atomic**: Each step should be a single logical unit of work.
+- **Verb-first**: "Add...", "Refactor...", "Verify...".
+- **Concrete**: Name specific files or modules when possible.