cdp-skill 1.0.0

package/SKILL.md

@@ -0,0 +1,543 @@
+---
+name: cdp-skill
+description: Automate Chrome browser interactions via JSON piped to a Node.js CLI. Use when you need to navigate websites, fill forms, click elements, take screenshots, extract data, or run end-to-end browser tests. Supports accessibility snapshots for resilient element targeting.
+license: MIT
+compatibility: Requires Chrome/Chromium running with --remote-debugging-port=9222 and Node.js.
+---
+
+# CDP Browser Automation Skill
+
+Automate Chrome browser interactions via JSON piped to a Node.js CLI. Produce JSON step definitions, not JavaScript code.
+
+## Purpose
+
+This skill enables **AI-powered browser automation**. The intended workflow:
+
+1. **Test definitions** are written as markdown files describing what to test
+2. **An agent** reads the definition, discovers page elements dynamically, and executes using this skill
+3. The agent interprets intent and adapts to page changes - making automation resilient without brittle hardcoded selectors
+
+## Quick Start
+
+Chrome must be running with remote debugging:
+```bash
+# macOS
+/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222
+
+# Linux
+google-chrome --remote-debugging-port=9222
+
+# Windows
+chrome.exe --remote-debugging-port=9222
+```
+
+Execute steps:
+```bash
+echo '{"steps":[{"goto":"https://example.com"}]}' | node src/cli.js
+```
+
+### Tab Reuse (Critical)
+
+The first invocation creates a new tab and returns a `targetId`. **Include this in ALL subsequent calls** to reuse the same tab:
+
+```bash
+# First call - extract targetId from response
+RESULT=$(echo '{"steps":[{"goto":"https://example.com"}]}' | node src/cli.js)
+TARGET_ID=$(echo "$RESULT" | jq -r '.tab.targetId')
+
+# All subsequent calls - include targetId
+echo "{\"config\":{\"targetId\":\"$TARGET_ID\"},\"steps\":[{\"click\":\"#btn\"}]}" | node src/cli.js
+```
+
+Omitting `targetId` creates orphan tabs that accumulate until Chrome restarts.
+
+
+## Input Schema
+
+```json
+{
+  "config": {
+    "host": "localhost",
+    "port": 9222,
+    "targetId": "ABC123...",
+    "timeout": 30000
+  },
+  "steps": [...]
+}
+```
+
+Config is optional on first call. `targetId` required on subsequent calls.
+
+## Output Schema
+
+```json
+{
+  "status": "passed|failed|error",
+  "tab": { "targetId": "ABC123...", "url": "...", "title": "..." },
+  "steps": [{ "action": "goto", "status": "passed", "duration": 1234 }],
+  "outputs": [{ "step": 2, "action": "query", "output": {...} }],
+  "errors": [{ "step": 3, "action": "click", "error": "Element not found" }]
+}
+```
+
+Exit code: `0` = passed, `1` = failed/error.
+
+Error types: `PARSE`, `VALIDATION`, `CONNECTION`, `EXECUTION`
+
+
+## Auto-Waiting
+
+All interaction actions (`click`, `fill`, `hover`, `type`) automatically wait for elements to be actionable before proceeding. Retries use exponential backoff with jitter (1.9-2.1x random factor) to avoid thundering herd issues.
+
+| Action | Waits For |
+|--------|-----------|
+| `click` | visible, enabled, stable, not covered, pointer-events |
+| `fill`, `type` | visible, enabled, editable |
+| `hover` | visible, stable |
+
+**State definitions:**
+- **visible**: In DOM, not `display:none`, not `visibility:hidden`, has dimensions
+- **enabled**: Not disabled, not `aria-disabled="true"`
+- **editable**: Enabled + not readonly + is input/textarea/select/contenteditable
+- **stable**: Position unchanged for 3 consecutive animation frames
+- **not covered**: Element at click coordinates matches target (detects overlays/modals)
+- **pointer-events**: CSS `pointer-events` is not `none`
+
+**Force options:**
+- Use `force: true` to bypass all checks immediately
+- **Auto-force**: When actionability times out but element exists, automatically retries with `force: true`. This helps with overlays, cookie banners, and loading spinners that may obscure elements. Outputs include `autoForced: true` when this occurs.
+
+**Performance optimizations:**
+- Browser-side polling using MutationObserver (reduces network round-trips)
+- Content quads for accurate click positioning with CSS transforms
+- InsertText API for fast form fills (like paste)
+- IntersectionObserver for efficient viewport detection
+
+
+## Element References
+
+The `snapshot` step returns an accessibility tree with refs like `[ref=e4]`. Use refs in subsequent actions:
+
+```json
+{"steps":[{"snapshot": true}]}
+// Response includes: - button "Submit" [ref=e4]
+
+{"config":{"targetId":"..."},"steps":[{"click":{"ref":"e4"}}]}
+```
+
+Refs work with: `click`, `fill`, `hover`.
+
+
+## Step Reference
+
+### Navigation
+
+**goto** - Navigate to URL
+```json
+{"goto": "https://example.com"}
+```
+
+**back** / **forward** - History navigation
+```json
+{"back": true}
+{"forward": true}
+```
+Returns: `{url, title}` or `{noHistory: true}` if no history entry exists.
+
+**waitForNavigation** - Wait for navigation to complete
+```json
+{"waitForNavigation": true}
+{"waitForNavigation": {"timeout": 5000, "waitUntil": "networkidle"}}
+```
+Options: `timeout`, `waitUntil` (commit|domcontentloaded|load|networkidle)
+
+**Note:** For click-then-wait patterns, the system uses a two-step event pattern to prevent race conditions - it subscribes to navigation events BEFORE clicking to ensure fast navigations aren't missed.
+
+
+### Frame/iFrame Navigation
+
+**listFrames** - List all frames in the page
+```json
+{"listFrames": true}
+```
+Returns: `{mainFrameId, currentFrameId, frames: [{frameId, url, name, parentId, depth}]}`
+
+**switchToFrame** - Switch to an iframe
+```json
+{"switchToFrame": "iframe#content"}
+{"switchToFrame": 0}
+{"switchToFrame": {"selector": "iframe.editor"}}
+{"switchToFrame": {"index": 1}}
+{"switchToFrame": {"name": "myFrame"}}
+```
+Options: CSS selector (string), index (number), or object with `selector`, `index`, `name`, or `frameId`
+
+Returns: `{frameId, url, name}`
+
+**switchToMainFrame** - Switch back to main frame
+```json
+{"switchToMainFrame": true}
+```
+Returns: `{frameId, url, name}`
+
+**Note:** After switching to a frame, all subsequent actions execute in that frame context until you switch to another frame or back to main.
+
+
+### Waiting
+
+**wait** - Wait for element
+```json
+{"wait": "#content"}
+{"wait": {"selector": "#loading", "hidden": true}}
+{"wait": {"selector": ".item", "minCount": 10}}
+```
+
+**wait** - Wait for text
+```json
+{"wait": {"text": "Welcome"}}
+{"wait": {"textRegex": "Order #[A-Z0-9]+"}}
+```
+
+**wait** - Wait for URL
+```json
+{"wait": {"urlContains": "/success"}}
+```
+
+**wait** / **delay** - Fixed time (ms)
+```json
+{"wait": 2000}
+{"delay": 500}
+```
+
+**Network idle detection:** The `networkidle` wait condition uses a precise counter-based tracker that monitors all network requests. It considers the network "idle" when no requests have been pending for 500ms.
+
+
+### Interaction
+
+**click** - Click element
+```json
+{"click": "#submit"}
+{"click": {"selector": "#btn", "verify": true}}
+{"click": {"ref": "e4"}}
+{"click": {"x": 450, "y": 200}}
+```
+Options: `selector`, `ref`, `x`/`y`, `verify`, `force`, `debug`, `timeout`
+
+Returns: `{clicked: true}`. With `verify`: adds `{targetReceived: true/false}`. With navigation: adds `{navigated: true, newUrl: "..."}`.
+
+**fill** - Fill input (clears first)
+```json
+{"fill": {"selector": "#email", "value": "user@example.com"}}
+{"fill": {"ref": "e3", "value": "text"}}
+```
+Options: `selector`, `ref`, `value`, `clear` (default: true), `react`, `force`, `timeout`
+
+**fillForm** - Fill multiple fields
+```json
+{"fillForm": {"#firstName": "John", "#lastName": "Doe"}}
+```
+Returns: `{total, filled, failed, results: [{selector, status, value}]}`
+
+**type** - Type text (no clear)
+```json
+{"type": {"selector": "#search", "text": "query", "delay": 50}}
+```
+Returns: `{selector, typed, length}`
+
+**press** - Keyboard key/combo
+```json
+{"press": "Enter"}
+{"press": "Control+a"}
+{"press": "Meta+Shift+Enter"}
+```
+
+**select** - Select text in input
+```json
+{"select": "#input"}
+{"select": {"selector": "#input", "start": 0, "end": 5}}
+```
+Returns: `{selector, start, end, selectedText, totalLength}`
+
+**hover** - Mouse over element
+```json
+{"hover": "#menu"}
+{"hover": {"selector": "#tooltip", "duration": 500}}
+```
+
+**drag** - Drag element from source to target
+```json
+{"drag": {"source": "#draggable", "target": "#dropzone"}}
+{"drag": {"source": {"x": 100, "y": 100}, "target": {"x": 300, "y": 200}}}
+{"drag": {"source": "#item", "target": "#container", "steps": 20, "delay": 10}}
+```
+Options: `source` (selector or {x,y}), `target` (selector or {x,y}), `steps` (default: 10), `delay` (ms, default: 0)
+
+Returns: `{dragged: true, source: {x, y}, target: {x, y}, steps}`
+
+
+### Scrolling
+
+```json
+{"scroll": "top"}
+{"scroll": "bottom"}
+{"scroll": "#element"}
+{"scroll": {"deltaY": 500}}
+{"scroll": {"x": 0, "y": 1000}}
+```
+Returns: `{scrollX, scrollY}`
+
+
+### Data Extraction
+
+**query** - Find elements by CSS
+```json
+{"query": "h1"}
+{"query": {"selector": "a", "limit": 5, "output": "href"}}
+{"query": {"selector": "div", "output": ["text", "href"]}}
+{"query": {"selector": "button", "output": {"attribute": "data-id"}}}
+```
+Options: `selector`, `limit` (default: 10), `output` (text|html|href|value|tag|array|attribute object), `clean`, `metadata`
+
+Returns: `{selector, total, showing, results: [{index, value}]}`
+
+**query** - Find by ARIA role
+```json
+{"query": {"role": "button"}}
+{"query": {"role": "button", "name": "Submit"}}
+{"query": {"role": "heading", "level": 2}}
+{"query": {"role": ["button", "link"], "refs": true}}
+```
+Options: `role`, `name`, `nameExact`, `nameRegex`, `checked`, `disabled`, `level`, `countOnly`, `refs`
+
+Supported roles: `button`, `textbox`, `checkbox`, `link`, `heading`, `listitem`, `option`, `combobox`, `radio`, `img`, `tab`, `tabpanel`, `menu`, `menuitem`, `dialog`, `alert`, `navigation`, `main`, `search`, `form`
+
+**queryAll** - Multiple queries at once
+```json
+{"queryAll": {"title": "h1", "links": "a", "buttons": {"role": "button"}}}
+```
+
+**inspect** - Page overview
+```json
+{"inspect": true}
+{"inspect": {"selectors": [".item"], "limit": 3}}
+```
+Returns: `{title, url, counts: {links, buttons, inputs, images, headings}, custom: {...}}`
+
+**console** - Browser console logs
+```json
+{"console": true}
+{"console": {"level": "error", "limit": 20, "stackTrace": true}}
+```
+Options: `level`, `type`, `since`, `limit`, `clear`, `stackTrace`
+
+Returns: `{total, showing, messages: [{level, text, type, url, line, timestamp, stackTrace?}]}`
+
+Note: Console logs don't persist across CLI invocations.
+
+
+### Screenshots & PDF
+
+**screenshot**
+```json
+{"screenshot": "./result.png"}
+{"screenshot": {"path": "./full.png", "fullPage": true}}
+{"screenshot": {"path": "./element.png", "selector": "#header"}}
+```
+Options: `path`, `fullPage`, `selector`, `format` (png|jpeg|webp), `quality`, `omitBackground`, `clip`
+
+Returns: `{path, viewport: {width, height}, format, fullPage, selector}`
+
+**pdf**
+```json
+{"pdf": "./report.pdf"}
+{"pdf": {"path": "./report.pdf", "landscape": true, "printBackground": true}}
+```
+Options: `path`, `selector`, `landscape`, `printBackground`, `scale`, `paperWidth`, `paperHeight`, margins, `pageRanges`, `validate`
+
+Returns: `{path, fileSize, fileSizeFormatted, pageCount, dimensions, validation?}`
+
+
+### JavaScript Execution
+
+**eval** - Execute JS in page context
+```json
+{"eval": "document.title"}
+{"eval": {"expression": "fetch('/api').then(r=>r.json())", "await": true}}
+```
+Options: `expression`, `await`, `timeout`, `serialize`
+
+**Shell escaping tip:** For complex expressions with quotes or special characters, use a heredoc or JSON file:
+```bash
+# Heredoc approach
+node src/cli.js <<'EOF'
+{"steps":[{"eval":"document.querySelectorAll('button').length"}]}
+EOF
+
+# Or save to file and pipe
+cat steps.json | node src/cli.js
+```
+
+Returns typed results:
+- Numbers: `{type: "number", repr: "Infinity|NaN|-Infinity"}`
+- Date: `{type: "Date", value: "ISO string", timestamp: N}`
+- Map: `{type: "Map", size: N, entries: [...]}`
+- Set: `{type: "Set", size: N, values: [...]}`
+- Element: `{type: "Element", tagName, id, className, textContent, isConnected}`
+- NodeList: `{type: "NodeList", length: N, items: [...]}`
+
+
+### Accessibility Snapshot
+
+**snapshot** - Get accessibility tree
+```json
+{"snapshot": true}
+{"snapshot": {"root": "#container", "maxElements": 500}}
+{"snapshot": {"root": "role=main", "includeText": true}}
+{"snapshot": {"includeFrames": true}}
+```
+Options: `mode` (ai|full), `root` (CSS selector or "role=X"), `maxDepth`, `maxElements`, `includeText`, `includeFrames`
+
+Returns YAML with: role, "name", states (`[checked]`, `[disabled]`, `[expanded]`, `[required]`, `[invalid]`, `[level=N]`), `[name=fieldName]` for form inputs, `[ref=eN]` for clicking.
+
+```yaml
+- navigation:
+  - link "Home" [ref=e1]
+- main:
+  - heading "Welcome" [level=1]
+  - textbox "Email" [required] [invalid] [name=email] [ref=e3]
+  - button "Submit" [ref=e4]
+```
+
+Use `includeText: true` to capture static text (error messages, etc.). Elements with `role="alert"` or `role="status"` always include text.
+
+Use `includeFrames: true` to include same-origin iframe content in the snapshot. Cross-origin iframes are marked with `crossOrigin: true`.
+
+
+### Viewport & Device Emulation
+
+**viewport** - Set viewport size
+```json
+{"viewport": "iphone-14"}
+{"viewport": {"width": 1280, "height": 720}}
+{"viewport": {"width": 375, "height": 667, "mobile": true, "hasTouch": true, "isLandscape": true}}
+```
+Options: `width`, `height`, `deviceScaleFactor`, `mobile`, `hasTouch`, `isLandscape`
+
+Returns: `{width, height, deviceScaleFactor, mobile, hasTouch}`
+
+Presets: `iphone-se`, `iphone-14`, `iphone-15-pro`, `ipad`, `ipad-pro-11`, `pixel-7`, `samsung-galaxy-s23`, `desktop`, `desktop-hd`, `macbook-pro-14`, etc.
+
+
+### Cookie Management
+
+**cookies** - Get/set/clear cookies
+```json
+{"cookies": {"get": true}}
+{"cookies": {"get": ["https://example.com"], "name": "session_id"}}
+{"cookies": {"set": [{"name": "token", "value": "abc", "domain": "example.com", "expires": "7d"}]}}
+{"cookies": {"delete": "session_id"}}
+{"cookies": {"clear": true}}
+```
+
+Set options: `name`, `value`, `url` or `domain`, `path`, `secure`, `httpOnly`, `sameSite`, `expires`
+
+Expiration formats: `30m`, `1h`, `7d`, `1w`, `1y`, or Unix timestamp.
+
+Returns: get → `{cookies: [...]}`, set → `{set: N}`, delete/clear → `{count: N}`
+
+
+### Form Validation
+
+**validate** - Check field validation state
+```json
+{"validate": "#email"}
+```
+Returns: `{valid, message, validity: {valueMissing, typeMismatch, ...}}`
+
+**submit** - Submit form with validation
+```json
+{"submit": "form"}
+{"submit": {"selector": "#login-form", "reportValidity": true}}
+```
+Returns: `{submitted, valid, errors: [{name, type, message, value}]}`
+
+
+### Assertions
+
+**assert** - Validate conditions
+```json
+{"assert": {"url": {"contains": "/success"}}}
+{"assert": {"url": {"matches": "^https://.*\\.example\\.com"}}}
+{"assert": {"text": "Welcome"}}
+{"assert": {"selector": "h1", "text": "Title", "caseSensitive": false}}
+```
+
+URL options: `contains`, `equals`, `startsWith`, `endsWith`, `matches`
+
+
+### Tab Management
+
+**listTabs** - List open tabs
+```json
+{"listTabs": true}
+```
+Returns: `{count, tabs: [{targetId, url, title}]}`
+
+**closeTab** - Close a tab
+```json
+{"closeTab": "ABC123..."}
+```
+Returns: `{closed: "<targetId>"}`
+
+
+### Optional Steps
+
+Add `"optional": true` to continue on failure:
+```json
+{"click": "#maybe-exists", "optional": true}
+```
+
+
+## Debug Mode
+
+Capture screenshots/DOM before and after each action:
+```json
+{
+  "config": {
+    "debug": true,
+    "debugOptions": {"outputDir": "./debug", "captureScreenshots": true, "captureDom": true}
+  },
+  "steps": [...]
+}
+```
+
+
+## Not Supported
+
+Handle via multiple invocations:
+- Conditional logic / loops
+- Variables / templating
+- File uploads
+- Dialog handling (alert, confirm)
+
+
+## Troubleshooting
+
+| Issue | Solution |
+|-------|----------|
+| Tabs accumulating | Include `targetId` in config |
+| CONNECTION error | Start Chrome with `--remote-debugging-port=9222` |
+| Element not found | Add `wait` step first |
+| Clicks not working | Scroll element into view first |
+
+## Best Practices
+
+1. **Discover before interacting** - Use `inspect` and `snapshot` to understand page structure
+2. **Use website navigation** - Click links and submit forms; don't guess URLs
+3. **Be persistent** - Try alternative selectors, add waits, scroll first
+4. **Prefer refs** - Use `snapshot` + refs over brittle CSS selectors
+
+## Feedback
+
+If you encounter limitations, bugs, or feature requests that would significantly improve automation capabilities, please report them to the skill maintainer.
+If you spot opportunities for speeding things up raise this in your results as well.

package/install.js

@@ -0,0 +1,92 @@
+#!/usr/bin/env node
+
+import { fileURLToPath } from 'url';
+import { dirname, basename, join } from 'path';
+import { homedir } from 'os';
+import { existsSync, lstatSync, mkdirSync, symlinkSync, unlinkSync, rmSync, cpSync } from 'fs';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+
+const packageRoot = __dirname;
+const skillName = basename(packageRoot);
+const isDevMode = !packageRoot.includes('node_modules');
+
+const targets = [
+  { name: 'claude', path: join(homedir(), '.claude', 'skills', skillName) },
+  { name: 'codex', path: join(homedir(), '.codex', 'skills', skillName) },
+];
+
+const filesToCopy = [
+  'SKILL.md',
+  'src',
+];
+
+function ensureParentDir(targetPath) {
+  const parent = dirname(targetPath);
+  if (!existsSync(parent)) {
+    mkdirSync(parent, { recursive: true });
+  }
+}
+
+function removeExisting(targetPath) {
+  if (!existsSync(targetPath)) {
+    return;
+  }
+
+  try {
+    const stat = lstatSync(targetPath);
+    if (stat.isSymbolicLink()) {
+      unlinkSync(targetPath);
+    } else {
+      rmSync(targetPath, { recursive: true });
+    }
+  } catch (err) {
+    console.warn(`  Warning: Could not remove existing ${targetPath}: ${err.message}`);
+  }
+}
+
+function createSymlink(source, target) {
+  const type = process.platform === 'win32' ? 'junction' : 'dir';
+  symlinkSync(source, target, type);
+}
+
+function copyFiles(targetPath) {
+  mkdirSync(targetPath, { recursive: true });
+
+  for (const file of filesToCopy) {
+    const sourcePath = join(packageRoot, file);
+    const destPath = join(targetPath, file);
+
+    if (!existsSync(sourcePath)) {
+      console.warn(`  Warning: ${file} not found, skipping`);
+      continue;
+    }
+
+    cpSync(sourcePath, destPath, { recursive: true });
+  }
+}
+
+console.log(`Installing skill: ${skillName}`);
+console.log(`Mode: ${isDevMode ? 'development (symlink)' : 'production (copy)'}`);
+console.log();
+
+for (const target of targets) {
+  try {
+    ensureParentDir(target.path);
+    removeExisting(target.path);
+
+    if (isDevMode) {
+      createSymlink(packageRoot, target.path);
+      console.log(`✓ Dev symlink: ${target.path} -> ${packageRoot}`);
+    } else {
+      copyFiles(target.path);
+      console.log(`✓ Installed to ${target.name}: ${target.path}`);
+    }
+  } catch (err) {
+    console.warn(`✗ Failed to install to ${target.name}: ${err.message}`);
+  }
+}
+
+console.log();
+console.log('Installation complete.');

package/package.json

@@ -0,0 +1,47 @@
+{
+  "name": "cdp-skill",
+  "version": "1.0.0",
+  "description": "Browser automation skill using Chrome DevTools Protocol for Claude Code and Codex",
+  "type": "module",
+  "main": "src/index.js",
+  "bin": {
+    "cdp-skill": "src/cli.js"
+  },
+  "exports": {
+    ".": "./src/index.js",
+    "./utils": "./src/utils.js"
+  },
+  "engines": {
+    "node": ">=22.0.0"
+  },
+  "scripts": {
+    "postinstall": "node install.js",
+    "preuninstall": "node uninstall.js",
+    "test": "node --test --test-force-exit src/tests/*.test.js",
+    "test:run": "node --test --test-force-exit --test-reporter spec src/tests/*.test.js"
+  },
+  "files": [
+    "install.js",
+    "uninstall.js",
+    "SKILL.md",
+    "src/"
+  ],
+  "keywords": [
+    "cdp",
+    "chrome",
+    "devtools",
+    "browser",
+    "automation",
+    "claude-code",
+    "skill"
+  ],
+  "skill": {
+    "name": "cdp-skill",
+    "description": "Automate Chrome browser interactions via CDP",
+    "entry": "./SKILL.md"
+  },
+  "author": "Cezar Lotrean",
+  "license": "MIT",
+  "dependencies": {},
+  "devDependencies": {}
+}